diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/architecture.md | 4 | ||||
| -rw-r--r-- | docs/dashboard.md | 104 | ||||
| -rw-r--r-- | docs/measurements.md | 13 | ||||
| -rw-r--r-- | docs/openapi.yaml | 38 | ||||
| -rw-r--r-- | docs/operations.md | 3 | ||||
| -rw-r--r-- | docs/peers.md | 5 |
6 files changed, 51 insertions, 116 deletions
diff --git a/docs/architecture.md b/docs/architecture.md index 0f0bad7..f4ce611 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -77,7 +77,7 @@ it switched to the right document without trusting anyone. ## Modules **[`transport/`](../src/transport/)** — [`mod.rs`](../src/transport/mod.rs) defines the message types (HELLO, HELLO_REPLY, sync -envelope, STATUS/STATUS_REPLY, PING/PONG, blob messages, LINKS/LINKS_REPLY, PROBE/PROBE_REPLY) and the `Transport` trait; +envelope, STATUS/STATUS_REPLY, PING/PONG, blob messages, PROBE/PROBE_REPLY) and the `Transport` trait; [`wire.rs`](../src/transport/wire.rs) is the wire format (a version byte, bumped with every change to the format, and a kind byte, then the message in MessagePack, derived from the types in [`mod.rs`](../src/transport/mod.rs) with codecs for hashes and IDs); [`mycelium/`](../src/transport/mycelium/) (with the `mycelium` feature) is the REST @@ -118,7 +118,6 @@ ignored 25-node soak test with a propagation bound (`cargo test soak -- --ignore | `tree` | writers' trees: passing each writer's changes on to the members after us in its tree, the same on every node | [`peers.md`](peers.md) | | `reach` | whom each neighbour reaches, as it says in its STATUS: the 2-hop neighbourhood trees are built on | [`peers.md`](peers.md) | | `relay` | announcing heads where trees don't reach (at once) and to everyone else (slow round), and pulling what's missing | module docs | -| `links` | answering LINKS with this node's links, for the dashboard's map | [`dashboard.md`](dashboard.md) | | [`topology/`](../src/swarm/topology/) | which members are neighbours: ring plus low-score links, stable members preferred | module docs | | `members` | the member registry: aliases, alive stamps, tiers, the electorate | module docs | | `stability` | a node's own connectivity record and its stable/transient tier | module docs | @@ -237,7 +236,6 @@ replies, admit (swarm key), mark the peer seen, record its epoch (ignoring late then by type: HELLO answered, HELLO_REPLY establishes membership and discovers peers, sync queued on its lane if on our epoch (held back from everyone but the source while catching up), STATUS answered with a STATUS_REPLY (an announcement only if the receiver lacks some of its heads) and compared for divergence, blobs served or collected, -LINKS answered with the node's links if the asker is a member, and again on each change while it subscribes, or with other members' links it asks for the asker, which can't reach them (`swarm::links`, for the dashboard); PING and PONG (the paths layer answered the PING already) only mark the peer seen, and after a PONG the peer's round-trip time is its path's again. diff --git a/docs/dashboard.md b/docs/dashboard.md index 5a66afd..05069d1 100644 --- a/docs/dashboard.md +++ b/docs/dashboard.md @@ -2,8 +2,8 @@ A page showing one node's view of the swarm, live: a map of the members and the links between them, the node's peers, its figures, and the document, which it can also edit. -This page is about using it; how it works underneath (events, LINKS, what is sent -when) is at the end, [Underneath](#underneath). +This page is about using it; how it works underneath (events, subscriptions, what is +sent when) is at the end, [Underneath](#underneath). ## Opening it @@ -67,10 +67,10 @@ are collapsed is kept in the browser too. Every member of the swarm, by alias (or a shortened node ID): * **This node**: filled blue, slightly larger. -* **Filled**: the member reports its links to this node. -* **Hollow**: it reports none: it doesn't answer this node's LINKS, or its last answer - is more than 90 s old (it is unreachable from here, stopped, or doesn't yet take this - node for a member). +* **Filled**: the page has the member's links: this node's own, and those of the nodes + it subscribes to (below). +* **Hollow**: it has none of its links: the page doesn't subscribe to it, or doesn't + reach it. * **Black centre, ring of this node's blue**: subscribed: the page reads its links from it, directly (below); the ring grey while the browser doesn't reach it (why, in its tooltip). @@ -94,10 +94,9 @@ Every member of the swarm, by alias (or a shortened node ID): Settled but not decided yet, the coordinator is hollow, with a golden question mark in it. Of the remakes in this node's document: every member's vote shows, as it reaches this node. * **A golden glow**, growing from it and fading, once: it just entered a new epoch (a - remake, [`remakes.md`](remakes.md)). For every member that reports its links (they - say its epoch), and this node's peers, as they change while the page is open: so - after a remake, a wave across the map as the members switch. With `links: every - 30 s`, up to that late. + remake, [`remakes.md`](remakes.md)). For this node's peers and the nodes subscribed + to (their links say their epoch), as they change while the page is open: so after a + remake, a wave across the map as the members switch. ### Links @@ -118,7 +117,7 @@ else the fresher. Then: * **Solid**: direct. * **Dashed**: through a relay. -* **Dotted**: one end reports no links: what is drawn may be out of date. +* **Dotted**: the page has one end's links only: drawn as that end reports it. * **Narrow, over a red band**: either end holds the other dormant, or the path in use is down (no path to the peer answers; `/peers` shows its connection as `null`). @@ -145,12 +144,7 @@ legend and the sliders greyed meanwhile); the choice is kept in the browser. A label on the map that changes, a round trip or a node's name, breathes once: it glows around its new text and fades (not where the system asks for reduced motion). -The map's controls are at its top right, over the map as the legend is (below): -`links: live` says the members tell this node of their links' -changes as they happen. Clicked, it's `links: every 30 s`: the members are only asked -every 30 s, which costs the swarm fewer messages and leaves the map up to 30 s behind -(this node's own links stay current). It's the node's choice, for every page open on -it, until clicked again or the node restarts. +The map's controls are at its top right, over the map as the legend is (below). The legend, over the top of the map, names the transports that appear; its contents are on an underlay of their own shape, as the map's labels, so they read over what is @@ -205,7 +199,6 @@ measured. they were (on a dynamic map they then move to their places among the other node's links, smoothly; new ones are placed among them), the view, the pins, what is in focus, the legend shown or hidden, what is open in the figures and in the document, - `links: live` or `every 30 s` (the other node's choice, for every page open on it), and this browser's settings for the page (sizes, collapsed boxes, the layout, the messages' size and speed, the console's filter), which are per host otherwise. They travel in the URL's fragment, which no request carries, and the page removes it @@ -263,7 +256,7 @@ The box is as tall as its rows, up to eight; then it scrolls. The figures of what is in focus, as a tree, its keys at the top level and how many beside the title: this node's `/admin/stats`, or a peer's `/peers` entry and, after it, -the links the member reports (`links`). +the links it reports (`links`), where the page has them. Branches stay as opened across updates, and ages count on between them. What each figure means: [`measurements.md`](measurements.md). Some values get a reading after them, muted, in parentheses: a node ID its alias @@ -316,7 +309,7 @@ miss the commit before the others switch, and rejoin). The answer shows below: t The messages this node exchanged with its peers, one by one: when (on the 24-hour clock, to the millisecond), which way (`→` sent, `←` received, `↛` refused: the transport wouldn't send it), with whom, the type (`HELLO`, `SYNC`, `STATUS_REPLY`, -`LINKS`, …, in its kind's colour, as on the map) and the size. The latest 500, the +`PING`, …, in its kind's colour, as on the map) and the size. The latest 500, the newest at the bottom; scrolled along while at the bottom, which `following…` beside the title says. `clear` empties it; what comes from then on fills it again. They arrive with every update (every 2 s). The PINGs the paths layer sends on its own, and the PONGs it answers @@ -338,7 +331,7 @@ the map or in the peers, goes back. Pointing at a message marks the others of its exchange (a lighter grey, and a bar on their left); while one is shown, its exchange stays marked. An exchange is a PING and its PONG (by their nonce, wherever they were listed), a HELLO and its replies (by the -run of the node that sent the HELLO), a LINKS and its replies (by their nonce), a blob's +run of the node that sent the HELLO), a blob's requests and answers, from whichever peers (by the blob), and a delta offered in a SYNC with the blob it is fetched as. A STATUS and its reply carry nothing that ties them, nor does an ordinary SYNC. @@ -379,15 +372,14 @@ again. * **The status stays at `connecting…`, or the page doesn't load**: the node isn't built with `dashboard` (404), or the user isn't in `--htpasswd` (401). -* **A hollow node**: no links report from it. If it is also dormant here, it is - probably down or unreachable; if not, it may not take this node for a member yet - (it hasn't registered it), or its answers don't arrive. A node that just joined - through others shows as soon as one of them reports a link to it, hollow until its - registration reaches this node and it is asked for its links (within 30 s). +* **A hollow node**: the page has none of its links: subscribe to it (double-click). If + the ring stays grey, the browser doesn't reach its dashboard (why, in its tooltip). + A node that just joined through this one shows at once, hollow; through others, once + its registration reaches this node (the graph is computed again within 30 s). * **Only dashed grey between two nodes**: the member list wants them as neighbours, but neither reports a link to the other: they haven't reached each other (yet). -* **A dotted link**: one end has stopped reporting; the link is as it was last - reported. +* **A dotted link**: the page has one end's links only (subscribe to the other to + see its side). * **A red band**: one end holds the other dormant. If the other end still reports the link, the trouble is likely in one direction only. @@ -408,42 +400,16 @@ them; the page reconnects once the node is back, and reloads if the node now ser another version of it (the state's `page`). **The map's graphs.** The intended one is the neighbours the member list gives every -member (`swarm::topology`), as this node computes them. The measured one is every -member's links as it reports them in reply to a LINKS (sent to every member, at most -512, while the producer runs), with the epoch it is in. The LINKS subscribes: the -member replies again whenever its links change (a peer, a transport, relayed, dormant, -or a round trip moved by a fifth; it looks every 2 s) or it enters a new epoch, for 90 s, renewed every 30 s (each renewal answered), and -ended when the producer stops; a node that is gone stops it by not renewing. A reply -older than 90 s is left out. A member keeps 64 subscribers at most. With `links: every -30 s` (`POST /admin/dashboard/links?live=false`; `true` back), the subscriptions end -and the members are only asked every 30 s, for every page open on this node, until -asked otherwise or the node restarts. - -A member this node can't reach (on none of its transports: one on Tor alone, from a -node on libp2p alone; or one that doesn't answer its HELLOs, as behind NAT) is asked -through a peer that can: an active member it reaches -that, by the member list, shares a transport with the member, else one on a transport -this node isn't on (the lowest pair score of several). The LINKS names the members to -ask (`of`): the peer asks them on this node's behalf, itself or again through a peer -of its own (three nodes at most), for as long as this node asked, and passes their -links on in its replies (`relayed`, each with its age), several together (64 members' -at most, and 256 KiB as encoded but for one member's alone): it holds what arrives a -while. The asker's window (250 ms, at most 5 s) is for the whole way: -each hop holds its share of what is left, the window divided among it and the hops -that may follow (at least 10 ms), and passes the rest on, so links reach the asker -within the window however long the chain. Such a subscription isn't ended by a message: every hop's runs -out unless renewed, so it ends within 90 s of the page closing or the node going away. -A hop renews its own as soon as it is renewed, for the same time, so the hops run in -step, each a message's delay behind the one before (it never runs out first), and a -renewal keeps its nonce, so what is on its way under the last one still counts. A -renewal lost on the way leaves the hops behind it to run out at the old time: with 90 s -renewed every 30 s, one lost costs nothing. - -A way of asking a member that brings none of its links within 20 s (the member doesn't -answer this node after all, or the peer relaying doesn't pass anything on) is left out -for that member for 5 minutes: the next renewal goes the next way (directly, then the -peer picked as above, then the next one), and the old way runs out by itself. Once -every way has failed, all are tried again. Every hop does the same for what it asks on. +member (`swarm::topology`), as this node computes them, every 30 s. The measured one is +this node's links, in the state, and those of the nodes the page subscribes to, which +it reads from each node itself: `GET /admin/dashboard/links/own/events` there, a +`links` event with its links at once and again whenever they change (a peer, a +transport, relayed, dormant, or a round trip moved by a fifth; it looks every 2 s) or +it enters a new epoch, with the epoch it is in. The page asks with a token from this +node that reads the member list only (`Authorization: Bearer`, [`access.md`](access.md), +Tokens), refreshed half way through its life; the other node allows the page's origin +by CORS where it is a member's dashboard (`--cors-origins` for others). A stream that +fails is opened again, waiting longer each time (up to a minute). The links a member reports: its neighbours and the peers it holds dormant (a neighbour that goes dormant is one no more, but stays listed until it is heard from again), by @@ -451,13 +417,11 @@ which transport, whether only through a relay, the round trip (as the paths laye measures it), whether it holds the peer dormant or the path is down, the peer's other transports with a path up (each with its fastest path's round trip, and whether it goes only through a relay), and the peer's clock against the member's ([`measurements.md`](measurements.md), -Per peer, `clock`). A member answers a LINKS from members only, at most every 5 s, and -asks for each peer that has it relay at most 512 members at once and 32 a second on -average (the rest of a request is dropped); every build answers, with or without the -feature (`swarm::links`). +Per peer, `clock`). -**Clocks.** The clocks in the links chain every member's clock to this node's, along -the narrowest bounds, and the map takes each against their median: a member more than +**Clocks.** The clocks in the links on the map (this node's and the subscribed nodes') +chain every member's clock to this node's, along the narrowest bounds, and the page +takes each against their median: a member more than `--clock-skew-warn` off it, beyond the chained bound, is ringed. With fewer than three members compared, none is. diff --git a/docs/measurements.md b/docs/measurements.md index a444862..9036031 100644 --- a/docs/measurements.md +++ b/docs/measurements.md @@ -2,8 +2,7 @@ What a node measures about itself and its peers, where to read it, and what the numbers mean. Everything here comes from traffic and work the node has anyway; it -sends no probes of its own and nothing is measured by anyone else, but for the -dashboard's map (LINKS, only while someone looks at it). +sends no probes of its own and nothing is measured by anyone else. Four places to read: @@ -179,7 +178,7 @@ clocks"); setting them is the operating system's job (NTP). Messages and payload bytes by direction (`sent`, `received`, `refused`) and kind (`hello`, `helloReply`, `sync`, `status`, `statusReply`, `blob`, `identity` for WHO and -ME, `paths` for PING and PONG, `links` for LINKS and their replies, `probe` for PROBE +ME, `paths` for PING and PONG, `probe` for PROBE and PROBE_REPLY, `plumtree` for PRUNE and GRAFT, `hyparview` for NEIGHBOUR, NEIGHBOUR_REPLY and DISCONNECT). The layers below the node count WHO and ME, and the paths layer's own PINGs and PONGs, under the address they went to or came from; a peer @@ -302,13 +301,13 @@ outgrowing the full save (busy swarm, or a `--document-rewrite` set low). In builds with the `dashboard` cargo feature, `GET /admin/dashboard` is a page showing `/admin/stats`, `/peers` and a map of the swarm, live: [`dashboard.md`](dashboard.md), and how it works -underneath (events, LINKS) at its end. Its LINKS are the one thing a node sends only to -measure: the links every member reports, while someone looks. +underneath (events, subscriptions) at its end. Other members' links it reads from +them, over HTTP, from the nodes the page subscribes to: no messages in the swarm. ## What is not measured -* Nothing about other nodes but what they send us: no probes, no remote figures (but - the dashboard's LINKS). +* Nothing about other nodes but what they send us: no probes, no remote figures (the + dashboard's page reads those of the nodes it subscribes to from them). * Clock skew between nodes that aren't each other's peers: only peers' clocks are compared (`clock`, per peer). * Remake durations and outcomes: the remake reports (`GET /admin/remakes`) hold them. diff --git a/docs/openapi.yaml b/docs/openapi.yaml index 20cbf80..4b82ff7 100644 --- a/docs/openapi.yaml +++ b/docs/openapi.yaml @@ -992,20 +992,18 @@ paths: The dashboard's latest snapshot: `node` (`id`, `alias`), `page` (the page's version, a hash: a page showing another reloads), `stats` (as `/admin/stats`), `peers` (as `/peers`) and `graph`: `members` (by node ID, - those of the member list, and any a link reported goes to: `alias`, `self`, + those of the member list, and any of this node's links goes to: `alias`, `self`, `dashboard`: its `--dashboard-url`), `intended` (by node ID: the neighbours the member list gives - it, as this node computes them), `measured` (by node ID: `secsAgo`, `epoch` - (the epoch the member said it is in: `generation`, `id`) and `links`, each with `peer`, `transport`, `relayed`, `down`, `rttMs`, `dormant` and - `others` (the peer's other transports: `transport`, `rttMs`, `relayed`), as the member - answered a LINKS; ours always, the others' while fresh) and `live` (whether the - members' links are subscribed to, `/admin/dashboard/links`), and `remakes` + it, as this node computes them) and `measured` (this node's own, by its node ID: + `secsAgo`, `epoch` (`generation`, `id`) and `links`, each with `peer`, `transport`, `relayed`, `down`, `rttMs`, `dormant`, `clock` and + `others` (the peer's other transports: `transport`, `rttMs`, `relayed`); other + nodes' the page reads from them, `/admin/dashboard/links/own/events`), and `remakes` (those of this node's epoch: `id`, `coordinator`, `phase` as in `/admin/remakes`, `voters` (node IDs), `fromGeneration`, the epoch's, and once committed `switchDelayMs` and `switchInMs`, this node's wait to switch and what is left of it). One producer per node makes it every 2 s while anyone looks (and for a minute after this - request), subscribed to the members' links (each tells of a change as it - happens; renewed every 30 s); every client gets the same. Only with the `dashboard` cargo feature. + request); every client gets the same. Only with the `dashboard` cargo feature. responses: '200': description: The snapshot. @@ -1076,30 +1074,11 @@ paths: summary: Dashboard refresh description: | The dashboard's producer builds the state now rather than at its next tick, and - asks the members for their links again; what changed comes as events. Only with + computes the member list's graph again; what changed comes as events. Only with the `dashboard` cargo feature. responses: '204': description: Done. - /admin/dashboard/links: - post: - tags: [Admin] - summary: Dashboard links, live or asked - description: | - Whether the dashboard's producer subscribes to the members' links (`live=true`, - the default: each member tells of a change as it happens) or only asks for them - every 30 s (`live=false`: fewer messages in the swarm, a map up to 30 s - behind). For every client of this node, until set again or the node restarts; - the state's `graph.live` says which. Only with the `dashboard` cargo feature. - parameters: - - name: live - in: query - required: true - schema: - type: boolean - responses: - '204': - description: Done. /admin/dashboard/tree: get: tags: [Admin] @@ -2075,9 +2054,6 @@ components: paths: $ref: '#/components/schemas/Counter' description: PING and PONG; the paths layer's own PINGs counted under the address (so only in the totals). - links: - $ref: '#/components/schemas/Counter' - description: LINKS and LINKS_REPLY, asked for and answered for a map of the swarm (the dashboard). probe: $ref: '#/components/schemas/Counter' description: PROBE and PROBE_REPLY, asking others whether they reach a member before recording a link to it as unreached. diff --git a/docs/operations.md b/docs/operations.md index 2fd8e7f..f8dcdef 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -249,7 +249,6 @@ And from the log, at warn or error: since its last rewrite (`--document-rewrite`); the state file (small); Tor's state directory. * **Network**: per neighbour, about two keepalives per `--keepalive` interval when - quiet ([`measurements.md`](measurements.md), Traffic), plus sync for every write, and LINKS while a - dashboard is open; a joining node downloads the document once (`savedBytes`, about). + quiet ([`measurements.md`](measurements.md), Traffic), plus sync for every write; a joining node downloads the document once (`savedBytes`, about). * **Processes and ports**: the HTTP API (`--bind-addr`); a supervised Mycelium daemon and its overlay port; libp2p's listen addresses. diff --git a/docs/peers.md b/docs/peers.md index 85d65a9..4aa1334 100644 --- a/docs/peers.md +++ b/docs/peers.md @@ -246,7 +246,7 @@ stateDiagram-v2 - Dormancy is only judged while the node is ready, so joining nodes keep fast retries. A peer's farewell (it is shutting down: a STATUS with `leaving`) makes it dormant at once, whatever our phase; it isn't counted as an incident (`dormancies`). What that - run still sends while it stops (sync messages, PINGs, a LINKS_REPLY) doesn't make it + run still sends while it stops (sync messages, PINGs) doesn't make it active again: only a HELLO or HELLO_REPLY from its next run does. - **A suspended machine** (a laptop asleep) counts its peers as silent for as long as it slept: the monotonic clock the timers run on stops meanwhile, so without that every @@ -376,8 +376,7 @@ what they know, contacting nobody: an active session that answers; a session tha doesn't, held dormant or unanswered HELLOs; or can't tell): if some don't and none does, the member is away and nothing is recorded (asked again after another hold); otherwise, or with no answer within 10 s, the link is recorded. A node that reaches nobody at all is the one away, and -records nothing. PROBEs are their own message, apart from the dashboard's -LINKS, which a node may leave unanswered. +records nothing. A writer's changes pass on along its tree ([`swarm/tree.rs`](../src/swarm/tree.rs)), the same on every node: built on the graph the member list gives (as if every member reached every other), less |