# What to watch What tells an operator, or a party, that a federated swarm needs them. Watch what every node needs watching ([`operations.md`](../../operations.md), [What to alert on](../../operations.md#what-to-alert-on)), and besides it, the node's store, through its API and its log. ## Through the API `continuum-party status` says everything about the federation at the node but the values (`get`, `values`), with what needs a look in colour on a terminal: ``` store 6d1e8a0c4f2b9d77… checkpoint 6d1e8a0c4f2b9d77… 3 heads, 120 events taken after the checkpoint the node lass's, voting for it by itself knows the base revision, revision 1 (merge-events) version 3 (a7c2e81b9d0453f6…) federation 1, the base revision 3 parties, 2 passing a change parties lass keys the version names (votes, proposals, key set, node list): ssh-ed25519 SHA256:J1nQ… keys of its key set (writes): the same nodes: continuum:9c07e2a4b6d8f0a1… (beta) this node; wire 52; revision 1 (merge-events) tv keys the version names (votes, proposals, key set, node list): ssh-ed25519 SHA256:pX4c… keys of its key set (writes): ssh-ed25519 SHA256:r82M… (--key) nodes: continuum:5be1a3c09f2e4d6b… (alpha) heard 12 s ago; wire 52; the base revision proposals on version 3 8d02f1e3a5c7b9d0…: can still pass, 1 more `true` passing it (2 of 3) true: tv; false: nobody; not yet: lass, makefu proposed by tv: version 4 on 3 lets makefu in: 1 key (ssh-ed25519 SHA256:2kq…); 1 node (continuum:e2a8…) refused no version event of its own you: tv --key ssh-ed25519 SHA256:r82M…: of the key set: writes record tv.record: agrees with the node ``` IDs and hashes are cut short here; the client writes them whole, the shortest prefix naming a proposal in bold, and keys by their fingerprints. It says of every party's nodes the names they gave themselves (`--alias`), how long ago the node heard from each, and what each runs (its wire versions and the newest revision it knows: the nodes to upgrade before a revision passes); of every proposal how it stands, who voted what and who not yet, and what it does, as `show` says it; and of the party it works as (its record's, `--party`'s, or else the node's), its `--key`, its record held against the votes the node shows, what `again` signs again, and what may keep votes away. What people read may change from release to release; `status --json` says the same as one JSON object, whose `format` says which: a later release raises it where the object changes otherwise than by more members. * **A version the node refused** (`own` in `GET /events/refused` not empty): keys voted twice, and nodes may be apart. Act on it at once ([`when-things-go-wrong.md`](when-things-go-wrong.md)). * **A stop** (`stopped` in `GET /events` not `null`): a version passed that the node's continuum doesn't read, and it takes nothing until upgraded, or until its operator has it drop that version event ([`when-things-go-wrong.md`](when-things-go-wrong.md), [A node stopped](when-things-go-wrong.md#a-node-stopped)). * **No store** (`404` from `GET /events` for longer than a start takes): the node can't read its store ([`when-things-go-wrong.md`](when-things-go-wrong.md), [A node doing less than it should](when-things-go-wrong.md#a-node-doing-less-than-it-should)). * **Proposals on the current version** (`proposals` in `GET /events/counts`): each with who voted what, and whether it passed or is dead; the party's record says what it voted. * **The version** (`.version.number` in `GET /events/counts`): a change nobody expected is worth a look at what passed and who voted for it. * **The members** (`members` in `GET /events/counts`): a node of your party missing there syncs with nobody. * **What the nodes run** (`knows` of each party in `GET /events/counts`): the wire versions each node speaks and the newest revision it knows, as far as the node heard from it: the nodes still to upgrade before a revision passes ([`upgrades.md`](upgrades.md), [Moving to a revision](upgrades.md#moving-to-a-revision)). * **Events beyond the checkpoint** (`taken` in `GET /events`): what a move to a checkpoint would let nodes forget; a node voting by itself proposes one at a count you set ([`voting-by-itself.md`](voting-by-itself.md)). ## On the dashboard A node's dashboard (`/admin/dashboard`) shows the same in its document box, read every 5 s: the version, where the store stopped, the parties and which of their nodes are members, the values, the proposals with who voted what and what each changes, and what the node refused ([`dashboard.md`](../../dashboard.md), [Federation](../../dashboard.md#federation)). ## In the log At warn or error: * `Refused an event its history takes, by what this node took before: kept so`: a version the node refused, as above; * `The store stopped at an event this continuum can't judge: it takes none until it runs one that does, or its operator has it drop that one`: a stop, as above; `The store takes events again` once its operator had it drop that event; * `This node isn't a member of its store, as it reads now: its party doesn't list it, and members don't sync with it`; * `The pinned checkpoint isn't in the document (yet): holding no store until it is`; * `The document's root is no store's: not reading it`, `Reading the store from the document failed`; * `Couldn't write the store file: a restart may take what this node refused`: a full or read-only disk; * `Couldn't vote by itself`, `Voting by itself, without its party's record: votes none`; * `Couldn't put what it signed for its party: put again later`, `Couldn't write a version that passed: put again later`, `Couldn't put again events the document lacks`, `Couldn't put a merge event: put again later`; * `Dropped events, as this node's operator says`: whoever may use the node's API did it. At info, worth knowing: `The store's members, as it reads now`, `Writing a version that passed`, `Voting for its party`, `Proposing a checkpoint for its party`, `This node is a member of its store`, `Merged heads beyond what an event may name`.