aboutsummaryrefslogtreecommitdiffstats
path: root/docs/guides/federation/watching.md
blob: a54dd7f2611d8e4b28e13b23835f1467882fd61d (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
# What to watch

What tells an operator, or a party, that a federated swarm needs them. Watch what every
node needs watching ([`operations.md`](../../operations.md),
[What to alert on](../../operations.md#what-to-alert-on)), and besides it, the node's
store, through its API and its log.

## Through the API

* **A version the node refused** (`own` in `GET /events/refused` not empty): keys voted
  twice, and nodes may be apart. Act on it at once
  ([`when-things-go-wrong.md`](when-things-go-wrong.md)).
* **A stop** (`stopped` in `GET /events` not `null`): a version passed that the node's
  continuum doesn't read, and it takes nothing until upgraded, or until its operator has
  it drop that version event
  ([`when-things-go-wrong.md`](when-things-go-wrong.md),
  [A node stopped](when-things-go-wrong.md#a-node-stopped)).
* **No store** (`404` from `GET /events` for longer than a start takes): the node can't
  read its store ([`when-things-go-wrong.md`](when-things-go-wrong.md),
  [A node doing less than it should](when-things-go-wrong.md#a-node-doing-less-than-it-should)).
* **Proposals on the current version** (`proposals` in `GET /events/counts`): each with
  who voted what, and whether it passed or is dead; the party's record says what it
  voted.
* **The version** (`.version.number` in `GET /events/counts`): a change nobody expected
  is worth a look at what passed and who voted for it.
* **The members** (`members` in `GET /events/counts`): a node of your party missing
  there syncs with nobody.
* **Events beyond the checkpoint** (`taken` in `GET /events`): what a move to a
  checkpoint would let nodes forget; a node voting by itself proposes one at a count you
  set ([`voting-by-itself.md`](voting-by-itself.md)).

## On the dashboard

A node's dashboard (`/admin/dashboard`) shows the same in its document box, read every
5 s: the version, where the store stopped, the parties and which of their nodes are
members, the values, the proposals with who voted what and what each changes, and what
the node refused
([`dashboard.md`](../../dashboard.md), [Federation](../../dashboard.md#federation)).

## In the log

At warn or error:

* `Refused an event its history takes, by what this node took before: kept so`: a version
  the node refused, as above;
* `The store stopped at an event this continuum can't judge: it takes none until it runs
  one that does, or its operator has it drop that one`: a stop, as above; `The store
  takes events again` once its operator had it drop that event;
* `This node isn't a member of its store, as it reads now: its party doesn't list it, and
  members don't sync with it`;
* `The pinned checkpoint isn't in the document (yet): holding no store until it is`;
* `The document's root is no store's: not reading it`, `Reading the store from the
  document failed`;
* `Couldn't write the store file: a restart may take what this node refused`: a full or
  read-only disk;
* `Couldn't vote by itself`, `Voting by itself, without its party's record: votes none`;
* `Couldn't put what it signed for its party: put again later`, `Couldn't write a version
  that passed: put again later`, `Couldn't put again events the document lacks`;
* `Dropped events, as this node's operator says`: whoever may use the node's API did it.

At info, worth knowing: `The store's members, as it reads now`, `Writing a version that
passed`, `Voting for its party`, `Proposing a checkpoint for its party`, `This node is a
member of its store`.