aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authortv <tv@krebsco.de>2026-11-17 13:09:42 +0000
committertv <tv@krebsco.de>2026-11-17 14:23:21 +0100
commitd054476f7dc71b84c4ee5b707806032d567226e6 (patch)
tree321b0a6d2515399dd77ebed4bbfe9ce57163fd8d
parentc5dd5c4b04da76753e351b8ea1e5a1a7e64f3659 (diff)
guides: when things go wrong in a federation
A version a node refused by what it took before, finding from the parties' records who voted twice, having a node drop one side's version and take the other's, answering by a fork, and what keeps a node from holding its store, syncing, taking an event, or starting.
-rw-r--r--docs/guides/federation/README.md2
-rw-r--r--docs/guides/federation/when-things-go-wrong.md81
2 files changed, 83 insertions, 0 deletions
diff --git a/docs/guides/federation/README.md b/docs/guides/federation/README.md
index e4c52f4..021e8e4 100644
--- a/docs/guides/federation/README.md
+++ b/docs/guides/federation/README.md
@@ -16,3 +16,5 @@ by them, in its words.
and a stolen key.
* [`checkpoints.md`](checkpoints.md): what a checkpoint does, proposing and backing one,
signing again after a move, and nodes that weren't there.
+* [`when-things-go-wrong.md`](when-things-go-wrong.md): a version a node refused, a party
+ that voted twice, choosing a side, and a node doing less than it should.
diff --git a/docs/guides/federation/when-things-go-wrong.md b/docs/guides/federation/when-things-go-wrong.md
new file mode 100644
index 0000000..bf79377
--- /dev/null
+++ b/docs/guides/federation/when-things-go-wrong.md
@@ -0,0 +1,81 @@
+# When things go wrong
+
+What to do when a node reports a version it refused, when a party voted twice, and when
+a node doesn't do what it should. The rules are the design's
+([`federation.md`](../../federation.md), Changing the policy; [`store.md`](../../store.md),
+A node's own).
+
+## A node refused a version
+
+Two versions changing one are written only where keys voted twice: a party's `true` and
+`false` on one proposal, each shown to some, or its `true` on two. Each node takes the
+first it judges and refuses the other, with everything built on it, and keeps so across
+restarts (`--store-file`); its log says so (`Refused an event its history takes, by what
+this node took before: kept so`), and `GET /events/refused` lists it under `own`:
+
+```sh
+curl -s -u tv:secret localhost:3000/events/refused | jq .own
+curl -s -u tv:secret localhost:3000/events/<ID> | jq .version # the version it names
+curl -s -u tv:secret localhost:3000/events/counts | jq .version # the one it took
+```
+
+Nodes that took different versions are apart: each refuses what is built on the other's.
+Nothing about the node that put the refused version is held against it: the version was
+valid at its history.
+
+## Finding who voted twice
+
+The votes show it: a party's `true` and `false` on one proposal, or its `true` on two
+proposals of one version, neither dead by all the votes. The parties' clients keep their
+votes, each as signed, and the votes that freed them (`--record`); gather the records
+and ask the client:
+
+```sh
+continuum-party twice tv.record lass.record makefu.record
+# lass voted twice on version 1f4c…: `true` on 8d02… and 3b71…
+```
+
+That proves the party's keys signed both, not that it meant to: two of its clients, two
+of its nodes each voting by itself, a vote by hand beside a node voting for it, or a
+stolen key look the same. Keep the records: once a checkpoint left those votes out, they
+show nowhere else.
+
+## Choosing a side
+
+Which version a node keeps is its operator's choice, and so is joining the other side.
+To take the other side's version, have the node drop what it took that parts it, with
+everything built on it, and take back what it refused:
+
+```sh
+curl -u tv:secret -X POST -H 'Content-Type: application/json' \
+ --data '{"ids": ["<the version event it took>"]}' localhost:3000/admin/events/drop
+# {"dropped": ["…", "…"]}
+curl -u tv:secret -X POST -H 'Content-Type: application/json' \
+ --data '{"ids": ["<the version event it refused>"]}' localhost:3000/admin/events/allow
+```
+
+The node refuses what it dropped from then on, from whoever brings it, and writes no
+version naming what it dropped, until it is allowed again. A node drops only what it
+took after its checkpoint: a version that moved it to a checkpoint it undoes only by
+taking a checkpoint its operator pins (`--store-checkpoint`).
+
+The parties answer keys that voted twice by forking, without the party that voted twice
+or with its keys named anew ([`forking-and-following.md`](forking-and-following.md)).
+Tell the other parties what your node took and what it refused, and send them your
+record.
+
+## A node doing less than it should
+
+* **It holds no store** (`404` from `GET /events`): it hasn't read its document yet, its
+ pinned checkpoint isn't in its document yet (`The pinned checkpoint isn't in the
+ document (yet)`), or its document's root is no store's (`The document's root is no
+ store's: not reading it`).
+* **Nobody syncs with it** (`This node isn't a member of its store`): its party's node
+ list doesn't name it, as the store reads now. The party writes its list
+ (`continuum-party nodes`), through any member.
+* **An event waits** (`waiting` when put): the node lacks an event it builds on. It
+ takes it once that one arrives.
+* **It doesn't start**: its store file doesn't read, or an option of a swarm that isn't
+ federated is set ([`running-a-node.md`](running-a-node.md), Its options).
+* **A write is refused**: [`writing-and-reading.md`](writing-and-reading.md), Why a write
+ is refused.