aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--docs/federation.md7
-rw-r--r--docs/guides/federation/upgrades.md43
-rw-r--r--docs/guides/federation/watching.md4
-rw-r--r--docs/guides/federation/writing-and-reading.md5
-rw-r--r--docs/openapi.yaml39
-rw-r--r--docs/peers.md7
-rw-r--r--src/api/events.rs46
-rw-r--r--src/federation/client.rs66
-rw-r--r--src/federation/party.rs42
-rw-r--r--src/swarm/speaking.rs26
10 files changed, 260 insertions, 25 deletions
diff --git a/docs/federation.md b/docs/federation.md
index 8dc23b3..615bc05 100644
--- a/docs/federation.md
+++ b/docs/federation.md
@@ -904,8 +904,11 @@ swarm's root names holds no store, and says so.
So a revision is rolled out, then passed: every party's operators restart its nodes on a
continuum that knows it, and the parties pass a version naming it once theirs do, as a
-party's client backs no version it doesn't read. Who hasn't by then stops at that version,
-and takes part again once it has. Once a store moved to a checkpoint at a version naming
+party's client backs no version its node doesn't read. Each node tells its peers the
+newest revision it knows (SPEAKS, [`peers.md`](peers.md),
+[Handshake and join](peers.md#handshake-and-join)), and a party's client warns, backing
+one, of every node a party lists not known to know its revision. Who hasn't upgraded by
+then stops at that version, and takes part again once it has. Once a store moved to a checkpoint at a version naming
a revision, nodes join it only from a pinned checkpoint, and nobody judges what came
before again: a later continuum may hold only stores at that revision or after, and leave
out the rules of the revisions before.
diff --git a/docs/guides/federation/upgrades.md b/docs/guides/federation/upgrades.md
index 0a1a094..55dd921 100644
--- a/docs/guides/federation/upgrades.md
+++ b/docs/guides/federation/upgrades.md
@@ -15,7 +15,9 @@ the same Automerge on every node, and one with another rejoins instead of switch
continuum's first release, take every upgrade as such a change.
Nobody runs every node of a federated swarm: each party's operators upgrade its nodes.
-Agree on when, among the parties, and upgrade close together.
+Agree on when, among the parties, and upgrade close together. Which wire versions each
+node the parties list speaks, as far as a node heard from it, shows in
+`GET /events/counts` (`knows`, below: `wire`, the oldest and the newest).
## The federation's number and its revision
@@ -61,7 +63,22 @@ adds is refused everywhere until a version names it.
```
and the revision of the current version, in `GET /events/counts`
- (`.version.revision.named`).
+ (`.version.revision.named`). There too, of each node the parties list, the newest
+ revision it said it knows (`knows`; [`peers.md`](../../peers.md),
+ [Handshake and join](../../peers.md#handshake-and-join)), as far as this node heard
+ from it: the nodes still to upgrade, every party's.
+
+ ```sh
+ curl -s -u tv:secret localhost:3000/events/counts | jq -r '.parties | to_entries[]
+ | .key as $party | .value.knows | to_entries[]
+ | "\($party) \(.key): \(.value.revision.named // "said nothing")"'
+ # lass continuum:9c07…: the base revision
+ # tv continuum:5be1…: revision 1 (merge-events)
+ # tv continuum:77d3…: said nothing
+ ```
+
+ A node says nothing on a build from before it said this, or where this node didn't
+ hear from it since it started: each says it with each handshake.
2. **Propose.** A party begins the next version following it, by the feature it adds;
`next` says so where its node knows a later revision than the current version's:
@@ -74,8 +91,9 @@ adds is refused everywhere until a version names it.
# proposal 5e81…
```
- It may already hold what the revision adds to a version: a version is read by the
- revision it names. Its proposal and the votes on it are judged under the current
+ Where nodes the parties list aren't known to know the revision, `next` warns, and
+ names them. It may already hold what the revision adds to a version: a version is read
+ by the revision it names. Its proposal and the votes on it are judged under the current
version, and what the revision adds to events counts from the version after.
3. **Vote.** Each party backs it once every node of the party knows the revision:
@@ -90,10 +108,19 @@ adds is refused everywhere until a version names it.
--record tv.record vote 5e81… yes
```
- The client backs it only where its node knows the revision, and says so otherwise; it
- can't see the party's other nodes, which its operators check. A node voting by itself
- backs no version changing the revision: vote on it by hand, through that node. A party
- not ready leaves it be, or votes `false` to end it.
+ The client backs it only where its node knows the revision, and says so otherwise. It
+ backs it where other nodes, of any party, aren't known to know it, and warns, naming
+ them and what each said (`knows`, above): once it passes, those stop at it till
+ upgraded.
+
+ ```
+ Warning: these nodes aren't known to know revision 1 (merge-events), as far as the
+ node heard from them; once a version of it passes, each stops at it till upgraded:
+ tv: continuum:77d3… said nothing: a build from before SPEAKS, or not heard from
+ ```
+
+ A node voting by itself backs no version changing the revision: vote on it by hand,
+ through that node. A party not ready leaves it be, or votes `false` to end it.
4. **After.** Once two thirds passed it, what the revision adds counts from that version
on. A node that doesn't know it stops at that version (`stopped` in `GET /events`,
diff --git a/docs/guides/federation/watching.md b/docs/guides/federation/watching.md
index da4a93f..9db3399 100644
--- a/docs/guides/federation/watching.md
+++ b/docs/guides/federation/watching.md
@@ -25,6 +25,10 @@ store, through its API and its log.
is worth a look at what passed and who voted for it.
* **The members** (`members` in `GET /events/counts`): a node of your party missing
there syncs with nobody.
+* **What the nodes run** (`knows` of each party in `GET /events/counts`): the wire
+ versions each node speaks and the newest revision it knows, as far as the node heard
+ from it: the nodes still to upgrade before a revision passes
+ ([`upgrades.md`](upgrades.md), [Moving to a revision](upgrades.md#moving-to-a-revision)).
* **Events beyond the checkpoint** (`taken` in `GET /events`): what a move to a
checkpoint would let nodes forget; a node voting by itself proposes one at a count you
set ([`voting-by-itself.md`](voting-by-itself.md)).
diff --git a/docs/guides/federation/writing-and-reading.md b/docs/guides/federation/writing-and-reading.md
index 057aad7..d45bf0d 100644
--- a/docs/guides/federation/writing-and-reading.md
+++ b/docs/guides/federation/writing-and-reading.md
@@ -97,7 +97,10 @@ Votes and proposals aren't signed again: their version is behind.
`GET /events/counts` is what counts, as the node reads it now:
* `version`: the current version, its `number`, its `hash`, and the version itself;
-* `parties`: each party's `keys` and `nodes`, as they count;
+* `parties`: each party's `keys` and `nodes`, as they count, and of those nodes what
+ each said it speaks and knows (`knows`: the wire versions, the newest revision; null
+ where the node heard nothing from it), for upgrades
+ ([`upgrades.md`](upgrades.md), [Moving to a revision](upgrades.md#moving-to-a-revision));
* `values`: each place holding a value, with it;
* `members`: each node of the swarm, with its party, `[node, party]`;
* `proposals`: the proposals on the current version, by hash, each with the events
diff --git a/docs/openapi.yaml b/docs/openapi.yaml
index 05e29d5..b1e1e28 100644
--- a/docs/openapi.yaml
+++ b/docs/openapi.yaml
@@ -1550,7 +1550,12 @@ paths:
description: |
What counts, as the store reads now (docs/federation.md): the current version
(its number, its hash, its revision, by number and ID and as people read it, and
- the version itself), each party's keys and nodes, the
+ the version itself), each party's keys and nodes, what each of those nodes said
+ it speaks and knows (`knows`, by node ID: the wire versions it speaks, the oldest
+ and the newest, and the newest revision of the federation it knows, by number
+ and ID and as people read it, its number null where this node doesn't know it;
+ null where the node said nothing, a build from before SPEAKS or one this node
+ didn't hear from; this node's own as it is), the
value at each place, the members, each `[node, party]`, and the proposals on the
current version, by their hash, each with the IDs of the events proposing it that
a vote may build on (`proposals`: of a version naming a checkpoint, those above
@@ -1576,6 +1581,38 @@ paths:
type: object
parties:
type: object
+ additionalProperties:
+ type: object
+ required: [keys, nodes, knows]
+ properties:
+ keys:
+ type: array
+ items:
+ type: string
+ nodes:
+ type: array
+ items:
+ type: string
+ knows:
+ type: object
+ additionalProperties:
+ type: [object, 'null']
+ required: [wire, revision]
+ properties:
+ wire:
+ type: array
+ items:
+ type: integer
+ revision:
+ type: [object, 'null']
+ required: [number, id, named]
+ properties:
+ number:
+ type: [integer, 'null']
+ id:
+ type: string
+ named:
+ type: string
values:
type: object
members:
diff --git a/docs/peers.md b/docs/peers.md
index c63bebe..d2f822f 100644
--- a/docs/peers.md
+++ b/docs/peers.md
@@ -99,9 +99,10 @@ Before each HELLO and each HELLO_REPLY, a node sends SPEAKS: the wire versions i
and the newest revision of the federation it knows, as its build has them. It stands
outside the wire versions, laid out as no build changes, so that a peer on another build
reads it too, once that peer reads the node's WHO and ME
-([Node IDs and addresses](#node-ids-and-addresses)). A build from before SPEAKS takes it
-for a message of another wire version, and logs that once of the peer
-([`operations.md`](operations.md), [Upgrading](operations.md#upgrading)).
+([Node IDs and addresses](#node-ids-and-addresses)): what each peer said shows in `knows`
+of `GET /events/counts`, for the nodes a federated store's parties list. A build from
+before SPEAKS takes it for a message of another wire version, and logs that once of the
+peer ([`operations.md`](operations.md), [Upgrading](operations.md#upgrading)).
Every HELLO carries a random id of the sender's current run, and the HELLO_REPLY echoes
it. A reply with another id answers a HELLO from an earlier run (delivered late, or
diff --git a/src/api/events.rs b/src/api/events.rs
index 7bb322e..f49c540 100644
--- a/src/api/events.rs
+++ b/src/api/events.rs
@@ -106,6 +106,41 @@ async fn summary(State(state): State<AppState>) -> Response {
})
}
+/// What `node` says it speaks and knows (SPEAKS, `swarm::speaking`), this node's own as
+/// it is: the wire versions it speaks, the oldest and the newest, and the newest revision
+/// of the federation it knows, by number and ID and as people read it, its number null
+/// where this continuum doesn't know it. Null where the node said nothing: a build from
+/// before SPEAKS, or one not heard from.
+fn knows(state: &AppState, node: &str) -> JsonValue {
+ let speaks = if node == state.local_id() {
+ Some(crate::swarm::speaking::own())
+ } else {
+ state.speaking.of(node)
+ };
+ let Some(speaks) = speaks else {
+ return JsonValue::Null;
+ };
+ let revision = speaks.revision.map(|id| {
+ let id = Id(id);
+ match Revision::of(&id) {
+ Some(revision) => json!({
+ "number": revision.0,
+ "id": id.to_string(),
+ "named": revision.to_string(),
+ }),
+ None => json!({
+ "number": null,
+ "id": id.to_string(),
+ "named": format!("{id}, a revision this continuum doesn't know"),
+ }),
+ }
+ });
+ json!({
+ "wire": [speaks.oldest, speaks.newest],
+ "revision": revision,
+ })
+}
+
/// The revisions this continuum knows, the base revision first: each its number, its ID,
/// the feature it adds and what that adds, and as people read it.
fn revisions() -> Vec<JsonValue> {
@@ -143,7 +178,8 @@ async fn event(State(state): State<AppState>, Path(id): Path<String>) -> Respons
}
/// `GET /events/counts`: what counts, as the store reads now: the current version, each
-/// party's keys and nodes, the values at places, the members, and the proposals on the
+/// party's keys and nodes, what each of those nodes says it speaks and knows (`knows`),
+/// the values at places, the members, and the proposals on the
/// current version, each with the events proposing it a vote may build on (`proposing`:
/// for a checkpoint, those above its heads), the votes on it, the parties
/// voting `true` and `false`, whether it passed and whether it is dead, and why a party's
@@ -157,11 +193,17 @@ async fn counts(State(state): State<AppState>) -> Response {
.parties
.keys()
.map(|party| {
+ let nodes = read.nodes_of(party, current);
+ let knows: serde_json::Map<String, JsonValue> = nodes
+ .iter()
+ .map(|node| (node.clone(), knows(&state, node)))
+ .collect();
(
party.clone(),
json!({
"keys": read.keys_of(party, current),
- "nodes": read.nodes_of(party, current),
+ "nodes": nodes,
+ "knows": knows,
}),
)
})
diff --git a/src/federation/client.rs b/src/federation/client.rs
index 7b9d3d7..45f5019 100644
--- a/src/federation/client.rs
+++ b/src/federation/client.rs
@@ -22,6 +22,7 @@ use serde::{Deserialize, Serialize};
use super::event::{Does, Event, Signed};
use super::jcs::{self, Json};
+use super::revision::Revision;
use super::store::Id;
use super::version::Version;
@@ -480,6 +481,41 @@ pub(crate) fn changes(current: &Json, proposed: &Json) -> Vec<String> {
lines
}
+/// Of the parties' nodes, those not known to know `revision`, by what the node a client
+/// talks to heard each say it knows (`knows` in `counts`, from `GET /events/counts`): a
+/// line for each party with any, naming them and what each said. Once a version of
+/// `revision` passes, such a node stops at it till upgraded. None where every node said
+/// it knows it.
+pub(crate) fn unready(counts: &serde_json::Value, revision: Revision) -> Vec<String> {
+ let mut lines = Vec::new();
+ for (party, of) in counts["parties"].as_object().into_iter().flatten() {
+ let behind: Vec<String> = of["knows"]
+ .as_object()
+ .into_iter()
+ .flatten()
+ .filter_map(|(node, knows)| {
+ let newest = &knows["revision"];
+ let named = newest["named"].as_str().unwrap_or("?");
+ match newest["number"].as_u64() {
+ Some(number) if usize::try_from(number).is_ok_and(|n| n >= revision.0) => None,
+ Some(_) => Some(format!("{node} knows {named} at newest")),
+ None if newest.is_object() => Some(format!("{node} knows {named}")),
+ None if knows.is_object() => Some(format!(
+ "{node} knows no revision: built without the federation"
+ )),
+ None => Some(format!(
+ "{node} said nothing: a build from before SPEAKS, or not heard from"
+ )),
+ }
+ })
+ .collect();
+ if !behind.is_empty() {
+ lines.push(format!("{party}: {}", behind.join("; ")));
+ }
+ }
+ lines
+}
+
/// The parties that voted twice, by the votes `records` keep: of each version they name,
/// a party whose `true` and `false` are on one proposal, or whose `true` is on two
/// proposals of which neither is dead by all the votes. Each vote counts as checked from
@@ -908,4 +944,34 @@ mod tests {
)]
);
}
+
+ /// Of the nodes `knows` names, those that didn't say they know a revision are named,
+ /// with what each said; a party whose nodes all did, not.
+ #[test]
+ fn nodes_not_known_to_know_a_revision_are_named() {
+ let one = Revision(1);
+ let id = one.id().to_string();
+ let counts = serde_json::json!({"parties": {
+ "lass": {"knows": {
+ "a": {"wire": [52, 52], "revision": {"number": 1, "id": id, "named": "one"}},
+ }},
+ "tv": {"knows": {
+ "b": {"wire": [52, 52], "revision": {"number": 0, "named": "the base"}},
+ "c": null,
+ "d": {"wire": [52, 52], "revision": {"number": null, "named": "x, unknown"}},
+ "e": {"wire": [52, 52], "revision": null},
+ "f": {"wire": [52, 53], "revision": {"number": 2, "named": "two"}},
+ }},
+ }});
+ assert_eq!(
+ unready(&counts, one),
+ [
+ "tv: b knows the base at newest; c said nothing: a build from before SPEAKS, \
+ or not heard from; d knows x, unknown; e knows no revision: built without \
+ the federation"
+ ]
+ );
+ assert!(unready(&counts, Revision(0)).len() == 1);
+ assert!(unready(&serde_json::json!({}), one).is_empty());
+ }
}
diff --git a/src/federation/party.rs b/src/federation/party.rs
index a12dc7f..2c8dc4b 100644
--- a/src/federation/party.rs
+++ b/src/federation/party.rs
@@ -314,7 +314,15 @@ pub(crate) async fn main() -> Result<()> {
moves to it, once every party's nodes run a continuum that knows it",
current.revision
);
+ warn_unready(&node.counts().await?, later);
}
+ } else if let Some(revision) = next["revision"]
+ .as_str()
+ .and_then(Id::from_hex)
+ .and_then(|id| Revision::of(&id))
+ && revision != current.revision
+ {
+ warn_unready(&node.counts().await?, revision);
}
println!("{}", serde_json::to_string_pretty(&next)?);
return Ok(());
@@ -755,18 +763,21 @@ async fn vote(
})) if hash == proposal => version,
_ => bail!("the node's {proposing} isn't the proposal it names"),
};
- // A later revision: the node knows it, or it stops at the version once it passes.
+ // A later revision: the node knows it, or it stops at the version once it passes; and
+ // so does every other node not known to know it.
if yes
&& let Ok(version) = Version::read(&proposed)
&& version.revision != current.revision
- && !node.revisions().await?.contains(&version.revision.id())
{
- bail!(
- "{proposal} moves to {}, which the node doesn't know: once it passed, a node that \
- doesn't stops at it, and votes nothing till upgraded; upgrade this node and every \
- other of the party to a continuum that knows it, then back it",
- version.revision
- );
+ if !node.revisions().await?.contains(&version.revision.id()) {
+ bail!(
+ "{proposal} moves to {}, which the node doesn't know: once it passed, a node \
+ that doesn't stops at it, and votes nothing till upgraded; upgrade this node \
+ and every other of the party to a continuum that knows it, then back it",
+ version.revision
+ );
+ }
+ warn_unready(&counts, version.revision);
}
// The votes on its party's earlier proposals it backed, for the nays among them.
let mut nays = Vec::new();
@@ -816,6 +827,21 @@ async fn vote(
node.put(&cast).await
}
+/// Warns of the nodes not known to know `revision`, by `counts` (`client::unready`).
+fn warn_unready(counts: &JsonValue, revision: Revision) {
+ let behind = client::unready(counts, revision);
+ if behind.is_empty() {
+ return;
+ }
+ eprintln!(
+ "Warning: these nodes aren't known to know {revision}, as far as the node heard from \
+ them; once a version of it passes, each stops at it till upgraded:"
+ );
+ for line in behind {
+ eprintln!(" {line}");
+ }
+}
+
/// Signs again what a move refused: of the latest event it signed at each place, its key
/// set and its node list, those the node refuses as not above its checkpoint's heads, or
/// as built on one it refuses.
diff --git a/src/swarm/speaking.rs b/src/swarm/speaking.rs
index dcbfac4..e1a8d2c 100644
--- a/src/swarm/speaking.rs
+++ b/src/swarm/speaking.rs
@@ -43,4 +43,30 @@ impl Speaking {
}
heard.insert(peer.to_owned(), speaks);
}
+
+ /// What `peer` said last, if it said anything.
+ pub(crate) fn of(&self, peer: &str) -> Option<Speaks> {
+ self.0.lock_or_recover().get(peer).copied()
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn a_peer_is_heard_as_it_spoke_last() {
+ let speaking = Speaking::default();
+ assert_eq!(speaking.of("a"), None);
+ speaking.heard("a", own());
+ let older = Speaks {
+ oldest: 1,
+ newest: 1,
+ revision: None,
+ };
+ speaking.heard("b", older);
+ speaking.heard("a", older);
+ assert_eq!(speaking.of("a"), Some(older));
+ assert_eq!(own().newest, wire::VERSION);
+ }
}