add documentation

1 files changed, 306 insertions, 0 deletions

diff --git a/docs/architecture.md b/docs/architecture.md
new file mode 100644
index 0000000..e0e2c94
--- /dev/null
+++ b/docs/architecture.md
@@ -0,0 +1,306 @@
+# Architecture
+
+Continuum is a replicated Automerge document, read and written by path: every node
+holds the same document, exposes it over an HTTP API, and syncs it with its peers over one or more
+transports (Mycelium, libp2p, Tor; [`transports.md`](transports.md)). There
+is no leader and no central service; nodes find each other through seeds and a member
+registry kept in the document itself. Everything the swarm agrees on, it agrees on
+through that document.
+
+This page is the map: what the parts are, how they fit, and where the complexity sits.
+The details live in the other pages ([`lifecycle.md`](lifecycle.md), [`operations.md`](operations.md), [`peers.md`](peers.md), [`transports.md`](transports.md),
+[`remakes.md`](remakes.md), [`api.md`](api.md), [`configuration.md`](configuration.md), [`measurements.md`](measurements.md), [`dashboard.md`](dashboard.md), [`trust.md`](trust.md),
+[`glossary.md`](glossary.md)) and
+in the module docs at the top of each source file.
+
+## Layers
+
+```
+ operators, applications
+          │ HTTP (basic auth)
+ ┌────────▼────────────────────────────────────────────────────────────────┐
+ │ api/        /store, /stream, /peers, /admin/*, /debug/*                 │
+ ├─────────────────────────────────────────────────────────────────────────┤
+ │ state/      AppState: the replica, lifecycle, peer states, timings,     │
+ │             shared tables; the Host impl for remakes                    │
+ ├──────────────────────────────┬──────────────────────────────────────────┤
+ │ swarm/                       │ remake/                                  │
+ │ peers, sync, relay, topology,│ epochs: proposal, votes, decision,       │
+ │ members, blobs, catch-up,    │ snapshot, switch, rejoin, quarantine     │
+ │ stability, bootstrap, files  │ (reaches the node only through `Host`)   │
+ ├──────────────────────────────┴──────────────────────────────────────────┤
+ │ document/   Replica (document + epoch), the store view, the document    │
+ │             file                                                        │
+ ├─────────────────────────────────────────────────────────────────────────┤
+ │ transport/  message types and wire format, traffic counting; the        │
+ │             identity layer (node IDs) over the paths layer (addresses,  │
+ │             PING/PONG, the debug firewall) over the transports:         │
+ │             Mycelium (REST client, daemon), libp2p, Tor                 │
+ └─────────────────────────────────────────────────────────────────────────┘
+          Mycelium daemon · libp2p peers and relays · the Tor network
+```
+
+Dependencies point downwards, with two deliberate exceptions: [`state/`](../src/state/) is the hub
+everything else reaches (it holds the shared state and is reached by most modules), and
+[`remake/`](../src/remake/) never touches `AppState`, peers or sync directly; it sees the node through
+the `Host` trait ([`remake/mod.rs`](../src/remake/mod.rs)), which `AppState` implements. That boundary is what
+keeps the epoch machinery testable with plain documents.
+
+## The document
+
+One Automerge document per node (`document::replica::Replica`: the document, its epoch,
+cached heads). Automerge is used directly only in `document` and in what remakes do to
+documents ([`remake/`](../src/remake/): `snapshot`, `delta`, `catchup`, `kind`, `quarantine`); the rest of
+the node names its types as `document` re-exports them (`Document`, `ChangeHash`,
+`Change`, `ActorId`) and works through `document`'s functions. Two views of it:
+
+* **The store** (`document::store`): JSON over slash-separated paths, with Automerge's
+  concurrent values merged into one view (maps union, scalars newest-wins) and
+  reported on request (`?conflicts=true`). The HTTP API and the remake kinds work
+  through it; nothing else reads Automerge objects directly. Within a list, a path
+  segment is an index (`servers/0/name`); a negative one counts from the end. A key with `/` in it, or
+  an empty one, can be written inside a value but not named in a path: it is read and
+  written through its parent.
+* **Protocol data** under `_continuum`: the swarm ID, the member registry, forget
+  records, remake records, the votes register and epoch metadata. The nodes coordinate
+  through these keys and nothing else: there is no control channel besides the
+  document and the handshake. `--protocol-writers` may edit them for repairs; the
+  swarm ID and epoch metadata never.
+
+**Epochs.** The document's history can't be dropped in place, so a *remake* moves the
+whole swarm to a new document (same content, no history; or new content), the next
+epoch. Nodes on different epochs don't sync. The epoch is `{generation, id}`; the ID
+is the hash of the new document's first change, which every node builds itself, byte
+for byte the same ([`remake/snapshot.rs`](../src/remake/snapshot.rs)). That determinism is what lets a node verify
+it switched to the right document without trusting anyone.
+
+## Modules
+
+**[`transport/`](../src/transport/)** — [`mod.rs`](../src/transport/mod.rs) defines the message types (HELLO, HELLO_REPLY, sync
+envelope, STATUS/STATUS_REPLY, PING/PONG, blob messages, LINKS/LINKS_REPLY) and the `Transport` trait;
+[`wire.rs`](../src/transport/wire.rs) is the wire format (a version byte, bumped with every change to the format,
+and a kind byte, then the message in MessagePack, derived from the types in [`mod.rs`](../src/transport/mod.rs)
+with codecs for hashes and IDs); [`mycelium/`](../src/transport/mycelium/) (with the `mycelium` feature) is the REST
+client (polling inbox, outbound POST, one topic, `continuum`, for every message), with
+[`mycelium/daemon.rs`](../src/transport/mycelium/daemon.rs) running the daemon (an external one by its URL, or one it starts
+and supervises); [`libp2p/`](../src/transport/libp2p/) (with the `libp2p` feature) is libp2p as a transport,
+direct connections with one message per stream, relays and hole punching ([`driver.rs`](../src/transport/libp2p/driver.rs),
+[`behaviour.rs`](../src/transport/libp2p/behaviour.rs)), the addresses known per peer ([`book.rs`](../src/transport/libp2p/book.rs)), and which of them are
+ignored or unroutable ([`ignore.rs`](../src/transport/libp2p/ignore.rs), [`routes.rs`](../src/transport/libp2p/routes.rs)); [`tor.rs`](../src/transport/tor.rs) (with the `tor` feature) is Tor as a
+transport, an onion service per node and one connection per pair, authenticated by the
+challenge handshake of [`challenge.rs`](../src/transport/challenge.rs); [`paths.rs`](../src/transport/paths.rs) makes one address space
+(`<scheme>:<target>`) of the node's transports (one or more), routing each send to the
+transport of its address, measuring the paths to peers on several (PING, PONG) and
+sending over the best, the next when one fails; [`identity.rs`](../src/transport/identity.rs) is the identity layer above it, so the node
+deals in node IDs only (WHO and ME prove which node is at an address; [`peers.md`](peers.md)); [`traffic.rs`](../src/transport/traffic.rs) counts messages and
+bytes per peer; [`stats.rs`](../src/transport/stats.rs) the transports' own figures; [`firewall.rs`](../src/transport/firewall.rs) is the debug firewall (`/debug/firewall`), which the paths layer applies to
+everything, by the node at each address; [`sim.rs`](../src/transport/sim.rs) (tests only) is a
+simulated network: many nodes in one process, with per-link delay, loss and
+duplication from a seed, nodes taken down and brought up, and messages injected as if
+a node had sent them. It sends the wire encoding, so tests exercise the codec, or a
+transport's envelope around it ([`development.md`](development.md)).
+
+**[`simtest/`](../src/simtest/)** (tests only) — whole nodes on the simulated network, built as `main`
+builds them, with a 250 ms keepalive and a 1 s sync reply timeout: joining and
+converging, dormancy and return, late messages, a stalled catch-up source; and an
+ignored 25-node soak test with a propagation bound (`cargo test soak -- --ignored`). How to build and test quickly: [`development.md`](development.md).
+
+**[`swarm/`](../src/swarm/)** — the live protocol between nodes:
+
+| Module | Does | Documented in |
+|---|---|---|
+| `bootstrap` | genesis, joining through seeds (forever), starting on a document | [`lifecycle.md`](lifecycle.md) |
+| [`peer/`](../src/swarm/peer/) | handshake, keepalives and healing, peers' epochs, admission with a swarm key, forget records, inbound dispatch, the peer worker (a file each) | [`peers.md`](peers.md) |
+| `sync` | the sync worker, Automerge sync messages with retries and reply deadlines | [`peers.md`](peers.md) |
+| `changes` | the change publisher: the store paths each change touched (`change_events`) | module docs |
+| `document_file` | the worker keeping the document file (`document::file`) up to date | module docs |
+| `source` | the one peer a catching-up node syncs with, and when to give it up | [`peers.md`](peers.md) |
+| `relay` | announcing heads to neighbours and pulling what's missing (partial mesh) | module docs |
+| `links` | answering LINKS with this node's links, for the dashboard's map | [`dashboard.md`](dashboard.md) |
+| [`topology/`](../src/swarm/topology/) | which members are neighbours: ring plus low-score links, stable members preferred | module docs |
+| `members` | the member registry: aliases, alive stamps, tiers, the electorate | module docs |
+| `stability` | a node's own connectivity record and its stable/transient tier | module docs |
+| `auth` | the pre-shared swarm key and handshake tags | [`trust.md`](trust.md) |
+| `blob` | chunked content-addressed transfers from several peers | module docs |
+| `rtt`, `throughput` | round-trip times (of the paths, from HELLOs until measured) and sync throughput, for the node's own choices | [`measurements.md`](measurements.md) |
+| `skew` | this node's clock against its peers' (from their PONGs), and a warning while it is off | [`measurements.md`](measurements.md) |
+| `pair` | which side of a pair leads (sends the first keepalive) | module docs |
+| `stats` | node-wide figures for `/admin/stats` | [`measurements.md`](measurements.md) |
+| `state_file` | what a node remembers across restarts | [`lifecycle.md`](lifecycle.md) |
+
+**[`remake/`](../src/remake/)** — epochs and the move between them ([`remakes.md`](remakes.md) for operators):
+
+| Module | Does |
+|---|---|
+| `records` | remake records and the votes register in the document |
+| `api` | `/admin/remakes`: proposing, reports, aborting, superseding |
+| `proposal` | the checks before a proposal, superseding, the electorate and quorum |
+| `vote` | who votes for what, tallies, the one-vote-per-epoch rule |
+| `coordinator` | counts votes, writes the decision, builds the snapshot on commit |
+| `kind` | `compact`, `replace`, `import`: how the new document is built and where later writes go |
+| `snapshot` | the deterministic build and its verification; epoch metadata |
+| `switch` | the remake worker: switching this node to a committed epoch, replaying writes since the base |
+| `catchup` | rejoining a newer epoch from peers (download, verify, replay own writes) |
+| `delta` | replaying changes between two sets of heads into another document |
+| `quarantine` | writes a remake had no place for |
+| `content` | the new document of a `replace`/`import`, fetched by voters |
+| `local` | in-memory per-node remake state: prebuilt snapshots, archive, own actors |
+
+**[`document/`](../src/document/)** — [`replica.rs`](../src/document/replica.rs) (the document with its epoch), [`store.rs`](../src/document/store.rs) (the JSON
+view), [`file.rs`](../src/document/file.rs) (the document file: full save plus appended increments), [`changes.rs`](../src/document/changes.rs)
+(changes as bytes: one by its hash, and deltas between heads), [`sync.rs`](../src/document/sync.rs) (one sync with
+a peer, `SyncSession`: Automerge's sync state with our workarounds for unreliable
+delivery), [`meta.rs`](../src/document/meta.rs) (the protocol's paths under `_continuum`: the only place their
+names are spelled).
+
+**[`api/`](../src/api/)** — axum handlers: `store` (`/store`, `/stream`), `peers`, [`stats/`](../src/api/stats/) (JSON and
+Prometheus), `document` (backup export), `log` (filter at runtime), `config`
+(`/admin/config`), `health` (`/health`, `/ready`), `debug`, `auth` (htpasswd with
+Argon2), [`dashboard/`](../src/api/dashboard/) (with the `dashboard` feature: the page and its events;
+[`dashboard.md`](dashboard.md)), and [`mod.rs`](../src/api/mod.rs) (router, protocol-path rules). The remake routes
+come from [`remake/api.rs`](../src/remake/api.rs). [`docs/openapi.yaml`](openapi.yaml) describes it all; nodes serve it at
+`/openapi.yaml`, with a Swagger UI at `/swagger-ui`.
+
+**Top level** — [`main.rs`](../src/main.rs) reads the configuration, picks the starting document and
+runs the shutdown sequence; [`node.rs`](../src/node.rs) builds a node's state and starts its workers
+(also used by the simulator);
+[`config/`](../src/config/) loads options from defaults, files and arguments ([`configuration.md`](configuration.md));
+[`force.rs`](../src/force.rs) is the one format for overriding checks; [`files.rs`](../src/files.rs) reads and writes the
+node's own files and counts them; [`clock.rs`](../src/clock.rs) is the node's wall clock (every Unix
+timestamp it stamps or judges by goes through it, so tests can skew one node's);
+[`locks.rs`](../src/locks.rs) takes std locks without poisoning (`lock_or_recover` and the like);
+[`backoff.rs`](../src/backoff.rs), [`logging.rs`](../src/logging.rs); [`testrand.rs`](../src/testrand.rs) (tests only) is a seeded generator and a
+runner for property tests over many seeds (topology, heads summaries, replay).
+
+## Runtime
+
+**Tasks.** Started by [`node.rs`](../src/node.rs), running for the life of the process:
+
+* the **consumer**: every transport's inbound messages, through the paths and the
+  identity layer (Mycelium polls its inbox, two polls at once so HTTP round trips
+  overlap; libp2p and Tor read their connections), handed one at a time to
+  `peer::handle_message`;
+* the **sync lanes**: four tasks that apply inbound sync messages, each sender's on one
+  lane in arrival order, so a slow apply doesn't hold up the consumer
+  (`peer::lanes`);
+* the **sync worker**: one queue of dirty peers, coalesced, one Automerge sync
+  message generated per peer per pass and sent by a task of its own, so a slow send
+  holds up no other peer (`swarm::sync`);
+* the **peer worker**: when the earliest is due, keepalives, dormancy, HELLO retries,
+  overdue sync replies; every 5 s (and at once when the node becomes ready), forget records
+  (also when they changed), registry renewal, stability, measurements, the state file
+  (`swarm::peer::run_peer_worker`);
+* the **topology worker**: recomputes the neighbours when the member list or a peer's
+  reachability changes (`swarm::topology::run_worker`);
+* the **remake worker**: every second, on record changes and when a vote is due,
+  voting, switching, rejoining (`remake::switch::run_worker`);
+* the **change publisher**: diffs the document after each change for `/stream`;
+* the **document file** writer, if configured;
+* **bootstrap**, until the node is ready, then the HTTP server.
+
+Short-lived tasks: a coordinator per remake this node proposed, blob serving and
+fetching, the snapshot download of a rejoin, announcement rounds.
+
+**Locks.** The document is an `RwLock<Replica>`; what changes it (writes, applying
+sync messages and fetched changes, the epoch switch) takes the write lock, and what
+only reads it (generating sync messages, looking changes up, saving) the read lock,
+both on the blocking pool. Peers are a `RwLock<HashMap<_, Arc<Mutex<PeerState>>>>`: the map lock
+is held briefly, each peer's mutex for one operation. Lock order is document, then
+peer, wherever both are held (sync generation, the epoch switch). Whether a write may
+go in is checked under the write lock too: a rejoin's fresh document takes none until
+the rejoin is done (`Replica::writable`), so a write checked as ready just before the
+rejoin can't land in it; it goes into the document before the rejoin (`StaleView`),
+which is replayed when it ends. A rejoin begins and ends under the write lock, with
+the lifecycle (taken after the document's) and the stale view. The small shared
+tables (RTT, throughput, neighbours, stability, catch-up source, remake state) sit
+behind std mutexes and are never held across an await.
+
+**Lifecycle.** `Joining` → `Ready`, and back to `Joining` (keeping the swarm ID) while
+a rejoin runs. Only a ready node registers, votes, serves writes and is followed as a
+catch-up source. The phases, what each serves and how the files come in: [`lifecycle.md`](lifecycle.md).
+
+## The paths that matter
+
+**A write.** `PUT /store/{path}` → `AppState::mutate_doc` (one Automerge change under
+the write lock, stamped with the clock) → `publish_change` (bumps the version, emits
+`DocumentChanged`) → the sync worker generates a message per neighbour → each
+neighbour applies it, relays its new heads to *its* neighbours, which pull what they
+lack. In a full mesh (up to `k + 1` members) the writer reaches everyone directly.
+
+**An inbound message.** Consumer → `handle_message`: drop our own and stale-run
+replies, admit (swarm key), mark the peer seen, record its epoch (ignoring late ones),
+then by type: HELLO answered, HELLO_REPLY establishes membership and discovers peers,
+sync queued on its lane if on our epoch (held back from everyone but the source while catching
+up), STATUS answered with a STATUS_REPLY and compared for divergence, blobs served or collected,
+LINKS answered with the node's links if the asker is a member, and again on each change while it subscribes (`swarm::links`, for the dashboard);
+PING and PONG (the paths layer answered the PING already) only mark the peer seen, and
+after a PONG the peer's round-trip time is its path's again.
+
+**A remake.** Proposal written by the coordinator → every electorate member votes once
+it holds the content → the coordinator commits on a majority (checking the deadline
+first, writing the decision atomically with a re-check) → after the switch delay
+every node builds the snapshot, verifies the epoch ID, replays writes since the base
+into it and swaps documents → nodes that missed it rejoin: download the snapshot in
+chunks, sync, replay their own writes. [`remakes.md`](remakes.md) has the operator's view,
+[`remake/*.rs`](../src/remake/) the rules.
+
+## What is kept, and where
+
+Nothing in the document is durable unless `--document-file` says so: a node that
+restarts without one starts empty and syncs. The optional files ([`files.rs`](../src/files.rs)):
+
+| File | Holds | Written |
+|---|---|---|
+| state file | swarm ID, members to rejoin through | once ready, when the selection changes, at shutdown |
+| document file | the document, this node's actor IDs | appended every few seconds, rewritten on epoch change and growth |
+| backup | `GET /admin/document`, restored with `--recover-from` or imported | by the operator |
+
+Everything else (archive of old epochs, quarantine, prebuilt snapshots, measurements,
+peer states) is in memory and gone at restart.
+
+## Design rules
+
+The decisions that recur throughout the code; changing one means revisiting the
+places that rely on it.
+
+* **The document is the only shared state.** Membership, forgetting, remakes and votes
+  are all keys under `_continuum`. Nodes never agree on anything out of band.
+* **One writer per key.** Each part of a remake record, each registry entry, each
+  vote-register entry has exactly one node that writes it, so concurrent writes never
+  fight. Where two writers are possible anyway (decisions after expiry), the record
+  is marked contested and nobody acts on it alone.
+* **Deterministic builds, verified by hash.** A snapshot's bytes depend only on the
+  base document and the proposal, so the epoch ID doubles as proof of the content.
+* **Messages may be late, lost or duplicated.** Every message carries epoch and
+  epoch-change count; sync is idempotent and repaired by deadlines; keepalives only
+  inform; a HELLO resets the session on both sides.
+* **Peers on different epochs don't sync.** Ever. A node moves epochs by switching
+  (building) or rejoining (downloading), never by merging.
+* **Writes since the base are replayed as state, not history.** Per map key, whole
+  lists and texts; by every switching node, and only by their author on a rejoin.
+* **Members are trusted with everything** ([`trust.md`](trust.md)). The swarm key is the
+  boundary; nothing inside it is signed.
+* **Conflicts are errors; overrides are named.** A contradiction in the configuration
+  or at the API refuses with a message that names the `force` value that would
+  override it, in one camelCase vocabulary everywhere.
+* **Wire formats grow by appending.** New fields default when absent, so a swarm can
+  be upgraded one node at a time (until the first release lifts that constraint).
+
+## Where the complexity sits
+
+Roughly by size and by how many rules interact there:
+
+| Area | Lines (tests included) | Why it is complex |
+|---|---|---|
+| [`remake/`](../src/remake/) | 6,900 | Thirteen modules, each simple, but the sequence spans the whole node: proposal, votes, decision, snapshot, switch, rejoin, replay, quarantine, and every failure in between. Most of the project's rules live here. |
+| [`swarm/peer/`](../src/swarm/peer/) + [`sync.rs`](../src/swarm/sync.rs) + [`source.rs`](../src/swarm/source.rs) | 2,300 | The protocol state machine: handshake, liveness, sync sessions with deadlines, who syncs with whom while catching up. Timing-dependent; tested on the simulated network ([`simtest/`](../src/simtest/)). |
+| [`state/`](../src/state/) | 1,600 | `AppState` and its write helpers ([`mod.rs`](../src/state/mod.rs)), `PeerState` ([`peer.rs`](../src/state/peer.rs)), `Timing`, reads ([`read.rs`](../src/state/read.rs)) and the `Host` impl ([`host.rs`](../src/state/host.rs)). A hub, not a layer. |
+| [`transport/`](../src/transport/) | 11,500 | libp2p ([`libp2p/`](../src/transport/libp2p/), 4,400: addresses, relays, hole punching, which paths to keep), the paths and identity layers (2,300: measuring, failing over, proving node IDs), Mycelium and its daemon ([`mycelium/`](../src/transport/mycelium/), 1,200), Tor (700), the wire format ([`wire.rs`](../src/transport/wire.rs), derived), the simulator. The transports are self-contained behind the `Transport` trait; libp2p's choice of addresses is the subtle part. |
+| [`config/`](../src/config/) | 2,800 | Layered sources, importers and placeholders, conflicts, the report. Self-contained. |
+| [`document/store.rs`](../src/document/store.rs) | 1,200 | Conflict-aware JSON over Automerge. Subtle (merged maps, canonical writes), well covered by the rest relying on it. |
+| [`swarm/topology/`](../src/swarm/topology/), [`blob.rs`](../src/swarm/blob.rs) | 1,500 | Pure algorithms with tests; the topology hashes each member once, then scores the N² pairs by cheap mixing. |
+| [`api/`](../src/api/) | 3,600 | Handlers; the Prometheus rendering is the longest file. The dashboard's page ([`dashboard.html`](../src/api/dashboard/dashboard.html), 2,000 more) is one file of HTML, CSS and script. |
+
+The two places where a change needs the most care are [`peer/`](../src/swarm/peer/)/[`sync.rs`](../src/swarm/sync.rs) (every
+rule about when to send what interacts with every other) and [`remake/catchup.rs`](../src/remake/catchup.rs)
+(a rejoin touches the lifecycle, the document swap, blobs, sync and replay at once).