From 29cad923a5e2d7dc3207c8d456440f8c37bfd7b6 Mon Sep 17 00:00:00 2001 From: tv Date: Mon, 2 Nov 2026 12:03:02 +0100 Subject: add documentation --- README.md | 108 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 108 insertions(+) create mode 100644 README.md (limited to 'README.md') diff --git a/README.md b/README.md new file mode 100644 index 0000000..e626f69 --- /dev/null +++ b/README.md @@ -0,0 +1,108 @@ +# continuum + +A replicated [Automerge](https://automerge.org) document for fleets of machines that +aren't always online, read and written as JSON by path (`settings/theme`, +`servers/0/name`) at any depth. Every node holds the whole document, takes reads and +writes on its own, and syncs with its peers whenever it can reach them. Think +Syncthing, but for structured data behind an HTTP API. + +There is no leader and no quorum for writes: a node keeps working through a network +partition, on a laptop on a train, or as the only one left. Concurrent writes are +merged, and conflicts are reported rather than hidden. + +A node is its key, not its address. Nodes find and reach each other over any of three +transports, one or several at once, each a cargo feature: + +* **[Mycelium](https://github.com/threefoldtech/mycelium)**: an encrypted overlay + network addressed by keys, so NAT, changing IP addresses and firewalls don't need + configuring. +* **[libp2p](https://libp2p.io)**: direct connections over TCP and QUIC, with relays + (members of the swarm, or a relay-only continuum), hole punching, UPnP and LAN + discovery for nodes behind NAT. +* **Tor**: an onion service per node (arti, embedded), reachable from anywhere without + a forwarded port. For reachability, not anonymity. + +A node on several transports reaches each peer over the fastest path they share, falls +back to the next when one fails, and carries the traffic between nodes that share +none. + +The one thing that takes a quorum is a remake: compacting the history, or another move +to a new document. A majority of the members must vote for it, so it waits, and in the +end aborts, while too many of them are out of reach; reads and writes go on meanwhile +([`docs/remakes.md`](docs/remakes.md)). + +## What it isn't + +Continuum is eventually consistent. Reads are local and may be behind; there are no +compare-and-set, transactions, leases, locks or leader election. Don't build +coordination on it. + +## Quick start + +```sh +# The transports are cargo features: mycelium, libp2p, tor (any, at least one); +# dashboard adds the live page (`/admin/dashboard`). +cargo build --release --features mycelium,dashboard + +# An HTTP API user (Argon2 hash; e.g. with the `argon2` command-line tool). +printf '%s' 'secret' | argon2 "$(openssl rand -hex 16)" -id -e | sed 's/^/alice:/' > htpasswd + +# The first node starts a new swarm. --node-key-create makes the node's key on its first +# start (a new key is a new node; keep the file). +./target/release/continuum --genesis --htpasswd htpasswd --state-file state.json \ + --node-key-file node.key --node-key-create --mycelium-enable + +# Further nodes, on other machines, join through any node of it: one of its addresses +# (`addresses` in `GET /admin/stats`: `mycelium:`, `libp2p:`, +# `tor:.onion:`), or for Mycelium its public key or IPv6 address, or a +# host name with an AAAA record for one. +./target/release/continuum --seeds
--htpasswd htpasswd \ + --state-file state.json --node-key-file node.key --node-key-create --mycelium-enable +``` + +A node uses the transports enabled, one at least (`--mycelium-enable`, +`--libp2p-enable`, `--tor-enable`). With Mycelium, each node runs its own Mycelium +daemon, unless pointed at one with `--mycelium-rest-url`; `--mycelium-peers` names +Mycelium nodes to connect through. With libp2p, `--libp2p-listen` says where to listen +(`/ip4/0.0.0.0/tcp/9661`), and seeds are multiaddrs +(`libp2p:/ip4/192.0.2.7/tcp/9661`); with Tor (`--tor-enable`), `--tor-state-dir` +keeps the onion service's key, and seeds are onion addresses. [`docs/transports.md`](docs/transports.md) +has the rest. The HTTP API listens on `127.0.0.1:3000`: + +```sh +curl -u alice:secret -X PUT localhost:3000/store/config/feature \ + -H 'Content-Type: application/json' -d '{"enabled": true}' +curl -u alice:secret localhost:3000/store/config/feature +curl -u alice:secret -N localhost:3000/stream/config # follow changes +curl -u alice:secret localhost:3000/peers # who is out there +``` + +## Documentation + +**Running nodes** + +* [`docs/configuration.md`](docs/configuration.md): options, files and imports (`continuum --help` lists them); + [`docs/example.toml`](docs/example.toml) has every option, explained. +* [`docs/transports.md`](docs/transports.md): Mycelium, libp2p and Tor: what each is for, how they combine, + relays, and their options. +* [`docs/lifecycle.md`](docs/lifecycle.md): starting, joining, rejoining and stopping a node. +* [`docs/operations.md`](docs/operations.md): adding and removing nodes, keys, backups, upgrades, what to + alert on. +* [`docs/remakes.md`](docs/remakes.md): compacting history and other moves to a new document. +* [`docs/trust.md`](docs/trust.md): who can do what. +* [`docs/measurements.md`](docs/measurements.md): `/admin/stats` and its Prometheus metrics. +* [`docs/dashboard.md`](docs/dashboard.md): the dashboard, a live page of a node's view of the swarm. + +**Writing applications** + +* [`docs/api.md`](docs/api.md): the store API for applications: consistency, concurrent writes, + streams. +* [`docs/openapi.yaml`](docs/openapi.yaml): the HTTP API (also served at `/openapi.yaml`, with a Swagger + UI at `/swagger-ui`). + +**Working on continuum** + +* [`docs/architecture.md`](docs/architecture.md): the map of the code, and where to read on. +* [`docs/development.md`](docs/development.md): building and testing quickly. +* [`docs/peers.md`](docs/peers.md): the protocol between nodes. +* [`docs/glossary.md`](docs/glossary.md): the words used in the code, logs, API and docs. -- cgit v1.3.1