aboutsummaryrefslogtreecommitdiffstats
path: root/README.md
diff options
context:
space:
mode:
Diffstat (limited to 'README.md')
-rw-r--r--README.md108
1 files changed, 108 insertions, 0 deletions
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..e626f69
--- /dev/null
+++ b/README.md
@@ -0,0 +1,108 @@
+# continuum
+
+A replicated [Automerge](https://automerge.org) document for fleets of machines that
+aren't always online, read and written as JSON by path (`settings/theme`,
+`servers/0/name`) at any depth. Every node holds the whole document, takes reads and
+writes on its own, and syncs with its peers whenever it can reach them. Think
+Syncthing, but for structured data behind an HTTP API.
+
+There is no leader and no quorum for writes: a node keeps working through a network
+partition, on a laptop on a train, or as the only one left. Concurrent writes are
+merged, and conflicts are reported rather than hidden.
+
+A node is its key, not its address. Nodes find and reach each other over any of three
+transports, one or several at once, each a cargo feature:
+
+* **[Mycelium](https://github.com/threefoldtech/mycelium)**: an encrypted overlay
+ network addressed by keys, so NAT, changing IP addresses and firewalls don't need
+ configuring.
+* **[libp2p](https://libp2p.io)**: direct connections over TCP and QUIC, with relays
+ (members of the swarm, or a relay-only continuum), hole punching, UPnP and LAN
+ discovery for nodes behind NAT.
+* **Tor**: an onion service per node (arti, embedded), reachable from anywhere without
+ a forwarded port. For reachability, not anonymity.
+
+A node on several transports reaches each peer over the fastest path they share, falls
+back to the next when one fails, and carries the traffic between nodes that share
+none.
+
+The one thing that takes a quorum is a remake: compacting the history, or another move
+to a new document. A majority of the members must vote for it, so it waits, and in the
+end aborts, while too many of them are out of reach; reads and writes go on meanwhile
+([`docs/remakes.md`](docs/remakes.md)).
+
+## What it isn't
+
+Continuum is eventually consistent. Reads are local and may be behind; there are no
+compare-and-set, transactions, leases, locks or leader election. Don't build
+coordination on it.
+
+## Quick start
+
+```sh
+# The transports are cargo features: mycelium, libp2p, tor (any, at least one);
+# dashboard adds the live page (`/admin/dashboard`).
+cargo build --release --features mycelium,dashboard
+
+# An HTTP API user (Argon2 hash; e.g. with the `argon2` command-line tool).
+printf '%s' 'secret' | argon2 "$(openssl rand -hex 16)" -id -e | sed 's/^/alice:/' > htpasswd
+
+# The first node starts a new swarm. --node-key-create makes the node's key on its first
+# start (a new key is a new node; keep the file).
+./target/release/continuum --genesis --htpasswd htpasswd --state-file state.json \
+ --node-key-file node.key --node-key-create --mycelium-enable
+
+# Further nodes, on other machines, join through any node of it: one of its addresses
+# (`addresses` in `GET /admin/stats`: `mycelium:<public key>`, `libp2p:<multiaddr>`,
+# `tor:<onion>.onion:<port>`), or for Mycelium its public key or IPv6 address, or a
+# host name with an AAAA record for one.
+./target/release/continuum --seeds <address of the first node> --htpasswd htpasswd \
+ --state-file state.json --node-key-file node.key --node-key-create --mycelium-enable
+```
+
+A node uses the transports enabled, one at least (`--mycelium-enable`,
+`--libp2p-enable`, `--tor-enable`). With Mycelium, each node runs its own Mycelium
+daemon, unless pointed at one with `--mycelium-rest-url`; `--mycelium-peers` names
+Mycelium nodes to connect through. With libp2p, `--libp2p-listen` says where to listen
+(`/ip4/0.0.0.0/tcp/9661`), and seeds are multiaddrs
+(`libp2p:/ip4/192.0.2.7/tcp/9661`); with Tor (`--tor-enable`), `--tor-state-dir`
+keeps the onion service's key, and seeds are onion addresses. [`docs/transports.md`](docs/transports.md)
+has the rest. The HTTP API listens on `127.0.0.1:3000`:
+
+```sh
+curl -u alice:secret -X PUT localhost:3000/store/config/feature \
+ -H 'Content-Type: application/json' -d '{"enabled": true}'
+curl -u alice:secret localhost:3000/store/config/feature
+curl -u alice:secret -N localhost:3000/stream/config # follow changes
+curl -u alice:secret localhost:3000/peers # who is out there
+```
+
+## Documentation
+
+**Running nodes**
+
+* [`docs/configuration.md`](docs/configuration.md): options, files and imports (`continuum --help` lists them);
+ [`docs/example.toml`](docs/example.toml) has every option, explained.
+* [`docs/transports.md`](docs/transports.md): Mycelium, libp2p and Tor: what each is for, how they combine,
+ relays, and their options.
+* [`docs/lifecycle.md`](docs/lifecycle.md): starting, joining, rejoining and stopping a node.
+* [`docs/operations.md`](docs/operations.md): adding and removing nodes, keys, backups, upgrades, what to
+ alert on.
+* [`docs/remakes.md`](docs/remakes.md): compacting history and other moves to a new document.
+* [`docs/trust.md`](docs/trust.md): who can do what.
+* [`docs/measurements.md`](docs/measurements.md): `/admin/stats` and its Prometheus metrics.
+* [`docs/dashboard.md`](docs/dashboard.md): the dashboard, a live page of a node's view of the swarm.
+
+**Writing applications**
+
+* [`docs/api.md`](docs/api.md): the store API for applications: consistency, concurrent writes,
+ streams.
+* [`docs/openapi.yaml`](docs/openapi.yaml): the HTTP API (also served at `/openapi.yaml`, with a Swagger
+ UI at `/swagger-ui`).
+
+**Working on continuum**
+
+* [`docs/architecture.md`](docs/architecture.md): the map of the code, and where to read on.
+* [`docs/development.md`](docs/development.md): building and testing quickly.
+* [`docs/peers.md`](docs/peers.md): the protocol between nodes.
+* [`docs/glossary.md`](docs/glossary.md): the words used in the code, logs, API and docs.