add documentation

1 files changed, 240 insertions, 0 deletions

diff --git a/docs/configuration.md b/docs/configuration.md
new file mode 100644
index 0000000..d3f7e56
--- /dev/null
+++ b/docs/configuration.md
@@ -0,0 +1,240 @@
+# Configuration
+
+Every option can be given on the command line, in configuration files, or both.
+Precedence, lowest first:
+
+1. built-in defaults;
+2. configuration files, in the order of `--config` (repeatable; later files win);
+3. command-line arguments.
+
+An argument that replaces a value from a file is logged (info) with both values and
+the file. Imported values are never logged, only where they come from; text with
+placeholders is logged as it ends up. Secrets (`mycelium-rest-url`, which may carry
+credentials) are never logged, however they are given.
+
+`continuum --help` lists all options with their defaults.
+
+## Durations and sizes
+
+Durations (`--keepalive`, `--dormant-probe-max`, `--document-append`,
+`--document-rewrite`, `--electorate-window`, `--clock-skew-warn`, and the API's
+`settle`, `switchDelay` and retention `age`) are a bare number of seconds (`30`, `0.5`)
+or components with units in descending order: `ms`, `s`, `m`, `h`, `d`, e.g. `500ms`,
+`500s`, `1h30m`, `1d2h3s`, `1.5h`. Precision is a millisecond; anything finer is an
+error. Sizes (`--blob-chunk`) are a bare number of bytes or a number with `B`, `KiB`,
+`MiB` or `GiB`, e.g. `64KiB`, `1.5MiB`; they must come to whole bytes. In configuration
+files both may also be numbers (seconds, bytes).
+
+What the node reports (stats, `/peers`, remake reports) stays numeric, with the unit
+in the name: `uptimeSecs`, `dormantMs`, `savedBytes`. Timestamps are Unix
+milliseconds, named `…At` or `…Until`.
+
+## Files
+
+TOML (`.toml`) for people, JSON (`.json`) for generated configuration. Keys are the
+long option names. Nested tables join their keys with `-`, so these are the same:
+
+```toml
+mycelium-peers = ["tls://203.0.113.5:9651"]
+
+[mycelium]
+peers = ["tls://203.0.113.5:9651"]
+```
+
+Unknown keys are an error.
+
+An example ([`example.toml`](example.toml) has every option, explained):
+
+```toml
+include = ["/run/secrets/continuum.json"]
+
+bind-addr = "[::1]:3000"
+htpasswd = "/etc/continuum/htpasswd"
+node-key-file = "/var/lib/continuum/node.key"
+seeds = ["node1.example.org", "node2.example.org"]
+alias = { command = ["hostname", "-s"] }
+keepalive = "1m"
+neighbours = 12
+
+[mycelium]
+enable = true
+key-file = "/var/lib/continuum/mycelium.key"
+peers = ["tls://203.0.113.5:9651", "quic://203.0.113.6:9651"]
+```
+
+The same as JSON, as a system like NixOS would generate it:
+
+```json
+{
+  "include": ["/run/secrets/continuum.json"],
+  "bind-addr": "[::1]:3000",
+  "htpasswd": "/etc/continuum/htpasswd",
+  "node-key-file": "/var/lib/continuum/node.key",
+  "seeds": ["node1.example.org", "node2.example.org"],
+  "alias": { "command": ["hostname", "-s"] },
+  "mycelium": {
+    "enable": true,
+    "key-file": "/var/lib/continuum/mycelium.key",
+    "peers": ["tls://203.0.113.5:9651"]
+  }
+}
+```
+
+### Includes
+
+`include` (a path or a list) loads other files first, in the order listed; an entry
+`{ path = "…", optional = true }` is skipped where the file is missing. Relative paths
+are relative to the including file. The including file's own values win over its
+includes, later includes over earlier ones (with a warning, since two includes setting
+the same option is usually a mistake). A cycle is an error.
+
+This keeps public settings in version control and secrets elsewhere:
+
+```toml
+# continuum.toml, committed: the secrets, then local settings over them
+include = ["/run/secrets/continuum.toml", { path = "local.toml", optional = true }]
+seeds = ["node1.example.org"]
+```
+
+## Transports
+
+A node uses every transport enabled: `--mycelium-enable`, `--libp2p-enable` and
+`--tor-enable`, each in builds with its cargo feature (`--features mycelium,libp2p`,
+none by default). None is on by default: a node needs one at least, and without, the
+error says which this build has. As each is its own option, a file that configures a
+transport can enable it too, and including it is all it takes:
+
+```toml
+# continuum.toml
+include = ["libp2p.toml"]
+
+# libp2p.toml
+[libp2p]
+enable = true
+listen = ["/ip4/0.0.0.0/tcp/9661", "/ip4/0.0.0.0/udp/9661/quic-v1"]
+```
+
+`--transport-order` is their order of preference (`mycelium,libp2p,tor` by default),
+whether enabled or not; those it leaves out come after, in the default order. How a node
+uses several: [`transports.md`](transports.md), Several transports. Options
+of a transport the node doesn't use (not enabled), or this build doesn't have, are
+ignored, and the node says which at start (`Options of a transport this node doesn't use are ignored`,
+with the transport, whether it's built, and the options); a configuration shared
+between nodes may carry them on purpose. A transport's options are checked when the
+configuration is loaded if the node uses it (Mycelium's always, where it is built), so
+`--print-config` catches a wrong one too.
+
+What each transport is for and its options: [`transports.md`](transports.md).
+
+## Unsetting and conflicting options
+
+A later source can set an option back to its default: `--unset OPTION` on the command
+line (repeatable), `unset = ["option", …]` in a file (overriding its includes and
+earlier files), or `null` for the option in a JSON file.
+
+Some options can't be combined: `mycelium-rest-url` (use a Mycelium daemon running
+elsewhere) excludes the options of a supervised daemon (`mycelium-key-file`,
+`mycelium-key-create`, `mycelium-network-key-file`, `mycelium-network-name`,
+`mycelium-tcp-listen-port`).
+When a later source sets one side, values for the other side from earlier sources are
+dropped (and logged), so `--mycelium-rest-url` on the command line just works with a
+file that configures a supervised daemon. Setting both sides in the same source is an
+error.
+
+Other combinations contradict each other and are an error wherever their values come
+from: `genesis` with `seeds` (a new swarm has nobody to join) or with `recover-from`.
+To start a node with `--genesis` whose configuration file lists seeds, unset them:
+`--genesis --unset seeds`.
+
+## Forcing
+
+Some checks refuse to start the node because something looks wrong, but can be
+overridden when you know better: `--force` lists them by name, or is `true` for all.
+An unknown name is an error. The API's `?force=` parameters use the same format.
+`--force` is accepted on the command line only: it is meant for the one start it was
+given for, and in a configuration file it would apply to every start.
+
+| Name | Check |
+|---|---|
+| `swarmId` | the document to start on (`--recover-from`, `--document-file`) is of another swarm than the state file remembers; with it, the document's swarm wins |
+| `badDocumentFile` | the document file can't be read; with it, it is moved aside and the node starts without it |
+| `replaceDocumentFile` | `--recover-from` while the document file holds a document; with it, the backup wins |
+
+Names are camelCase everywhere, on the command line too, like the API's
+(`switchDelay`, `supersede`, `alive`).
+
+## Importing values
+
+A value can be taken from elsewhere instead of written out:
+
+| In a file | On the command line | Value |
+|---|---|---|
+| `{ env = "NAME" }` | `env:NAME` | environment variable |
+| `{ file = "PATH" }` | `file:PATH` | file contents |
+| `{ command = "COMMAND" }` | `cmd:COMMAND` | output of `sh -c COMMAND` |
+| `{ command = ["prog", "arg"] }` | | output of the program, no shell |
+
+Commands run as the node's user, at startup: whoever can write the configuration can run
+them ([`trust.md`](trust.md)).
+
+The content is text with trailing line breaks removed, read like a command-line
+value: numbers for numeric options, `true`/`false` for switches, comma- or
+line-separated items for lists. With `format = "json"` in a file, or `+json` on the
+command line (`env+json:SEEDS`, `file+json:PATH`, `cmd+json:COMMAND`), it is parsed
+as JSON instead (e.g. a list as `["a", "b"]`).
+
+In files, relative paths are relative to the file, and commands run in its directory.
+
+Any other text value can mix fixed text with imported values: each `${env:NAME}`,
+`${file:PATH}` or `${cmd:COMMAND}` in it is replaced by that content, as text (no
+`+json` in a placeholder). `$$` is a `$`, and any other `$` outside a placeholder is
+an error (a shell-style `$NAME` would otherwise silently stay as it is). A placeholder
+ends at the `}` that matches its `{`: braces inside it balance, and in a command, braces
+in shell quotes (`'...'`, `"..."`) or after a backslash don't count, so a command may
+hold `$`, quotes and braces of its own. Placeholders don't nest. `literal:` on the
+command line turns this off too.
+
+```sh
+continuum --alias 'node-${env:HOSTNAME}'
+```
+
+```toml
+alias = "${env:SITE}-${cmd:hostname -s}"
+# A command with both kinds of quotes: a TOML literal string ('''...''') needs no escapes.
+libp2p-listen = ['''/ip6/${cmd:ip -j addr show wireguard | jq -r '.[].addr_info[]|select(.family=="inet6" and .scope=="global").local'}/tcp/9661''']
+```
+
+A command that prints several lines (above: several addresses) doesn't give one item
+per line: as a list element the text is one item, line breaks and all, and as a whole
+list value it is split at commas and line breaks, the fixed text going with the first
+and last item only. Make it print one (`| head -n 1`).
+List elements can be imported one by one:
+
+```toml
+seeds = ["node1.example.org", { env = "EXTRA_SEED" }]
+```
+
+On the command line, `literal:` passes a value that happens to start with one of the
+prefixes: `--alias literal:env:prod`.
+
+```sh
+continuum --config /etc/continuum.toml --alias env:HOSTNAME --seeds cmd+json:'my-inventory --seeds'
+```
+
+Imports are read once, at start.
+
+## Checking the configuration
+
+`--print-config` prints the effective configuration as JSON and exits: every option
+with its value and where it came from (`default`, `file` with the file, or
+`argument`; `unset` if it was set back to its default there). Imported values are left
+out and their importers listed instead (`imported`), and secrets are left out however
+they were given (`secret`), so the output can be shared. Text with placeholders is
+shown as it ends up, with the text as written under `templates`: keep secrets of
+other options out of placeholders, and import them whole instead. A running node
+answers the same at `GET /admin/config`; with `?values=original`, text with
+placeholders is shown as written instead.
+
+```sh
+continuum --config /etc/continuum.toml --print-config | jq .options.seeds
+```