add documentation

1 files changed, 247 insertions, 0 deletions

diff --git a/docs/api.md b/docs/api.md
new file mode 100644
index 0000000..5b4dac4
--- /dev/null
+++ b/docs/api.md
@@ -0,0 +1,247 @@
+# The store API, for applications
+
+What an application can rely on when it reads and writes through a node's HTTP API. The
+endpoints and their answers, one by one, are in [`openapi.yaml`](openapi.yaml) (served by every node at
+`/openapi.yaml`, with a Swagger UI at `/swagger-ui`); this page is about what they mean
+together.
+
+## The model
+
+The store is one document, the same on every node once they have synced, read and
+written as JSON. A path names a place in it: `settings/theme` is the key `theme` in the
+map `settings`, and within a list a segment is an index (`servers/0/name`; `-1` the last
+item). Every node holds all of it and answers from its own copy: there is no leader to
+ask, and no request goes to another node.
+
+```sh
+curl -u alice:secret -X PUT localhost:3000/store/settings \
+  -H 'Content-Type: application/json' -d '{"theme": "dark", "fontSize": 14}'
+curl -u alice:secret localhost:3000/store/settings/theme        # "dark"
+curl -u alice:secret -H 'Accept: text/plain' localhost:3000/store/settings/theme  # dark
+curl -u alice:secret -X DELETE localhost:3000/store/settings/fontSize
+curl -u alice:secret -X POST localhost:3000/store/queue/-1 -d '"job 7"'   # append
+curl -u alice:secret -X POST 'localhost:3000/store/stats/visits?increment=1'  # count
+```
+
+* `GET /store/<path>` reads, `PUT` writes (what is there then reads as the value
+  written; the maps on the way are created), `DELETE` deletes (`?prune=true` also
+  removes the maps it leaves empty), and `POST` inserts into a list at an index, the
+  items from there moving up, or with `?increment=n` adds `n` to a counter (Concurrent
+  writes), or with `?cast=<type>` casts what is there to another type (Types).
+* Bodies are JSON (`application/json`, the default), text (`text/plain`, stored as a
+  string), or a value of a type of its own (Types); answers are JSON, or as `Accept`
+  asks. Numbers in JSON are stored as integers where they fit, as floats otherwise.
+* Every request needs a user from the node's `--htpasswd` (basic authentication). Users
+  are all alike: each can read and write everything but protocol data (below).
+* A key with `/` in it, or an empty key, can be written inside a value at its parent,
+  but not named in a path: read it through the parent.
+
+## Types
+
+Every value is stored as one of Automerge's types: a map, a list, a text, or a scalar:
+a string, bytes, a signed or unsigned integer, a float, a boolean, null, a counter or a
+timestamp (milliseconds since 1970). Every read says which in `x-continuum-type`
+(`map`, `list`, `text`, `string`, `bytes`, `int`, `uint`, `float`, `boolean`, `null`,
+`counter`, `timestamp`), and so does `?shallow=true` for each entry.
+
+Each has a JSON form, so any value reads as JSON: bytes as base64, counters and the
+integers as numbers, timestamps as RFC 3339 strings (`2026-11-01T21:50:20.345Z`), or as
+`Accept: application/json; timestamp-format=…` says: `rfc3339`, `unix` (seconds, with
+the milliseconds as a fraction), `unix-ms` or `http-date` (RFC 9110). JSON alone
+doesn't tell them apart: written as JSON, values are maps, lists, strings, signed
+integers, floats, booleans and nulls.
+
+A scalar is also read and written as its own media type, its value in the body:
+
+| Type | Media type | Body |
+|---|---|---|
+| string, text | `text/plain` | the characters |
+| bytes | `application/octet-stream` | the bytes |
+| int | `application/vnd.continuum.int` | decimal |
+| uint | `application/vnd.continuum.uint` | decimal |
+| float | `application/vnd.continuum.float` | decimal |
+| boolean | `application/vnd.continuum.boolean` | `true` or `false` |
+| counter | `application/vnd.continuum.counter` | decimal (a `PUT` sets it) |
+| timestamp | `application/vnd.continuum.timestamp; format=…` | as the format says (`rfc3339` by default) |
+
+```sh
+curl -u alice:secret -X PUT localhost:3000/store/logo -H 'Content-Type: application/octet-stream' --data-binary @logo.png
+curl -u alice:secret -H 'Accept: application/octet-stream' localhost:3000/store/logo > logo.png
+curl -u alice:secret -X PUT localhost:3000/store/deploy/at \
+  -H 'Content-Type: application/vnd.continuum.timestamp; format=unix' -d 1793569820
+curl -u alice:secret -H 'Accept: application/vnd.continuum.timestamp; format=http-date' \
+  localhost:3000/store/deploy/at        # Sun, 01 Nov 2026 21:50:20 GMT
+```
+
+Typed JSON (`application/vnd.continuum.typed+json`, for reads and writes) keeps every
+type: it is JSON, but for what JSON would show as another type, which is a single-key
+object whose key is a tag: `{"$uint": 5}`, `{"$counter": 5}`, `{"$bytes": "<base64>"}`,
+`{"$timestamp": "2026-11-01T21:50:20.345Z"}` (in its `timestamp-format`) and
+`{"$text": "…"}` (a text); a float is a number, as in plain JSON. A map whose only key
+starts with `$` is `{"$map": {…}}`, so nothing else is taken for a tag; any other
+single `$` key is `400`. A whole map with typed values in it is written at once this
+way, and reads back as written. A text written over a text is changed where it differs
+(a diff, as splices), so concurrent edits elsewhere in it stay.
+
+Types are sticky: plain JSON (or text) written over a value takes its type where it is
+that type's JSON form. A string over a timestamp that reads as one (RFC 3339, or as the
+body's `Content-Type: application/json; timestamp-format=…` says, where numbers are
+read as timestamps too), over bytes that is base64, over a text (then changed where it
+differs); an integer over a float, an unsigned integer (if it isn't negative) or a
+counter (which it sets). So what a client read as JSON writes back as it was stored,
+whole maps included. Typed JSON and a type's own media type say the type themselves,
+but for an integer over a float, which stays a float in typed JSON too.
+
+A write that changes the type of what it writes over (a string over a timestamp that
+doesn't read as one, a map over a list) says which in `x-continuum-retyped`
+(`deploy/at: timestamp -> string`, comma-separated); with `?retype=false` it is refused
+instead (`409`, naming the first). A `PUT` answers with what is there after it, read
+back, so its `x-continuum-type` is what the value was stored as.
+
+`POST <path>?cast=<type>` casts what is there to a type (as `x-continuum-type` names
+it), and answers as a `PUT` does: numbers (integers, floats, counters, timestamps as
+milliseconds since 1970) to numbers where they are exactly that, any scalar but bytes
+to a string or a text as its media type shows it (a timestamp in `?format=`,
+`rfc3339` by default) and back, and bytes to a string or a text and back as UTF-8.
+Anything else, or a value that isn't one of the type (`"soon"` to a timestamp, `2.5`
+to an integer), is `409`, saying why; maps, lists and nulls aren't cast.
+
+```sh
+curl -u alice:secret -X POST 'localhost:3000/store/deploy/at?cast=timestamp&format=unix'
+```
+
+`Accept` is taken in its order (quality values aren't weighed): the first media type the
+value can be shown as, JSON for `*/*` or without an `Accept`, and `406` (naming those it
+can be shown as) if none. `text/plain` shows a string as it is and anything else as
+JSON.
+
+## Consistency
+
+**Reads are local.** A node answers from its copy, which may lack writes other nodes
+made a moment ago: they arrive by sync, as fast as messages get from the writer to
+this node (a hop between neighbours, a few in a large swarm; [`measurements.md`](measurements.md),
+Propagation), and after a partition once it heals. There is
+no read that waits for the rest of the swarm.
+
+**Writes are local too, and never refused for want of others.** A write is applied to
+the node's copy at once (the answer holds the value written) and spreads from there.
+Two nodes cut off from each other both go on taking writes; when they meet again, the
+writes merge. So a write that was answered `200` is durable on that node (with a
+`--document-file`, within `--document-append`, 5 s by default) and reaches the others
+eventually, but it isn't confirmed by anyone else when answered.
+
+**Read your own writes** on the node you wrote to: its later reads include the write.
+On another node, they include it once synced. `?heads=true` on a read gives the
+document's heads (`x-continuum-heads`, the hashes of its latest changes): two reads with
+the same heads read the same document. Every write answers with the heads after it, in
+the same header. `?at=<heads>` reads the document as it was at those heads, on any node
+that has them (`404` on one that doesn't, yet), however it changed since: a client that
+wrote through one node and reads its write through another reads with the write's
+heads, and retries on `404`.
+
+**One request is one change.** What a `PUT` writes is applied whole or not at all, also
+on the other nodes: a map written in one `PUT` never shows half its keys. There are no
+transactions across requests: to change several values together, write their common
+parent.
+
+## Concurrent writes
+
+Writes made on different nodes before they synced are concurrent. They merge, the same
+way on every node:
+
+* **Different keys** both stay: two nodes writing `settings/theme` and
+  `settings/fontSize` get both.
+* **The same key**: one write wins, the same one everywhere. "Newest" means by
+  Automerge's operation order (the write made after seeing more changes; between equals,
+  by the writer's actor ID), not by any clock: of two truly concurrent writes, which wins
+  is arbitrary, but it is the same on every node.
+* **Maps written at the same key** merge into one: their keys are united, each key as
+  above.
+* **A write and a delete** of the same key: the write stays (a delete removes only what
+  the deleting node had seen).
+* **List inserts** both stay, in an order every node agrees on. Writing a list item
+  (`PUT` at its index) overwrites it: two nodes writing the same item concurrently are
+  two writes of the same key.
+* **Counters** (`POST <path>?increment=n`, made by the first increment) add up: two
+  nodes incrementing one counter by 1 concurrently leave it 2 higher. Two nodes making
+  the same counter concurrently make two, and one wins with its increments.
+* **Strings** are values, not texts: concurrent writes to one string don't merge
+  character by character; one wins.
+
+A write changes only what differs from what is there: a map written over a map keeps
+the keys that stay (and what is inside them), gains those it lacks and loses those the
+value doesn't have; a list written over a list does the same item by item, by
+position; a value equal to the one there isn't written again. So a `PUT` of `settings`
+as a whole, concurrent with a `PUT` of `settings/theme` elsewhere, keeps the theme,
+unless the whole write changed the theme too (then one wins) or left it out (then the
+theme's write stays, as against a delete). Where the path holds concurrent values, or
+one of another kind (a map written over a list), it is written anew.
+
+`?conflicts=true` on a read answers `{"value": …, "conflicts": [...]}`: every place on
+the way where concurrent values were resolved, with all candidates and which was
+selected (`mergedMaps` or `newestWins`). A later write at that path ends the conflict.
+
+## Following changes
+
+`GET /stream/<path>` (or `/stream` for everything) is newline-delimited JSON: one event
+with the current value, then one whenever the value there changes, by a write on this
+node or by sync:
+
+```
+{"key":"settings/theme","value":"dark"}
+{"key":"settings/theme","value":"light"}
+{"key":"settings/theme","deleted":true}
+```
+
+An event carries the value as a read would return it then. Changes in quick succession
+may come as one event with the latest value: a stream tells the value, not every write.
+A change that leaves the value as it was sends nothing. The connection stays open; after
+it breaks, reconnect, and the first event is the current value again.
+
+## While a node isn't ready
+
+A node serves the store once it holds the swarm's document ([`lifecycle.md`](lifecycle.md)):
+
+* **Joining** (a new node, or one restarted without its document file): `503` for
+  reads, writes and streams, but for `GET /store`, the whole store, which answers with
+  what has arrived so far. `GET /ready` (no authentication) says `200` once it is ready.
+* **Rejoining** a newer epoch after a remake it missed: reads answer from the document
+  before, with `x-continuum-stale` naming that epoch; writes are answered as usual and
+  go into that document (so reads show them), carried over into the new epoch when the
+  rejoin ends, unless newer writes touched their paths (Concurrent writes); streams are
+  `503`.
+* **Remakes** ([`remakes.md`](remakes.md)) don't interrupt anything else: a node switches to the new
+  document between two requests, and writes made meanwhile are carried over (at
+  their path, or where the remake's rules put them; [`remakes.md`](remakes.md), Writes around a
+  remake).
+
+Load balancers can send requests to any ready node: `/ready` is the check.
+
+## Who wrote what
+
+Every change records the node that committed it (as its Automerge author) and, for a
+write through the API, the user who wrote it (in its message, as a trailer like Git's
+`Signed-off-by:`: `User: alice`), with when (by the writer's clock). User names are
+each node's own (`--htpasswd`), so who wrote it is that user at that node: `alice` on
+two nodes is one person only where their operators make it so, which nothing checks.
+`GET /admin/changes` lists them,
+newest first, with the paths each touched; `GET /admin/changes/settings` those that
+touched what is at `settings`.
+Every node holds every change, so user names are known to the whole swarm, and so is
+who wrote what, until a remake drops the history ([`remakes.md`](remakes.md)).
+
+## Protocol data
+
+`_continuum` holds what the nodes coordinate through: the swarm ID, the member registry,
+remakes and votes ([`architecture.md`](architecture.md), The document). It is part of the store and shows
+in `GET /store`, but isn't application data: only users in `--protocol-writers` may
+write it, for repairs. Keep application data out of it, and leave it out when reading
+the whole store if it's in the way.
+
+## Large stores
+
+`?shallow=true` on a read answers what is there one level deep (a map's keys with their
+types and sizes, short scalars' values) instead of the whole value, to browse a large
+store a level at a time. Requests are limited to 2 MiB. Every write keeps its history
+until a remake compacts it; a document of many small writes grows faster than its
+content ([`remakes.md`](remakes.md)).