1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
|
# Users and what they may do
Who the users are, and what each may do, is the swarm's: a registry in the document,
the same on every node. How a user proves who it is stays each node's: its
`--htpasswd`, or a reverse proxy naming the user (`--user-header`, [`trust.md`](trust.md)).
## The registry
`_continuum/users/<name>`, one entry per user. It is protocol data: every member holds
it, a remake's snapshot carries it into the next epoch, and it holds no secrets (no
password hashes: every member reads the document).
```json
{ "since": 1762400000000, "by": "continuum:3f…",
"caps": { "8c41e2…": { "path": [], "rights": ["read", "write"],
"by": "continuum:3f…", "since": 1762400000000 } },
"disabled": false }
```
* **Names** are what `--htpasswd` and the proxy give: 1 to 64 of `A-Z a-z 0-9 . _ @ -`.
* **Capabilities** (`caps`): each a path, as segments as the store's API takes them,
and the rights it gives there and below: `read`, `write`. A capability covers its
path segment by segment (`servers/eu` covers `servers/eu/0`, not `servers/europe`).
One outside protocol data never covers protocol data: `[]` is all of the swarm's
data, `["_continuum"]` protocol data, managing users included (`_continuum/users`).
They are kept by a random ID, so two capabilities granted at once both hold, and
revoking one removes it alone. Each names who granted it (`by`: a user as
`user:<name>`, or the node that recorded it) and when (`since`).
* **`disabled`**: the entry stays (who wrote what stays readable), and gives nothing.
An entry that doesn't read as above gives less, never more: a capability that
doesn't read is left out, and anything but `false` disables.
* **`keys`**: the user's SSH keys, each by a name of its choosing, as an
`authorized_keys` line has it (`"laptop": "ssh-ed25519 AAAA…"`; Ed25519 only): what
it signs values with (Values signed by their users, below).
## A new swarm
A swarm gets a registry from its start if its genesis node is given its first users,
`--bootstrap-users`: it records them, with the swarm ID, in its first change, each with
everything (`[]` and `["_continuum"]`). Other users, of its `--htpasswd` or any node's,
may do nothing until added (the genesis node warns of those of its `--htpasswd`). With
`--bootstrap-keys alice=ssh-ed25519\ AAAA…\ laptop` it records their keys too, so they
log in by key and sign values from the start: a swarm whose first user then writes the
policy (Values signed by their users, below) is strict from its start.
Without it, the swarm has no registry: its users are all alike (A swarm without a
registry, below), until someone gives it one. `--protocol-writers` counts in
both: without a registry, as who may write protocol data; with one, to set it right
should nobody be left who may manage users (Keeping the swarm manageable, below).
## At the HTTP API
Every request is checked against the registry, as the node serving it holds it: a
change to it counts on a node as soon as it has reached it.
* **The store** (`/store`, `/stream`): writing a path (`PUT`, `POST`, `DELETE`) takes
`write` on it; `?prune=true` deletes maps left empty only up to the highest one the
user may write. Reading takes `read`: of a place the user may read only partly, it
sees the keys on the way to what it may read, as a directory one may pass through
but not list (a capability on `servers/eu` shows `servers` with `eu` alone in it),
and nothing else; a list on the way is left out whole, as its items have no paths of
their own. Native forms (`text/plain` of a map, a counter as its own type) only of
what it may read whole. What a write answers with is what was written. A stream
shows what the user could read when it opened.
* **Protocol data** is in the store too, under `_continuum`: `read` and `write` on it
as on any path.
* **Looking at the node** (`/admin/stats`, `GET /peers`, the dashboard): any user with
a capability. The dashboard's document box is read through the store, as the user
may; the page is told only of changes where its user may read.
* **Remakes** (`/admin/remakes`): `read` on `_continuum/remakes` to look, `write` to
propose or abort one. **Forgetting a peer** (`DELETE /peers/…`): `write` on
`_continuum/forgotten`.
* **Everything else an operator does** (`/admin/config`, `/admin/log`,
`/admin/document`, `/admin/changes`, `/admin/epochs`, `/admin/quarantine`, `/debug`):
`read` on `_continuum` to look, `write` on it to act.
A user the node lets in but the registry doesn't have may do nothing (`403`).
`[]` and `["_continuum"]` are two capabilities on purpose: protocol data acts on the
whole swarm (its members, its remakes, its users), so it is given apart, never by
accident with the data. A user who may do both has both.
## Tokens
A user logs in on any node, by what that node checks (its `--htpasswd`, or a proxy's
`--user-header`), and gets a token every node of the swarm takes, sent as
`Authorization: Bearer <token>` instead of a password:
```sh
curl -u alice -X POST http://localhost:3000/auth/login # {"token": "ct1.…", …}
curl -H "Authorization: Bearer ct1.…" http://other-node:3000/store
```
* **Who signs it**: the node it logged in on, with its node key. Every node takes a
token signed by a member of the swarm (the member list gives the keys: a node ID is
its key), and none signed by another or since forgotten.
* **What it says**: who, how it proved it, when it ends, and optionally less than the
user may do (`limit`, capabilities as `POST /admin/users/{name}` takes them): never
what the user may do. Every node judges a request by the registry it holds then, so
a capability granted or revoked counts on a node as soon as the change reaches it.
* **How long**: `--token-lifetime` (12 hours), refreshed (`POST /auth/refresh`, on any
node, which signs the new one) until `--session-lifetime` (7 days) after the login;
then the user logs in again. A refreshed token may say less (`limit`), never more,
and live less (`lifetime`, in seconds): a dashboard opening another member's hands it
one reading only, for five minutes ([`dashboard.md`](dashboard.md)).
* **Revoking**: a user disabled or removed is refused at once; all of a user's tokens,
by the user (`POST /auth/logout`) or by one who may manage users (`DELETE
/admin/users/{name}/tokens`), which sets `_continuum/logouts/<name>`: tokens issued
before aren't taken any more. A token of a node forgotten by the swarm isn't taken
either. In a swarm without a registry, a token is taken until it expires.
* **What it means to trust**: any member could sign a token for any user, as it could
write anything as anyone ([`trust.md`](trust.md)). A token is a bearer's: whoever holds it is
the user until it ends, so it goes over TLS only, as passwords do.
A user with keys in its entry (`keys`, The registry, above) logs in on any node without
a password: it asks for a challenge, signs it with one of its keys, and gets the same
token (`login: "key"`).
```sh
c=$(curl -s -X POST localhost:3000/auth/challenge | jq -r .challenge)
printf %s "$c" | ssh-keygen -Y sign -f ~/.ssh/id_ed25519 -n continuum-login > sig
jq -n --arg c "$c" --rawfile s sig '{user: "alice", challenge: $c, signature: $s}' |
curl -s -X POST localhost:3000/auth/login/key -H 'Content-Type: application/json' -d @-
```
A challenge is the node's, signed by it, valid for five minutes, and taken once;
the signature must be by one of the user's keys, as the registry the node holds then
says, and the user enabled (`401` if not, saying why). A swarm without a registry has
no keys (`409`).
A browser keeps a token as its session: `POST /auth/session`, with a token, answers as
`/auth/refresh` does and sets a cookie holding the new one (`HttpOnly`,
`SameSite=Strict`: no script reads it, no other site's page sends it), which lets in
requests without `Authorization` that read (`GET`, `HEAD`) or renew or end the session; one
that changes anything brings the token itself, which no other page, of another site or
of another port of the same host, can make a browser send. `DELETE /auth/session`
forgets it. The login page
(`/login`, without authentication, with the dashboard) begins one by password or from a
token pasted, from a login by key, say; the dashboard renews it while open
([`dashboard.md`](dashboard.md)).
`GET /auth/whoami` says who a request's user is, how it proved it, and what it may do.
Pages of other origins, the dashboards of other members, may call a node with a token:
its own links (`/admin/dashboard/links/own`, and its events) and messages
(`/admin/dashboard/messages/own/events`), `/auth/refresh` and `/auth/whoami`, nothing
else. A node allows the origins of the dashboards its member
list names (`--dashboard-url`) and those of `--cors-origins`; without credentials (no
cookie or password crosses origins: the page sends its token).
## Managing users
Through `/admin/users` ([`openapi.yaml`](openapi.yaml), Users): looking needs `read` on
`_continuum/users`, changing `write` on it. Each change is one change to the document,
logged and named by who made it. `POST /admin/users` adds several at once (below).
```sh
curl -u alice http://localhost:3000/admin/users # who, and what each may do
curl -u alice -X POST -H 'Content-Type: application/json' \
--data '{"caps": [{"path": ["servers", "eu"], "rights": ["read"]}]}' \
http://localhost:3000/admin/users/bob # add bob, reading servers/eu
curl -u alice -X POST -H 'Content-Type: application/json' \
--data '{"path": [], "rights": ["read", "write"]}' \
http://localhost:3000/admin/users/bob/caps # grant: answers its ID
curl -u alice -X DELETE http://localhost:3000/admin/users/bob/caps/ID # revoke it
curl -u alice -X PUT -H 'Content-Type: application/json' --data true \
http://localhost:3000/admin/users/bob/disabled # disable bob
```
Disabling a user keeps who wrote what readable; removing it (`DELETE`) doesn't. The
registry is protocol data in the store too, so a user who may write `_continuum/users`
may also edit its entries there directly; the same checks apply.
## A swarm without a registry
In a swarm whose document has no registry (`_continuum/users`), started without
`--bootstrap-users`, every user a node lets in may read and write the swarm's data and
use the admin endpoints, and only those in `--protocol-writers` write protocol data. It
gets a registry once a protocol writer adds its users:
```sh
curl -u alice -X POST -H 'Content-Type: application/json' --data '{
"alice": {"caps": [{"path": [], "rights": ["read", "write"]},
{"path": ["_continuum"], "rights": ["read", "write"]}]},
"bob": {"caps": [{"path": [], "rights": ["read", "write"]}]}
}' http://localhost:3000/admin/users
```
From that change on, every node serves only the users the registry has, so the caller
must be among them, with `write` on `_continuum` (else `409`): list every node's users
first.
## Keeping the swarm manageable
A user may not lock itself out: a write that would keep its writer from managing users
(enabled, with `write` on `_continuum/users`) when it could, deleting, disabling or
narrowing its own entry, is refused (`409`), and nothing of it is written; so is one
that would remove the registry (`_continuum/users`). Others are not kept from removing
the last user who may manage users: each node can only judge what it holds, and two
such writes on two nodes at once would leave nobody once they merge anyway. Then,
while nobody may manage users, the users in a node's `--protocol-writers` may do
everything there again, to set it right.
## Changes judged on every node
What a user may do is checked where it writes, by the node it writes through. A swarm
can also have every node judge every change it receives before applying it, so a node
whose operator lets anyone write (or that claims another's users) gets nothing past
the others: the policy, at `_continuum/access/policy`, written by a user who may write
protocol data.
```sh
curl -X PUT localhost:3000/store/_continuum/access/policy \
-H 'Content-Type: application/json' -d '{"version": 1, "signatures": "optional"}'
```
A change is judged by what it touches, by the node that committed it, and by the user
it names, as the document stood when it was written (at its dependencies): a
revocation since refuses nothing it allowed then, and every node judges alike.
* A member's entry (`_continuum/members/<id>`) and its votes are written by that member
only; removing an entry, also by a user who may write there.
* Epochs and the swarm ID (`_continuum/epochs`, `_continuum/swarm`) by an epoch's first
change only (the genesis change, a remake's snapshot).
* A remake's decision by the coordinator its proposal names; the rest of a record by
any member.
* A user's tokens revoked by that user too (logging out).
* A member's forget record cancelled by that member too (forgotten while alive).
* Everything else, protocol data or not, by a user the registry lets write there.
* A node that isn't a member may only register itself; a banned node nothing (Banning
a node, below). So a node registers before anything else it writes, through the API
or its own: a change refused stays out with every change built on it.
With `"signatures": "required"`, a change also needs a signature by the node it names
([`api.md`](api.md), Signed changes): every node then signs every change it makes, and
one no signature vouches for yet waits, with whatever builds on it, until one does
(they come with the changes, or within a second on their own). A member can then no
longer write as another member; the user a change names is still its node's word.
In a swarm without a registry, whatever a user would need, any member may. A refused
change stays out, and so does whatever builds on it; the node logs it (`Refused a
change`) and lists the last thousand (`GET /admin/refused`). Offered again, as its
writer's node keeps offering what is never taken, it is refused by its hash, not judged
or logged again, and a peer offering nothing else is answered later each time, up to a
minute apart.
With signatures required, a member whose own change broke a rule (its signature proves
it its own) is refused as a whole: every node sends it nothing more, as all it commits
from then on builds on what was refused, and logs it once. It shows in `GET /peers`
(`refused`, with the change and why), in `/admin/stats` (`sync.refusedPeers`) and on the
dashboard. A node that only passed a refused change on isn't: the committer is, whoever
brought it. It lasts until a change of the member's is taken again (it rejoined and
writes anew), or the epoch changes; forgetting or banning it is the operators'.
A node takes a change the others refuse only if it didn't judge it as they do: under a
policy it didn't know yet (it receives without judging then), or before the policy
came. Everything it writes from then on builds on that change, and is refused too. So
a node refusing changes a peer brought tells it which broke a rule (REFUSALS, at most
every 30 seconds). The peer doesn't take the word for it: it judges each it holds
itself, by its rules as they are now. If they refuse one too, it drops it with
everything built on it, its document becoming what it would be without them, and puts
its own writes among them in the quarantine (`GET /admin/quarantine`), to be replayed
or discarded; what it writes from then on goes everywhere. If its rules take the
change, the two policies disagree, which it logs. A change whose
dependencies haven't come yet waits for them.
What it doesn't do yet:
* **Prove the user.** A change is its node's (with signatures required, proven) and its
claimed user's: a member could claim a user its operator doesn't have.
* **Judge a joining node's first sync.** An empty document has no policy: what it gets
from the node it joins through is taken as it is, as that node judged it.
* **Agree at once.** The policy applies as each node gets it: a change received between
the policy's arrival on one node and on another may be refused by one and kept by
the other.
A node that doesn't know the policy (another version, or a setting it lacks) takes no
writes through the API (`409`) and receives without judging, until it is upgraded.
## Banning a node
A node can be banned: `_continuum/banned/<node ID>`, written by a user who may write
there (protocol data). Every node then leaves its handshakes unanswered, so it is
neither synced with nor counted a member, and with the policy every node refuses the
changes it commits from then on, judged as the ban stood when they were made (what it
wrote before stays). Deleting the entry lifts the ban. A ban doesn't remove the node
from the member list: forget it too (`DELETE /peers/<node ID>`).
```sh
curl -X PUT localhost:3000/store/_continuum/banned/<node ID> \
-H 'Content-Type: application/json' -d '{"by": "alice", "reason": "compromised"}'
```
## Values signed by their users
A member's node signs its changes, but the user a change names is its node's word: a
member could write as any user. A swarm can have its users sign the values that
matter themselves, with their SSH keys, so no member can write them but as a user
signed them: the policy's `"values": "signed"` (strict; it requires signatures too).
```sh
curl -X PUT localhost:3000/store/_continuum/access/policy \
-H 'Content-Type: application/json' -d '{"version": 1, "values": "signed"}'
```
Signed then are what decides who may do what (every user's entry, every entry of
`_continuum/access`, every ban), and the paths listed at `_continuum/access/userSigned`,
a list of paths as capabilities name them, `*` for any one key:
`[["config"], ["servers", "*", "secret"]]`. Each such value is an envelope, one string
(`cv1.<payload>.<signature>`): the payload names the path, the value (`null`: deleted),
the user, when, and the envelope it replaces (`prev`), and the signature is the user's
over it, as `ssh-keygen -Y sign -n continuum-value` makes it, with one of its `keys`.
`scripts/continuum-signed` is a reference client, with `ssh-keygen`, `curl` and `jq`:
it logs in by key, writes and deletes signed values (`put config '{"mode": "fast"}'`),
and checks one against its user's keys (`verify config`).
What a client reads to check a signature, the signer's keys, is in the registry, and
whom to seal for (Values sealed for their readers, below) in `_continuum/access`: both
protocol data, which a user's capabilities cover only if given so. Neither holds
secrets (every member holds the document), so give the users who check or seal reading
them:
```sh
curl -u alice -X POST -H 'Content-Type: application/json' \
--data '{"path": ["_continuum", "users"], "rights": ["read"]}' \
http://localhost:3000/admin/users/bob/caps
curl -u alice -X POST -H 'Content-Type: application/json' \
--data '{"path": ["_continuum", "access"], "rights": ["read"]}' \
http://localhost:3000/admin/users/bob/caps
```
Under a strict policy, the same, as part of bob's entry, signed (`continuum-signed put
_continuum/users/bob '{…}'`).
A node takes one only from that user, for that path, replacing what is there, by one
of the user's keys, and if the user may write there ([`api.md`](api.md), Signed values);
it writes nothing else there, nor within or above such a value (`409`), and the users
API changes nothing (`409`). Values there before the policy stay as they are until
replaced: a registry whose entries hold the users' keys turns strict as it is. Without
the policy, signed values may be written all the same, and plain ones too.
Every node judges the changes it receives alike (Changes judged on every node, above),
as their writer saw the registry: at such a place, only a string that is an envelope
that checks, replacing the one there then; nothing within one; and above one, nothing
replacing or deleting what holds one (a map made where there is none yet is fine). So
a member writing to its document directly gets nothing there past the others: the
registry, the policy and the bans change only as a user who may change them signed.
## Values sealed for their readers
Anyone may read the whole document: every member holds it. A value only some may read
is sealed for them: encrypted with age to their SSH keys (`keys`, The registry,
above), so only they open it. Every node syncs, stores and remakes it as the string it
is, and none decrypts it: `cs1.<the age file, base64url>` (binary, not armored).
```sh
jq -r '.keys[]' <(curl -s -u alice localhost:3000/store/_continuum/users/bob) > readers
printf %s "$secret" | age -R readers | basenc --base64url -w0 | tr -d = |
sed 's/^/cs1./' | jq -R . | curl -u alice -X PUT localhost:3000/store/secrets/db -d @-
```
Any value anywhere may be sealed, for anyone. Where the swarm declares it, every value
must be: `_continuum/access/sealed`, a list of `{"path": [...], "readers": [...]}`, `*`
in a path for any one key, written as any of `_continuum/access` (under a strict
policy, signed).
```json
[{"path": ["secrets", "*"], "readers": ["alice", "bob"]}, {"path": ["notes"]}]
```
Without `readers`, a value there must be sealed, for anyone; with them, sealed for
exactly their keys, as the registry has them when it is written. A node tells without
decrypting: an age file names each `ssh-ed25519` recipient by a tag, the first four
bytes of the SHA-256 of its key. A write leaving a value there plain, or sealed for a
reader's key too few or a stranger's too many, is refused (`409`), and with the policy
every node refuses such a change on receipt (Changes judged on every node, above). A
value signed by its user (Values signed by their users) holds the sealed value; the
signature is over the ciphertext, so every node checks who wrote it without reading
it.
`scripts/continuum-signed` seals and opens them too, with age: `seal secrets/db
'{"password": "…"}'` seals for the readers declared there (or `CONTINUUM_READERS`),
signed, and `open secrets/db` opens it with the user's key.
It guards against mistakes, not against a writer: one who means to leak a value has
it. A reader removed keeps what it read: values stay sealed for the readers they were
written for until written again, sealed for those declared then. Which values there
are, their sizes, when and by whom they were written stay visible to every member.
|