| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Setting a node's log filter took `write` on `_continuum`, all of the protocol data:
whoever could change one node's logging could change the swarm's members, remakes and
users too. Now what a node holds alone has paths of its own for capabilities, under
`_continuum/local/<node ID>/`: `log` (`/admin/log`), `config` (`/admin/config`) and
`debug` (`/debug`, always `write`), `read` to look and `write` to act
(`api::access::needed`, which takes the node's ID). So `_continuum/local/<node ID>` is
everything on one node, `_continuum/local/*/log` the log filter on every node, and
holders of `_continuum` keep it all.
Nothing is stored there: the store refuses writes under `_continuum/local`, and the
rules refuse a change touching it, as for epoch metadata. The quarantine stays with
`_continuum`: its entries are everyone's writes, which a node-local capability
mustn't show.
|
| |
|
|
|
|
| |
Since load_delta is for tests (1190), nothing else in changes.rs uses
anyhow's Context, and builds without tests warned of an unused import.
load_delta imports it itself.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A sync message carried the signatures its peer wasn't sent yet, by a
cursor per peer. Changes sent again, as after a message taken as
delivered and lost on the way, went without them: the peer was past
them. Under a policy requiring signatures they waited until the
writer's next signed change, or the next session.
A message now carries the signatures of the changes it carries
(Signatures::for_changes), whatever the peer was sent before: changes
sent again arrive vouched for as the first time. Past 256 (a message
carrying the batches of many writers), each writer's latest instead,
as a delta carries. The cursor goes, and with it resetting it after a
failed send or a new session.
|
| |
|
|
|
|
|
|
|
| |
delivered_messages_are_answered_with_changes_we_lack failed now and
then: after the answer, the node still lacked the peer's change.
Automerge's sync tells what each side has by a Bloom filter, whose
false positives take more rounds. The test now exchanges up to four
rounds more, each asking for what is still missing, until both are
level.
|
| |
|
|
|
|
|
|
|
|
|
| |
A sync message took the signatures its peer wasn't sent yet whenever it
had a body, an Automerge message saying only what we have included. A
signature belongs with the changes it vouches for: they go along the
trees, and a peer that got them elsewhere got the signatures with them.
Signatures now go in a message that carries changes (pushed, offered
as a delta, or in an Automerge message carrying some: SyncSession::
carrying), and in no other.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A node signed its changes no rule signs a while after them (--sign-every
1s): the change left at once, unsigned, and its signature followed on
its own a second later, to every neighbour, over every link. A few
writes made a cascade of SIGNATURES. Under a policy requiring
signatures it signed every change as it was made instead.
A node now signs its latest change, if no signature vouches for it, as
a sync message takes it (--sign-every sent, the default): the
signature leaves in the same message, and goes where the change goes.
One signature per message, however many changes it carries, each
vouching for those before it: a slow writer's every change is signed,
a fast writer's batches are, at no threshold to pick. A policy
requiring signatures signs what is sent too, even with `none`. `all`
still signs every change as it is made; changes to protocol data,
listed paths and ?signed=true writes are signed as they are made, as
before. The interval, and the task signing on it, go.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
A node commits a change, keeps it in its document file, and may stop
before any signature vouches for it. It starts again under a new
actor and only ever signed its current actor's latest change: the old
one stays unsigned for good. Peers pulling it then wait for a
signature under a policy requiring them, and so does everything the
node writes after, building on it. Signing every change at commit hid
this under such a policy; signing what is sent, next, won't.
At start, before anything leaves, a node signs each of its own
changes at the document's heads that no signature vouches for:
nothing it made since builds on them, so they are heads.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 3, R4: whether a batch was judged followed the receiving
document's policy now (and whether the batch writes one). After the
policy was removed, or replaced by a version the node doesn't know,
changes written under it and received late went in unjudged, while
nodes that had them earlier judged them; REFUSALS gave up likewise.
Screening now remembers that the document held a policy, known or not
(Screening::policed), and judges what it receives while it does, each
change by the policy at its dependencies; REFUSALS judges then too
(not under a policy the node doesn't know, as before). Not across a
restart, which access.md says.
|
| |
|
|
|
|
|
|
|
|
| |
With every message's signatures offered now (1205), Signatures::offer
runs for each, and it asked get_change_meta_by_hash whether the
document holds each signature's change, and each one waiting, up to
4096 of them: Automerge rebuilds the change's metadata from the
document's operations to answer. Ask changes::holds instead (1197),
there and in forks and kept_for; the metadata is still read where it
is used.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the policy's reads kept, judging a change still costs about one
read (some 3.5 µs), and so does judging without a policy: what is left
is At::new, which asks get_change_by_hash for each head. Automerge
builds the change again from the document's operations to answer, at
a cost growing with its history. Screening asks it again for every
candidate and every dependency, and sync for heads.
Ask by the missing dependencies instead (changes::holds), which look
the hash up: one the document holds is not among them. A change queued
in the document, waiting for its dependencies, isn't either; but then
some are missing, and only then is it asked the slow way. Every check
of whether a document holds a change goes through it.
|
| |
|
|
|
|
|
|
| |
The breakdown judges at the heads after 400 changes. Whether a read
at given heads costs more in a document with a long history decides
what a read the rules don't keep costs a node with a big document.
For the workloads of more than 10000 changes, break it down a second
time, judging the last 1000 at the heads after the rest.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bench_screening shows judging at about 17 µs of a change's 25: some
two and a half reads at its heads (the policy, bans, its node's member
entry, whether there is a registry, the sealed declarations), each
costing as much as working out what the heads cover. Screening judges
most of a batch at the same heads, and those reads are the same for
every change judged there.
Keep them (rules::Reads), by node and by user where they depend on
one, while the heads stay the same; screening keeps one for a batch.
What the heads hold doesn't change as the document takes changes, so
nothing goes stale but by other heads. The benchmark breakdown shows
judging so and afresh.
|
| |
|
|
|
|
|
|
|
| |
bench_screening tells what screening costs in all, not where. Print,
per change judged, what decoding it, placing its operations
(inspect::Inspector) and judging it (rules::judge) cost; of judging,
what reading the policy costs, and one read for scale. Judged as
screening judges most changes: at the heads after the first ones,
whose maps flush the batch.
|
| |
|
|
|
|
|
|
|
|
| |
A write every node would refuse is rolled back by forking the document
at the heads before it, under the write lock, at a cost that grows with
the document's history. A user could ask for such writes in a loop.
Count each user's refused writes over the last minute. Past
--refused-writes-per-minute (default 10), its writes are answered 429
without being tried, until the oldest refusal leaves the minute.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 2, N7: a node screened received changes only while its own
document had a policy it enforces. A joiner's empty document has none,
so its first sync, as much history as the node it joins through chose,
was applied unjudged, and under signatures required unvouched; the same
for any node while the policy's own change arrived with later ones.
Sync messages, deltas and downloaded changes are now always screened,
each change judged by the policy at its dependencies (1135), so the
verdicts are the swarm's. While neither the document nor the changes
hold a policy (none touches a "policy" key: inspect::touch_key), only
an epoch's root is judged and nothing is placed: bench_screening, after
1188, measured placing everything at about as much again as applying
(564 ms + 543 ms for 100 000 changes), judging by the rules at about
five times (+2.5 s), which a document with a policy pays anyway.
access.md no longer lists the joiner's first sync, nor nodes agreeing
only as the policy reaches them (judged at the dependencies, they
agree), as left to do; changes::load_delta is for tests now.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 2, N9: the API checked a write's capability on the path asked
(servers/list/3/name), the rules on the list it is in (servers/list):
a user whose capability lay inside a list had every such write pass the
API and refused by the node's own judgement (403, after rolling the
document back).
inspect::placed places a write's path in the document by the rule the
inspector places operations by (Place::child: cut after the first list
or text on the way), and the store's writes check capabilities there.
One rule for both; a test checks placed against the paths the inspector
finds in the change such a write makes.
|
| |
|
|
|
|
|
|
|
|
|
| |
Review 2, N7, measured first: a node whose document has no policy (a
joiner's empty one) applies what it receives unjudged. Screening it too
would cost finding every change's paths and judging it. bench_screening
takes 1 000, 10 000 and 100 000 changes into an empty document, as one
delta brings them: unjudged, as Automerge applies them, against
screened (access::screen::apply), with no policy in them (each passes,
read at its dependencies, after its paths were found) and with the
policy their first change wrote (each judged by the rules).
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Review 3.1, part of finding 12: a signed value's payload bound path,
user, time and prev, but not the swarm. A user with the same name and
key in two swarms could have an envelope replayed from one into the
other, wherever the path holds no envelope yet (prev "").
The payload now names the swarm (`swarm`), and a node takes an envelope
only for its own. /auth/whoami says the swarm, and continuum-signed and
the example in api.md sign it. Wire version 44: nodes before this
refuse such envelopes, and those after refuse the old ones.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review finding 9: once 4096 signatures waited for their changes, new
ones were dropped, and the waiting changes were cut at 10 000 keeping
the oldest. A member could send junk once (signatures of changes that
never come, changes no signature will vouch for) and fill both for
good. Every honest signature arriving before its change was then lost,
so under signatures required its changes waited for the actor's next
signature: a stall for quiet actors.
Both now keep the newest: the oldest go. A share per sender, so a member
sending junk without pause can't push out what others sent either, is
in future-work.md.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Review 2.2: swap_document forgot every signature. After dropping
changes taken unjudged (or a rejoin into the same epoch), the node
could vouch for none of the changes it kept: under signatures required,
those building on an actor waited for that actor's next signature, and
for an actor that stopped writing, forever.
Signatures::kept_for keeps the signatures of the changes the new
document holds, checks each actor's latest again, and keeps those
waiting for their changes. This node's replayed changes are signed at
once, as before.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review finding 5: vouching by (actor, seq) isn't bound to a hash. A
member can make a change under another member's actor at a seq that
member's signature covers. The change is then taken as vouched, and if
it breaks a rule, the honest member is refused as a whole and logged as
the one whose own change broke the rules.
A signature now vouches for its change and the changes of its actor it
builds on: in the document, as before; among received changes, those
it reaches through the batch's dependencies. A received change under an
actor at a seq whose change the document holds (a signature verified up
to it) is a fork: refused as forged, never as its actor's node's, not
told back in REFUSALS, and logged with the peer that brought it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The API checks the path a request asks for; the rules judge everything
the change does. Where they disagreed, the honest node's change was
refused by every other node, and with signatures required the node was
refused as a whole. The plan had both gates run the same judgement.
Under a policy, each change a node makes (its own writes and those
through the API) is now judged by rules::judge as receivers will judge
it, before it is signed or sent; one every node would refuse is rolled
back (the document forked at the heads before it, same actor: the
change never left) and the request refused with 403 (StoreError::
Refused). rules::judge_held, judging a change the document holds,
moves from swarm::refusals to access::rules for both.
The simulations that play a member past its own checks write with a
helper that bypasses them (forge).
Review finding 3 (the same judgement at both gates).
|
| |
|
|
|
|
|
|
|
| |
A node refusing changes a peer brought tells it which broke a rule; the
peer judges them itself, and if its rules refuse them too, drops them
with all built on them (its document forked without them, a fresh
actor), quarantining its own writes among them. Wire version 42.
Auth plan M6 (rejoin on taint).
|
| |
|
|
|
|
|
|
| |
A refused change is remembered by its hash: offered again it is refused
without decoding, judging or logging, so is one built on it, and a peer
offering nothing else is answered later each time, up to a minute.
Auth plan M6 (re-sending).
|
| |
|
|
|
|
|
|
|
| |
Inspection gives each operation's kind; under strict every node
refuses, at a signed place, anything but an envelope that checks and
replaces the one there (deleting first as replacing does), anything
within one, and above one anything replacing what holds one.
Auth plan M7.
|
| |
|
|
|
|
|
|
|
| |
"values": "signed" (strict, requiring signatures too): the registry,
_continuum/access, bans and the paths in _continuum/access/userSigned
(* for any key) hold values their users signed. Envelopes read as the
values they hold where the node reads protocol data.
Auth plan M7.
|
| |
|
|
|
|
|
|
| |
_continuum/logouts/<name> holds when a user's tokens were last revoked,
written by the user itself or by one who may manage users, so users'
entries can be signed by their managers alone.
Auth plan M7.
|
| |
|
|
|
|
|
|
|
| |
access::envelope: cv1.<payload>.<signature>, the payload naming the
path, value, user, time and the envelope it replaces (prev), signed
with one of the user's SSH keys from the registry (keys). Checked
against the registry as the writer saw it.
Auth plan M7.
|
| |
|
|
| |
future-work brought up to date with M6.
|
| |
|
|
|
|
|
|
| |
_continuum/banned/<node ID>: every node refuses the banned node's
changes from then on (judged as the ban stood) and leaves its
handshakes unanswered.
Auth plan M6.
|
| |
|
|
|
|
|
|
| |
"signatures": "required": a change counts only once a signature by its
committer vouches for it, and waits until then; every node then signs
every change it makes.
Auth plan M6.
|
| |
|
|
|
|
|
|
| |
_continuum/access/policy ({"version": 1}) has every node screen every
change it receives, on every path changes come in. A policy a node
doesn't know stops its writes through the API. GET /admin/refused.
Auth plan M6.
|
| |
|
|
|
|
|
|
|
| |
access::screen: a batch of received changes judged change by change,
those that change what rules read applied before the next is judged;
a refused change stays out with whatever builds on it, kept aside for
operators; changes missing dependencies wait.
Auth plan M6.
|
| |
|
|
|
|
|
|
| |
A remake's replay writes each value as the user who wrote it,
splitting where several users wrote under a path, so replays are
judged as the original writes.
Auth plan M6 (remakes, 2.4).
|
| |
|
|
|
|
|
|
|
|
| |
access::rules::judge: a change judged as its writer saw the document
(at its dependencies): committer a member or registering itself;
members' entries and votes their own; epochs and swarm ID from roots;
a remake's decision by its coordinator; the rest by the registry for
the user the change names.
Auth plan M6.
|
| |
|
|
|
|
|
|
| |
document::inspect: each change's paths read from its operations before
it is applied, the same on every node: an object's place as made, cut
at lists and text; unknown objects at the root.
Auth plan M6.
|
| |
|
|
|
|
|
| |
Finding the paths changes touch, from documents and from raw change
bytes, measured before M6 is built.
Auth plan M6 (measure first).
|
| |
|
|
|
|
|
| |
GET /admin/changes shows whether a signature by its node vouches for
each change.
Auth plan M5.
|
| |
|
|
|
|
|
| |
Sync messages carry, for each actor's writes, the latest signature the
sender checked; the receiver checks it once it holds the change.
Auth plan M5.
|
| |
|
|
|
|
|
| |
A node's signatures are saved with its document (--document-file) and
loaded with it.
Auth plan M5.
|
| |
|
|
|
|
|
|
| |
Changes left unsigned are vouched for by the node's next signature:
--sign-every 1s (default) signs the latest within a second, all signs
every change, none only those that must be.
Auth plan M5.
|
| |
|
|
|
|
|
|
|
|
| |
Every node signs the changes it commits with its node key, an Ed25519
signature over the change's hash, kept beside the document. One
signature on an actor's latest change vouches for its earlier ones.
Protocol data and paths listed in _continuum/access/signed are signed,
and writes with ?signed=true.
Auth plan M5.
|
| |
|
|
|
|
|
| |
Signing and checking change signatures, per change and per actor's
latest change, measured before M5 is built (development.md).
Auth plan M5 (measure first).
|
| |
|
|
|
|
|
|
| |
registry::exists counted any value at _continuum/users as a registry;
keys_at answers None where there is no map, and its users (records,
members, forget records, remakes) say what they make of it.
Auth plan M1 (fix).
|
| |
|
|
|
|
|
|
|
| |
Every request is judged by the registry the node holds: store reads
pruned to what the user may read, writes refused where it may not,
admin endpoints by protocol capabilities. Without a registry, as
before (--protocol-writers for protocol data).
Auth plan M1.
|
| |
|
|
|
|
|
|
|
| |
_continuum/users/<name>: one entry per user, with capabilities by path
(read, write) granted by whom and when, and a disabled flag. Read
leniently: a damaged entry gives less, never more. Protocol data stays
apart: a capability outside _continuum never covers it.
Auth plan M1.
|
| | |
|
| | |
|
| | |
|
| | |
|
