| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 3, R1: what a node received was vouched for only while its own
document's policy required signatures. A joiner's first sync, and
every node's message bringing `required`, went in by the rules alone:
a malicious first peer could plant changes under other members' actors
at their next seqs, their real ones then failing as duplicates for
good.
The signatures a message carries are always offered now, and the
screen always has them: a change waits for one where the policy at the
heads it is judged at requires it (rules::Reads::required, read once
per heads), whatever the receiving document holds. Changes written
under a policy not requiring signatures wait for none, as on every
other node; the check for forged changes runs always. Changes waiting
for a signature are judged again whenever signatures come.
The test of signatures required now takes the history into an empty
document; a new one takes changes written under a policy not requiring
them without any.
|
| |
|
|
|
|
|
|
| |
Each change taken but not yet applied got its own set of the heads it
is judged at, built in an ordered set and copied into a vector, and
kept so for those building on it. Most build on one change alone and
are judged at its heads: share them instead (reference counted), and
build a set only for a change with several dependencies.
|
| |
|
|
|
|
|
| |
Screening copied every change it took, and every one left waiting for
a signature, bytes and all, from the batch it had ordered and then
dropped. Move them out of the batch instead; only a refused change,
recorded with why, is still copied.
|
| |
|
|
|
|
|
| |
in_order is about a seventh of screening in the profile, most of it
hashing change hashes: each change went into and out of three maps,
and each dependency into two. Look each hash up once, in a map from
hash to place, and keep the rest by place in vectors.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The profile of bench_screening puts screening at about 21% of the
samples, half of it Automerge applying the changes. Of the other half,
reading the policy for every change taken is the largest part (3.4%):
reaches_signed reads it, and what users sign, to tell whether the
change must go in before the next is judged.
Read them once, and again only after the document took the accepted
ones; a change to the policy or to what users sign is relevant, taken
at once. A change touching nothing reaches nothing.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the policy's reads kept, judging a change still costs about one
read (some 3.5 µs), and so does judging without a policy: what is left
is At::new, which asks get_change_by_hash for each head. Automerge
builds the change again from the document's operations to answer, at
a cost growing with its history. Screening asks it again for every
candidate and every dependency, and sync for heads.
Ask by the missing dependencies instead (changes::holds), which look
the hash up: one the document holds is not among them. A change queued
in the document, waiting for its dependencies, isn't either; but then
some are missing, and only then is it asked the slow way. Every check
of whether a document holds a change goes through it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bench_screening shows judging at about 17 µs of a change's 25: some
two and a half reads at its heads (the policy, bans, its node's member
entry, whether there is a registry, the sealed declarations), each
costing as much as working out what the heads cover. Screening judges
most of a batch at the same heads, and those reads are the same for
every change judged there.
Keep them (rules::Reads), by node and by user where they depend on
one, while the heads stay the same; screening keeps one for a batch.
What the heads hold doesn't change as the document takes changes, so
nothing goes stale but by other heads. The benchmark breakdown shows
judging so and afresh.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 2, N7, the care it raised: a joining node's empty document took
as the first epoch's root whatever dependency-less change came first,
from whichever peer it joined through; the first epoch's ID is all
zeros, so nothing pinned it. A remake's epoch has its ID, the snapshot
change's hash, which the rules hold its root to.
* Every node names its epoch's root in HELLO and HELLO_REPLY (the
epoch's ID for a remake's, its genesis change for the first; wire
version 45). A node whose document starts from another root of the
same epoch refuses the handshake, logging it: another history under
the swarm's ID, a forged root or a genesis started again.
* A joining node with an empty document takes as its root only the one
it was given (--genesis-root, new), else the one its peers name, and
none while they name different ones (access::screen::RootPin, kept
in Screening across documents): one peer hands it another root only
if it is all the joiner hears, or never, with --genesis-root. A root
refused is listed with the refused, what builds on it with it.
* The genesis node logs the root (`The swarm's root`); GET
/admin/epochs shows it (`root`).
A genesis started again under a remembered swarm ID used to merge with
the nodes of it still running; they now refuse it (another root), and
the docs say to join them with seeds instead.
The epoch's identity stays as it is (generation 0 all zeros): records
and epoch metadata name it so, and every comparison of epochs with
them.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 2, N7: a node screened received changes only while its own
document had a policy it enforces. A joiner's empty document has none,
so its first sync, as much history as the node it joins through chose,
was applied unjudged, and under signatures required unvouched; the same
for any node while the policy's own change arrived with later ones.
Sync messages, deltas and downloaded changes are now always screened,
each change judged by the policy at its dependencies (1135), so the
verdicts are the swarm's. While neither the document nor the changes
hold a policy (none touches a "policy" key: inspect::touch_key), only
an epoch's root is judged and nothing is placed: bench_screening, after
1188, measured placing everything at about as much again as applying
(564 ms + 543 ms for 100 000 changes), judging by the rules at about
five times (+2.5 s), which a document with a policy pays anyway.
access.md no longer lists the joiner's first sync, nor nodes agreeing
only as the policy reaches them (judged at the dependencies, they
agree), as left to do; changes::load_delta is for tests now.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bench_screening showed screening growing quadratically: 1 000 changes
took 29 ms on top of applying them, 10 000 took 2.8 s, and 100 000
didn't finish in minutes. in_order placed changes in rounds, each
taking those, in hash order, whose dependencies were placed: a chain
received in one delta, its hashes in no order of their own, took about
as many rounds as changes, each over all of them.
It now counts, for each change, its dependencies among those received
not placed yet, and places the smallest hash of those with none left,
then those building on it as their counts reach zero (Kahn's
algorithm): n log n, and still the same order on every node, whatever
order the changes came in. One with a dependency neither in the
document nor among them, and what builds on it, still wait.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 2, N3: 1150 let any member write an abort decision once a remake
had expired a grace period ago, judged by the change's own time, the
writer's word. A member could race the coordinator's legitimate commit
with an abort dated as it liked; both taken, Automerge shows
one, and nodes that acted on the commit split from those that didn't.
The two writers of such aborts are operator requests whose endpoints
already need write on _continuum/remakes: aborting an expired remake
(DELETE /admin/remakes/{id}) and superseding one (POST
/admin/remakes?force=supersede). Both now commit as the user who asks
(the proposal and its vote with them, for a supersede), which the rules
take as one who may write the decision. The clause judging an abort by
its time goes, and with it the time the rules carried. An expired
remake is still aborted through any member, by an operator.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 2, N6: finding 2's class survived for two reads the frontier
didn't cover: whether a map a change makes is there (maps made on the
way, 1137; holders made where none, 1147, for any path), and
_continuum/votes, which `relevant` didn't list. A crafted pair, the
second making a map the first made, was taken by nodes receiving both
in one batch (judged at the frontier, where the map isn't yet) and
refused by nodes receiving them apart.
The screen now keeps the paths the accepted, not applied yet, touch;
a change making a map at one of them is judged after they go in. Votes
join what `relevant` lists. A crafted pair needs hand-made changes,
which the tests don't build.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A node told (REFUSALS) that it holds a change the swarm's rules refuse,
and whose own rules refuse it too, dropped it and everything built on
it on one peer's word, however much that was; its own writes among them
went to the quarantine, the others' nowhere.
* A drop taking more than 100 changes, or more than ten and a tenth of
the document, now
waits until a second member names the same change: the node's rules
and one peer's word could both be off. Logged as an error, counted in
/admin/stats (sync.dropsHeld).
* The others' changes dropped are listed with the refused (GET
/admin/refused, "taken unjudged, dropped"), for operators to see.
The cheap half of asking the others (future-work.md, Telling a
malicious change from a wrong policy).
|
| |
|
|
|
|
|
|
|
|
|
| |
A change waiting for a dependency that hadn't come stayed waiting when
another of its dependencies was refused meanwhile, until the bound
pushed it out. Changes building on one refused before were refused one
generation per batch.
Building on a refused change is now followed through at once, before
judging and again after (refuse_building): whatever waits on a change
refused in the batch, or on one of those, is refused then and there.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Review section 7: Automerge takes the changes judged in a batch all or
none, so one it won't take (a fork of an actor's changes, under a
policy without signatures) fails the whole batch, and every batch
carrying it. The log said who sent the message (`Failed to process sync
message`, `Delta download failed`, with the peer) but not what failed.
The error now names the changes it was given, the first eight, each
with its seq and the node it says made it, so whoever sends the
offending one shows in the log beside what it sent.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Review 3.1, part of finding 12: a signed value's payload bound path,
user, time and prev, but not the swarm. A user with the same name and
key in two swarms could have an envelope replayed from one into the
other, wherever the path holds no envelope yet (prev "").
The payload now names the swarm (`swarm`), and a node takes an envelope
only for its own. /auth/whoami says the swarm, and continuum-signed and
the example in api.md sign it. Wire version 44: nodes before this
refuse such envelopes, and those after refuse the old ones.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review finding 9: once 4096 signatures waited for their changes, new
ones were dropped, and the waiting changes were cut at 10 000 keeping
the oldest. A member could send junk once (signatures of changes that
never come, changes no signature will vouch for) and fill both for
good. Every honest signature arriving before its change was then lost,
so under signatures required its changes waited for the actor's next
signature: a stall for quiet actors.
Both now keep the newest: the oldest go. A share per sender, so a member
sending junk without pause can't push out what others sent either, is
in future-work.md.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Under a policy, a remake's decision was taken from its coordinator only.
A node aborting an expired remake whose coordinator is gone
(DELETE /admin/remakes, after abortableAt) was refused by its own
judgement (403), so such a remake could never be decided.
The decision is now also taken from a user who may write it (the
protocol writers), and an abort from any member once the remake expired
a grace period ago (abortableAt, by the coordinator's grace period in
the proposal). The rules have no clock of their own, so that is judged
by the time the change says it was made: the writer's word, so a member
could abort a remake early, as it could ask its coordinator to, but
never commit one. Before then, an abort stays a request the coordinator
applies.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review finding 5: vouching by (actor, seq) isn't bound to a hash. A
member can make a change under another member's actor at a seq that
member's signature covers. The change is then taken as vouched, and if
it breaks a rule, the honest member is refused as a whole and logged as
the one whose own change broke the rules.
A signature now vouches for its change and the changes of its actor it
builds on: in the document, as before; among received changes, those
it reaches through the batch's dependencies. A received change under an
actor at a seq whose change the document holds (a signature verified up
to it) is a fork: refused as forged, never as its actor's node's, not
told back in REFUSALS, and logged with the peer that brought it.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
A node refusing a member as a whole sent it nothing, and lifted that
only when a change of the member's was taken again. But a member that
dropped what was refused (REFUSALS) starts its sync sessions over, and
a fresh session sends no changes until the other side answers: the two
waited for each other, and the member stayed refused.
The refusal now notes the member's run and epoch_seq when it began; a
member with another document since (it dropped what was refused, or
rejoined) is synced with again.
|
| |
|
|
|
|
|
|
|
|
|
| |
Under a strict policy the rules read user data: the value at a signed
place (prev) and what holds one. A batch judged at its frontier missed
a signed write earlier in the same batch, so two successive signed
writes at one path received together were refused on some nodes and
taken on others. A change at, within or above a signed path now goes
in before the next is judged.
Review finding 2.
|
| |
|
|
|
|
|
|
|
|
| |
The rules read the policy at the change's dependencies only for strict
mode, so history from before the policy was judged by today's rules
when a late node caught up, or when REFUSALS had it judged again. A
change written while the document had no policy, or one this node
doesn't know, is now judged by none.
Review finding 6.
|
| |
|
|
|
|
|
|
|
| |
A node refusing changes a peer brought tells it which broke a rule; the
peer judges them itself, and if its rules refuse them too, drops them
with all built on them (its document forked without them, a fresh
actor), quarantining its own writes among them. Wire version 42.
Auth plan M6 (rejoin on taint).
|
| |
|
|
|
|
|
|
|
| |
With signatures required, a member whose own change broke a rule is
refused as a whole: sent nothing, shown in /peers (refused), in stats
(sync.refusedPeers) and on the dashboard, until a change of its is
taken again or the epoch changes.
Auth plan M6 (diverged standing).
|
| |
|
|
|
|
|
|
| |
A refused change is remembered by its hash: offered again it is refused
without decoding, judging or logging, so is one built on it, and a peer
offering nothing else is answered later each time, up to a minute.
Auth plan M6 (re-sending).
|
| |
|
|
|
|
|
|
|
| |
Inspection gives each operation's kind; under strict every node
refuses, at a signed place, anything but an envelope that checks and
replaces the one there (deleting first as replacing does), anything
within one, and above one anything replacing what holds one.
Auth plan M7.
|
| |
|
|
|
|
|
|
| |
"signatures": "required": a change counts only once a signature by its
committer vouches for it, and waits until then; every node then signs
every change it makes.
Auth plan M6.
|
| |
|
|
|
|
|
|
| |
_continuum/access/policy ({"version": 1}) has every node screen every
change it receives, on every path changes come in. A policy a node
doesn't know stops its writes through the API. GET /admin/refused.
Auth plan M6.
|
|
|
access::screen: a batch of received changes judged change by change,
those that change what rules read applied before the next is judged;
a refused change stays out with whatever builds on it, kept aside for
operators; changes missing dependencies wait.
Auth plan M6.
|
