| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A sync message larger than the threshold for its peer (16 chunks, lower
on a slow link or after a message to it failed whole) goes as a delta:
the peer fetches its changes in chunks, from any node holding them. Its
signatures, each writer's latest with changes in it, went in the
envelope offering it: up to 65536 of them, 6 MiB, in one message, the
size the delta exists to avoid. Nor did an Automerge message count its
signatures (one per writer in a whole document pulled) towards the
threshold.
A delta's signatures are now a blob of their own: the offerer packs and
keeps them (Blobs::keep, the latest 64), and the spec names them by
BLAKE3 hash and size (DeltaSpec::signatures, None where no signature
vouches for its changes). They are served as content (BlobId::Content),
so their hash checks them. The receiver fetches them from the offerer
alone, as another node holding the changes holds signatures of its own,
before the changes, and offers them under the write lock just before
judging the changes, so they wait for them as before. The envelope of a
delta carries none. An Automerge message's signatures count towards its
size: past the threshold, it goes as a delta.
Wire version 49.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Under a policy requiring signatures, a node takes a change only once a signature
vouches for it: one of the change itself, or of a later change of the same writer. A
writer is a node's run, each start, rejoin or drop making a fresh actor, so an epoch
gathers writers for as long as it lasts, and a transfer of its whole history (a joiner
pulling the document, a delta) must carry a signature for each of them, or the joiner
waits for the rest for good.
A message's signatures are now chosen by what it carries:
* changes, up to 256 signed among them: each one's signature;
* more (a whole document pulled): each writer's latest signed change among them, found
by the document's change metadata (`Signatures::for_changes`), vouching for the rest;
* a delta: each writer with changes in it (`get_changes_meta(from)`), its latest
verified signature (`for_delta`).
A message carries up to 65536 (6 MiB), and as many wait at the receiver for their
changes (a delta's come after it). So a transfer leaves writers unvouched only in an
epoch with more than 65536 runs writing since it began; a remake starts the count again.
Wire version 48.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 4, F5: a delta, and a message whose changes held more than 256 signatures (a
whole document pulled, of any real history), carried each actor's latest signature,
the 256 that became latest last. An actor is a node's run, so past 256 runs writing in
an epoch, some actors' changes arrived unvouched at the puller under a policy requiring
signatures, waited, and every resend carried the same 256: a joiner never finished.
A message now carries up to 4096 signatures (`MAX_PER_MESSAGE`, as many as wait for
their changes, `MAX_PENDING`: 384 KiB at most), all of its changes' up to there, each
actor's latest past it and for a delta, and a receiver takes as many (wire version
47). An epoch with more than 4096 runs writing would still leave some unvouched in
such a transfer, which the constant says; sending each actor of the transfer its own
latest would need the actors of a delta's changes, which only rebuilding their
metadata gives.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Review 4, F3: the console writes each message out with `{:#?}`, and `SyncBody` derived
`Debug`, so `Changes` (changes pushed as they are) printed every byte of every change.
Pushes are the small messages, and Automerge deflates no change under 256 bytes: a
short value's path and value went into the console in clear, which any user with any
capability reads (the page's `messages` events, and since 1266 a subscribed node's
`messages/own/{seq}` too), paths it may not read included.
`SyncBody` now writes itself out with sizes for the changes, as `SyncPayload` does
for an Automerge message; heads and a delta's spec stay as they were.
|
| |
|
|
|
|
|
|
|
|
| |
A subscribed node's messages come without being written out (their stream leaves the
text out: up to 8 KiB each, 500 at a time), so the figures box said only when, both
ends, type and size of one. Now the node answers for one message by its number,
`GET /admin/dashboard/messages/own/{seq}` (`{"text": …}`, `404` once it isn't kept),
which pages of other origins may call as they do its streams (`cors::open`), and the
page reads it when the message is shown, with its narrow token, and keeps it on the
message. Where it can't be read, the box says why.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The console said with whom a message went, not how: the message log was noted above
the identity layer (`CountingTransport`), where a message is to or from a node, and the
address it went to (the paths layer picks one, `Paths::send_first`) or came from is
gone. Only the paths layer's own PINGs and PONGs, logged under their addresses, had one.
Now the identity layer notes the node's messages (`MessageLog::set_addressed`), with
the address each went to (`IdentityTransport::send_proven` returns it) or came from;
`CountingTransport` leaves them to it, and still notes them where there is no identity
layer (tests). A console line ends with the transport (the address's scheme, in its
colour on the map, the address in the tooltip), and the figures box shows the address.
A subscribed node's messages carry theirs too, and the map runs every message along
its transport's line, not only the PINGs and PONGs.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The node's keepalives take turns between the two sides (`PeerState::keepalive_due_at`),
but the paths layer pinged every path in use every `probe_every` (the keepalive interval,
30 s at least) whatever the peer did: a PING from the peer over the path didn't count.
So both sides pinged each path every interval, often within a second of each other
(seen on the dashboard: a PING answered, and our own PING right after it).
Now a PING from the peer over a path is noted (`PathState::peer_probe`), and a path is
pinged again (`rested`) an interval after the peer's PING, if that came last; after
ours, an interval and a half while the peer pings too (its PING within two intervals
before ours), so its turn comes first; else an interval. One exchange per interval
goes over a path, as the measurements' "two keepalives per neighbour per interval"
says, and each side measures every other interval; a shorter `--keepalive` measures
more often. A path whose way out broke while the peer's PINGs still come in is found by
our own PINGs half as often; a failed send or unanswered traffic (`suspect`) still
pings at once.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The node sent its messages to the page every 2 s (`messages` events,
this node's and a subscribed node's), and this node's ran on the map
from its counts, which come with the state: the map showed a message 2
to 4 s after it went, the messages of each 2 s spread evenly over the
next.
The message log now tells those waiting of each message logged
(MessageLog::logged_after), and both streams send what was logged a
tenth of a second after the first of it (MESSAGES_GATHERED): a busy
node about ten events a second, a quiet one each message soon after it
went. The page runs messages as their streams bring them, this node's
from its own log too (with the addresses its PINGs went to), as far
apart as they were sent; more than six on a link and way at once are
shared out by kind over the moment, and after a gap only the last 2 s
run. The counts still give the rates and what was delivered.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Signatures sent on their own (SIGNATURES, every second, to each peer
not sent them yet) flooded every link: a signature made after its
change left, or one that came with changes, went to every neighbour,
whether or not it got the change along with its signature elsewhere.
With every change of a node signed as it leaves (1217) and signatures
going where their changes go (1218), none is left for them to carry:
the message, its task and its traffic counter go (wire version 46).
Signatures waiting for their changes are offered as before when sync
messages bring them, and the changes waiting for signatures judged
again with the next batch.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review 2, N7, the care it raised: a joining node's empty document took
as the first epoch's root whatever dependency-less change came first,
from whichever peer it joined through; the first epoch's ID is all
zeros, so nothing pinned it. A remake's epoch has its ID, the snapshot
change's hash, which the rules hold its root to.
* Every node names its epoch's root in HELLO and HELLO_REPLY (the
epoch's ID for a remake's, its genesis change for the first; wire
version 45). A node whose document starts from another root of the
same epoch refuses the handshake, logging it: another history under
the swarm's ID, a forged root or a genesis started again.
* A joining node with an empty document takes as its root only the one
it was given (--genesis-root, new), else the one its peers name, and
none while they name different ones (access::screen::RootPin, kept
in Screening across documents): one peer hands it another root only
if it is all the joiner hears, or never, with --genesis-root. A root
refused is listed with the refused, what builds on it with it.
* The genesis node logs the root (`The swarm's root`); GET
/admin/epochs shows it (`root`).
A genesis started again under a remembered swarm ID used to merge with
the nodes of it still running; they now refuse it (another root), and
the docs say to join them with seeds instead.
The epoch's identity stays as it is (generation 0 all zeros): records
and epoch metadata name it so, and every comparison of epochs with
them.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Review 3.1, part of finding 12: a signed value's payload bound path,
user, time and prev, but not the swarm. A user with the same name and
key in two swarms could have an envelope replayed from one into the
other, wherever the path holds no envelope yet (prev "").
The payload now names the swarm (`swarm`), and a node takes an envelope
only for its own. /auth/whoami says the swarm, and continuum-signed and
the example in api.md sign it. Wire version 44: nodes before this
refuse such envelopes, and those after refuse the old ones.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review finding 7, second part: a record carried into a new epoch whole
(proposal and decision in one change) was taken from any member, for
any remake ID. A member could write a forged record, decision included,
for a fresh ID in a single change.
The snapshot's epoch metadata now names the remake's coordinator, as
the quorum built it. The carried-record exception holds only in a
later epoch, only for the remake that made it, and only from the
coordinator its snapshot names, where there is no record yet. Wire
version 43: the snapshot, and so the epoch ID, changes.
The review suggested carrying the record inside the snapshot instead.
The snapshot can't hold the decision, which names the snapshot's own
hash. So the snapshot vouches for who may carry the record instead.
|
| |
|
|
|
|
|
|
|
| |
A node refusing changes a peer brought tells it which broke a rule; the
peer judges them itself, and if its rules refuse them too, drops them
with all built on them (its document forked without them, a fresh
actor), quarantining its own writes among them. Wire version 42.
Auth plan M6 (rejoin on taint).
|
| |
|
|
|
|
|
| |
Signatures no sync message took go on their own within a second, to
each peer not sent them yet (wire version bumped).
Auth plan M5.
|
| |
|
|
|
|
|
| |
Sync messages carry, for each actor's writes, the latest signature the
sender checked; the receiver checks it once it holds the change.
Auth plan M5.
|
| |
|
|
|
|
|
|
|
| |
The dashboard reads every member's links from the members themselves;
LINKS and LINKS_REPLY, swarm::links and their wire tags go (wire
version bumped). Nodes the browser can't reach are drawn from what the
reachable ones measure.
Auth plan M3.
|
| |
|
|
|
|
|
| |
Each tick's messages of a subscribed node, listed in the console with
this node's own, from its own stream.
Auth plan M3 (live data from every reachable node).
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
A neighbour now answers an announcement only if it lacks some of the
announced heads. Heads that merely differ are no longer enough, since
they do while writes go on. The check reads the document, so it runs
off the message consumer, as the pull check already does.
|
| | |
|
| | |
|
| | |
|
| | |
|
