aboutsummaryrefslogtreecommitdiffstats
path: root/docs/store.md
Commit message (Collapse)AuthorAgeFilesLines
* docs: the store at a node's API is builttv22 hours1-10/+12
| | | | | The store's design says that the store at a node's API is built: putting events, and showing what counts, what it refused, and its operator's drops.
* docs: members by the fold are builttv23 hours1-10/+12
| | | | | | The store's design says that members by the fold are built: each node of the party its operator names, syncing with members only, its neighbours among them and some among its own party's and those its operator names, and taking no token another node issued.
* docs: what of the store on Automerge is builttv23 hours1-9/+14
| | | | | | | | The store's design says what of it is built: on Automerge, a swarm holding a store, started from its root or joined by its ID, each change judged by what it writes, and the store a node reads from its document, by what is its own, putting again what the document lacks, from a pinned checkpoint where there is one; not yet members by the fold, the store at the API, and remakes, of which such a swarm takes no records or acks.
* docs: the federation's fold is builtfederation-2-stage-2tv24 hours1-9/+10
| | | | | store.md, federation.md and federation-tests.md say what of them is built: beside the store held in memory, the federation as its fold, with its byte formats, and their tests.
* store: a fork's later change ends once its events reach 1 MiBtv26 hours1-4/+5
| | | | | | | | | An event went into the change being filled unless it would take the events there past 1 MiB: whether a change was full hung on the next event. A node taking the root on its operator's word couldn't tell a change cut short, sent first, from the one every node builds, and by then held a change of that writer and number. Events now go into a change until their bytes there reach 1 MiB, and the next event begins the next change: each change is judged whole by itself.
* store: documents parted by a writer's two changes come together at the next ↵tv26 hours1-2/+4
| | | | | | | | remake The text said such documents "remake apart". By its own rules they come together at the next remake: one side decides, and the other rejoins its epoch; both decide, and of the two, the lower ID.
* store: grinding epoch IDs moves the nodes again for each lower onetv26 hours1-1/+1
| | | | | | | The row said nodes rejoin the lower epoch once a generation. Each lower ID found moves them again: of N tries, about ln N are lower than all before, so a billion hashes of a small snapshot make some twenty rejoins of the whole swarm, each node putting again what the new document lacks. Still slow, not wrong.
* store: a node follows no epoch with another policy, and reads nothing closed ↵tv26 hours1-8/+10
| | | | | | | | | in one An epoch's first change, which a rejoining node can't build, holds a snapshot: a made-up one may hold another policy, or what is closed in a swarm holding a store (users, bans, the topology). A node follows no epoch whose policy isn't its swarm's, and reads nothing closed in one it follows.
* store: the fold says how large an event may be, and the document judges by ittv26 hours1-2/+6
| | | | | | | | The document took any bytes that hash to their key, while the fold bounds events (the federation: 256 KiB): one change of any size filled every node until the next remake. How large an event may be is a question of the change alone, so the fold says it (`Fold::LARGEST`), the document refuses a change putting a larger one, and the store takes such bytes as none.
* store: where a pinned checkpoint comes from, and what a node keeps across a ↵tv26 hours1-8/+16
| | | | | | | | | | | | | | | | restart A pin is an ID, and the document holds a checkpoint only at `_continuum/checkpoint`, which only an epoch's first change writes: a swarm that moved once, and never remade, held its checkpoint nowhere, and a node told to join from a pinned one had nothing to start from. It finds it in its document: the document's checkpoint, or one the fold moves to at the history of an event the document holds, built at those heads from what the document holds, as the ID fixes its bytes whoever builds them. The document holds what that takes, as no remake leaves it out. And across a restart, a node reads its store from its document anew, from the document's checkpoint or the one its operator pinned, by its own refusals and drops, which it keeps in its file; it keeps nothing else of the store.
* store: a node follows no epoch whose snapshot holds an earlier checkpointtv26 hours1-8/+12
| | | | | | | | | Two listed nodes advertising a newer epoch make a ready node rejoin it, and a made-up epoch's snapshot may hold the root and nothing else. A node keeping its own checkpoint reads what is above it, but the new document then holds events whose history nobody puts again, and no node joining or restarting from it reaches the checkpoint the swarm holds. A node follows an epoch only where the snapshot's checkpoint is its document's, one it builds from that document, or one its operator pins; a row for the attack table.
* store: one rule for putting events againtv26 hours1-24/+29
| | | | | | | | | | | | | | | | | Events were put again by three rules: of a change continuum refused, every event of it and of what was built on it; after a switch, what a node held beyond the base that the new document lacked; after a rejoin, what it held. Between them an event could go: at a remake whose base came before a checkpoint, switched after nodes moved to it, the events between the base and its heads that the checkpoint didn't keep were in no node's store and left the document, and a store starting at the snapshot's checkpoint could reach the nodes' no more, pinned or not. One rule now: a node puts again every event its document lacks that a store starting at the document's checkpoint would take, but for its own refusals and drops, and that it has: one it holds, one in a change it refused, or one the document it left held. So no remake leaves out what a store starting at the new document's checkpoint takes, and the document holds what a node forgot until a remake whose snapshot holds that checkpoint or a later one.
* store: the snapshot is the store read from the document up to the basetv26 hours1-7/+9
| | | | | | | | | | The snapshot held the store "as the node reads it at the base". A node that moved to a later checkpoint since the base forgot what came before it, and reads the store at the base no more; one that didn't move yet still does: the two built different snapshots from one base, and the checkpoint the swarm held could leave the document with no node able to give it back. The snapshot is the store a node reads from the document up to the base, starting at the document's checkpoint, by its own refusals and drops: every node holding the base builds the same, whenever it moved.
* federation: every event's shape, and the fold's nametv26 hours1-1/+1
| | | | | | | | `keys`, `nodes` and `proposal` events had no shape written, a vote's was written as if it were the event, and the fold had no name, though the genesis change is built byte for byte from the policy naming it. Each field an event may have is written with what it holds, a version's hash as an ID is, and the fold is named `federation`: `{"version": 2, "store": "federation"}`.
* federation: groups that parted come together as parties come in, not by mergestv26 hours1-18/+16
| | | | | | | | | | | | | A merge, a fork at heads of several stores, named every party's keys anew, so that a party wrote its key set again before anything a key of its set signed counted there; and kept each side's values side by side, so that a value a party replaced on one side came back where the other side's write had the lower ID. Letting the other side's parties in, by a version or a fork naming them, who sign again what they keep as a party's client signs again what a move refused, does the same with nothing new. So a fork is at heads of one store: the fold makes its root from that store's state, and the store from its own heads (`Store::fork`). Merges' rows go, those after them move up, and Roads not taken says why, with merging in place.
* docs: the store and the federation say what of them is builttv26 hours1-9/+10
| | | | | store.md, federation.md and federation-tests.md said none of them was built; the store held in memory is, with a fold made for its tests and their properties.
* store: what is above some heads names nothing before themtv27 hours1-8/+7
| | | | | | | | | An event above some heads has each of its dependencies among them or above them, not only them all in its history. The store holds after its checkpoint only events whose dependencies are its heads and the events after them, and so it holds every event above them: one naming an event before them as well is refused by a node that moved, and never taken by one starting from the checkpoint, which never held what it names. Tests show it for an event offered after the move and one taken before it.
* federation: the feature, and the store's foldtv27 hours1-0/+2
| | | | | | | | | | | | `federation`, a feature of its own, builds what `docs/federation.md` and `docs/store.md` design, stage by stage; without it continuum builds and behaves as it does. The aliases build with it, and `docs/development.md` says so. Its first part is the store's: an event's ID, the SHA-256 of its bytes written as 64 lower-case hex digits and read back only so, and the `Fold` trait as `docs/store.md` gives it. The trait reads a checkpoint's parts too, the heads it was built at and the events it keeps, which a node taking a pinned checkpoint needs; the design names it. Nothing uses either yet, but a test.
* docs/store: an epoch ID ground below the honest onetv28 hours1-0/+1
| | | | | | | As any member may decide a compaction, and the lower of two epochs for one is followed, a member can grind a remake's ID until its epoch's is lower, and have every node on the honest epoch rejoin, once a generation. No event is lost, as a node that rejoins puts again what it held; the attack table says so.
* docs/store: the changes every node buildstv28 hours1-11/+19
| | | | | | | | | | | An epoch's first change, and a fork's changes after it, carry no node's signature, so the rule that a change carries one now sets them aside. Their writer, derived from the root or from the remake, is one anyone can compute, so it writes exactly the changes every node builds, and any other change under it is refused: nobody commits a change of that writer and number before the one every node builds, and parts the nodes. The genesis change writes the policy (a map, keys sorted), the swarm's ID (a string) and the root, in that order; a fork's kept events fill its later changes in the order of their IDs, each up to 1 MiB of events.
* docs/store: forks in the fold, and the bytes of genesis changestv28 hours1-12/+26
| | | | | | | | | | The fold makes a fork's root, from the states at heads of one store or several and what makes the fork; a node checks a fork's root by it. The state carries what judging needs of history, as the federation's carries the graph of its events, and the state is the same for the set a node takes, which its own refusals may make depend on order. The root is taken before the fold sees any event. A genesis change's writer is the first 16 bytes of the root's ID, and a fork's kept events follow its genesis change in changes of up to 1 MiB, in the order of their IDs.
* docs/store: checkpoint heads, drops around them, and joiningtv28 hours1-25/+36
| | | | | | | | | | | | A checkpoint's heads are events after the store's checkpoint, none in another's history, and are held by their IDs after the move, so that what builds on them is taken; "above" says so more plainly, and that no head is above its own heads. A node drops only events after its checkpoint, as every later event builds on what the checkpoint keeps, and undoes a move only by a pin. Until the next remake the document holds what a node forgot, and a node away syncs it; after it, a node that can't build the checkpoint pins it. A node joins from a pinned checkpoint, and from the root only a store that never moved: from the root, what came before a checkpoint can be grown anew by what the fold refuses since.
* docs/store: remakes a store needs nothing fromtv28 hours1-32/+36
| | | | | | | | | | | A remake changes no event, so of two epochs decided for one a node follows the lower ID, and a node on the other rejoins it: no operator has to name one. A node follows a decision whichever node coordinated it, as a decision reaches it only through a member. Old epochs are no longer synced: a node that lacks the base rejoins, keeping its own checkpoint and reading what is above its heads, and a later checkpoint it can't build it takes where its operator pins it. After a switch a node puts again only what the new document lacks, after a random wait. An epoch's first change writes the policy too, and the snapshot leaves out events nobody takes.
* docs/store: the document judges structure, the store judges eventstv28 hours1-41/+45
| | | | | | | | | | | | | | A change of a swarm holding a store is judged by what it is alone: it puts new keys under `_continuum/events` whose bytes hash to them, writes its node's own protocol data, carries its node's signature and is one Automerge writes. The store reads the events the document holds and takes them as it would from anywhere, each once it holds its dependencies, judged at its history; the rest stays in the document until the next remake. So a fold's verdict, a node's own refusal or a checkpoint refuses no change, nodes no longer drop and rebuild their documents at a move, and honest nodes hold one document whatever each reads from it. Membership is no longer judged at a change's history; a node keeps its own refusals and drops in a file of its own. Events are put again only where continuum refuses a change on its own, as of a writer's two changes of one number.
* docs/federation-tests: how the store and the federation are testedtv28 hours1-1/+2
| | | | | | | | | | | A plan, the contract the code is built to: the fold and the store tested as pure code, events in and verdicts and states out; a model written plainly from the design, which the store is checked against; every rule by its smallest scenarios; every attack row a test named by its table and number, and a test that every row has one; the promises as properties over generated histories, every order for small ones and seeded ones beyond, cut down before they are printed; the store on Automerge checked against the store in memory; simulations with an adversary node; fixed vectors for every encoding; fuzzing and mutants in longer runs. All of it behind the `federation` feature, by stage.
* docs/store: a remake whose base comes before a checkpointtv28 hours1-7/+11
| | | | | | | | | A node that moved to a checkpoint after a remake's base, and switches to it, would have refused the snapshot for the events it forgot, and stayed apart from the nodes that switched. It keeps its own checkpoint instead, and reads none of what the snapshot holds that its checkpoint forgot, as it reads none of it in the old document; only its own refusals and its operator's drops keep it where it is. A snapshot's checkpoint before its own is one it takes.
* docs/store: merges, forks at heads of several storestv29 hours1-9/+11
| | | | | | | A fork's root may be a checkpoint at heads of several stores, which merges them: it keeps what its fold takes of each. A node holding the history up to all those heads builds it and checks it, and one that doesn't takes it on its operator's word, as with any fork; neither the fork nor the stores it came from take each other's events.
* docs/store: checkpointstv29 hours1-76/+147
| | | | | | | | | | | | | | | A store holds a checkpoint, the state at some heads written down (those heads, the events before them its fold keeps, and what the fold says of the rest), and the events after it. The fold says when a node moves to a later checkpoint: the node builds it itself, forgets what it doesn't keep and every event whose history lacks its heads, and refuses such events from then on, so that it holds what is relevant and nothing else. A checkpoint a node can't build, as it was away too long or joins, it takes only where its operator pins it. A store's root is its first checkpoint, and a fork's root a checkpoint at the heads it forks at, which nodes holding the history up to them check, and others take on their operator's word. On Automerge, a remake's snapshot holds the store as it is at the base, so what a checkpoint forgot leaves the document there.
* docs/store: a store of events, generic over its foldtv29 hours1-0/+306
A set of events that only grows, each named by the SHA-256 of its bytes and naming those it builds on, judged at that history so that every node judges it alike, and read together as one state. What the events are, how one is judged and what they read as is a fold's, a trait the federation is the first to implement. Its first home is a swarm's Automerge document: each event is bytes at `_continuum/events/<ID>`, a change writes events and continuum's protocol data only, and a node puts again every event it doesn't refuse of a change it refuses or drops, so that which change carries an event changes nothing. A compaction's snapshot holds the events at its base, and every event beyond it is put again: remakes, rejoins and a writer's two changes of one number change no event a node holds, and the fold never sees them. It says what is promised, a node's own refusals and drops, forks carrying their root's history, members by the fold, where Automerge sets limits, its attacks, and what its tests check. Nothing of it is built; all of it is to be behind the `federation` feature.