aboutsummaryrefslogtreecommitdiffstats
path: root/docs/federation-tests.md
diff options
context:
space:
mode:
Diffstat (limited to 'docs/federation-tests.md')
-rw-r--r--docs/federation-tests.md38
1 files changed, 22 insertions, 16 deletions
diff --git a/docs/federation-tests.md b/docs/federation-tests.md
index c22f300..eeb90b6 100644
--- a/docs/federation-tests.md
+++ b/docs/federation-tests.md
@@ -124,22 +124,26 @@ and a key set `ssh-keygen` signed with it. A vector that changes is a change to
every node must agree on ([`federation.md`](federation.md),
[Changing the federation](federation.md#changing-the-federation)).
-## The corpus
+## Against released revisions
-What each revision takes, refuses and reads, as fixed bytes: for each revision this
-continuum knows, a file in `src/federation/corpus/` of histories generated at that
-revision (of every kind the properties generate, those of the first two seeds, and of the
-next few hundred those showing what the ones kept don't: of each kind of event, what
-became of one, a version written, a move, a stop; eight at most), each event's bytes with
-what became of it, by kind (taken, refused and why, by kind, waiting, held back, gone with
-a move), and what the store reads after. Together they show a version written, a move to
-a checkpoint, a stop, and a merge event taken where the revision has merge events, else
-refused. A revision's file is written once, as the
-revision is made (`write_the_corpus`, run by hand), and never edited after: every later
-continuum judges it alike (`every_revision_judges_as_its_corpus_says`), so that a change
-to judging that alters an earlier revision's comes as a later revision. Its first line
-names the revision by its ID: a feature renamed fails as a judgment changed does. It holds
-only what its histories exercise.
+A release judges every history under the revisions an earlier release knows as that one
+did, the earlier one's continuum built and asked. Each release that makes a revision is
+tagged `federation-<F>.<R>`, the federation's number and the revision's
+(`federation-1.1`, revision 1 of federation 1); `scripts/federation-check` judges the same
+histories with the continuum the latest such tag names (or one given) and with the tree it
+runs in, and says where they differ. The histories: of each revision this tree knows and
+each kind the properties generate, those of a few seeds (`write_histories`, from a random
+seed it says, `CONTINUUM_TEST_SEED` to rerun them), and the attack tests' crafted ones, as
+each test offered them where nothing but offers made them (`attacks::recording`); those
+whose root the tagged continuum reads as none, of a revision after it, it leaves out. Both
+judge with `continuum-party judge`, whose format no release changes (`federation::judge`):
+what became of each event, by kind and never by a refusal's words, and what the store
+reads after.
+
+So a change to judging that alters what an earlier revision does comes as a later
+revision, and the check, run before a release, says so where one doesn't. It needs the
+tags and a build of the tagged commit, kept between runs; a tree without git's history
+can't run it, nor a release before `judge`.
## Fuzzing and mutants
@@ -154,8 +158,10 @@ only what its histories exercise.
* `cargo quick`: the rules and attacks by example, the properties at small sizes and a
few hundred seeds, the store on Automerge by crafted changes and seeded runs, the
- vectors, the corpus, and the check that every row has its test.
+ vectors, and the check that every row has its test.
* `cargo sim`: the simulations.
+* Before a release: `scripts/federation-check`, against the latest `federation-<F>.<R>`
+ tag ([Against released revisions](#against-released-revisions)).
* Longer runs, by hand or on a schedule: the properties and the store on Automerge with
many more seeds (a count from the environment), fuzzing, mutants.