1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
|
# Development
## The loop
```sh
cargo check --features mycelium,libp2p,tor,dashboard # does it compile: seconds
cargo quick # every test but the simulations: seconds
cargo sim # the simulations (src/simtest/): under a minute
cargo test --features mycelium,libp2p,tor,dashboard # both
cargo benchmarks # the benchmarks, in release builds
```
`cargo quick`, `cargo sim` and `cargo benchmarks` are aliases ([`.cargo/config.toml`](../.cargo/config.toml)),
with every transport and the dashboard (`dashboard`, `GET /admin/dashboard`;
[`docs/dashboard.md`](dashboard.md)). Each transport is a feature, none of them default, and a build
needs at least one: a transport left out takes its tests and dependencies with it
(`--features libp2p` builds neither `reqwest` and the daemon supervision of Mycelium,
nor arti; among Mycelium's tests is a swarm simulated on its envelope).
The benchmarks are ignored tests named `bench_*`: the document's (writes and sync
messages, `document/bench.rs`), the remakes' replay (`remake/bench.rs`), and what
passing changes on costs over the simulated network (`simtest/bench.rs`: messages and
bytes per write by kind, propagation delays, for a full mesh, a writer behind NAT (in a full mesh, and in a partial one), two
transports, partial meshes of 50 and 100 nodes, lossy links (with delays at random,
and with Mycelium's round trips, each link its own; and Tor's, with some slow circuits), a member away for a while, and
members joining during the writes; several minutes, or one scenario alone
with `BENCH_SCENARIO`, part of its name: `BENCH_SCENARIO=Tor`; several, comma-separated,
at once: `BENCH_SCENARIO=Tor,Mycelium`; 100 writes per workload
unless `BENCH_WRITES` says otherwise (more, to see what grows with them); each
scenario run several times, one run after another, with `BENCH_REPEAT=3`, and a
summary of each workload's mean delay and messages per write, run by run (runs
differ: one can't tell a small change from chance); with
`SIM_DISSEMINATION=plumtree` (or `flood`), every simulated node passes changes on by
PlumTree (or by flooding: `--dissemination`), the benchmark's views of the trees still
being the writers' trees'; with `SIM_TOPOLOGY=hyparview`, every simulated node chooses
its neighbours by HyParView (`--topology`; with flooding unless `SIM_DISSEMINATION`
says otherwise, as `--dissemination` defaults to then); all at once with
`BENCH_PARALLEL=1`, faster on many cores, though sharing them makes delays come out
somewhat higher; and in `bench_bridge`, large values from a writer on a fast transport
through a bridge to a node on a slow one: what that node gets as sync messages and as
blobs against what was written, 200 values unless `BENCH_BRIDGE_WRITES` says
otherwise, over a fast link of 1 GiB/s and 5 ms round trips and a slow one of 50
MiB/s and 500 ms unless `BENCH_BRIDGE_FAST_RATE`, `BENCH_BRIDGE_SLOW_RATE` (MiB/s),
`BENCH_BRIDGE_FAST_RTT` and `BENCH_BRIDGE_SLOW_RTT` (ms) say otherwise), and
the same estimated on the member list's graph alone for up to 2000 members, with some
links down, as now and along the writer's tree (`swarm/topology/tests.rs`,
`bench_writer_trees`). One of
them alone: `cargo test --release --features mycelium,libp2p,tor,dashboard
bench_propagation -- --ignored --nocapture`.
The dashboard's page has browser checks (`src/api/dashboard/check.js`): it loads
without errors, its peers table and map draw what the state says, and a map of 300
members doesn't hold the page up. They need Node.js with Playwright's library and its
Chromium, so they run only when asked; `src/api/dashboard/check.nix` gives all three
from nixpkgs (Playwright's library and browsers of one version, nothing from npm):
```
nix-shell src/api/dashboard/check.nix --run \
'cargo test --features dashboard dashboard_in_a_browser -- --ignored'
```
The simulations send the wire encoding alone, as libp2p and Tor do; a simulated net
can put messages into a transport's envelope instead (`SimNet::with_codec`), and every
message kind is round-tripped through each codec (`wire::samples`).
The dev profile ([`Cargo.toml`](../Cargo.toml)) keeps the loop short: the crate gets line tables only
(backtraces still name files and lines), and the dependencies are built optimised and
without debug info. Automerge's work dominates the tests, so they run several times
faster; the binaries are a quarter of the size, so they link faster. The dependencies
build once (a few minutes, longer on few cores); after that, a change rebuilds in
seconds. To step through a dependency in a debugger, override it locally (`debug =
true` for that package).
Measured on 2 cores, `--features libp2p,tor` (before Mycelium became a feature), after
touching one file, with Cargo's default dev profile and with this one:
| | Default profile | This profile |
|---|---|---|
| `cargo check` | — | 3 s |
| test build | 8 s | 6 s |
| binary build | 13 s | 6 s |
| tests but the simulations | 6 s | 1.4 s |
| all tests | 40 s | 31–35 s |
| test binary | 550–670 MB | 100–155 MB |
## The simulations
The simulations ([`src/simtest/`](../src/simtest/)) run whole nodes on a simulated network
([`transport/sim.rs`](../src/transport/sim.rs)), in real time: tokio's clock isn't paused, because a node's
document work runs on the blocking pool, which a paused clock doesn't wait for. So they
take as long as the protocol's timeouts make them, however fast the machine, and on a
busy machine some fail now and then (a node's timer fires before the work it waits for
is done). One failing alone (`cargo sim <name>`) and passing then is that, not a
regression; one failing again and again is a regression.
A node on several transports is a node on several simulated nets, one per scheme
(`Sim::start_on`, under the identity layer): nodes on different nets can't reach each
other, as one on Tor alone and one on libp2p alone can't, so a swarm whose transports
don't all meet is simulated with the simulated transport alone.
|
