refusals: a drop taking much waits for a second member, and shows

A node told (REFUSALS) that it holds a change the swarm's rules refuse,
and whose own rules refuse it too, dropped it and everything built on
it on one peer's word, however much that was; its own writes among them
went to the quarantine, the others' nowhere.

* A drop taking more than 100 changes, or more than ten and a tenth of
  the document, now
  waits until a second member names the same change: the node's rules
  and one peer's word could both be off. Logged as an error, counted in
  /admin/stats (sync.dropsHeld).
* The others' changes dropped are listed with the refused (GET
  /admin/refused, "taken unjudged, dropped"), for operators to see.

The cheap half of asking the others (future-work.md, Telling a
malicious change from a wrong policy).

2 files changed, 14 insertions, 0 deletions

diff --git a/src/api/stats.rs b/src/api/stats.rs
index 1f0701d..90b4fc2 100644
--- a/src/api/stats.rs
+++ b/src/api/stats.rs
@@ -130,6 +130,10 @@ struct SyncStats {
     /// Peers refused as a whole, their own changes having broken the swarm's rules
     /// (`access::screen`): sent nothing.
     refused_peers: usize,
+    /// Changes peers say the swarm's rules refuse, which this node took unjudged and its
+    /// rules refuse too, whose drop takes much: held for a second member to say so
+    /// (`AppState::drop_taken`).
+    drops_held: usize,
     /// The largest sync message received since start (its Automerge part). Larger
     /// transfers come as deltas (blobs).
     largest_received_bytes: u64,
@@ -378,6 +382,10 @@ pub(crate) async fn collect(state: &AppState, list: &PeerListResponse) -> StatsR
             last_from_neighbour_secs,
             diverged_peers,
             refused_peers,
+            drops_held: {
+                use crate::locks::MutexExt;
+                state.screening.lock_or_recover().drops_held()
+            },
             largest_received_bytes: node.largest_sync_received,
         },
         relay: RelayStats {
diff --git a/src/api/stats/prometheus.rs b/src/api/stats/prometheus.rs
index 41d224a..dd49751 100644
--- a/src/api/stats/prometheus.rs
+++ b/src/api/stats/prometheus.rs
@@ -640,6 +640,12 @@ pub(super) fn render(stats: &StatsResponse, list: &PeerListResponse) -> String {
         "Peers refused as a whole, their own changes having broken the swarm's rules.",
         stats.sync.refused_peers as f64,
     );
+    out.single(
+        "drops_held",
+        "gauge",
+        "Changes peers say the swarm's rules refuse, ours too, whose drop takes much: held for a second member to say so.",
+        stats.sync.drops_held as f64,
+    );
     let relay = &stats.relay;
     out.single(
         "announce_rounds_total",