diff options
| author | tv <tv@krebsco.de> | 2026-11-12 02:07:10 +0000 |
|---|---|---|
| committer | tv <tv@krebsco.de> | 2026-11-12 02:07:10 +0000 |
| commit | 1c7621261c4e9df8d572aef5eb6369d20eea23aa (patch) | |
| tree | 1ffd240897052146a2c1cebe54b699b052dbd237 /pkgs/populate | |
| parent | 3c062a788580a846acb16a5a9ad519bbee2c3747 (diff) | |
tests: isolate flake scenarios, check eval errors against controls
Several weaknesses let checks pass although the feature under test is
broken:
- Nix restores fetched inputs from its fetcher cache, so evaluating a
delivered flake offline after collecting garbage succeeds even if
populate rooted none of its inputs. nix_eval uses an empty cache
directory.
- Scenarios share target directories and stores, so store paths and roots
of earlier scenarios stand in for missing ones, and uploads of flakes
never run in a passing scenario. Scenarios of the flake, flake-input,
and features tests are isolated now: they start without target
directories and with a collected store on both machines.
- Phases expecting evaluation to fail pass on any evaluation failure,
e.g. a misspelt option. Each such phase has a control now: sources
differing only in what makes evaluation fail, which must evaluate.
Further, populate runs with a time limit, so a hang fails its scenario
instead of the whole test, phases can run steps on the deploying machine
before and after, and evaluating the scenarios fails if the remote
target counts as local for whoever evaluates them.
Diffstat (limited to 'pkgs/populate')
0 files changed, 0 insertions, 0 deletions
