| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
populate's tests are in pkgs/populate/tests, independent of krops.
They run the program, with the sources as JSON from populate's Nix
library, and target strings for the remote targets, leaving the user
and whether the target is local to populate, so they cover its command
line too. Phases expecting populate to refuse its arguments, e.g. a
source name or a target path that can't be used, expect the program's
refusal, with refused, the message, since nothing checks them during
evaluation there.
builder.nix is what builds the scenarios, which krops' tests use as
well, with a function running the phases, and whether that checks the
arguments during evaluation. vm.nix takes a module for both machines,
for what else the scenarios run, and fixtures.nix and smoke.nix are
what both use of the fixtures and the smoke test.
krops' tests are deploy, writeDeploy's arguments to nixos-rebuild, with
the stand-in for nixos-rebuild in the module, and populate, populate's
scenarios that expect refusals, which pkgs.populate makes evaluation
fail with, and those of populate's smoke test, all through
pkgs.populate with target attribute sets as mkTarget returns them.
-A only selects from all of populate's scenarios and krops' own.
|
| |
|
|
|
|
|
| |
The canary manifest records modification times, so changing a canary's
time shows, as rsync does to a hard-linked file it takes for unchanged.
And entries beside the target directories whose names start like the
canaries' count as canaries, so a file created next to one shows, too.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Populate replaces what's at the target paths of sources, removes what
sources don't deliver anymore, and prunes. Nothing of that may reach
beyond the target directory, whatever names sources have and whatever the
target directory contains:
- Source names that aren't file names in the target directory don't
evaluate: the empty name, ., .., .populate, and names with slashes or
control characters. E.g. a git source named "" removes the target
directory, and a file source named .. syncs into its parent.
- Target paths that aren't absolute, that are the root directory, or that
contain .. don't evaluate.
- A symlink at the target path of a source gets replaced, not followed,
and populate says so. rsync-based and flake sources do that already.
git sources replace the link as well, and also a work tree whose .git is
a symlink or a file naming a repository elsewhere, instead of fetching,
resetting, and cleaning there. derivation sources replace a link that
doesn't point into the store, instead of failing.
- Populate refuses to use the sentinel, or a state directory therein, if
it is a symlink, instead of changing and removing what it points to.
- What populate removes recursively gets removed as a link if it is one:
chmod -R follows a link named on the command line.
- Flake and flake-input sources that get patched, get their lock file
rewritten, or are materialized replace a symlink at their target path
like every source does, and say so.
- Nix rewrites lock files in place, which changes other hard links to
them as well, e.g. in a backup or outside the target directory, and
what a lock file that is a symlink points to. Populate replaces a lock
file by a copy of its own before rewriting it.
The target directory itself may be a symlink. Nix refuses path: URLs with
symlinks on the way, so the flakes step resolves the paths it hands to
Nix.
Backups work on a sentinel that is still a file, which becomes a
directory, like for everything else that keeps state.
The tests create canaries outside the target directories: a file and a
directory beside them, and a directory elsewhere with a read-only
directory and a Git work tree with uncommitted changes. The check of
every phase of every scenario fails if a canary changed, or if a store
path that the target directory links to doesn't match its hash anymore.
A new test, safety, points links at the canaries: at the target path of
every source type, inside what sources replace, in place of populate's
state, and among what it removes and prunes. It also covers the names
and target paths that don't evaluate, a target directory that is a
symlink, and a lock file with another hard link.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Several weaknesses let checks pass although the feature under test is
broken:
- Nix restores fetched inputs from its fetcher cache, so evaluating a
delivered flake offline after collecting garbage succeeds even if
populate rooted none of its inputs. nix_eval uses an empty cache
directory.
- Scenarios share target directories and stores, so store paths and roots
of earlier scenarios stand in for missing ones, and uploads of flakes
never run in a passing scenario. Scenarios of the flake, flake-input,
and features tests are isolated now: they start without target
directories and with a collected store on both machines.
- Phases expecting evaluation to fail pass on any evaluation failure,
e.g. a misspelt option. Each such phase has a control now: sources
differing only in what makes evaluation fail, which must evaluate.
Further, populate runs with a time limit, so a hang fails its scenario
instead of the whole test, phases can run steps on the deploying machine
before and after, and evaluating the scenarios fails if the remote
target counts as local for whoever evaluates them.
|
|
|
tests/default.nix provides one VM test per source type, and one for
features spanning sources. Each test runs scenarios from
tests/scenarios.nix, which cover every combination of a source type's
options, deployed to the deploying machine itself, to a remote target as
root, and to a remote target using sudo. A scenario prepares the target
directory, deploys, checks the result, and deploys and checks again.
Scenarios expecting populate to fail check its error message, or check
at evaluation time if populate is expected not to evaluate.
The deploying machine creates fixtures using tests/fixtures.sh and serves
them via HTTP and git daemon, so flakes can be fetched on the target
without further network access. Checks evaluate delivered flakes without
network access to the fixtures after collecting garbage, so they verify
that the delivered store paths are complete and rooted.
|
