| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The matrices cross every option of a source type with every other one,
also where the code handling one never looks at the other. Scenarios
for such combinations are exhaustive, and run in their own tests, which
nix-build tests doesn't build, but nix-build tests -A exhaustive does:
- file and git: drawing inputs combined with rsync's or Git's options,
since drawing inputs happens after delivery.
- flake: materialize combined with how the flake gets there, except for
flakes in Git work trees and in subdirectories, since materializing
copies what got delivered.
- flake-input: materialize for inputs other than dep.
- deploy: /etc/nixos/flake.nix combined with deploying a flake, since it
only matters without one.
- edge-cases: generated flakes where only delivery matters, and remote
targets where drawing inputs alone matters, which happens on the target
alike for every kind of target.
That's 181 of 707 scenarios.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Several checks would pass with the feature they cover broken, or cover
it only by chance:
- Phases can state what a succeeding populate prints. That covers the
warnings of insecureLinks, the lines pruning prints, the one about a
replaced symlink, and what rsync logs about each item.
- The insecureLinks scenarios take their expectations from the version of
rsync, so with an rsync older than 3.5.0 they pass without the check
ever running. New tests, rsync-3-4-1 and rsync-3-5-0, run them with
these versions, which tests/rsync.nix builds from their release
tarballs, with ACLs and extended attributes, which backups use.
- Pruning has a leftover upload to remove, and a source that stops
having inputs to root leaves no empty directory of roots.
- A full clone has every commit from the start, so the fetch for a commit
it lacks never runs. A git scenario deploys a commit that appears on
the server after cloning.
- The jq test also runs the Nix function that finds a flake's
subdirectory on the target, over both shapes of nix flake metadata's
output.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Populate replaces what's at the target paths of sources, removes what
sources don't deliver anymore, and prunes. Nothing of that may reach
beyond the target directory, whatever names sources have and whatever the
target directory contains:
- Source names that aren't file names in the target directory don't
evaluate: the empty name, ., .., .populate, and names with slashes or
control characters. E.g. a git source named "" removes the target
directory, and a file source named .. syncs into its parent.
- Target paths that aren't absolute, that are the root directory, or that
contain .. don't evaluate.
- A symlink at the target path of a source gets replaced, not followed,
and populate says so. rsync-based and flake sources do that already.
git sources replace the link as well, and also a work tree whose .git is
a symlink or a file naming a repository elsewhere, instead of fetching,
resetting, and cleaning there. derivation sources replace a link that
doesn't point into the store, instead of failing.
- Populate refuses to use the sentinel, or a state directory therein, if
it is a symlink, instead of changing and removing what it points to.
- What populate removes recursively gets removed as a link if it is one:
chmod -R follows a link named on the command line.
- Flake and flake-input sources that get patched, get their lock file
rewritten, or are materialized replace a symlink at their target path
like every source does, and say so.
- Nix rewrites lock files in place, which changes other hard links to
them as well, e.g. in a backup or outside the target directory, and
what a lock file that is a symlink points to. Populate replaces a lock
file by a copy of its own before rewriting it.
The target directory itself may be a symlink. Nix refuses path: URLs with
symlinks on the way, so the flakes step resolves the paths it hands to
Nix.
Backups work on a sentinel that is still a file, which becomes a
directory, like for everything else that keeps state.
The tests create canaries outside the target directories: a file and a
directory beside them, and a directory elsewhere with a read-only
directory and a Git work tree with uncommitted changes. The check of
every phase of every scenario fails if a canary changed, or if a store
path that the target directory links to doesn't match its hash anymore.
A new test, safety, points links at the canaries: at the target path of
every source type, inside what sources replace, in place of populate's
state, and among what it removes and prunes. It also covers the names
and target paths that don't evaluate, a target directory that is a
symlink, and a lock file with another hard link.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
rsync 3.5.0 opens merge files, including per-directory .rsync-filter
files, walking the path one directory at a time, refusing symlinks that
neither root nor the user running rsync owns, and opening each directory
for reading. It treats the files it can't open that way as missing,
without saying so. That happens, e.g., below a home directory with mode
711, which the user may enter but not read, or behind another user's
symlink. Such a source then got synced without its .rsync-filter rules:
excluded files got transferred, and --delete-excluded deleted nothing.
Before syncing with rsync 3.5.0 or later a source containing .rsync-filter
files, populate asks rsync which of them it reads: a dry run with
--debug=FILTER2. If it reports an existing one as missing, the file
source type's new insecureLinks option decides: with null, the default,
populate fails with an error naming the file; with true, rsync reads such
files anyway (--insecure-links); with false, rsync syncs without them.
Both print a warning.
File scenarios sync a source through a symlink owned by nobody, with each
setting.
|
| |
|
|
|
|
|
|
|
| |
Depending on the version of Nix, lock files name subdirectories as
attribute dir or as query parameter dir of the URL. The VM tests run
one version, so only one shape gets covered there. The jq programs for
flakes move to pkgs/populate/flake-jq.nix, and a new test, jq, runs them
without VMs over lock files of both shapes, with follows, removed and
cyclic follows, and inputs given by relative paths.
|
| |
|
|
|
|
|
| |
The git scenarios deployed only the commit a clone checks out anyway, so
populate never fetched, reset, or updated submodules in a full clone, and
fetchAlways made no observable difference. They deploy HEAD's parent
first now.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A new test, edge-cases, covers paths that work but have no scenario yet:
a flake given by an indirect reference, which the target fetches by its
locked reference; a remote flake in a subdirectory; a flake in a
subdirectory drawing inputs from sources; a dirty Git work tree; a flake
whose lock file needs updating; drawing inputs from a source that isn't
a directory; nested names of missing inputs; several sources sharing one
flake; and an input locked to a path that the target has.
The deploying machine gets a registry entry for the indirect reference.
Resolving it fails if the global registry can't be downloaded, which it
can't in the VM, and disabling the global registry with flake-registry
needs flakes, so the deploying machine enables them, like any deploying
machine using indirect references does. The target doesn't, so checks
for flake support on targets still mean something.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
For targets using sudo, writeDeploy runs nixos-rebuild on the target via
sudo, so --use-remote-sudo has no effect there. It only matters when
nixos-rebuild runs on a separate build target and activates the system
on the target, so writeDeploy passes it only then.
A new VM test, deploy, runs writeDeploy with a stand-in for
nixos-rebuild, which records the user it runs as, NIX_PATH, and its
arguments, for deploying flakes and other sources, with and without a
separate build target, to every kind of target.
|
|
|
tests/default.nix provides one VM test per source type, and one for
features spanning sources. Each test runs scenarios from
tests/scenarios.nix, which cover every combination of a source type's
options, deployed to the deploying machine itself, to a remote target as
root, and to a remote target using sudo. A scenario prepares the target
directory, deploys, checks the result, and deploys and checks again.
Scenarios expecting populate to fail check its error message, or check
at evaluation time if populate is expected not to evaluate.
The deploying machine creates fixtures using tests/fixtures.sh and serves
them via HTTP and git daemon, so flakes can be fetched on the target
without further network access. Checks evaluate delivered flakes without
network access to the fixtures after collecting garbage, so they verify
that the delivered store paths are complete and rooted.
|
