| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
pkgs/populate/README.md documents populate on its own: building it,
what it needs, its command line, the sources as JSON and in Nix with
its Nix library, the target directory, the source types, flake inputs
from sources, what it prints, and its tests. The sections about the
target directory, the source types, and flake inputs from sources move
there from krops' README, which links to them from sections of the same
names, and from the first mention of populate. docs/design.md keeps
how populate works.
krops' README describes krops' own tests, and CHANGELOG what populate
being a program of its own means for krops.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The program takes populate's arguments on its command line, like other
programs: populate [OPTION]... SOURCES [TARGET], with the sources as
JSON in the file SOURCES, or on standard input for -. TARGET has the
syntax of krops' lib's mkTarget, [USER@][HOST][:PORT][PATH], parsed the
same way, and what it leaves out gets filled in when populate runs, as
mkTarget would, from LOGNAME and the machine's name. Options set what
the syntax can't express, or override its parts: --sudo, --ssh-arg,
--user, --host, --port, --path, and --local or --remote for whether the
target is the machine populate runs on, which populate otherwise
decides like isLocalTarget. The sources carry no version.
pkgs.populate passes a target attribute set, as mkTarget returns it,
part by part, with --local or --remote as evaluation decides, so
nothing changes for krops. A target string goes to the program as it
is.
The program also refuses target hosts and users starting with -, which
ssh would take for options, now that a command line can give them.
|
| |
|
|
|
|
|
| |
git sources refused work trees with hooks, but populate's Git doesn't
run hooks, so git-foreign's work tree with a hook is fine to use, as it
expects. Only Nix's Git runs them, in a git+file: flake, so drawing
inputs refers to a work tree with hooks as path: instead.
|
| |
|
|
|
|
|
|
| |
DerivedState::parse reads what derived_state prints, and key_text
builds the text whose hash is a derived copy's key, the same as before,
both with unit tests, instead of decoding fields by position inside
Flakes::derive. Fields from the target that don't end in a null byte
fail, instead of getting dropped.
|
| |
|
|
|
|
|
|
| |
For specs whose target's local is null, the program decides whether the
target is local like krops' lib's isLocalTarget: an empty LOGNAME or
HOSTNAME counts as unset, as with lib.maybeEnv, and /etc/hostname's
labels can be of any length, as with lib.types.label. is_local_with,
pure, has unit tests.
|
| |
|
|
|
|
|
|
|
|
|
| |
The shell implementation runs under set -e, so it exits with the status
of the command that failed, e.g. 255 if ssh can't reach the target,
which wrappers retrying on connection failures rely on, or 128 if git
fetch fails. The program does the same, instead of exiting with 1, and
with 128 plus the signal for a command a signal ended.
A phase's expect.status is the exit status it expects populate to fail
with; git-status checks git fetch's.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The program checks that each source's settings are those
lib/types/populate.nix defines for its type, with values of the kinds
it defines, for specs from elsewhere than default.nix: a misspelt
setting fails, as design.md promises, instead of being ignored, and so
does a setting the program would treat as its default because of its
kind. This subsumes the check of absolute paths.
ln gets a symlink source's target after --, so it can't be taken for an
option either.
|
| |
|
|
|
|
|
|
|
|
| |
Looking for .rsync-filter files that apply to a file source, a failing
find counted as finding none, and only happened with rsync 3.5.0 and
later. Like in the shell implementation, find runs before each
transfer with filter files, and its failure stops populate with find's
message, e.g. for a source that's missing, or contains a directory find
can't read, which would also have switched off the check of
insecureLinks. A signal during find takes effect right after it.
|
| |
|
|
|
|
| |
rsync gives sizes in units of 1000 for two apostrophes, and in units of
1024 for three, which the shell implementation asks for. rsync-units
checks it.
|
| |
|
|
|
|
|
|
|
| |
The program checks that the paths of file, pass, and pipe sources are
absolute, as lib/types/populate.nix demands, for specs from elsewhere,
so none can be taken for an option, e.g. of rsync.
A cycle of sources drawing inputs from each other gets named with the
way into it, like checks.nix names it.
|
| |
|
|
|
|
|
|
| |
A pipe source's command is an absolute path, which a shell runs as a
script if it isn't a program, e.g. lacks #!. The program ran it
directly, which fails for such a command.
pipe-script runs a pipe command without #!.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Writing the lock file of a git+file: flake, Nix runs Git, with the
repository's configuration and hooks, which may run commands. A .git
in a source's directory can come from anywhere, e.g. one planted in a
materialized flake, or one a file source keeps. So drawing inputs
refers to a flake as git+file: only if git_problems finds nothing wrong
with its .git directory, the way git sources check their work trees,
and else as path:.
lock-foreign-git plants a .git in a materialized flake and in a file
source that keeps it, whose configuration names an fsmonitor command.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
git submodule update uses the .git of a submodule as it finds it in the
work tree: a repository of its own, with configuration that runs
commands, or a file naming a repository elsewhere. So git_problems
also requires each .git below the work tree to be a file naming a
repository in .git/modules. And it refuses hooks in .git/hooks, which
populate's Git doesn't run, but Nix's does, writing the lock file of a
flake in a git source.
git-submodule plants a submodule's .git in the work tree: a repository
of its own whose configuration names a filter, and a file naming a
repository elsewhere. The fixture withsub has gitsrc as its submodule
sub.
|
| |
|
|
|
|
|
|
|
|
| |
A git source's url and ref are strings, and git clone, git remote, and
git fetch took ones starting with - for options, e.g. --upload-pack,
which runs a command where the repository is local. They come after
-- now, and so does a pass source's name for git log and pass show.
git-ref-option gives a ref that git fetch would take for
--upload-pack.
|
| |
|
|
|
|
| |
The program needs Rust 1.71, the oldest version its dependencies support,
which Cargo.toml states, so clippy points out what newer versions add.
The lock file is of version 3, which Cargo reads from version 1.53 on.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The program decides how to draw inputs and derive copies of flake and
flake-input sources itself. The target runs fixed snippets, which send
back what it needs to know: where paths resolve to and what's there, the
lock files of flakes as nix flake metadata prints them, and what tells
whether a derived copy needs to be made again. The program checks that
drawn inputs exist in the lock files, computes the keys of derived
copies, and writes generated flakes.
Derived copies get the same keys as with the shell implementation, so
switching implementations doesn't make them again.
|
| |
|
|
|
|
|
|
|
|
|
| |
src/flake_input.rs reads what nix flake metadata and nix flake archive
print as JSON itself: it selects the flake or input, decides what the
target fetches or gets copied, and computes the store paths to root,
which are what selected.jq, selfIsRemote.jq, selfLocked.jq, and the Nix
expressions of the shell implementation do there. On the target, fixed
snippets in sh::flake run Nix, getting values as positional parameters,
and send JSON or store paths back. The jq test's cases are unit tests of
the Rust equivalents.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The program delivers flake and flake-input sources, draws inputs from
sources, and removes state that sources don't need anymore, so it does
everything the shell implementation does, and passes all scenarios.
src/flake.rs builds the same shell code as shell.nix generates for
these, with every value quoted, so both implementations behave alike
while that code moves to Rust piece by piece.
The ssh scripts in the temporary directory run the ssh that was in PATH
when populate started, so one can be ssh in PATH for nix copy. The
other one is for nix copy's ssh:// store, without the control socket.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The program delivers derivation, file, git, pass, passage, pipe, and
symlink sources, makes backups, and prunes, as the shell implementation
does, with the same messages. Commands on the target are fixed shell
snippets in src/sh.rs, which get values as positional parameters, run
directly for local targets and over ssh, sharing one master connection,
for remote ones. rsync's transfers and its check for .rsync-filter files
it can't read are in src/rsync.rs. Given flake or flake-input sources, or
sources drawing inputs from others, the program fails before changing
anything, saying that these aren't implemented yet.
SIGHUP, SIGINT, and SIGTERM make populate stop once the command it waits
for ends, stop the ssh master, remove its temporary directory, and exit
with 128 plus the signal's number.
The spec's target gets local, which rust.nix sets during evaluation, like
the other functions of krops decide, and which the program decides when
it runs if it's null. The spec comes on file descriptor 3, so the
commands populate runs keep standard input, and the tools populate runs
come first in PATH.
|
|
|
pkgs.populate takes implementation: "shell", the default, generates a
script from populate's arguments, and "rust" passes them as JSON to the
program in pkgs/populate, built from the crate there by program.nix.
The program checks the spec as checks.nix does during evaluation, with the
same messages, and reports that delivering sources isn't implemented yet.
rustProgram replaces the program, e.g. with one built some other way.
checks.nix holds the checks of populate's arguments during evaluation.
Both implementations check the names of sources and the target path with
it, and the Rust one's wrapper also checks drawing inputs, which the shell
implementation checks where it draws them.
nix-build tests --argstr implementation rust runs the scenarios with the
Rust implementation. They expect it to refuse copying flakes to targets
using sudo when it runs, not during evaluation, because only then it
decides whether the target is local.
|
