diff options
| -rw-r--r-- | CHANGELOG.md | 10 | ||||
| -rw-r--r-- | README.md | 466 | ||||
| -rw-r--r-- | pkgs/populate/README.md | 620 | ||||
| -rw-r--r-- | pkgs/populate/docs/design.md | 93 | ||||
| -rw-r--r-- | pkgs/populate/src/check.rs | 14 |
5 files changed, 668 insertions, 535 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md index 8a59dc4..79c5ca2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,10 +2,12 @@ ## Unreleased -- Populate is a program written in Rust, which reads populate's arguments - as JSON. The store gets one program for each version of krops, instead - of a script for each set of arguments. `pkgs.populate`, `writeDeploy`, - and the other functions keep their arguments. +- Populate is a program of its own, written in Rust, in `pkgs/populate`, + with a command line, a Nix library, a README, and tests of its own. The + store gets one program for each version of krops, instead of a script for + each set of arguments. `pkgs.populate-unwrapped` is the program; + `pkgs.populate`, `writeDeploy`, and the other functions keep their + arguments, and `pkgs.populate` also takes a target string. ## 1.31.1 @@ -58,9 +58,9 @@ and run `$(nix-build --no-out-link krops.nix)` to deploy the target machine. ## Requirements The deploying machine needs Nix and nixpkgs 23.11 or later. krops builds -populate, the program that delivers the sources, with nixpkgs' Rust the -first time it's used, which fetches the program's dependencies from -[crates.io](https://crates.io). +[populate](pkgs/populate/README.md), the program that delivers the +sources, with nixpkgs' Rust the first time it's used, which fetches the +program's dependencies from [crates.io](https://crates.io). The target needs Nix, a POSIX shell, coreutils, findutils, rsync, and `patch` for sources with `patches`, and a remote target ssh access as @@ -197,7 +197,7 @@ build` step. Name of the source containing the flake to deploy, optionally followed by `#` and the name of the NixOS configuration, e.g. `"myconf#myhost"`. If the source draws inputs from other sources through a generated flake, that flake gets -deployed, see [Flake inputs from sources](#flake-inputs-from-sources). This +deployed, see [Flake inputs from sources](pkgs/populate/README.md#flake-inputs-from-sources). This makes `nixos-rebuild` use `--flake`, and requires Nix with flake support (Nix 2.4 or later) on the build target. Without the name of the NixOS configuration, `nixos-rebuild` uses the host name of the build target. @@ -316,461 +316,49 @@ sets `-t` on the ssh command. ## The target directory -Populate owns the target directory, `/var/src` by default: it replaces -whatever is at the target path of a source, and with `prune` it removes -everything else. It doesn't change or remove anything outside: - -* Target paths must be absolute, must not be the root directory, and must not - contain `..`. Source names must consist of letters, digits, and the - characters `._-`, not start with `-`, and be neither `.`, `..`, nor - `.populate`. Anything else fails when evaluating. - -* A symlink at the target path of a source gets replaced by what the source - delivers, not followed, and populate says so. To keep a source elsewhere, - e.g. on another disk or in memory, mount that place at the target path, or - make the target directory itself a symlink. A Git work tree whose `.git` - is a symlink, or a file naming a repository elsewhere, gets replaced by a - `git` source as well. - -* A `git` source only uses a work tree whose `.git` directory contains no - symlinks, no `commondir` file and no linked work trees, and only the - configuration that `git clone` and populate write, and in which each - `.git` of a submodule is a file naming a repository in `.git/modules`. Otherwise Git's - commands could change what's elsewhere, or run commands, so populating - fails, telling what's wrong, unless the source's `reclone` is set. Git - runs no hooks. - -* A file in the target directory that has other hard links, e.g. one in - `/etc/nixos`, gets replaced by a copy of its own before rsync syncs it, so - the other ones keep their mode and time. Populate writes its own files, - and replaces lock files it rewrites, by renaming new files into place, and - only makes directories writable that it removes. - -* Populate keeps its state in the sentinel, `.populate`, which becomes a - directory once there is state to keep. Populating fails if the sentinel, - or a directory populate needs therein, is a symlink. +populate owns the target directory, and doesn't change anything outside it. +[populate's README](pkgs/populate/README.md#the-target-directory) says what +that means. ## Source Types -### `derivation` - -Nix expression to be built at the target machine. - -Supported attributes: - -* `text` - - Nix expression to be built. - - -### `file` - -The file source type transfers local files (and folders) to the target -using [`rsync`](https://rsync.samba.org/). - -Supported attributes: - -* `path` - - absolute path to files that should by transferred. - -* `useChecksum` (optional) - - boolean that controls whether file contents should be checked to decide - whether a file has changed. This is useful when `path` points at files - with mangled timestamps, e.g. the Nix store. - - The default value is `true` if `path` is a derivation, and `false` otherwise. - -* `filters` (optional) - List of filters that should be passed to [`rsync`](https://rsync.samba.org/). - Filters are specified as attribute sets with the attributes `type` and - `pattern`. Supported filter types are `include` and `exclude`. - Checkout the filter rules section in the - [rsync manual](https://download.samba.org/pub/rsync/rsync.html) - for further information. - -* `deleteExcluded` (optional) - boolean that controls whether the excluded directories should be deleted - if they exist on the target. This is passed to the `--delete-excluded` option - of rsync. Defaults to `true`. - -* `insecureLinks` (optional) - - what to do if rsync can't read `.rsync-filter` files that apply to `path`: - the ones therein, and the ones in the directories above it. rsync 3.5.0 - and later open them more strictly than the files they transfer, and ignore - the ones they can't open, e.g. because a symlink on the way belongs to - neither root nor the user deploying, or, with rsync 3.5.0, because a - directory on the way may be entered but not read, like a home directory - with mode 711. By default, with `null`, populating fails with an error - naming the file. With `true`, rsync reads such files anyway - (`--insecure-links`), which also lets it pass such symlinks on the way to - the target path of a local target; with `false`, rsync syncs without them. - Both print a warning. Older versions of rsync aren't affected. - -* `inputs` (optional) - - inputs of the flake in this source to draw from other sources, see - [Flake inputs from sources](#flake-inputs-from-sources). - -* `rewriteLock` (optional) - - boolean that controls whether inputs get drawn from other sources by - rewriting the lock file of the flake in this source, or else through a - generated flake, see [Flake inputs from sources](#flake-inputs-from-sources). - Defaults to `true`. - - -### `flake` - -The flake source type delivers a flake to the target. The flake gets -archived locally using `nix flake archive`, which includes only files tracked -by Git, so use a `file` source for work trees with untracked files. The -flake and its inputs get copied into the target's Nix store using `nix copy`, -and the target path becomes a symlink to the flake's store path, registered -as garbage collector root. For flakes in a subdirectory, e.g. -`git+file:/src/mono?dir=sub`, the symlink points at that subdirectory of the -store path. The inputs get garbage collector roots in -`.populate/flake-inputs/<name>`, which turns the sentinel file into a -directory. - -Supported attributes: - -* `fetchOnTarget` (optional) - - boolean that controls whether the target fetches the flake's inputs itself - instead of receiving them from the deploying machine. The target then uses - the arguments recorded in the lock file, so inputs that refer to local paths - can only be fetched if the target has these paths as well; populating fails - otherwise, naming the input. If the flake reference names a scheme that - doesn't refer to the deploying machine, e.g. `github:`, the target handles - the flake entirely on its own, and the deploying machine doesn't fetch it. - Otherwise the target fetches the flake itself as well if its locked - reference doesn't refer to the deploying machine, e.g. for indirect - references resolving to `github:` flakes, or else the flake gets - transferred using `rsync`, unless the target's Nix store has it already, - and added to the target's Nix store, where it gets the same store path as - on the deploying machine. This requires Nix with flake support (Nix 2.4 or - later) on the target, and works for targets using `sudo`. Defaults to - `false`. - -* `url` - - flake reference, e.g. the absolute path of a directory containing a - `flake.nix`. Its lock file is used as is; populating fails if the lock - file needs to be updated. - -* `inputs` (optional) - - inputs of the flake in this source to draw from other sources, see - [Flake inputs from sources](#flake-inputs-from-sources). - -* `materialize` (optional) - - boolean that controls whether the target path becomes a writable copy - instead of a link into the store. The copy gets made again only if what - it derives from changes: the delivered store path, the patches, or which - inputs get drawn from which sources. Until then, changes made to it on the - target stay. Nix copies it into the store whenever evaluating a flake - changed it, which is noticeable for big flakes like nixpkgs. For a flake - in a subdirectory, only that subdirectory gets copied. Defaults to - `false`. - -* `patches` (optional) - - list of paths of patches on the deploying machine, which get applied in - order with `patch -p1` on the target, in the directory the target path - shows. Populating fails if a patch doesn't apply. Defaults to `[]`. - -* `rewriteLock` (optional) - - boolean that controls whether inputs get drawn from other sources by - rewriting the lock file of the flake in this source, or else through a - generated flake, see [Flake inputs from sources](#flake-inputs-from-sources). - Defaults to `true`. - -Instead of an attribute set, the flake reference can be given as a string. - -The flake commands on the deploying machine run using krops' own Nix, so -flakes don't need to be enabled there. Without `fetchOnTarget`, the target -only needs `nix-store`. Because the copied store paths aren't signed, the -target user must be allowed to import them, i.e. be `root` or a trusted user, -and targets using `sudo` aren't supported. - - -### `flake-input` - -The flake-input source type delivers an input of a flake, as recorded in the -flake's lock file, to the target. This makes the lock file of a flake the -single place that pins, e.g., nixpkgs, for flakes as well as for `NIX_PATH`. -It works like the `flake` source type, but delivers the selected input -instead of the flake, so the target path becomes a symlink to the input's -store path. The inputs that the input's own lock file pins, which evaluating -the input needs, get delivered as well, with garbage collector roots in -`.populate/flake-inputs/<name>`. An input without a lock file comes without -its inputs, because locking it would change it. - -Because the target path points at the store path named by the lock file, it -can be used in `NIX_PATH` as well as by flakes evaluated on the target, -without network access. - -Supported attributes: - -* `url` - - flake reference of the flake whose input gets delivered, e.g. the absolute - path of a directory containing a `flake.nix`. Its lock file is used as is; - populating fails if the lock file needs to be updated. - -* `name` - - name of the input to deliver. Inputs of inputs are separated by `/`, - e.g. `stockholm/nixpkgs`, and resolved as the flake's lock file resolves - them, so an input that follows another one names the other one. - -* `requireInputs` (optional) - - boolean that controls whether populating fails if the inputs that the - input's own lock file pins can't be delivered, e.g. because the lock file - pins something unreachable, or lacks an input its `flake.nix` has. With - `false`, populate prints a warning and delivers the input with the inputs - Nix locks when it adds what the lock file lacks, or else without its - inputs. Evaluating the input as a flake then needs network access. - Defaults to `true`. - -* `fetchOnTarget` (optional) - - boolean that controls whether the target fetches the input itself instead - of receiving it from the deploying machine. The target then uses the - arguments recorded in the lock file, so inputs that refer to local paths - can only be fetched if the target has these paths as well; populating fails - otherwise, naming the input. This requires Nix with flake support (Nix 2.4 - or later) on the target, and works for targets using `sudo`. Defaults to - `false`. - -* `inputs` (optional) - - inputs of the flake in this source to draw from other sources, see - [Flake inputs from sources](#flake-inputs-from-sources). - -* `materialize` (optional) - - boolean that controls whether the target path becomes a writable copy - instead of a link into the store. The copy gets made again only if what - it derives from changes: the delivered store path, the patches, or which - inputs get drawn from which sources. Until then, changes made to it on the - target stay. Nix copies it into the store whenever evaluating a flake - changed it, which is noticeable for big flakes like nixpkgs. For a flake - in a subdirectory, only that subdirectory gets copied. Defaults to - `false`. - -* `patches` (optional) - - list of paths of patches on the deploying machine, which get applied in - order with `patch -p1` on the target, in the directory the target path - shows. Populating fails if a patch doesn't apply. Defaults to `[]`. - -* `rewriteLock` (optional) - - boolean that controls whether inputs get drawn from other sources by - rewriting the lock file of the flake in this source, or else through a - generated flake, see [Flake inputs from sources](#flake-inputs-from-sources). - Defaults to `true`. - -Without `fetchOnTarget`, the same requirements as for the `flake` source type -apply. - -Example: - -```nix -{ - nixpkgs.flake-input = { - url = toString ./.; - name = "nixpkgs"; - }; -} -``` - - -### `git` - -Git sources that will be fetched on the target machine. - -Supported attributes: - -* `url` - - URL of the Git repository that should be fetched. - -* `ref` - - Branch / tag / commit that should be fetched. A branch or tag gets fetched - on every run, so the work tree follows it. A commit, given by its hash or a - prefix thereof of at least four hex digits, only gets fetched if the work - tree lacks it, unless `fetchAlways` is set. So a branch or tag whose name - consists of hex digits only counts as a commit. - -* `clean.exclude` - - List of patterns that should be excluded from Git cleaning. - -* `shallow` (optional) - boolean that controls whether only the requested commit ref. should be fetched - instead of the whole history, to save disk space and bandwith. Defaults to `false`. - -* `reclone` (optional) - - boolean that controls whether a work tree populate can't use, see - [The target directory](#the-target-directory), gets replaced by a fresh - clone, instead of failing. Defaults to `false`. - -* `inputs` (optional) - - inputs of the flake in this source to draw from other sources, see - [Flake inputs from sources](#flake-inputs-from-sources). - -* `rewriteLock` (optional) - - boolean that controls whether inputs get drawn from other sources by - rewriting the lock file of the flake in this source, or else through a - generated flake, see [Flake inputs from sources](#flake-inputs-from-sources). - Defaults to `true`. - - -### `pass` - -The pass source type transfers contents from a local -[password store](https://www.passwordstore.org/) to the target machine. - -Supported attributes: - -* `dir` - - absolute path to the password store. - -* `name` - - sub-directory in the password store. - - -### `passage` - -The passage source type decrypts files from a local -[passage store](https://github.com/FiloSottile/passage) -and transfers them to the target using -[`rsync`](https://rsync.samba.org/). - -Supported attributes: - -* `dir` - - Path to the passage store. - For a partial transfer, this may point to a subdirectory. - Example: `~/.passage/store/hosts/MYHOSTNAME` - -* `identities_file` (optional) - - Path to the identities file. - Defaults to `~/.passage/identities`. - -* `age` (optional) - - Path of the age binary. - Defaults to `age` (absolute path gets resolved using `passage`'s search path.) - - -### `pipe` - -Executes a local command, capture its stdout, and send that as a file to the -target machine. - -Supported attributes: - -* `command` - - The (shell) command to run. - -### `symlink` - -Symlink to create at the target, relative to the target directory. -This can be used to reference files in other sources. - -Supported attributes: - -* `target` - - Content of the symlink. This is typically a relative path. - -* `inputs` (optional) - - inputs of the flake in this source to draw from other sources, see - [Flake inputs from sources](#flake-inputs-from-sources). +`lib.evalSource` turns sources, written in Nix, into what `pkgs.populate` +passes to populate. [populate's +README](pkgs/populate/README.md#source-types) describes the source types: +`derivation`, `file`, `flake`, `flake-input`, `git`, `pass`, `passage`, +`pipe`, and `symlink`. ## Flake inputs from sources -Sources of type `file`, `flake`, `flake-input`, `git`, and `symlink` can -declare which inputs of the flake they contain get drawn from other sources, -in the style of `flake.nix`, where `follows` names a source instead of an -input. This makes flakes on the target use what krops delivers there, much -like `NIX_PATH` does for Nix expressions. - -```nix -{ - nixpkgs.flake-input = { - url = toString ./.; - name = "nixpkgs"; - }; - crate2nix.flake = { - url = "github:nix-community/crate2nix"; - inputs.nixpkgs.follows = "nixpkgs"; - }; - myconf.file = { - path = toString ./.; - inputs = { - crate2nix.follows = "crate2nix"; - stockholm.inputs.nixpkgs.follows = "nixpkgs"; - }; - }; -} -``` - -Populate rewrites the lock file of the flake in each source that draws -inputs from other sources, using `nix flake lock --override-input`, so the -flake in the source itself uses the other sources. `file` and `git` sources -get their lock file rewritten in place. `flake` and `flake-input` sources -get delivered as a copy with the rewritten lock file, which gets added to the -store, unless materialized, so the target path links to that instead of the -delivered store path, as it does for patched ones. A source drawing an input -from a source that draws inputs itself uses that source as rewritten, so in -the example above, myconf's crate2nix uses nixpkgs from the `nixpkgs` source as -well. - -With `rewriteLock = false`, and for `symlink` sources, which point at other -sources, the source stays as it is, and populate generates a flake in -`.populate/flakes/<name>` instead, which re-exports the outputs of the flake -in the source, with those inputs drawn from the other sources. A source -drawing an input from a source with a generated flake uses that flake. -`writeDeploy`'s `flake` parameter uses a source's generated flake +Sources can draw inputs of the flakes they contain from other sources, as +[populate's README](pkgs/populate/README.md#flake-inputs-from-sources) +describes. `writeDeploy`'s `flake` uses a source's generated flake automatically. -Sources in a Git work tree are referred to as `git+file:` (with submodules if -there are any), all other sources as `path:`, with symlinks resolved. Nix -runs Git in a `git+file:` flake, with its configuration and hooks, so -that's only for work trees whose `.git` directory a `git` source would use, -see [The target directory](#the-target-directory), and has no hooks. Lock -files get rewritten or generated on every run, so they match the sources. -Copies of `flake` and `flake-input` sources get made again only if the -delivered store path, the patches, or the sources they draw inputs from -change. Inputs that get drawn from other sources don't get delivered with -a `flake` or `flake-input` source, and neither does what they pin. -Generated flakes and copies get removed once their sources no longer need -them. This requires Nix with flake support (Nix 2.4 or later) on the -target. Populating fails if a source that draws inputs from other sources -contains no `flake.nix`, or if its flake has no such input, before anything -changes. Sources that inputs get drawn from don't need to be flakes, as long -as the inputs drawing from them are declared with `flake = false`. - ## Tests -`tests/` contains NixOS VM tests for populate: one test per source type, one -for features spanning sources, such as flake inputs from sources, one for edge -cases of flakes, such as unusual lock files, one for staying inside the target -directory, whatever links it contains, and one for the arguments `writeDeploy` -passes to `nixos-rebuild`. After every deploy, all tests check that nothing -outside the target directory changed. The tests deploy combinations of a -source type's options to the deploying machine itself, to a remote target as -`root`, and to a remote target using `sudo`, and check the result after -deploying and after deploying again. They cover every combination that a code -path tells apart; the exhaustive tests add the remaining ones, e.g. drawing -inputs combined with each of rsync's options. Patches get covered by the -features test. The scenarios whose results depend on the version of rsync also -run with rsync 3.4.1 and 3.5.0, which `tests/rsync.nix` builds, whatever -version nixpkgs has. The tests require KVM and nixpkgs with -`pkgs.testers.runNixOSTest` (23.05 or later), and use `<nixpkgs>` by default. +`tests/` contains NixOS VM tests for krops, using the VM machinery and the +scenarios of [populate's tests](pkgs/populate/README.md#tests): `deploy`, +for the arguments `writeDeploy` passes to `nixos-rebuild`, which a stand-in +records, and `populate`, for `pkgs.populate`, running populate's scenarios +that expect populate to refuse its arguments, which `pkgs.populate` does +during evaluation, and those of populate's smoke test. The tests require +KVM and nixpkgs with `pkgs.testers.runNixOSTest` (23.05 or later), and use +`<nixpkgs>` by default. ```sh nix-build tests # all tests, except the exhaustive ones -nix-build tests -A flake # the test for one source type +nix-build tests -A deploy # the test of writeDeploy nix-build tests -A exhaustive # all exhaustive tests -nix-build tests -A exhaustive.git # the remaining combinations of one test -nix-build tests -A rsync-3-5-0 # the scenarios depending on rsync's version -nix-build tests -A smoke # a few scenarios of each kind, in one VM +nix-build tests -A smoke # a few scenarios of each, in one VM nix-build tests -A only --argstr only 'lock-hooks-.*' # scenarios by name ``` -Populate is a program in Rust, in `pkgs/populate`, which -[`pkgs/populate/docs/design.md`](pkgs/populate/docs/design.md) describes. -`cargo test` in that directory runs its unit tests, without VMs. +`-A only` selects from all of populate's scenarios and krops' own, all +running through `pkgs.populate`. ## References diff --git a/pkgs/populate/README.md b/pkgs/populate/README.md new file mode 100644 index 0000000..f96ef49 --- /dev/null +++ b/pkgs/populate/README.md @@ -0,0 +1,620 @@ +# populate + +populate fills a target directory, `/var/src` by default, on the machine it +runs on or on another one via ssh, with sources: directories and files, +Git work trees, secrets from [pass](https://www.passwordstore.org/) or +[passage](https://github.com/FiloSottile/passage), the output of commands, +Nix derivations, flakes, and inputs of flakes, which can draw their own +inputs from other sources. Running it again brings the target directory up +to date with the sources. It doesn't change anything outside the target +directory. + +[krops](../../README.md) deploys NixOS systems with it. + +## Building + +```sh +nix-build # in this directory, which builds result/bin/populate +cargo build # for development, with Rust 1.71 or later +``` + +`default.nix` takes nixpkgs' `rustPlatform`, from `<nixpkgs>` by default, +so `callPackage` builds it as well. It needs nixpkgs 23.11 or later, and +fetches the program's dependencies from [crates.io](https://crates.io). + + +## Requirements + +populate runs rsync, ssh, Git, Nix, pass, and passage as the sources need +them, found in `PATH`, as well as a POSIX shell, coreutils, and findutils. +The target needs a POSIX shell, coreutils, findutils, rsync, `patch` for +sources with `patches`, Git for `git` sources, and Nix, and, if remote, +ssh access as root, or, with `--sudo`, as a user who may use `sudo`. +NixOS has all of them but Git, and rsync as long as +`environment.defaultPackages` contains it. Features that say so need Nix +with flake support (Nix 2.4 or later) on the machine that uses them. + + +## Command line + +``` +populate [OPTION]... SOURCES [TARGET] +``` + +populate delivers the sources in the file `SOURCES`, or standard input if +`SOURCES` is `-`, to `TARGET`, which is `[USER@][HOST][:PORT][PATH]`. The +user ends at the first `@`, the host at a `:` or a `/`, and the port at a +`/`. The user defaults to `$LOGNAME`, the host to the name of the machine +populate runs on, from `$HOSTNAME` or `/etc/hostname`, and the path to +`/var/src`. A target is local if its user is `$LOGNAME`, and its host is +`localhost` or the name of the machine; populate reaches other targets via +ssh, as their user. + +| Option | Meaning | +|-------------------|---------------------------------------------------------| +| `--backup` | Copy the target directory to a backup first. | +| `--force` | Create the target directory's sentinel if it's missing. | +| `--prune` | Remove what isn't a source from the target directory. | +| `--sudo` | Deliver as root, via sudo on the target. | +| `--ssh-arg ARG` | Pass `ARG` to ssh; repeatable. | +| `--user USER` | The target's user; empty for ssh's default. | +| `--host HOST` | The target's host, e.g. one `TARGET` can't express. | +| `--port PORT` | The port ssh connects to. | +| `--path PATH` | The target directory. | +| `--local` | The target is the machine populate runs on. | +| `--remote` | The target isn't, even if it looks so. | +| `-h`, `--help` | Print the usage and exit. | +| `-V`, `--version` | Print populate's version and exit. | + +`--user`, `--host`, `--port`, and `--path` take precedence over `TARGET`'s +parts. Options take their argument as the next argument or after `=`, and +may come before, between, or after `SOURCES` and `TARGET`, up to `--`. + +Without the sentinel, `.populate` in the target directory, populate refuses +to touch the target directory, unless `--force` creates it; see +[The target directory](#the-target-directory). + +Exit status: + +- 0: populate delivered all sources. +- 1: populate refused the sources or the target, or delivering failed, + with a message of its own. +- The exit status of the command that failed, if delivering failed with + that: e.g. 255 if ssh can't reach the target, or 128 if `git fetch` + fails. +- 2: the command line is wrong, or the sources can't be read, aren't JSON, + or aren't sources. +- 129, 130, 143: populate got SIGHUP, SIGINT, or SIGTERM, and stopped. + + +## Sources + +The sources are a JSON object, of sources by name. Each source is an +object with the field `type`, the source's type, and a field named after +the type, holding its settings, which [Source types](#source-types) +describes: + +```json +{ + "nixpkgs": { + "type": "git", + "git": { "url": "https://github.com/NixOS/nixpkgs", "ref": "nixos-26.05" } + }, + "config": { "type": "file", "file": { "path": "/home/me/config" } }, + "nixos-config": { + "type": "symlink", + "symlink": { "target": "config/configuration.nix" } + } +} +``` +```sh +populate --force sources.json root@server +``` + +Settings with defaults may be left out. Fields and settings populate +doesn't know are errors, so a misspelt one doesn't go unnoticed. The +sources carry no version: this is version 0, and a later version of +populate that reads them differently will mark them. + +### Sources in Nix + +`lib` is populate's Nix library: the types of sources, which describe how +to write them in Nix, with shorthands, e.g. `config.file = "/home/me/config"`, +and `evalSource`, which turns sources written so, an attribute set or a list +of them, into the sources populate reads. It takes nixpkgs' lib, from +`<nixpkgs/lib>` by default. + +```sh +nix-instantiate --eval --strict --json -E ' + (import ./lib {}).evalSource { + nixpkgs.git = { + url = "https://github.com/NixOS/nixpkgs"; + ref = "nixos-26.05"; + }; + config.file = "/home/me/config"; + nixos-config.symlink = "config/configuration.nix"; + } +' | populate --force - root@server +``` + +Store paths in the sources, e.g. of a `file` source that is a derivation, +must exist when populate runs. `nix-instantiate --eval` doesn't build +them; building a file that contains the sources, e.g. with `pkgs.writeText +"sources.json" (builtins.toJSON sources)`, builds them too. + + +## The target directory + +Populate owns the target directory, `/var/src` by default: it replaces +whatever is at the target path of a source, and with `prune` it removes +everything else. It doesn't change or remove anything outside: + +* Target paths must be absolute, must not be the root directory, and must not + contain `..`. Source names must consist of letters, digits, and the + characters `._-`, not start with `-`, and be neither `.`, `..`, nor + `.populate`. populate refuses anything else, and so does krops' + `pkgs.populate` during evaluation. + +* A symlink at the target path of a source gets replaced by what the source + delivers, not followed, and populate says so. To keep a source elsewhere, + e.g. on another disk or in memory, mount that place at the target path, or + make the target directory itself a symlink. A Git work tree whose `.git` + is a symlink, or a file naming a repository elsewhere, gets replaced by a + `git` source as well. + +* A `git` source only uses a work tree whose `.git` directory contains no + symlinks, no `commondir` file and no linked work trees, and only the + configuration that `git clone` and populate write, and in which each + `.git` of a submodule is a file naming a repository in `.git/modules`. Otherwise Git's + commands could change what's elsewhere, or run commands, so populating + fails, telling what's wrong, unless the source's `reclone` is set. Git + runs no hooks. + +* A file in the target directory that has other hard links, e.g. one in + `/etc/nixos`, gets replaced by a copy of its own before rsync syncs it, so + the other ones keep their mode and time. Populate writes its own files, + and replaces lock files it rewrites, by renaming new files into place, and + only makes directories writable that it removes. + +* Populate keeps its state in the sentinel, `.populate`, which becomes a + directory once there is state to keep. Populating fails if the sentinel, + or a directory populate needs therein, is a symlink. + + +## Source Types + +### `derivation` + +Nix expression to be built at the target machine. + +Supported attributes: + +* `text` - + Nix expression to be built. + + +### `file` + +The file source type transfers local files (and folders) to the target +using [`rsync`](https://rsync.samba.org/). + +Supported attributes: + +* `path` - + absolute path to files that should by transferred. + +* `useChecksum` (optional) - + boolean that controls whether file contents should be checked to decide + whether a file has changed. This is useful when `path` points at files + with mangled timestamps, e.g. the Nix store. + + The default value is `true` if `path` is a derivation, and `false` otherwise. + +* `filters` (optional) + List of filters that should be passed to [`rsync`](https://rsync.samba.org/). + Filters are specified as attribute sets with the attributes `type` and + `pattern`. Supported filter types are `include` and `exclude`. + Checkout the filter rules section in the + [rsync manual](https://download.samba.org/pub/rsync/rsync.html) + for further information. + +* `deleteExcluded` (optional) + boolean that controls whether the excluded directories should be deleted + if they exist on the target. This is passed to the `--delete-excluded` option + of rsync. Defaults to `true`. + +* `insecureLinks` (optional) - + what to do if rsync can't read `.rsync-filter` files that apply to `path`: + the ones therein, and the ones in the directories above it. rsync 3.5.0 + and later open them more strictly than the files they transfer, and ignore + the ones they can't open, e.g. because a symlink on the way belongs to + neither root nor the user deploying, or, with rsync 3.5.0, because a + directory on the way may be entered but not read, like a home directory + with mode 711. By default, with `null`, populating fails with an error + naming the file. With `true`, rsync reads such files anyway + (`--insecure-links`), which also lets it pass such symlinks on the way to + the target path of a local target; with `false`, rsync syncs without them. + Both print a warning. Older versions of rsync aren't affected. + +* `inputs` (optional) - + inputs of the flake in this source to draw from other sources, see + [Flake inputs from sources](#flake-inputs-from-sources). + +* `rewriteLock` (optional) - + boolean that controls whether inputs get drawn from other sources by + rewriting the lock file of the flake in this source, or else through a + generated flake, see [Flake inputs from sources](#flake-inputs-from-sources). + Defaults to `true`. + + +### `flake` + +The flake source type delivers a flake to the target. The flake gets +archived locally using `nix flake archive`, which includes only files tracked +by Git, so use a `file` source for work trees with untracked files. The +flake and its inputs get copied into the target's Nix store using `nix copy`, +and the target path becomes a symlink to the flake's store path, registered +as garbage collector root. For flakes in a subdirectory, e.g. +`git+file:/src/mono?dir=sub`, the symlink points at that subdirectory of the +store path. The inputs get garbage collector roots in +`.populate/flake-inputs/<name>`, which turns the sentinel file into a +directory. + +Supported attributes: + +* `fetchOnTarget` (optional) - + boolean that controls whether the target fetches the flake's inputs itself + instead of receiving them from the deploying machine. The target then uses + the arguments recorded in the lock file, so inputs that refer to local paths + can only be fetched if the target has these paths as well; populating fails + otherwise, naming the input. If the flake reference names a scheme that + doesn't refer to the deploying machine, e.g. `github:`, the target handles + the flake entirely on its own, and the deploying machine doesn't fetch it. + Otherwise the target fetches the flake itself as well if its locked + reference doesn't refer to the deploying machine, e.g. for indirect + references resolving to `github:` flakes, or else the flake gets + transferred using `rsync`, unless the target's Nix store has it already, + and added to the target's Nix store, where it gets the same store path as + on the deploying machine. This requires Nix with flake support (Nix 2.4 or + later) on the target, and works for targets using `sudo`. Defaults to + `false`. + +* `url` - + flake reference, e.g. the absolute path of a directory containing a + `flake.nix`. Its lock file is used as is; populating fails if the lock + file needs to be updated. + +* `inputs` (optional) - + inputs of the flake in this source to draw from other sources, see + [Flake inputs from sources](#flake-inputs-from-sources). + +* `materialize` (optional) - + boolean that controls whether the target path becomes a writable copy + instead of a link into the store. The copy gets made again only if what + it derives from changes: the delivered store path, the patches, or which + inputs get drawn from which sources. Until then, changes made to it on the + target stay. Nix copies it into the store whenever evaluating a flake + changed it, which is noticeable for big flakes like nixpkgs. For a flake + in a subdirectory, only that subdirectory gets copied. Defaults to + `false`. + +* `patches` (optional) - + list of paths of patches on the deploying machine, which get applied in + order with `patch -p1` on the target, in the directory the target path + shows. Populating fails if a patch doesn't apply. Defaults to `[]`. + +* `rewriteLock` (optional) - + boolean that controls whether inputs get drawn from other sources by + rewriting the lock file of the flake in this source, or else through a + generated flake, see [Flake inputs from sources](#flake-inputs-from-sources). + Defaults to `true`. + +Instead of an attribute set, the flake reference can be given as a string. + +The flake commands on the deploying machine enable flakes themselves, so +flakes don't need to be enabled there. Without `fetchOnTarget`, the target +only needs `nix-store`. Because the copied store paths aren't signed, the +target user must be allowed to import them, i.e. be `root` or a trusted user, +and targets using `sudo` aren't supported. + + +### `flake-input` + +The flake-input source type delivers an input of a flake, as recorded in the +flake's lock file, to the target. This makes the lock file of a flake the +single place that pins, e.g., nixpkgs, for flakes as well as for `NIX_PATH`. +It works like the `flake` source type, but delivers the selected input +instead of the flake, so the target path becomes a symlink to the input's +store path. The inputs that the input's own lock file pins, which evaluating +the input needs, get delivered as well, with garbage collector roots in +`.populate/flake-inputs/<name>`. An input without a lock file comes without +its inputs, because locking it would change it. + +Because the target path points at the store path named by the lock file, it +can be used in `NIX_PATH` as well as by flakes evaluated on the target, +without network access. + +Supported attributes: + +* `url` - + flake reference of the flake whose input gets delivered, e.g. the absolute + path of a directory containing a `flake.nix`. Its lock file is used as is; + populating fails if the lock file needs to be updated. + +* `name` - + name of the input to deliver. Inputs of inputs are separated by `/`, + e.g. `stockholm/nixpkgs`, and resolved as the flake's lock file resolves + them, so an input that follows another one names the other one. + +* `requireInputs` (optional) - + boolean that controls whether populating fails if the inputs that the + input's own lock file pins can't be delivered, e.g. because the lock file + pins something unreachable, or lacks an input its `flake.nix` has. With + `false`, populate prints a warning and delivers the input with the inputs + Nix locks when it adds what the lock file lacks, or else without its + inputs. Evaluating the input as a flake then needs network access. + Defaults to `true`. + +* `fetchOnTarget` (optional) - + boolean that controls whether the target fetches the input itself instead + of receiving it from the deploying machine. The target then uses the + arguments recorded in the lock file, so inputs that refer to local paths + can only be fetched if the target has these paths as well; populating fails + otherwise, naming the input. This requires Nix with flake support (Nix 2.4 + or later) on the target, and works for targets using `sudo`. Defaults to + `false`. + +* `inputs` (optional) - + inputs of the flake in this source to draw from other sources, see + [Flake inputs from sources](#flake-inputs-from-sources). + +* `materialize` (optional) - + boolean that controls whether the target path becomes a writable copy + instead of a link into the store. The copy gets made again only if what + it derives from changes: the delivered store path, the patches, or which + inputs get drawn from which sources. Until then, changes made to it on the + target stay. Nix copies it into the store whenever evaluating a flake + changed it, which is noticeable for big flakes like nixpkgs. For a flake + in a subdirectory, only that subdirectory gets copied. Defaults to + `false`. + +* `patches` (optional) - + list of paths of patches on the deploying machine, which get applied in + order with `patch -p1` on the target, in the directory the target path + shows. Populating fails if a patch doesn't apply. Defaults to `[]`. + +* `rewriteLock` (optional) - + boolean that controls whether inputs get drawn from other sources by + rewriting the lock file of the flake in this source, or else through a + generated flake, see [Flake inputs from sources](#flake-inputs-from-sources). + Defaults to `true`. + +Without `fetchOnTarget`, the same requirements as for the `flake` source type +apply. + +Example: + +```nix +{ + nixpkgs.flake-input = { + url = toString ./.; + name = "nixpkgs"; + }; +} +``` + + +### `git` + +Git sources that will be fetched on the target machine. + +Supported attributes: + +* `url` - + URL of the Git repository that should be fetched. + +* `ref` - + Branch / tag / commit that should be fetched. A branch or tag gets fetched + on every run, so the work tree follows it. A commit, given by its hash or a + prefix thereof of at least four hex digits, only gets fetched if the work + tree lacks it, unless `fetchAlways` is set. So a branch or tag whose name + consists of hex digits only counts as a commit. + +* `clean.exclude` - + List of patterns that should be excluded from Git cleaning. + +* `shallow` (optional) + boolean that controls whether only the requested commit ref. should be fetched + instead of the whole history, to save disk space and bandwith. Defaults to `false`. + +* `reclone` (optional) - + boolean that controls whether a work tree populate can't use, see + [The target directory](#the-target-directory), gets replaced by a fresh + clone, instead of failing. Defaults to `false`. + +* `inputs` (optional) - + inputs of the flake in this source to draw from other sources, see + [Flake inputs from sources](#flake-inputs-from-sources). + +* `rewriteLock` (optional) - + boolean that controls whether inputs get drawn from other sources by + rewriting the lock file of the flake in this source, or else through a + generated flake, see [Flake inputs from sources](#flake-inputs-from-sources). + Defaults to `true`. + + +### `pass` + +The pass source type transfers contents from a local +[password store](https://www.passwordstore.org/) to the target machine. + +Supported attributes: + +* `dir` - + absolute path to the password store. + +* `name` - + sub-directory in the password store. + + +### `passage` + +The passage source type decrypts files from a local +[passage store](https://github.com/FiloSottile/passage) +and transfers them to the target using +[`rsync`](https://rsync.samba.org/). + +Supported attributes: + +* `dir` - + Path to the passage store. + For a partial transfer, this may point to a subdirectory. + Example: `~/.passage/store/hosts/MYHOSTNAME` + +* `identities_file` (optional) - + Path to the identities file. + Defaults to `~/.passage/identities`. + +* `age` (optional) - + Path of the age binary. + Defaults to `age` (absolute path gets resolved using `passage`'s search path.) + + +### `pipe` + +Executes a local command, capture its stdout, and send that as a file to the +target machine. + +Supported attributes: + +* `command` - + The (shell) command to run. + +### `symlink` + +Symlink to create at the target, relative to the target directory. +This can be used to reference files in other sources. + +Supported attributes: + +* `target` - + Content of the symlink. This is typically a relative path. + +* `inputs` (optional) - + inputs of the flake in this source to draw from other sources, see + [Flake inputs from sources](#flake-inputs-from-sources). + + +## Flake inputs from sources + +Sources of type `file`, `flake`, `flake-input`, `git`, and `symlink` can +declare which inputs of the flake they contain get drawn from other sources, +in the style of `flake.nix`, where `follows` names a source instead of an +input. This makes flakes on the target use what populate delivers there, +much like `NIX_PATH` does for Nix expressions. + +```nix +{ + nixpkgs.flake-input = { + url = toString ./.; + name = "nixpkgs"; + }; + crate2nix.flake = { + url = "github:nix-community/crate2nix"; + inputs.nixpkgs.follows = "nixpkgs"; + }; + myconf.file = { + path = toString ./.; + inputs = { + crate2nix.follows = "crate2nix"; + stockholm.inputs.nixpkgs.follows = "nixpkgs"; + }; + }; +} +``` + +Populate rewrites the lock file of the flake in each source that draws +inputs from other sources, using `nix flake lock --override-input`, so the +flake in the source itself uses the other sources. `file` and `git` sources +get their lock file rewritten in place. `flake` and `flake-input` sources +get delivered as a copy with the rewritten lock file, which gets added to the +store, unless materialized, so the target path links to that instead of the +delivered store path, as it does for patched ones. A source drawing an input +from a source that draws inputs itself uses that source as rewritten, so in +the example above, myconf's crate2nix uses nixpkgs from the `nixpkgs` source as +well. + +With `rewriteLock = false`, and for `symlink` sources, which point at other +sources, the source stays as it is, and populate generates a flake in +`.populate/flakes/<name>` instead, which re-exports the outputs of the flake +in the source, with those inputs drawn from the other sources. A source +drawing an input from a source with a generated flake uses that flake. +krops' `writeDeploy` uses a source's generated flake automatically. + +Sources in a Git work tree are referred to as `git+file:` (with submodules if +there are any), all other sources as `path:`, with symlinks resolved. Nix +runs Git in a `git+file:` flake, with its configuration and hooks, so +that's only for work trees whose `.git` directory a `git` source would use, +see [The target directory](#the-target-directory), and has no hooks. Lock +files get rewritten or generated on every run, so they match the sources. +Copies of `flake` and `flake-input` sources get made again only if the +delivered store path, the patches, or the sources they draw inputs from +change. Inputs that get drawn from other sources don't get delivered with +a `flake` or `flake-input` source, and neither does what they pin. +Generated flakes and copies get removed once their sources no longer need +them. This requires Nix with flake support (Nix 2.4 or later) on the +target. Populating fails if a source that draws inputs from other sources +contains no `flake.nix`, or if its flake has no such input, before anything +changes. Sources that inputs get drawn from don't need to be flakes, as long +as the inputs drawing from them are declared with `flake = false`. + + +## Output + +What populate prints on standard error is part of its interface: people +read it, and tests check it. For example: + +``` +populating 'host:/var/src/nixpkgs'... +pruning 'host:/var/src/old'... +replacing symlink 'host:/var/src/x', which points to '/elsewhere'... +replacing the Git work tree 'host:/var/src/x', because ... +``` + +A target path in a message starts with `user@host:` for remote targets, +without `user@` if the user is root or not set, and is a plain path for the +local target. Errors start with `error:` or `krops:`, warnings with +`warning:`. rsync's itemized changes appear as `%i %n%L %'''b/%'''l`, with +sizes in units of 1024. + + +## Tests + +`tests/` contains NixOS VM tests: one test per source type, one for +features spanning sources, such as flake inputs from sources, one for edge +cases of flakes, such as unusual lock files, and one for staying inside the +target directory, whatever links it contains. After every run, all tests +check that nothing outside the target directory changed. The tests deliver +combinations of a source type's settings to the machine populate runs on, +to a remote target as `root`, and to a remote target using `sudo`, and check +the result after delivering and after delivering again. They cover every +combination that a code path tells apart; the exhaustive tests add the +remaining ones, e.g. drawing inputs combined with each of rsync's options. +The scenarios whose results depend on the version of rsync also run with +rsync 3.4.1 and 3.5.0, which `tests/rsync.nix` builds, whatever version +nixpkgs has. The tests require KVM and nixpkgs with +`pkgs.testers.runNixOSTest` (23.05 or later), and use `<nixpkgs>` by +default. + +```sh +nix-build tests # all tests, except the exhaustive ones +nix-build tests -A flake # the test for one source type +nix-build tests -A exhaustive # all exhaustive tests +nix-build tests -A exhaustive.git # the remaining combinations of one test +nix-build tests -A rsync-3-5-0 # the scenarios depending on rsync's version +nix-build tests -A smoke # a few scenarios of each kind, in one VM +nix-build tests -A only --argstr only 'lock-hooks-.*' # scenarios by name +cargo test # the unit tests, without VMs +``` + +[`docs/design.md`](docs/design.md) describes how populate works. diff --git a/pkgs/populate/docs/design.md b/pkgs/populate/docs/design.md index 940ab5f..10e6f4b 100644 --- a/pkgs/populate/docs/design.md +++ b/pkgs/populate/docs/design.md @@ -36,74 +36,13 @@ tools populate runs in front of `PATH`, and depends on the store paths in the sources, so building it builds them. krops' `writeDeploy` and its other functions use `pkgs.populate`, so their arguments stay as they are. -## Invocation +## Command line and sources -``` -populate [OPTION]... SOURCES [TARGET] -``` - -`SOURCES` is the path of a file containing the sources, or `-` for -standard input. `TARGET` is `[USER@][HOST][:PORT][PATH]`, parsed like -krops' lib's `mkTarget` parses it: the user up to the first `@`, the host -up to a `:` or a `/`, the port up to a `/`, and the path from there. What -it leaves out gets filled in when the program runs, as `mkTarget` would: -the user from `LOGNAME`, the host with the machine's name, and the path -with `/var/src`. An empty user means ssh's default. - -| Option | Meaning | -|-----------------|---------------------------------------------------------| -| `--backup` | Copy the target directory to a backup first. | -| `--force` | Create the target directory's sentinel if it's missing. | -| `--prune` | Remove what isn't a source from the target directory. | -| `--sudo` | Deliver as root, via sudo on the target. | -| `--ssh-arg ARG` | Pass `ARG` to ssh; repeatable. | -| `--user USER` | The target's user, instead of `TARGET`'s. | -| `--host HOST` | The target's host, e.g. one `TARGET` can't express. | -| `--port PORT` | The port ssh connects to. | -| `--path PATH` | The target directory. | -| `--local` | The target is the machine populate runs on. | -| `--remote` | The target isn't, even if it looks so. | -| `-h`, `--help` | Print the usage and exit. | -| `-V`, `--version` | Print populate's version and exit. | - -Options take their argument as the next argument or after `=`, and may -come before, between, or after `SOURCES` and `TARGET`, up to `--`. Exit -status: - -- 0: populate delivered all sources. -- 1: a check of the sources or the target failed, or delivering did, with - a message of its own. -- The exit status of the command that failed, if delivering failed with - that, like the shell implementation does: e.g. 255 if ssh can't reach - the target, or 128 if git fetch fails. -- 2: the command line is wrong, or the sources can't be read, aren't JSON, - or aren't sources. -- 129, 130, 143: populate got SIGHUP, SIGINT, or SIGTERM, and stopped. - -## Sources - -The sources are a JSON object, of sources by name. Each source is an -object with the field `type`, the source's type, and a field named after -the type, holding the source's settings: - -```json -{ "nixpkgs": { "type": "git", "git": { "url": "https://...", "ref": "main" } } } -``` - -The types and their settings are those of `lib/types/populate.nix`, as -`evalSource` returns them: with shorthands expanded, e.g. `file = -"/etc/x"` as `{ "path": "/etc/x" }`, and without settings that are null. -Settings with defaults may be left out. The source types are -`derivation`, `file`, `flake`, `flake-input`, `git`, `pass`, `passage`, -`pipe`, and `symlink`. - -Fields and settings populate doesn't know are errors, so a misspelt option -doesn't go unnoticed. The sources carry no version: this is version 0, and -a later version of populate that reads them differently will mark them. - -Store paths in the sources, e.g. of a `file` source that is a derivation, -must exist when the program runs. The script krops' `pkgs.populate` -writes depends on them. Something producing sources at run time has to build them first. +[`../README.md`](../README.md) describes populate's command line, the +sources, and what populate prints, which are its interface. The program +reads the sources into `spec::Sources`, and parses the target with +`spec::TargetParts`, completing it with `run::Origin`, the user running +populate and the machine's name, when it runs. ## Checks @@ -129,24 +68,6 @@ refusing to copy flakes to remote targets using sudo. krops' `checks.nix` does that during evaluation too, where `pkgs.populate` decides whether the target is local. -## Output - -What populate prints on standard error is part of its interface: people -read it, and the tests check it. For example: - -``` -populating 'host:/var/src/nixpkgs'... -pruning 'host:/var/src/old'... -replacing symlink 'host:/var/src/x', which points to '/elsewhere'... -replacing the Git work tree 'host:/var/src/x', because ... -``` - -A target path in a message starts with `user@host:` for remote targets, -without `user@` if the user is root or not set, and is a plain path for -the local target. Errors start with `error:` or `krops:`, warnings with -`warning:`. rsync's itemized changes appear as `%i %n%L %'''b/%'''l`, with sizes in -units of 1024. - ## Safety Populate never changes or removes anything outside the target directory, @@ -190,6 +111,8 @@ Populate runs rsync, Git, ssh, and Nix as programs, found in `PATH`. - `nix-build tests -A smoke` runs the smoke test, a few scenarios covering each source type and each kind of target. - `nix-build tests` runs all scenarios but the exhaustive ones. +- krops' tests run populate's scenarios that expect refusals, and those of + the smoke test, through krops' `pkgs.populate`, with `tests/builder.nix`. ## Status diff --git a/pkgs/populate/src/check.rs b/pkgs/populate/src/check.rs index ebe5dbf..5b73deb 100644 --- a/pkgs/populate/src/check.rs +++ b/pkgs/populate/src/check.rs @@ -1,14 +1,14 @@ -//! Checks of the sources and the target, the same as checks.nix makes during -//! evaluation, so sources from anywhere else get checked as well. Messages -//! are the same. +//! Checks of the sources and the target, the same as krops' checks.nix +//! makes during evaluation, so sources from anywhere else get checked as +//! well. Messages are the same. use crate::spec::{Source, Spec}; use serde_json::{Map, Value}; use std::collections::BTreeMap; -/// The checks in the order checks.nix makes them, the first failing one's -/// message. The target's host and user, which a command line gives, come -/// first. +/// The checks in the order krops' checks.nix makes them, the first failing +/// one's message. The target's host and user, which a command line gives, +/// come first. pub fn spec(spec: &Spec) -> Result<(), String> { target_host(&spec.target.host, spec.target.user.as_deref())?; target_path(&spec.target.path)?; @@ -267,7 +267,7 @@ pub fn inputs(sources: &BTreeMap<String, Source>) -> Result<(), String> { } // Depth-first search, with the path of sources on the way, which repeats // a source if drawing goes round in circles. The message names the - // whole path, the way into the circle included, like checks.nix. + // whole path, the way into the circle included, like krops' checks.nix. fn visit<'a>( name: &'a str, drawn_from: &'a BTreeMap<&str, Vec<String>>, |