| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A .git directory names places elsewhere in more ways than being a
symlink or a file: core.worktree, symlinks to a configuration, to
objects, or to refs elsewhere, a commondir file naming another
repository, and linked work trees. Git's commands follow them, so
fetching, resetting, setting the remote's URL, and cleaning change what
is elsewhere: with core.worktree, git clean removes everything in the
work tree it names. And Git runs commands the configuration names, like
core.fsmonitor, and hooks.
Populate only uses a work tree whose .git directory has no symlinks, no
commondir file, no linked work trees, and only configuration that git
clone and populate write, where submodules' repositories may have their
work tree in the work tree. Otherwise populating fails with an error
telling what's wrong, unless the new option reclone replaces the work
tree by a fresh clone. Git runs without hooks and without fsmonitor in
any case.
The work tree also gets reset to the commit if HEAD is there already: a
lock file rewritten in place, or created and staged by Nix, stays
otherwise once the source stops drawing inputs.
A safety scenario refuses work trees with core.worktree, a symlinked
configuration, a commondir file, and core.fsmonitor, replaces them with
reclone, and checks that a hook doesn't run. A git scenario stops
drawing inputs and checks that the work tree is clean.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Names and paths with characters that mean something to shells, to find,
or in URLs get handled as what they are, or refused:
- Pruning, and removing state that sources don't need anymore, list
entries with find instead of a loop reading names, which strips blanks
at the ends of names and splits names at newlines. So an entry named
"trail " gets pruned instead of reported as pruned and left.
- Pruning doesn't make entries writable before removing them, which
would change the mode of files elsewhere that are hard links to files
therein.
- Source names are portable file names: letters, digits, and ._-, not
starting with -. nixpkgs' writers accept nothing else in the names of
the scripts populating the sources, which contain the source names, and
fail with an assertion that doesn't mention the source.
- Paths get percent-encoded where they go into URLs for Nix, which
refuses blanks there and takes ? and # for delimiters.
- rsync gets the target path protected from the target's shell
(--protect-args), which expands wildcards in it otherwise: a file source
delivered to /var/src/a?c ends up in /var/src/abc if that exists.
A safety scenario deploys every kind of state to a target path with a
blank, wildcards, #, and %. An edge case prunes entries with blanks at
the ends of their names, with a newline, and with wildcards, and the
scenario pruning links also prunes hard links to a read-only file
outside, and lists what's left in the C locale, where ls sorts .populate
first. The scenario with unusual source names also draws inputs from a
source named self.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
inputs
Delivering flakes and flake inputs gets more robust:
- With fetchOnTarget, the inputs get collected before the pipeline that
links them, so populate fails if the target can't fetch one, instead of
linking the flake without its inputs and succeeding.
- What the lock file pins gets taken from the store if it's there: the
store path is the one of a directory named source with the locked hash,
which nix-store --print-fixed-path computes. builtins.fetchTree
contacts where the input comes from even then, so redeploying unchanged
sources fails without that if a server is unreachable.
- With fetchOnTarget, a delivered input that is itself locked to a local
path the target lacks fails with populate's error naming the input, like
its inputs do.
- Inputs that get drawn from other sources are left out, with all they
pin: Nix archives an empty flake in their place. So nothing gets
fetched, copied, and rooted that no evaluation uses, and a pin that is
unreachable doesn't matter if it gets replaced anyway.
- flake-input sources get the option requireInputs. By default,
populating fails if the inputs that the input's own lock file pins
can't be delivered, e.g. because the lock file lacks an input its
flake.nix has. With false, populate warns and delivers the input with
what Nix locks now, or else without its inputs.
A remote flake that the target fetches entirely on its own collects its
inputs like everything else does. The Nix function finding its
subdirectory moves next to the jq programs.
New edge cases cover each of these.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Sources drawing inputs from other sources get the lock file of their
flake rewritten with nix flake lock --override-input, so the flake in
the source itself uses the other sources, and needs no generated flake
to be found. file and git sources get their lock file rewritten in
place. flake and flake-input sources get delivered as a copy with the
rewritten lock file, which gets added to the store with nix-store
--add-fixed, and the target path links to that. With rewriteLock set
to false, and for symlink sources, the source stays as it is and gets
a generated flake in .populate/flakes/<name>, which writeDeploy's flake
parameter uses.
flake and flake-input sources get two more options:
- patches: patches on the deploying machine, which get uploaded to
.populate/patches/<name> and applied in order with patch -p1 on the
target, in the directory the target path shows.
- materialize: the target path becomes a writable copy instead of a
link into the store.
Sources that get patched, rewritten, or materialized get delivered to
.populate/derived/<name>/original, which also roots the delivered store
path, and the flakes step derives the target path from there. Copies
get made again only if what they derive from changes, which a key in
.populate/derived/<name> records.
The matrices of the file, flake, flake-input, and git tests cover both
ways of drawing inputs, those of flake and flake-input materialize, the
deploy test a source with a rewritten lock file, and features and
edge cases run the scenarios drawing inputs both ways. New features
cover patches, a patch that doesn't apply, patches in a subdirectory,
reusing copies, and materializing.
|
| |
|
|
|
|
|
|
|
|
|
| |
Flake and flake-input sources take the flake reference as attribute url,
like git sources take their repository, and like inputs in flake.nix:
something.flake = "github:owner/something";
otherthing.flake = {
url = "github:owner/otherthing";
inputs.nixpkgs.follows = "nixpkgs";
};
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
rsync 3.5.0 opens merge files, including per-directory .rsync-filter
files, walking the path one directory at a time, refusing symlinks that
neither root nor the user running rsync owns, and opening each directory
for reading. It treats the files it can't open that way as missing,
without saying so. That happens, e.g., below a home directory with mode
711, which the user may enter but not read, or behind another user's
symlink. Such a source then got synced without its .rsync-filter rules:
excluded files got transferred, and --delete-excluded deleted nothing.
Before syncing with rsync 3.5.0 or later a source containing .rsync-filter
files, populate asks rsync which of them it reads: a dry run with
--debug=FILTER2. If it reports an existing one as missing, the file
source type's new insecureLinks option decides: with null, the default,
populate fails with an error naming the file; with true, rsync reads such
files anyway (--insecure-links); with false, rsync syncs without them.
Both print a warning.
File scenarios sync a source through a symlink owned by nobody, with each
setting.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With fetchOnTarget, the target fetches the flake's inputs using the
arguments recorded in the lock file, like it does for flake-input
sources. The flake itself has no lock entry, so the target fetches it
by its locked reference, unless that refers to the deploying machine,
e.g. a path or a Git work tree. Then the flake's tree gets transferred
using rsync and added to the target's store using nix-store --add-fixed,
which yields the same store path as on the deploying machine, because
flake store paths are content-addressed by their NAR hash. Adding paths
like this needs neither signatures nor a trusted user, so this works for
targets using sudo. The transfer gets skipped if the target's store has
the flake already, and populate fails if the added path differs from
the expected one.
rsync' gets a filterFiles option, so the flake's tree gets transferred
without applying .rsync-filter files, which would change it.
|
| |
|
|
|
|
|
| |
The flake-input source type delivers an input of a flake, as recorded in the
flake's lock file, and the flake source type delivers the flake itself.
fetchOnTarget belongs to flake-input, because only a locked input can be
fetched on the target.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
For each source that draws inputs from other sources, populate generates a
flake in .populate/flakes/<name>, which re-exports the outputs of the
source's flake with those inputs drawn from the other sources. The source
itself stays as it is, so NIX_PATH still refers to it.
Generated flakes refer to each other's generated flakes, so a source's own
declarations apply wherever it gets drawn from, and get generated in
dependency order. Evaluation fails for sources drawing inputs from missing
sources, from themselves, or from each other. Generated flakes get locked on
every run, and get removed once their sources no longer draw inputs from
other sources.
|
| |
|
|
|
| |
flakeFollows lists the inputs a source draws from other sources, with the
paths of nested inputs flattened.
|
| |
|
|
|
|
|
|
|
|
| |
With fetchOnTarget, the target fetches the input and its own inputs itself,
using fetchTree with the arguments recorded in the flake's lock file. Only
nix flake metadata runs on the deploying machine, so the inputs get
downloaded once, by the target. This needs Nix with flake support on the
target, which populate checks before fetching.
Because nothing gets copied, this also works for targets using sudo.
|
| |
|
|
|
|
| |
Shell code that enables flakes for the Nix commands following it, and fails
with an error message naming the host and the Nix version found, unless Nix
supports flakes.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Deliver a flake, or one of its locked inputs, to the target: archive the
flake locally, copy the selected input together with its own inputs into the
target's store, and link it at the target path, registered as garbage
collector root. The result can be used in NIX_PATH as well as by flakes
evaluated on the target, without network access.
nix copy runs with a separate ssh wrapper without the control socket, because
Nix waits for its LocalCommand, which may not run over a multiplexed
connection.
|
| | |
|
| |
|
|
|
| |
This is the expected behavior.
The SSH config is also implicitly used for other SSH-related settings.
|
| | |
|
| |
|
|
|
| |
This fixes an issue when trying to deploy using sudo from systems that
don't provide means to determine the real host name.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
- Added a new option for `file` sources, which allows to specify a list
of `include` and `exclude` filters
- Added a new option for `file` sources, which allows to disable the
`--delete-excluded` behaviour.
|
| | |
|
| |
|
|
|
|
|
| |
Let ssh use the user configured in its configuration files when the
target user is set to the empty string.
Closes https://github.com/krebs/krops/issues/2
|
| |
|
|
|
|
|
|
| |
This allows deployment of remote hosts without having ssh access to the
root user. Passwordless sudo is recommended since krops will use
multiple ssh sessions and the password will not be remembered.
Closes https://github.com/krebs/krops/issues/3
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
|
|
https://cgit.krebsco.de/stockholm 877b4104370c1ea9698a449e376e2842d7c372fd
|
