The commit metadata `version` key is well established but getting it for
a remote commit is cumbersome since the commit object needs to be
fetched and loaded. Including it in the summary additional metadata
allows a much more convenient view of what each of the remote refs
represents.
`kola` now follows symlinks when archiving an external test's `data/`
dir. So the recursive `data` symlink we have here breaks it.
Let's just move the shared files in its own directory and update the
symlinks.
Support for that file was added previously, but the testing lived in
rpm-ostree only. Let's add it here too.
In the process add a hidden `--lock-finalization` to `ostree admin
deploy` to make testing easier (though it could also be useful to update
managers driving OSTree via the CLI).
In fixing https://github.com/coreos/rpm-ostree/pull/3323
I felt that it was a bit ugly we're installing `/usr/bin/ostree-container`.
It's kind of an implementation detail. We want users to use
`ostree container`.
Let's support values outside of $PATH too.
For example, this also ensures that TAB completion for `ost` expands
to `ostree ` with a space.
Basically due to the glib structured logging rework we lost the
`noreturn` attribute on `g_error()`.
This is fixed in glib as of f97ff20adf
But we might as well just throw an error here.
This updates the test logic for CLI extensions, actually checking
for functional output from the subcommand.
It also cleans up some environmental leftover.
This adds some logic to detect and dispatch unknown subcommands to
extensions available in `$PATH`. Additional commands can be
implemented by adding relevant `ostree-$verb` binaries to the system.
As an example, if a `/usr/bin/ostree-extcommand` extension is provided,
the execution of `ostree extcommand --help` will be dispatched to that
as `ostree-extcommand extcommand --help`.
This reworks the var-mount destructive test in order to properly use
the datadir for the current stateroot instead of a duplicated one.
In turn, it ensures that the resulting `var.mount` after reboot is
correctly pointing to the same location which hosted `/var` on the
previous boot.
https://bugzilla.redhat.com/show_bug.cgi?id=1945274 is an issue where a privileged
kubernetes daemonset is writing a socket into `/etc`. This makes ostree upgrades barf.
Now, they should clearly move it to `/run`. However, one option is for us to
just ignore it instead of erroring out. Some brief investigation shows that
e.g. `git add somesocket` is a silent no-op, which is an argument in favor of ignoring it.
Closes: https://github.com/ostreedev/ostree/issues/2446
The logic for `--selinux-policy` ended up in the `--tree=dir`
path, but there's no reason for that. Fix the imported
labeling with `--tree=tar`. Prep for use with containers.
We had this bug because the previous logic was trying to avoid
duplicating the code for generic `--selinux-policy` and
the case of `--selinux-policy-from-base --tree=dir`.
It's a bit more code, but it's cleaner if we dis-entangle them.
Having to touch a global test counter when adding tests is
a recipe for conflicts between PRs.
The TAP protocol allows *ending* with the expected number of
tests, so the best way to do this is to have an explicit
API like our `tap_ok` which bumps a counter, then end with `tap_end`.
I ported one test as a demo.
This will be helpful for the "ostree native container" work in
https://github.com/ostreedev/ostree-rs-ext/
Basically in order to reuse GPG/signapi verification, we need
to support adding a remote, even though it can't be used via
`ostree pull`. (At least, not until we merge ostree-rs-ext into ostree, but
even then I think the principle stands)
We're waaay overdue for this, it's been the default
in rpm-ostree for years, and solves several important bugs
around not capturing `/etc` while things are running.
Also, `ostree admin upgrade --stage` (should) become idempotent.
Closes: https://github.com/ostreedev/ostree/issues/2389
There are some existing issues around fsck in unprivileged bare mode,
so this test does not really work at the moment. Leaving it as a FIXME
for the moment.
It cannot work to use `--no-xattrs` when SELinux is enabled
because we get a `security.selinux` attribute on created files
regardless. So just skip this test if true.
Also add some `ostree fsck`s in here which helped me debug
this.
We have a bunch of APIs to do GPG verification of a commit,
but that doesn't generalize to signapi. Further, they
require the caller to check the signature status explicitly
which seems like a trap.
This much higher level API works with both GPG and signapi.
The intention is to use this in things that are doing "external
pulls" like the ostree-ext tar import support. There we will
get the commitmeta from the tarball and we want to verify it
at the same time we import the commit.
As pointed out in the original review, `gpg-list-keys` fits better
alongside the existing `gpg-import`.
Changes were done with:
```
git grep -l list-gpg-keys | xargs sed -i 's/list-gpg-keys/gpg-list-keys/'
for src in $(git ls-files '*list-gpg-keys*'); do
dst=${src/list-gpg-keys/gpg-list-keys}
git mv "$src" "$dst"
done
```
Some distributions set this during build in order to have reproducible
builds from the same source code: for example, Debian uses the date
from debian/changelog.
However, some of our tests assume that `ostree commit` will result in
a commit with the current date/time, and SOURCE_DATE_EPOCH breaks that
assumption. Unset it for our build-time tests.
Resolves: https://github.com/ostreedev/ostree/issues/2405
Signed-off-by: Simon McVittie <smcv@collabora.com>
This provides a wrapper for the `ostree_repo_remote_get_gpg_keys`
function to show the GPG keys associated with a remote. This is
particularly useful for validating that GPG key updates have been
applied. Tests are added, which checks the
`ostree_repo_remote_get_gpg_keys` API by extension.
If we fail as a result of `set -x`, It's often not completely obvious
which command failed or how. Use a trap on ERR to show the command that
failed, and its exit status.
Signed-off-by: Simon McVittie <smcv@collabora.com>
[Originally from bubblewrap commits c5c999a7 "tests: test --userns"
and 3e5fe1bf "tests: Better error message if assert_files_equal fails";
separated into this commit by Simon McVittie.]
We struggled for a long time with enablement of our "internal units",
trying to follow the philosophy that units should only be enabled
by explicit preset.
See https://bugzilla.redhat.com/show_bug.cgi?id=1451458
and https://github.com/coreos/rpm-ostree/pull/1482
etc.
And I just saw chat (RH internal on a proprietary system sadly) where
someone hit `ostree-remount.service` not being enabled in CentOS8.
Thinking about this more, I realized we've shipped a systemd generator
for a long time and while its only role until now was to generate `var.mount`,
but by using it to force on our internal units, we don't require
people to deal with presets anymore.
Basically we're inverting things so that "if ostree= is on the kernel
cmdline, then enable our units" and not "enable our units, but have
them use ConditionKernelCmdline=ostree to skip".
Drop the weird gyrations we were doing around `ostree-finalize-staged.path`
too; forking `systemctl start` is just asking for bugs.
So after this, hopefully we won't ever again have to think about
distribution presets and our units.
This will be ignored, so let's make it very clear
people are doing something wrong. Motivated by a bug
in a build pipeline that injected `/var/lib/rpm` into an ostree
commit which ended up crashing rpm-ostree because it was an empty db
which it wasn't expecting.
It *also* turns out rpm-ostree is incorrectly dumping content in the
deployment `/var` today, which is another bug.
The semantics of multiple process locking are covered by
test-concurrency.py, but the semantics of the repository locking from a
single process aren't handled there.
This checks how the repository locking is handled from a single thread
with one OstreeRepo, a single thread with multiple OstreeRepos, and
multiple threads sharing an OstreeRepo.
If there's a locking issue in this test, then it's likely not going to
resolve after a few seconds of serializing access. Lower the default 30
second lock timeout to 5 seconds to prevent the test from hanging
unnecessarily.
This simplifies the lock state management considerably since the
previously pushed type doesn't need to be tracked. Instead, 2 counters
are kept to track how many times each lock type has been pushed. When
the number of exclusive locks drops to 0, the lock transitions back to
shared.
In gnupg 2.3.0[1], if a primary key is expired and a subkey does not
have an expiration or its expiration is older than the primary key, the
subkey's expiration will be reported as the primary's. Previously a
subkey without an expiration would not report one regardless of the
primary key's expiration.
This caused a regression in a test setting an expiration on a primary
key. The test was checking that the subkey was not expired by asserting
that there was no `Key expired` line in the signature verification
output. With gnupg 2.3.0+, it will show as expired, causing the test to
fail.
Remove the assertion since it's not consistent across gnupg versions. In
practice we don't care whether the subkey is considered expired or not
as long as the signature verification fails when the primary key is
expired.
1. https://dev.gnupg.org/T3343Fixes: #2359
Previous to this we'd trip an assertion `abort()` deep in the curl code if e.g.
a user did `ostree remote add foo htttp://...` etc.
Motivated by considering supporting "external remotes" where code outside
ostree does a pull, but we want to reuse the signing verification infrastructure.
Several tests generate summaries and then expect to use the generated
summary immediately. However, this can cause intermittent test failures
when they inadvertantly get a cached summary file. This typically
happens when the test is run on a filesystem that doesn't support user
extended attributes. In that case, the caching code can only use the
last modified time, which only has 1 second granularity. If tests don't
carefully manage the summary modification times or the repo cache then
they are likely subject to races in some test environments.
This introduces an environment variable `OSTREE_SKIP_CACHE` that
prevents the repo from using a cache directory. This is enabled by
default in tests and disabled for tests that are a explicitly trying to
test the caching behavior.
Fixes: #2313Fixes: #2351
If we fail as a result of `set -x`, It's often not completely obvious
which command failed or how. Use a trap on ERR to show the command that
failed, and its exit status.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Ideally in the future we change more of our unit tests to
support running installed; we've tried this in the past with
https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests
I'd like to pick that back up again. This takes a step
towards that by having our Rust tests.
To make this even easier, add a `tests/run-installed`
which runs the installed tests (uninstalled, confusingly
but conveniently for now).
rust-analyzer is happier with this because it understands
the project structure out of the box.
We aren't actually again adding a dependency on Rust/cargo in the core,
this is only used to make `cargo build` work out of the box to build
the Rust test code.
This API is push rather than pull, which makes it much more
suitable to use cases like parsing a tar file from external
code.
Now, we have a large mess in this area internally because
the original file writing code was pull based, but static
deltas hit the same problem of wanting a push API, so I added
this special `OstreeRepoBareContent` just for writing regular
files from a push API.
Eventually...I'd like to deprecate the pull based API,
and rework things so that for regular files the push API
is the default, and then `write_content_object()` would
be split up into archive/bare cases.
In this world the `ostree_repo_write_content()` API would
then need to hackily bridge pull to push and it'd be
less efficient.
Anyways for now due to this bifurcation, this API only
works on non-archive repositories, but that's fine for
now because that's what I want for the `ostree-ext-container`
bits.
Continuation of the addition of `ostree_repo_write_regfile_inline()`.
This will be helpful for ostree-rs-ext and importing from tar, it's
quite inefficient and awkward for small files to end up creating
a whole `GInputStream` and `GFileInfo` and etc. for small files.
When working on ostree-ext and importing from tar, it's
quite inefficient and awkward for small files to end up creating
a whole `GInputStream` and `GFileInfo` and etc. for small files.
Plus the gtk-rs binding API to map from `impl Read` to Gio
https://docs.rs/gio/0.9.1/gio/struct.ReadInputStream.html
requires that the input stream is `Send` but the Rust `tar` API
isn't.
This is only 1/3 of the problem; we also need similar APIs
to directly create a symlink, and to stream large objects via
a push-based API.
It was added for the collections bits, but we made that stable.
It's now just cruft and we're very unlikely to reuse the infrastructure
again.
Motivated by a unit test failure when running from a tarball:
https://github.com/ostreedev/ostree/issues/2313
Enable support for setting and getting xattrs. Allow modifications
to xattrs only on user.ima xattr.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Co-authored-by: Colin Walters <walters@verbum.org>
I was doing some rpm-ostree work and I wanted to compare two
OSTree commits to see if the kernel has changed. I think
this should be a lot more natural.
Add `ostree commit --bootable` which calls into a new generic
library API `ostree_commit_metadata_for_bootable()` that
discovers the kernel version and injects it as an `ostree.linux`
metadata key. And for extra clarity, add an `ostree.bootable`
key.
It's interesting because the "core" OSTree layer is all about
generic files, but this is adding special APIs around bootable
OSTree commits (as opposed to e.g. flatpak as well as
things like rpm-ostree's pkgcache refs).
Eventually, I'd like to ensure everyone is using this and
hard require this metadata key for the `ostree admin deploy`
flow - mainly to prevent accidents.
Dan Nicholson
Jonathan Lebon
Luca BRUNO
Colin Walters
Simon McVittie
Joseph Marrero
Simon McVittie
Dan Nicholson
Alexander Larsson
Timothée Ravier
Luca Bruno
Stefan Berger
OpenShift Merge Robot