This has a few fixes, mainly I want to get this in
as prep for fs-verity.
Update submodule: libglnx
```
Alex Kiernan (1):
macros: Add TEMP_FAILURE_RETRY for musl
Alexander Larsson (1):
Add glnx_open_anonymous_tmpfile_full() allowing you to specify the directory
Colin Walters (8):
Merge branch 'shutil-rm-rf-errprefix' into 'master'
Merge branch 'us-temp-failure-retry' into 'master'
Merge branch 'anonymous-tmpfile-dir' into 'master'
Merge branch 'meson-older-compilers' into 'master'
fdio: Add glnx_tmpfile_reopen_rdonly()
Merge branch 'reopen-rdonly' into 'master'
build-sys: Add libglnx-testlib.c to Automake
Merge branch 'testlib-automake' into 'master'
Jonathan Lebon (1):
Merge branch 'uchar' into 'master'
Simon McVittie (5):
missing: Remove unused <uchar.h>
Run the fdio test in its own temporary directory
meson: Define HAVE_DECL_FOO to 0 if foo isn't declared
Make the Meson build work on older compilers
CI: Target a Fedora stable release
Will Thompson (3):
Add meson.build files
Document using this as a Meson subproject
Add GitLab CI
```
Prep for fsverity, where I want to create a new group
`[fsverity]` in the keyfile that has default values. We should
treat the absence of a group the same as absence of a key
in these "with defaults" APIs.
This reverts commit 985a141002.
It turned out that some people have old bootloaders, and hence
get the "no entries" problem. That's much, much much worse
than double entries.
This way it's clearer this bit is only about the CLI entrypoint
also living in `ostree trivial-httpd`, not the underlying
`ostree-trivial-httpd` binary that's separate now.
Delete the automake conditional for this, and make the manpage
conditional use `if USE_LIBSOUP` the same way the C build does.
Suggested-by: Jonathan Lebon <jonathan@jlebon.com>
Same as https://github.com/coreos/rpm-ostree/pull/1923
Quoting that rationale:
> Since we're not using Homu anymore (and Tide instead looks at
> all statuses by default), let's just drop it. This brings down the
> number of statuses on PRs by one more (and so one less context to
> override when needed).
zipl is a bit special in that it parses the BLS config files
directly *but* we need to run the command to update the "boot block".
Hence, we're not generating a separate config file like the other
backends. Instead, extend the bootloader interface with a `post_bls_sync`
method that is run in the same place we swap the `boot/loader` symlink.
We write a "stamp file" in `/boot` that says we need to run this command.
The reason we use stamp file is to prevent the case where the system is
interrupted after BLS file is updated, but before zipl is triggered,
then zipl boot records are not updated.
This opens the door to making things eventually-consistent/reconcilable
by later adding a systemd unit to run `zipl` if we're interrupted via
a systemd unit - I think we should eventually take this approach
everywhere rather than requiring `/boot/loader` to be a symlink.
Author: Colin Walters <walters@verbum.org>
Tested-by: Tuan Hoang <tmhoang@linux.ibm.com>
Co-Authored-By: Tuan Hoang <tmhoang@linux.ibm.com>
More "scan-build doesn't understand GError and our out-param conventions"
AKA "these errors would be impossible with Rust's sum type Result<> approach".
I was trying to fix a clang `scan-build` error that jlebon
ended up tracking down in
https://github.com/ostreedev/ostree/pull/1939/commits/9344de1ce1e8c185e01988277606ba1ed7f9d16b
But in the process of tracing through this I found it
way easier to read as "new style" code, so this also ports the
code.
I added a `g_assert()` in there too to help assert that
`g_key_file_get_value` won't leak in the error path.
Got this error when trying to rebase libostree in RHEL:
```
Error: CLANG_WARNING: [#def1]
libostree-2019.2/src/libostree/ostree-repo-checkout.c:375:21: warning: Access to field 'disable_xattrs' results in a dereference of a null pointer (loaded from variable 'repo')
```
I think what's happening is it sees us effectively testing
`if (repo == NULL)` via the `while (current_repo)`. Let's
tell it we're sure it's non-null right after the loop.
I tried to balance reflecting the reality of who works on libostree
today with keeping some of the existing committers - particularly
committers from multiple organizations.
Part of switching libostree over to OpenShift Prow.
I'd like to add OpenShift's prow to this repository. Let's start
by adding a Dockerfile - it doesn't really do anything besides build.
However...I've lately been thinking about e.g. shipping the ostree tests
as an image, and then e.g. we could test FCOS by running that container
(which would orchestrate the *host's* ostree).
Anyways, not doing that right now but this is a start.
Also this cherry picks the fix from rpm-ostree CI for the sad
Fedora release package brokenness.
Closes: #1906
Approved by: cgwalters
I've seen people confused by this error in the case where
`/boot` isn't mounted or the BLS fragments were deleted, etc.
If you understand ostree deeply it's clear but, let's do
better here and a direct error message for the case where
we can't find `/boot/loader` which is the majority of these.
The other case could happen if e.g. just the BLS fragment
for the booted deployment was deleted; let's reword that
one a bit too.
Closes: #1905
Approved by: rfairley
This skips creating the default stuff in the physical sysroot.
I don't recall why I did that to be honest; it originated with
the first commit of this file. It might not have ever been
necessary.
In any case, it's not necessary now with Fedora CoreOS, so
prune it and let's have a clean `/`.
Keep the old behavior by default though to avoid breaking anyone.
Closes: #1894
Approved by: ajeddeloh
This way projects can dispatch at run-time based on ostree's
build time options, e.g. detect the availability of GPG.
Closes: #1890
Approved by: jlebon
When writing a delta to a file this may not always be recorded
in the filename, and it's useful data.
Ref: https://mail.gnome.org/archives/ostree-list/2019-February/msg00000.html
This also required teaching `show` to accept a file path.
Note...for some reason `test-deltas.sh` breaks when run from
a tty - we get `SIGTTIN` which implies something is reading from
the tty but it wasn't obvious to me what.
Closes: #1823
Approved by: jlebon
We have a `http2=[0|1]` remote config option; let's have the
`--disable-http2` build option define the default for that. This way
it's easy to still enable http2 for testing even if
we have it disabled by default.
Closes: #1798
Approved by: jlebon
This didn't quite work out, and is now always failing because Ansible
changed.
For now we have some OK coverage via the rpm-ostree suite, let's just
drop this and revisit later.
Closes: #1783
Approved by: jlebon
I personally am very opposed to the entire idea of `make distcheck`;
I think source code should canonically be *git* and not tarballs.
See e.g. https://github.com/cgwalters/git-evtag for some
rationale.
But anyways we are uploading classic tarballs since today that
what Debian/Fedora/etc consume sadly, so we need to test it.
We explicitly skip `make distcheck` since we don't want to rerun
the test suite.
Closes: #1766
Approved by: jlebon
See https://github.com/projectatomic/rpm-ostree/issues/1568
Basically for people on e.g. rotational media, the default 90
second timeout can be too small.
We're in a tough situation here, because delaying shutdown
can be problematic too if the user is trying to shut down their
laptop to put in a backpack, etc.
There's potential optimizations here to make; I think we
could pre-copy the kernel/initramfs for example.
I suspect for some people the grub2 os-prober is a factor here too,
if that tries to e.g. inspect attached USB rotational hard drives.
But hopefully we'll get rid of that soon.
Closes: #1755
Approved by: jlebon
I was debugging some rpm-ostree work and saw:
`openat: No such file or directory`
and it wasn't immediately obvious it was stderr from `rofiles-fuse`.
Use the `err` API which is better in many ways; in this case
it automatically prefixes with `argv0`.
Closes: #1747
Approved by: jlebon
Actually testing the patch to add `--force-copy-zerosized` to
rpm-ostree tripped over the fact that it uses `--union-identical`,
and we just hit an assertion failure with that combination.
Fix this by copying over the logic we have for the hardlink case.
Closes: #1753
Approved by: jlebon
In rpm-ostree we've hit a few cases where hardlinking zero-sized
files causes us problems. The most prominent is lock files in
`/usr/etc`, such as `/usr/etc/selinux/semanage.LOCK`. If there
are two zero-sized lock files to grab, but they're hardlinked,
then locking will fail.
Another case here is if one is using ostree inside a container
and don't have access to FUSE (i.e. `rofiles-fuse`), then the
ostree hardlinking can cause files that aren't ordinarily hardlinked
to become so, and mutation of one mutates all. An example where
this is concerning is Python `__init__.py` files.
Now, these lock files should clearly not be in the tree to begin
with, but - we're not gaining a huge amount by hardlinking these
files either, so let's add an option to disable it.
Closes: #1752
Approved by: jlebon
There are use cases for libostree as a local content store
for content derived or delivered via other mechanisms (e.g. OCI
images, RPMs, etc.). rpm-ostree today imports RPMs into OSTree
branches, and puts the RPM header value as commit metadata.
Some of these can be quite large because the header includes
permissions for each file. Similarly, some OCI metadata is large.
Since there's no security issues with this, support committing
such content.
We still by default limit the size of metadata fetches, although
for good measure we make this configurable too via a new
`max-metadata-size` value.
Closes: https://github.com/ostreedev/ostree/issues/1721Closes: #1744
Approved by: jlebon
Copying the xattrs on metadata objects is wrong in general, we
don't "own" them. Notably this would fail in the situation of
doing a pull from e.g. a `bare-user` source to a destination
that was on a different mount point (so we couldn't hardlink),
and the source had e.g. a `security.selinux` attribute.
Closes: #1734Closes: #1736
Approved by: jlebon
There's some subtlety to this, we don't handle all cases.
But the 99% cases are using `--sysroot deploy` to create an
initial deployment, and then doing upgrades from inside
a booted deployment.
It was only the latter case that didn't work with a separate `/var`.
Fixing all of them would probably require libostree to learn
how to e.g. look at `/etc/fstab` (or worse, systemd mount units?)
and handle the mounting. I don't think we want to do anything
like that right now, since there are no active drivers for the
use case.
Closes: https://github.com/ostreedev/ostree/issues/1729Closes: #1730
Approved by: akiernan
This is standard practice for units like this; e.g. it's what
`systemd-remount-fs.service` does. I think it may be part of
or the whole cause for
https://github.com/projectatomic/rpm-ostree/issues/1471
I haven't reproduced the problem exactly but it seems to me that
if the unit starts and is GC'd, then when systemd goes to execute
a later unit it might end up restarting it.
A noticeable side effect of this is that `systemctl status ostree-remount`
exits with code `0` as expected.
Closes: #1697
Approved by: jlebon
The reliablity has just not been what we need, and they haven't
yet caught any real bugs. Until I can carve off some time to
truly make this reliable let's just mark it as not required.
I'd like to gather more statistics on the failure scenarios.
Closes: #1686
Approved by: jlebon
We use the API, and not linking breaks the build with e.g.
`-fuse-ld=gold` in a Fedora 28 buildroot as gold doesn't do the
"search indirect dependencies" thing.
Closes: #1679
Approved by: jlebon
This is the inverse of https://github.com/ostreedev/ostree/pull/1558
aka commits cadece6c4f398ca61d21e497bd6e3fbb549f9cf6 and
3358698c86d80821d81443c906621c92672f99fb
Needed to fix `rpm-ostree kargs` test suite with default staging; skipping
a test here for now as eventually what we'll do is turn on the rpm-ostree
suite fully here.
Closes: #1677
Approved by: jlebon
For `rpm-ostree ex livefs` we have a use case of pushing a rollback
deployment. There's no reason this should require deleting the staged
deployment (and doing so actually breaks livefs which tries to access
it as a data source).
I was initially very conservative here, but I think it ends up
being fairly easy to retain the staged deployment. We need to handle
two cases:
First, when the staged is *intentionally* deleted; here, we just need
to unlink the `/run` file, and then everything will be sync'd up after
reloading.
Second, (as in the livefs case) where we're retaining it,
e.g. adding a deployment to the end. What I realized here is that
we can have the code keep `new_deployments` as view without staged,
and then when we do the final reload we'll end up re-reading it from
disk anyways.
Closes: #1672
Approved by: jlebon
Noticed as part of a random failure in this PR:
https://github.com/ostreedev/ostree/pull/1655
that we weren't actually testing the version of ostree
we built in git. Probably we should be generating RPMs but...later.
Closes: #1662
Approved by: jlebon
We were referencing the txn bits outside of the lock in the error
path. Generally shouldn't matter, but e.g. Rust wouldn't let us do this, and
race detector tooling will warn about it.
Closes: #1632
Approved by: jlebon
I generally like having variables include their units where applicable;
timer variables having `_secs` or `_ms`, etc.
Closes: #1632
Approved by: jlebon
The initial motivation for this is that the "staging" code currently
didn't rewrite the deployment refs, meaning that the staged commit
could be pruned.
Hence first, this new API ensures that deployments also
hold a strong ref to their commit, without relying on the magical
"deployment refs" that we inject. That has always been a weird
artifact of the strict layering separation between OstreeSysroot
and OstreeRepo.
I also plan to change rpm-ostree to start using this API to
hold references to base layers for client-side layering; it also
today generates various refs.
That said, if we still want to support multiple processes
writing to a single repo (as happens on EndlessOS today) we
still need to write refs; perhaps later we could add a concept
of "generators" or something that create refs based on whatever
logic?
Another minor thing this fixes is that we had a printf inside
the library; this propagates the pruned data to the higher level
which can log however it likes.
Closes: #1566
Approved by: jlebon
Prep for reworking how we do sysroot cleanup. We're going to
start doing more lowlevel pruning work there, and I wanted to avoid
duplicating the ref enumeration.
Closes: #1566
Approved by: jlebon
I feel like I'm drowning in a pile of experimental-but-almost-stable
features...
Anyways, since we made the feature opt-in in rpm-ostree in
https://github.com/projectatomic/rpm-ostree/pull/1352
let's mirror that a bit here with an environment variable so people
can play with it more easily.
The tests needed some tweaks; specifically we need to reload the
status fact after making changes. I'm still a bit uncertain
about the Ansible-as-tests.
But we add an upgrade test that uses the new environment variable.
Closes: #1583
Approved by: jlebon
For the same reason we do in the rpm-ostree tests. This also
made sure the test run worked when I was offline on a plane.
Closes: #1583
Approved by: jlebon
OK so I noticed that something was failing and we were missing
`set -xeuo pipefail` in our shells. That of course revealed
the ansible tests didn't actually work - my only defense
here is spending so much time fighting to get it through CI
and trying something new.
Anyways, to make the staged-deploy tests work we need a task
that actually uses `rpm-ostree override` rather than `usroverlay`.
Let's make this a bit saner and have a clean split between
tests that are "shell-script+usroverlay" and "ansible+override".
Closes: #1577
Approved by: jlebon
Colin Walters
