This is the new Fedora kernel standard layout; it has the advantage
of being in `/usr` like `/usr/lib/ostree-boot`, but it's not OSTree
specific.
Further, I think in practice forcing tree builders to compute the checksum is an
annoying stumbling block; since we already switched to e.g. computing checksums
always when doing pulls, the cost of doing another checksum for the
kernel/initramfs is tiny. The "bootcsum" becomes more of an internal
implementation detail.
Now, there is a transition; my current thought for this is that rpm-ostree will
change to default to injecting into both `/usr/lib/ostree-boot` and
`/usr/lib/modules`, and stop doing `/boot`, then maybe next year say we drop the
`/usr/lib/ostree-boot` by default.
A twist here is that the default Fedora kernel RPM layout (and what's in
rpm-ostree today) includes a kernel but *not* an initramfs in
`/usr/lib/modules`. If we looked only there, we'd just find the kernel. So we
need to look in both, and then special case this - pick the legacy layout if we
have `/usr/lib/modules` but not an initramfs.
While here, rework the code to have an `OstreeKernelLayout` struct which makes
dealing with all of the variables nicer.
Closes: #1079
Approved by: jlebon
In the production case since we used `daemon()` our stderr is `/dev/null`¹
there's not much use in logging errors from `FITHAW` or `exit(1)`, and doing so
breaks the test suite which checks the return from `waitpid()`. There's nothing
we can really do if `FITHAW` fails, and in most of those cases `EINVAL`,
`EOPNOTSUPP`, we *shouldn't* do anything anyways.
¹ Though perhaps we should set up the systemd journal, but let's not
go there right now.
Closes: #1084
Approved by: jlebon
I added `waitpid()`, but that didn't actually help because we were
`daemon()`izing. Don't daemonize if we're testing so that we can `waitpid()`.
Note I still haven't reproduced this race locally, but I'm pretty sure this will
fix it.
While here, actually check the return value from `waitpid()` just in case
something goes wrong there.
Closes: #1084
Approved by: jlebon
Saw this in a PR result; we need to wait for the child to have written its
result to stderr before we exit, otherwise the test suite may not read it in
time.
Closes: #1070
Approved by: jlebon
This will allow us to drop the awful hack in rpm-ostree where we watch our own
stdout. In general, libraries shouldn't write to stdout.
Also we can kill the systemd journal wrapper code. There's some duplication at
each call site now...but it's easier than trying to write a `sd_journal_send()`
wrapper.
I was originally going to have this emit all of the structured data too as a
`GVariant` but decided it wasn't worth it right now.
Closes: #1052
Approved by: jlebon
I'd like to move the new canonical kernel directory to `/usr/lib/modules/$kver`,
as Fedora has done. The `get_kernel_from_tree()` function now abstracts over
parsing the data (src vs destination filenames, as well as checksum) in
preparation for adding the new case.
In preparation for this, let's change the current test suite to use the
*current* directory of `/usr/lib/ostree-boot`, and also add coverage of `/boot`.
Closes: #1053
Approved by: jlebon
See: http://marc.info/?l=linux-fsdevel&m=149520244919284&w=2
XFS doesn't flush the journal on `syncfs()`. GRUB doesn't know how to follow the
XFS journal, so if the filesystem is in a dirty state (possible with xfs
`/boot`, extremely likely with `/`, if the journaled data includes content for
`/boot`, the system may be unbootable if a system crash occurs.
Fix this by doing a `FIFREEZE`+`FITHAW` cycle. Now, most people
probably would have replaced the `syncfs()` invocation with those two
ioctls. But this would have become (I believe) the *only* place in
libostree where we weren't safe against interruption. The failure
mode would be ugly; nothing else would be able to write to the filesystem
until manual intervention.
The real fix here I think is to land an atomic `FIFREEZETHAW` ioctl
in the kernel. I might try a patch.
In the meantime though, let's jump through some hoops and set up
a "watchdog" child process that acts as a fallback unfreezer.
Closes: https://github.com/ostreedev/ostree/issues/876Closes: #1049
Approved by: jlebon
Part of cleaning up our usage of libglnx; we want to use what's in GLib where we
can.
Had to change a few .c files to `#include ostree.h` early on to pick up
autoptrs for the core types.
Closes: #1040
Approved by: jlebon
I plan to at some point change rpm-ostree to read the journal messages from
libostree and render things like the time we spent in syncfs().
Closes: #1044
Approved by: jlebon
Mostly for the latest `-Wmaybe-uninitialized` fix, but while here also port some
places to newer APIs.
Update submodule: libglnx
Closes: #1027
Approved by: jlebon
There are a number of simple ports here. Prep for further work
in `/etc` merge.
I also stripped trailing whitespace globally.
Closes: #996
Approved by: jlebon
Having a failable accessor is annoying, since it's really common
to reference both. Instead, open the repo once when we load
the sysroot, and provide a non-failable accessor.
This is also prep for `ostree_repo_open_at()`, which collapses the separation
between `ostree_repo_new()` and `ostree_repo_open()`.
Closes: #886
Approved by: jlebon
This is a de-scoping of work I did in preparation for
rpm-ostree [live updates](https://github.com/projectatomic/rpm-ostree/pull/652).
Originally I was going to expose this as a public API.
However, I decided to do things differently, but the cleanup here for new code
style and fd-relative is nice to have anyways.
We rework things to use `OstreeDeployment*`, which the caller is expected to
already have, rather than `GFile*`s pointing to the config directories.
Closes: #741
Approved by: jlebon
If one wants to set up a mount for `/var` in `/etc/fstab`, it
won't be mounted since `ostree-prepare-root` set up a bind mount for
`/var` to `/sysroot/ostree/$stateroot/var`, and systemd will take
the already extant mount over what's in `/etc/fstab`.
There are a few options to fix this, but what I settled on is parsing
`/etc/fstab` in a generator (exactly like `systemd-fstab-generator` does),
except here we look for an explicit mount for `/var`, and if one *isn't* found,
synthesize the default ostree mount to the stateroot. Another nice property is
that if an admin creates a `var.mount` unit in `/etc` for example, that will
also override our mount.
Note that today ostree doesn't hard depend on systemd, so this behavior only
kicks in if we're built with systemd *and* libmount support (for parsing
`/etc/fstab`). I didn't really test that case though.
Initially I started writing this as a "pure libc" program, but at one point
decided to use `libostree.so` to find the booted deployment. That didn't work
out because `/boot` wasn't necessarily mounted and hence we couldn't find the
bootloader config. A leftover artifact from this is that the generator code
calls into libostree via the "cmd private" infrastructure. But it's an easy way
to share code, and doesn't hurt.
Closes: #859
Approved by: jlebon
This is a variant of the efforts in https://github.com/ostreedev/ostree/pull/741
Working on `rpm-ostree livefs`, I realized though I needed to just
check out *new* files directly into the live `/etc` (and possibly
delete obsolete files).
The way the current `/etc` merge works is fundamentally different from
that. So my plan currently is to probably do something like:
- Compute diff
- Check out each *new* file individually (as a copy)
- Optionally delete obsolete files
Also, a few other things become more important - in the current deploy code, we
copy all of the files, then relabel them. But we shouldn't expose to *live*
systems the race conditions of doing that, plus we should only relabel files we
checked out.
By converting the deploy's /etc code to use this, we fix the same TODO item
there around atomically having the label set up as we create files. And further,
if we kill the `/var` relabeling which I think is unnecessary since Anaconda
does it, we could delete large chunks of code there.
In the implementation, there are two types of things: regular files, and
symlinks. For regular files, in the `O_TMPFILE` case, we have the ability to
do *everything* atomically (including SELinux labeling) before linking it into
place. So let's just use that. For symlinks, we use `setfscreatecon()`.
Closes: #797
Approved by: jlebon
There aren't many users of `g_file_enumerator_iterate()` left - those
remaining are usually good candidates for porting. There's some more
porting to do in this file; a mix of trivial and harder. This
one is a good candidate for an individual commit.
Closes: #803
Approved by: jlebon
In particular the 26-variable monster 👹 in `install_deployment_kernel()` is
slain🗡. I didn't touch every function here, trying to keep things gradual.
Closes: #781
Approved by: jlebon
I happened to read this file and realized there's a lot of cruft left over from
the time when I liked `GFile` and `malloc()`ing like 50 times just to make a
pathname string. Delete it.
Closes: #767
Approved by: jlebon
In [this commit](6ce80f9685)
for some reason I added a `sepolicy` member to the sysroot. I
have no idea why I did that, and it's conceptually wrong
since the policy is specific to a *deployment*.
This bit me when I was working on [a pull request](https://github.com/ostreedev/ostree/pull/763)
elsewhere, since at that point it was `NULL`.
We already pass around the sepolicy in the deployment code, so just stop caching
it.
Closes: #764
Approved by: jlebon
There are a lot of things suboptimal about this approach, but
on the other hand we need to get our CI back up and running.
The basic approach is to - in the test suite, detect if we're on overlayfs. If
so, set a flag in the repo, which gets picked up by a few strategic places in
the core to turn on "ignore xattrs".
I also had to add a variant of this for the sysroot work.
The core problem here is while overlayfs will let us read and
see the SELinux labels, it won't let us write them.
Down the line, we should improve this so that we can selectively ignore e.g.
`security.*` attributes but not `user.*` say.
Closes: https://github.com/ostreedev/ostree/issues/758Closes: #759
Approved by: jlebon
More sophisticated users of libostree like rpm-ostree need control over things
like the system repository. Previously we introduced a "no cleanup" flag to
`ostree_sysroot_simple_write_deployment()`, but that's a high level API that
does filtering on its own.
Since rpm-ostree needs more control, let's expose the bare essentials of the
"sysroot commit" operation with an extensible options structure, where one of
the options is whether or not to do post-transaction repository operations.
Closes: #745
Approved by: jlebon
Use `g_auto()` more sanely with a struct implmenting the "is initialized"
pattern. This is way less ugly for callers, and fixes bugs like
us calling `setfscreatecon()` even if an error occurred beforehand.
Also fold in the logic for "NULL or not loaded" sepolicy into the setup rather
than requiring callers to inline it.
Prep for more users of this function.
Closes: #746
Approved by: jlebon
For future work I'm going to tweak how we handle cleanup, and
the private cleanup flags didn't really end up being used - we
only specify "prune repo or not". So fold that into a boolean for now.
The sysroot deploy logic then has a single "do_postclean" boolean, which is all
I want to expose as public API.
Closes: #744
Approved by: jlebon
https://github.com/ostreedev/ostree/pull/705 broke the build
on CentOS 7 which only has util-linux 2.23.
When I was thinking about this, I realized that there must really be a way to
make this safe even for older versions. Looking at that version of util-linux,
all we need to do is invert the order of frees so we `mnt_free_table()` *before*
`mnt_free_cache()`, like util-linux does:
https://github.com/karelzak/util-linux/blob/stable/v2.23/sys-utils/eject.c#L1131
We still use the `_unref()` versions if available. I also fixed
the ordering there too for double plus redundant safety.
Closes: #712
Approved by: jlebon
We saw a random ostree SEGV start popping up in our CI environment:
https://github.com/projectatomic/rpm-ostree/pull/641#issuecomment-281870424
Looking at this code more and comparing it to what util-linux does, I noticed we
had a write-after-free, since `mnt_unref_table()` will invoke
`mnt_unref_cache()` on its cache, and that function does:
```
if (cache) {
cache->rfcount--;
```
unconditionally.
Fix this by using `unref()`.
Closes: #705
Approved by: jlebon
It appears the result of assign_bootserials() is never actually used,
but I haven't changed it to return void right now.
Leak found with valgrind memcheck.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #556
Approved by: cgwalters
Just noticed this while inspecting the code. The deployments retrieved
by `_ostree_sysroot_list_deployment_dirs_for_os` will forcibly already
have a matching osname since it indirectly uses that same variable to
construct them. Having a check there makes it look like there may be
subtle corner cases, when there aren't.
Closes: #529
Approved by: cgwalters
While looking at a slow update issue (which I'm guessing is
unpredictable I/O latency in an OpenStack instance), I noticed
in one of the traces we were inside a fsync here.
Dropping the fsync here is just another of a long series of unwinding
them - we `syncfs()` the sysroot fd and `/boot` and we have a big
`sync()` anyways.
Closes: #508
Approved by: jlebon
More fsync pruning. Since we have a public API for writing the origin
file and it did a fsync before, let's preserve that. But when writing
deployments as part of a full transaction, we rely on the global
`syncfs()`, so add an internal function for origin file writing that
doesn't.
Closes: #509
Approved by: giuseppe
Since forever, we've been doing two cleanups. In
8ece4d6d51
I thought we were doing just one and wanted to go to zero (if specified),
but I actually just dropped one cleanup.
In https://github.com/projectatomic/rpm-ostree/pull/452
@jlebon pointed out the duplication. Fix this by creating a new internal
deploy wrapper that takes cleanup flags.
(Since we already had the "piecemeal cleanup" API internally, let's
frame it in terms of that, rather than passing down a boolean).
Closes: #500
Approved by: jlebon
Since we already had a "recursive copy" implementation here, let's
reuse it rather than the libgsystem `gs_shutil_cp_a()`. Part of the
libglnx porting.
Closes: #428
Approved by: jlebon
It handles ownership of the `DIR*` for us more cleanly, and
is just a better API.
This is in preparation for further changes to this code to do SELinux
labeling while copying.
Closes: #428
Approved by: jlebon
Since we're adding a new API, we have the opportunity to fix
the defaults. We expect clients to do a `syncfs()` or equivalent
on their own now, since it's way more efficient.
Flip the checkout fsync default to off.
Closes: #425
Approved by: giuseppe
In one case, we already had relative fds and hence this was
nicer. Unfortunately the other areas got uglier. More fd-relative
porting to do later.
Closes: #424
Approved by: giuseppe
Colin Walters
Jonathan Lebon
Simon McVittie
Giuseppe Scrivano