Followup to: https://github.com/ostreedev/ostree/pull/1503
After starting some more work on on this in rpm-ostree, it is
actually simpler if the staged deployment just shows up in the list.
It's effectively opt-in today; down the line we may make it the default,
but I worry about breaking things that e.g. assume they can mutate
the deployment before rebooting and have `/etc` already merged.
There's not that many things in libostree that iterate over the deployment
list. The biggest change here is around the
`ostree_sysroot_write_deployments_with_options` API. I initially
tried hard to support a use case like "push a rollback" while retaining
the staged deployment, but everything gets very messy because that
function truly is operating on the bootloader list.
For now what I settled on is to just discard the staged deployment;
down the line we can enhance things.
Where we then have some new gymnastics is around implementing
the finalization; we need to go to some effort to pull the staged
deployment out of the list and mark it as unstaged, and then pass
it down to `write_deployments()`.
Closes: #1539
Approved by: jlebon
Prep for handling staged deployments better; if we're not passed
the staged one back, then we just want to delete it but not
touch the bootloader config.
Closes: #1538
Approved by: jlebon
The reason we were returning a hashtable is a bit lost to history,
there's no reason to do so now anyways. Also port to declare-and-initialize
style and add more comments.
Closes: #1538
Approved by: jlebon
Add API to write a deployment state to `/run/ostree/staged-deployment`,
along with a systemd service which runs at shutdown time.
This is a big change to the ostree model for hosts,
but it closes a longstanding set of bugs; many, many people have
hit the "losing changes in /etc" problem. It also avoids
the other problem of racing with programs that modify `/etc`
such as LVM backups:
https://bugzilla.redhat.com/show_bug.cgi?id=1365297
We need this in particular to go to a full-on model for
automatically updated host systems where (like a dual-partition model)
everything is fully prepared and the reboot can be taken
asynchronously.
Closes: https://github.com/ostreedev/ostree/issues/545Closes: #1503
Approved by: jlebon
The main blocker for doing this before was the `goto out` handling
for remounting `/boot`. Handle that by factoring out the bits that
require it to a helper function, and do the C/GError equivalent of
"try/finally".
Not prep for anything right now, just decided to do this since I had the file
open.
Closes: #1515
Approved by: jlebon
For staged deploy, we want to pay the cost of creating copies from
`/usr/etc` → `/etc` at stage time, since it can be expensive. (We
want to minimize time spent during shutdown).
Split it up into two functions; the logic is also simply clearer.
Closes: #1514
Approved by: jlebon
Prep for deployment staging. We had the code to hande "explicit kargs" in one
place, but the "use merge deployment" karg bits mixed in with the "/etc merge"
logic. Those are separate things, and it's better to have karg handling in one
place.
Closes: #1514
Approved by: jlebon
Ensures it's labeled consistently. Prep for staged deployments which reworks the
logic around when the origin file is written.
Closes: #1505
Approved by: jlebon
Pulling some of this out of stage deploy work. It's generally better as it's
easier to change functions to have multiple callers.
Closes: #1505
Approved by: jlebon
When we changed around the kernel location in rpm-ostree, we
started installing the kernel into `/boot` as `modules_object_t`,
and the current policy didn't permit that. For maximum compatibility,
relabel installed kernel/initramfs/dtb as `boot_t`.
https://bugzilla.redhat.com/show_bug.cgi?id=1536991Closes: #1444
Approved by: jlebon
SPDX License List is a list of (common) open source
licenses that can be referred to by a “short identifier”.
It has several advantages compared to the common "license header texts"
usually found in source files.
Some of the advantages:
* It is precise; there is no ambiguity due to variations in license header
text
* It is language neutral
* It is easy to machine process
* It is concise
* It is simple and can be used without much cost in interpreted
environments like java Script, etc.
* An SPDX license identifier is immutable.
* It provides simple guidance for developers who want to make sure the
license for their code is respected
See http://spdx.org for further reading.
Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
Closes: #1439
Approved by: cgwalters
Much like the (optional) initramfs at
`/usr/lib/ostree-boot/initramfs-<SHA256>` or
`/usr/lib/modules/$kver/initramfs` you can now optionally include a
flattened devicetree (.dtb) file alongside the kernel at
`/usr/lib/ostree-boot/devicetree-<SHA256>` or
`/usr/lib/modules/$kver/devicetree`.
This is useful for embedded ARM systems which need the devicetree file
loaded by the bootloader for the kernel to discover and initialise
hardware. See https://en.wikipedia.org/wiki/Device_tree for more
information.
This patch was mostly produced by copy-pasting code for initramfs handling
and renaming `s/initramfs/devicetree/g`. It's not beautiful, but it is
fairly straightforward.
It may be useful to extend device-tree support in a number ways in the
future. Device trees dependant on many details of the hardware they
support. This makes them unlike kernels, which may support many different
hardware variants as long as the instruction-set matches. This means that
a ostree tree created with a device-tree in this manner will only boot on
a single model of hardware. This is sufficient for my purposes, but may
not be for others'.
I've tested this on my NVidia Tegra TK1 device which has u-boot running
in syslinux-compatible mode.
Closes: #1411
Approved by: cgwalters
Previously when initramfs-* was not found in a deployment's
boot directory, it was assumed that rootfs is prepared for
ostree booting by a kernel patch.
With this patch, the behaviour changes to be - if initramfs-*
is not found, assume that system is using a static
ostree-prepare-root as init process. Booting without initramfs
is a common use case on embedded systems. This approach is
also more convenient, than having to patch the kernel.
Closes: #1401
Approved by: cgwalters
This works around an (IMO) SpiderMonkey bug - it tries to
clean up in a shared library destructor, but doesn't install a
`pthread_atfork()` handler to unset its state.
Closes: https://github.com/ostreedev/ostree/issues/1262Closes: #1264
Approved by: dbnicholson
This ends up a lot better IMO. This commit is *mostly* just
`s/glnx_close_fd/glnx_autofd`, but there's also a number of hunks like:
```
- if (self->sysroot_fd != -1)
- {
- (void) close (self->sysroot_fd);
- self->sysroot_fd = -1;
- }
+ glnx_close_fd (&self->sysroot_fd);
```
Update submodule: libglnx
Closes: #1259
Approved by: jlebon
If the filesystem is already frozen, FIFREEZE returns EBUSY, and if the
filesystem is already thawed, FITHAW returns EINVAL. It's very unlikely
these issues would arise on a real ostree system since the sysroot would
be locked during the freeze/thaw cycle.
However, when multiple fake sysroots are used during the test suite (run
as root), the tests could race to run the freeze/thaw cycle without
locking. Furthermore, there's no reason why an independent process might
be trying to freeze the filesystem while ostree was deploying. Ignore
but warn for these errors since there's not much ostree can do about it,
anyways.
Closes: #1260
Approved by: cgwalters
The faster (OpenSSL/GnuTLS) code lived in a `GInputStream` wrapper, and that
adds a lot of weight (GObject + vtable calls). Move it into a simple
autoptr-struct wrapper, and use it in the metadata path, so we're
now using the faster checksums there too.
This also drops a malloc there as the new API does hexdigest in place to a
buffer.
Prep for more work in the commit path to avoid `GInputStream` for local file
commits, and ["adopting" files](https://github.com/ostreedev/ostree/pull/1255).
Closes: #1256
Approved by: jlebon
We added a `.dir-locals.el` in commit: 9a77017d87
There's no need to have it per-file, with that people might think
to add other editors, which is the wrong direction.
Closes: #1206
Approved by: jlebon
There were some important ones there like a random `syncfs()`. The remaining
users are mostly blocked on the "fstatat enoent" case, I'll wait to port those.
Closes: #1150
Approved by: jlebon
Follow up to <https://github.com/ostreedev/ostree/pull/1079>; I was working on
the rpm-ostree updates for this, and I think it's more consistent if we have
`.img` here, since that's a closer match to the "remove $kver" that results in
`vmlinuz`. Also just best practice to have file suffix types where they make
sense.
The astute reader might notice this sneaks in a change where we'd crash if the
legacy bootdir didn't have an initramfs...yeah, should probably have test
coverage of that.
Closes: #1095
Approved by: jlebon
This is the new Fedora kernel standard layout; it has the advantage
of being in `/usr` like `/usr/lib/ostree-boot`, but it's not OSTree
specific.
Further, I think in practice forcing tree builders to compute the checksum is an
annoying stumbling block; since we already switched to e.g. computing checksums
always when doing pulls, the cost of doing another checksum for the
kernel/initramfs is tiny. The "bootcsum" becomes more of an internal
implementation detail.
Now, there is a transition; my current thought for this is that rpm-ostree will
change to default to injecting into both `/usr/lib/ostree-boot` and
`/usr/lib/modules`, and stop doing `/boot`, then maybe next year say we drop the
`/usr/lib/ostree-boot` by default.
A twist here is that the default Fedora kernel RPM layout (and what's in
rpm-ostree today) includes a kernel but *not* an initramfs in
`/usr/lib/modules`. If we looked only there, we'd just find the kernel. So we
need to look in both, and then special case this - pick the legacy layout if we
have `/usr/lib/modules` but not an initramfs.
While here, rework the code to have an `OstreeKernelLayout` struct which makes
dealing with all of the variables nicer.
Closes: #1079
Approved by: jlebon
In the production case since we used `daemon()` our stderr is `/dev/null`¹
there's not much use in logging errors from `FITHAW` or `exit(1)`, and doing so
breaks the test suite which checks the return from `waitpid()`. There's nothing
we can really do if `FITHAW` fails, and in most of those cases `EINVAL`,
`EOPNOTSUPP`, we *shouldn't* do anything anyways.
¹ Though perhaps we should set up the systemd journal, but let's not
go there right now.
Closes: #1084
Approved by: jlebon
I added `waitpid()`, but that didn't actually help because we were
`daemon()`izing. Don't daemonize if we're testing so that we can `waitpid()`.
Note I still haven't reproduced this race locally, but I'm pretty sure this will
fix it.
While here, actually check the return value from `waitpid()` just in case
something goes wrong there.
Closes: #1084
Approved by: jlebon
Saw this in a PR result; we need to wait for the child to have written its
result to stderr before we exit, otherwise the test suite may not read it in
time.
Closes: #1070
Approved by: jlebon
This will allow us to drop the awful hack in rpm-ostree where we watch our own
stdout. In general, libraries shouldn't write to stdout.
Also we can kill the systemd journal wrapper code. There's some duplication at
each call site now...but it's easier than trying to write a `sd_journal_send()`
wrapper.
I was originally going to have this emit all of the structured data too as a
`GVariant` but decided it wasn't worth it right now.
Closes: #1052
Approved by: jlebon
I'd like to move the new canonical kernel directory to `/usr/lib/modules/$kver`,
as Fedora has done. The `get_kernel_from_tree()` function now abstracts over
parsing the data (src vs destination filenames, as well as checksum) in
preparation for adding the new case.
In preparation for this, let's change the current test suite to use the
*current* directory of `/usr/lib/ostree-boot`, and also add coverage of `/boot`.
Closes: #1053
Approved by: jlebon
See: http://marc.info/?l=linux-fsdevel&m=149520244919284&w=2
XFS doesn't flush the journal on `syncfs()`. GRUB doesn't know how to follow the
XFS journal, so if the filesystem is in a dirty state (possible with xfs
`/boot`, extremely likely with `/`, if the journaled data includes content for
`/boot`, the system may be unbootable if a system crash occurs.
Fix this by doing a `FIFREEZE`+`FITHAW` cycle. Now, most people
probably would have replaced the `syncfs()` invocation with those two
ioctls. But this would have become (I believe) the *only* place in
libostree where we weren't safe against interruption. The failure
mode would be ugly; nothing else would be able to write to the filesystem
until manual intervention.
The real fix here I think is to land an atomic `FIFREEZETHAW` ioctl
in the kernel. I might try a patch.
In the meantime though, let's jump through some hoops and set up
a "watchdog" child process that acts as a fallback unfreezer.
Closes: https://github.com/ostreedev/ostree/issues/876Closes: #1049
Approved by: jlebon
Part of cleaning up our usage of libglnx; we want to use what's in GLib where we
can.
Had to change a few .c files to `#include ostree.h` early on to pick up
autoptrs for the core types.
Closes: #1040
Approved by: jlebon
I plan to at some point change rpm-ostree to read the journal messages from
libostree and render things like the time we spent in syncfs().
Closes: #1044
Approved by: jlebon
Mostly for the latest `-Wmaybe-uninitialized` fix, but while here also port some
places to newer APIs.
Update submodule: libglnx
Closes: #1027
Approved by: jlebon
There are a number of simple ports here. Prep for further work
in `/etc` merge.
I also stripped trailing whitespace globally.
Closes: #996
Approved by: jlebon
Having a failable accessor is annoying, since it's really common
to reference both. Instead, open the repo once when we load
the sysroot, and provide a non-failable accessor.
This is also prep for `ostree_repo_open_at()`, which collapses the separation
between `ostree_repo_new()` and `ostree_repo_open()`.
Closes: #886
Approved by: jlebon
This is a de-scoping of work I did in preparation for
rpm-ostree [live updates](https://github.com/projectatomic/rpm-ostree/pull/652).
Originally I was going to expose this as a public API.
However, I decided to do things differently, but the cleanup here for new code
style and fd-relative is nice to have anyways.
We rework things to use `OstreeDeployment*`, which the caller is expected to
already have, rather than `GFile*`s pointing to the config directories.
Closes: #741
Approved by: jlebon
If one wants to set up a mount for `/var` in `/etc/fstab`, it
won't be mounted since `ostree-prepare-root` set up a bind mount for
`/var` to `/sysroot/ostree/$stateroot/var`, and systemd will take
the already extant mount over what's in `/etc/fstab`.
There are a few options to fix this, but what I settled on is parsing
`/etc/fstab` in a generator (exactly like `systemd-fstab-generator` does),
except here we look for an explicit mount for `/var`, and if one *isn't* found,
synthesize the default ostree mount to the stateroot. Another nice property is
that if an admin creates a `var.mount` unit in `/etc` for example, that will
also override our mount.
Note that today ostree doesn't hard depend on systemd, so this behavior only
kicks in if we're built with systemd *and* libmount support (for parsing
`/etc/fstab`). I didn't really test that case though.
Initially I started writing this as a "pure libc" program, but at one point
decided to use `libostree.so` to find the booted deployment. That didn't work
out because `/boot` wasn't necessarily mounted and hence we couldn't find the
bootloader config. A leftover artifact from this is that the generator code
calls into libostree via the "cmd private" infrastructure. But it's an easy way
to share code, and doesn't hurt.
Closes: #859
Approved by: jlebon
This is a variant of the efforts in https://github.com/ostreedev/ostree/pull/741
Working on `rpm-ostree livefs`, I realized though I needed to just
check out *new* files directly into the live `/etc` (and possibly
delete obsolete files).
The way the current `/etc` merge works is fundamentally different from
that. So my plan currently is to probably do something like:
- Compute diff
- Check out each *new* file individually (as a copy)
- Optionally delete obsolete files
Also, a few other things become more important - in the current deploy code, we
copy all of the files, then relabel them. But we shouldn't expose to *live*
systems the race conditions of doing that, plus we should only relabel files we
checked out.
By converting the deploy's /etc code to use this, we fix the same TODO item
there around atomically having the label set up as we create files. And further,
if we kill the `/var` relabeling which I think is unnecessary since Anaconda
does it, we could delete large chunks of code there.
In the implementation, there are two types of things: regular files, and
symlinks. For regular files, in the `O_TMPFILE` case, we have the ability to
do *everything* atomically (including SELinux labeling) before linking it into
place. So let's just use that. For symlinks, we use `setfscreatecon()`.
Closes: #797
Approved by: jlebon
Jonathan Lebon
Colin Walters
Marcus Folkesson
William Manley
Gatis Paeglis
Dan Nicholson
Jonathan Lebon