Commit Graph

2481 Commits

Author SHA1 Message Date
Jonathan Lebon 93e47f88f4 lib/commit: Directly use FICLONE for payload link
The idea of payload linking is to reflink between objects where
possible. Instead of relying on `glnx_regfile_copy_bytes` to hit the
`FICLONE` path, just call `FICLONE` directly. At that point in the code,
we've already established that the source and dest repos are on the same
filesystem and that it supports `FICLONE`.

Related: https://gitlab.gnome.org/GNOME/libglnx/-/merge_requests/41
Related: https://github.com/ostreedev/ostree/pull/2684#issuecomment-1204068437
2022-08-15 14:00:08 -04:00
Colin Walters 0a908a180f Move FIFREEZE/FITHAW ioctl invocations into linuxfsutil.c
Should help avoid conflicts between glibc and linux headers.

Closes: https://github.com/ostreedev/ostree/issues/2685
2022-08-03 10:44:51 -04:00
Colin Walters edba4b33be Remove unused `linux/fs.h` includes
Prep for fixing conflicts introduced by newer glibc.
cc https://github.com/ostreedev/ostree/issues/2685
2022-08-03 10:37:40 -04:00
Colin Walters 15740d042c Release 2022.5 2022-07-22 15:08:25 -04:00
Colin Walters 83e6357186 sign/ed25519: Verify signatures are minimum length
The ed25519 signature verification code does not
check that the signature is a minimum/correct length.
As a result, if the signature is too short, libsodium will end up
reading a few bytes out of bounds.

Reported-by: Demi Marie Obenour <demi@invisiblethingslab.com>
Co-authored-by: Demi Marie Obenour <demi@invisiblethingslab.com>

Closes: https://github.com/ostreedev/ostree/security/advisories/GHSA-gqf4-p3gv-g8vw
2022-07-14 17:13:51 -04:00
Colin Walters ed1146738b
Merge pull request #2669 from HuijingHei/fix-kargs
Fix `ostree admin kargs edit-in-place` fails issue
2022-07-13 16:35:28 -04:00
Colin Walters a3749efb1a
Merge pull request #2672 from cgwalters/initialize-sysroot-cleanups
sysroot: Have `ensure_writable` also always initialize
2022-07-13 16:35:17 -04:00
Colin Walters 60853219d5 sysroot: Add a few more assertions about `boot_fd`
These places are all safe, but it would catch bugs in the future
more clearly to trip an assertion here.
2022-07-13 15:38:59 -04:00
Colin Walters 75aa7a22f6 sysroot: Have `ensure_writable` also always initialize
For historical reasons we have a fair bit of distinct sysroot
initialization going on.  A lot of code is calling *just* the new
`ensure_writable()` API, which does basically what you'd expect...
except if we're not using a mount namespace.

Which is the case in unit tests and legacy setups.
Change this API to also ensure the sysroot is fully initialized
even in those cases.  Specifically we'll have `self->sysroot_fd`.

For now, callers that need `/boot` also need to separately
call `_ensure_boot_fd()`.
2022-07-13 15:35:00 -04:00
Colin Walters 2c71655205 deploy: Ensure sysroot is initialized for kargs in place
Even without a mount namespace set up.
2022-07-13 15:32:05 -04:00
Huijing Hei 7b7b6d741b Fix `ostree admin kargs edit-in-place` fails issue
Add func to set kernel arguments in place, instead of create new
deployment
Fix https://github.com/ostreedev/ostree/issues/2664
2022-07-12 16:27:00 +08:00
Saqib Ali d7107e3036 ostree-repo: bls-append-except-default followup
This PR is followup from https://github.com/coreos/coreos-assembler/pull/2863
Summary of changes:
- Moved bls-append-except-default parsing logic to reload_sysroot_config()
- Made sure heap allocated memory is being freed
2022-07-07 16:06:11 -04:00
Colin Walters becc18936f lib: Stop using old `ostree_sysroot_get_repo()` API
It's falliable, and in one place we were actually ignoring
the error and leaving a `NULL` repo object which is just a trap
for people coming along later since it's rarely nullable.

Quite a while ago we switched to loading the repo at the same time
as the sysroot; convert callers in the library to use this infallible
accessor.

Prep for another patch which will use the repo object.
2022-07-06 07:47:57 -04:00
Colin Walters 0d911bf91f
Merge pull request #2632 from saqibali-2k/pr/prune-commit-only
lib/prune: speed up pruning by retrieving only commits
2022-06-30 09:08:54 -04:00
Saqib Ali a984871237 lib/prune: speed up pruning by retrieving only commits
After landing the new --commit-only functionality, we still noticed
exceedingly long pruning times in large repos. Lets add an optimization
that will only retrieve commit objects when --commit-only flag is used.
2022-06-27 08:04:50 -04:00
Nikita Dubrovskii 14a7c0c74b s390x: rename sd-boot to sdboot
Signed-off-by: Nikita Dubrovskii <nikita@linux.ibm.com>
2022-06-24 15:15:57 +02:00
Huijing Hei 3bc59a5206 RFE: Add a hidden option to `ostree admin kargs edit-in-place` to
update all existing deployments in place

Example:
$ sudo ostree admin kargs edit-in-place --append-if-missing=rw
See https://github.com/ostreedev/ostree/issues/2617

This will not add duplicate key, if there is `TESTARG=VAL1` in the
kernel arguments, `--append-if-missing=TESTARG=VAL2` will be ignored.
2022-06-23 22:31:39 +08:00
Colin Walters a13d812368 repo: Document non-obvious way to list all commits
I was going to add an API for this and then realized the empty
string does it.
2022-06-09 18:34:51 -04:00
Joseph Marrero Corchado a6ecd96cda
Merge pull request #2638 from cgwalters/fix-symbol-versioning
lib: Fix symbol versioning inheritance
2022-06-08 17:17:28 -04:00
Colin Walters 145d91d1c9 lib: Fix symbol versioning inheritance
I messed this up; the last release should inherit from the previous
release (N-1) and not the previous to that (N-2).

I think (hope) this isn't an ABI break...

Just noticed this when I was going to add a new symbol.
2022-06-08 16:44:51 -04:00
Colin Walters fffb111633 prune: Also use object set API in `ostree_repo_prune_from_reachable()`
I missed the second prune path when working on
https://github.com/ostreedev/ostree/pull/2635
2022-06-08 16:22:26 -04:00
Colin Walters a71915e436 repo: Further optimize `ostree_repo_list_objects_set()`
In a prior change we discovered that for bad historical reasons
libostree was returning a mapping "object type+checksum" => "metadata"
but the "metadata" was redundant and pointless.

Optimize the prune API to use a (currently internal) object listing
API which returns a set, not a map.  This allows `GHashTable` to
avoid allocating a separate array for the values, neatly cutting
memory usage in half (from ~13MB to ~6MB) on my test case of a
dry-run prune of a FCOS build.
2022-06-08 10:18:55 -04:00
Colin Walters c2baa6d10b repo: Optimize memory use of `ostree_repo_list_objects()`
I was looking at https://github.com/ostreedev/ostree/pull/2632
and confused at the usage of
`GVariant *value = g_variant_new ("(b@as)", TRUE, g_variant_new_strv (NULL, 0));`
which looked strange - why the empty strv?

It turns out that this is a historical legacy of the time when
ostree had pack files.  And nothing actually cares about the values
of these variants; we should have an API that returns a proper set,
and not a hash.

But...since all of these things have exactly the same value, instead
of allocating lots of redundant copies on the heap, just have
them all hold a refcount on a shared value.

This cuts the heap usage from 20MB to 13MB on a test FCOS repository
build.
2022-06-07 20:35:37 -04:00
Colin Walters 5fa364d399 Release 2022.4 2022-06-06 11:04:44 -04:00
Joseph Marrero Corchado 8bd1c88013
Merge pull request #2625 from cgwalters/drop-features-introspection
Drop `OSTREE_BUILT_FEATURES` from introspection
2022-06-03 09:22:48 -04:00
Colin Walters 3e645ad950 Drop `OSTREE_BUILT_FEATURES` from introspection
It inherently depends on the individual build, and can't
really be an official stable API for introspection users.
I've noticed the value of this flip flop when doing local builds.

I'm fairly certain no one is trying to use it from a higher level
language.

It'd probably make sense to even drop from the official C API,
but I'm trying to be conservative with that.
2022-06-02 19:44:40 -04:00
Colin Walters 32ee82133b Rename ostree-cmdprivate to drop out of introspection
I was looking at our `.gir` and noticed we had the cmdprivate bits
because the pattern for excluding headers is `-private.h`, which
didn't match `cmdprivate.h`.
2022-06-02 17:13:17 -04:00
James Westman 03d592b116
Fix annotations on ostree_mutable_tree_lookup()
(nullable) and (optional) were missing on lookup()'s out parameters,
which caused the rust bindings for the function to not work. Due to the
missing (nullable), it would return a Result<(GString, MutableTree), _>,
not a Result<(Option<GString>, Option<MutableTree>), _>, which led to
panics.
2022-05-30 17:36:07 -05:00
Nikita Dubrovskii 972f00e483 s390x: do not unpack existing initrd, just append LUKS keys to its copy
Signed-off-by: Nikita Dubrovskii <nikita@linux.ibm.com>
2022-05-27 12:06:33 +02:00
Jonathan Lebon 9aca8816c6
Merge pull request #2615 from nikita-dubrovskii/handle_errors 2022-05-25 13:38:23 -04:00
Colin Walters 2363d3ad98
Merge pull request #2605 from saqibali-2k/pr/bls-append
ostree-systroot-deploy: parse bls-append-except-default key
2022-05-25 11:33:10 -04:00
Nikita Dubrovskii b03fa626f1 s390x: fail on error during reading of SecureExecution sysfs flag 2022-05-25 11:00:34 -04:00
Nikita Dubrovskii d000569859 s390x: ensure SecureExecution is enabled before sd-boot generation
Signed-off-by: Nikita Dubrovskii <nikita@linux.ibm.com>
2022-05-23 18:24:43 +02:00
Saqib Ali 326d89752a ostree-systroot-deploy: parse bls-append-except-default key
We want to parse a new "bls-append-except-default" key from ostree config. The
key-value pairs specified by this key will be added to the generated
BLS fragments of non-default deployments. They must follow the format
"key1,value1;key2,value2" and so on.

This change will allow us to land GRUB password support in FCOS.

Relevant: https://github.com/coreos/fedora-coreos-tracker/issues/134
2022-05-19 09:59:38 -04:00
Nikita Dubrovskii 7a5c604ca3 s390x: generate sd-boot at its own partition
Signed-off-by: Nikita Dubrovskii <nikita@linux.ibm.com>
2022-05-18 13:06:29 +02:00
Colin Walters 6a055f802f sysroot: Strengthen several `g_return_if_fail` into `g_assert()`
Similar to prior changes; motivated by testing out an updated
`gcc -fanalyzer`.
2022-05-16 11:00:08 -04:00
Colin Walters e27fed45fd sysroot: Quiet one gcc `-fanalyzer` warning
I don't think this can happen in practice, but add an assertion
to verify it.
2022-05-15 16:03:01 -04:00
Colin Walters f79b2cea91 Add APIs to get xattrs from disk
I'm aiming to do some more work on the Rust side around `fsck`
like functionality, and this is a useful primitive.  There isn't
a great Rust crate for xattrs, and I think it's better to share this
code.
2022-05-09 12:33:38 -04:00
Luca BRUNO 4b7fc4f1e9
lib/ostree-sysroot-deploy: add a missing include for stdint.h 2022-05-06 10:07:23 +00:00
Simon McVittie 71304e854c repo: Correctly initialize refcount of temporary transaction
Previously, the reference count was left uninitialized as a result of
bypassing the constructor, and the intended abort-on-error usually
wouldn't have happened.

Fixes: 8a9737a "repo/private: move OstreeRepoAutoTransaction to a boxed type"
Resolves: https://github.com/ostreedev/ostree/issues/2592
Signed-off-by: Simon McVittie <smcv@collabora.com>
2022-04-30 12:53:42 +01:00
Simon McVittie 540e60c3e3 repo: Factor out _ostree_repo_auto_transaction_new()
This will allow the direct allocation in
ostree_repo_prepare_transaction() to be replaced with a call to this
function, avoiding breaking encapsulation.

Signed-off-by: Simon McVittie <smcv@collabora.com>
2022-04-30 12:20:11 +01:00
Colin Walters d3d3e4ea13 Add an `ostree-boot-complete.service` to propagate staging failures
Quite a while ago we added staged deployments, which solved
a bunch of issues around the `/etc` merge.  However...a persistent
problem since then is that any failures in that process that
happened in the *previous* boot are not very visible.

We ship custom code in `rpm-ostree status` to query the previous
journal.  But that has a few problems - one is that on systems
that have been up a while, that failure message may even get
rotated out.  And second, some systems may not even have a persistent
journal at all.

A general thing we do in e.g. Fedora CoreOS testing is to check
for systemd unit failures.  We do that both in our automated tests,
and we even ship code that displays them on ssh logins.  And beyond
that obviously a lot of other projects do the same; it's easy via
`systemctl --failed`.

So to make failures more visible, change our `ostree-finalize-staged.service`
to have an internal wrapper around the process that "catches" any
errors, and copies the error message into a file in `/boot/ostree`.

Then, a new `ostree-boot-complete.service` looks for this file on
startup and re-emits the error message, and fails.

It also deletes the file.  The rationale is to avoid *continually*
warning.  For example we need to handle the case when an upgrade
process creates a new staged deployment.  Now, we could change the
ostree core code to delete the warning file when that happens instead,
but this is trying to be a conservative change.

This should make failures here much more visible as is.
2022-04-26 13:02:46 -04:00
Colin Walters 4fd0d6b4c1 fetcher/curl: Consistently check return value `curl_easy_setopt`
Static analyzers don't like when we only check it sometimes.

And we definitely want to know if any of these are failing.
2022-04-20 15:16:22 -04:00
Colin Walters 2346d5f4d5 libarchive: Handle `archive_entry_symlink()` returning NULL
The `archive_entry_symlink()` API can definitely return `NULL`,
reading through the libarchive sources.

I hit this in the wild when using old ostree-ext to try to unpack
a chunked archive.

I didn't try to characterize this more, and sorry no unit test right
now.
2022-04-04 10:25:35 -04:00
Colin Walters c58a4fe661 deploy: Be a bit more verbose about SELinux bits
Let's log when we don't find the expected CLI argument which
will help debug things.
2022-03-28 17:46:59 -04:00
Ondrej Mosnacek edb4f38934 deploy: Try to rebuild policy in new deployment if needed
Whenever the user has SELinux enabled and has any local
modules/modifications installed, it is necessary to rebuild the policy
in the final deployment, otherwise ostree will leave the binary policy
files unchanged from last deployment as it detects difference against
the base content (in rpm-ostree case this is the RPM content).

To avoid the situation where the policy binaries go stale once any local
customization of the policy is made, try to rebuild the policy as part
of sysroot_finalize_deployment(). Use the special
--rebuild-if-modules-changed switch, which detects if the input module
files have changed relative to last time the policy was built and skips
the most time-consuming part of the rebuild process if modules are
unchanged (thus making this a relatively cheap operation if the user
hasn't made any modifications to the shipped policy).

As suggested by Jonathan Lebon, this uses bubblewrap (via
g_spawn_sync()) to perform the rebuild inside the deployment's
filesystem tree, which also means that ostree will have a runtime
dependency on bubblewrap.

Partially addresses: https://github.com/coreos/fedora-coreos-tracker/issues/701

Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
2022-03-28 17:18:03 -04:00
Daniel Kolesa aca9e8e6a7 glib: bump glib requirement to 2.66 and port to GUri
This removes the old SoupURI copypasta from previous generation
of libsoup and opens up a path for a simple libsoup3 port.
2022-03-12 04:44:18 +01:00
Philip Withnall 8cc8e68768 ostree-repo-pull: Take correct out path on error
Like every other error return path in this function, jump to the `out`
label on error here. Returning directly will cause leaks.

Spotted by reading the code, not actually necessarily encountered in the
wild.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2022-03-08 15:36:27 +00:00
Simon McVittie a9a0110f44 s390x-se-luks-gencpio: Use interoperable path for bash
On OSs that do not consistently merge /usr/bin with /bin, the path to
bash has traditionally been /bin/bash.

Signed-off-by: Simon McVittie <smcv@debian.org>
2022-03-07 22:49:33 +00:00
Simon McVittie 084f8913f0 s390x-se-luks-gencpio: Fix shebang syntax
An indented `#!` is technically meaningless, although many shells will
run text files with the shell if asked to execute them.

Signed-off-by: Simon McVittie <smcv@debian.org>
2022-03-07 22:49:30 +00:00