Commit Graph

3630 Commits

Author SHA1 Message Date
Colin Walters cac4f2133b
Merge pull request #2414 from lucab/ups/cli-commit-modifier-autoptr
builtins/commit: move commit modifier to auto-cleanup
2021-08-18 09:10:17 -04:00
Luca BRUNO b079c11381
builtins/commit: move commit modifier to auto-cleanup
This reduces the usage of goto cleanup logic by porting the commit
modifier pointer to autoptr.
2021-08-18 09:06:26 +00:00
Luca BRUNO 5a3d5fb86f
builtins/commit: check for conflicting permissions options
This explicitly checks for commit command options asking for both
non-zero UID/GID and canonical permissions at the same time,
which are incompatible.
2021-08-18 08:16:26 +00:00
Jonathan Lebon 0f95e4e5ee ostree/dump: Fix free'ing a static string
Reported-by: Seth Arnold <seth.arnold@canonical.com>
2021-08-03 16:49:49 -04:00
Jonathan Lebon 738831c50b lib/sysroot: Fix error message about creating `/var/lib`
Reported-by: Seth Arnold <seth.arnold@canonical.com>
2021-08-03 16:49:49 -04:00
Jonathan Lebon 75b17937cf lib/sign-dummy: Handle incorrect signatures correctly
We need to check all signatures for one which passes, not just fail on
the first one.

Reported-by: Seth Arnold <seth.arnold@canonical.com>
2021-08-03 16:49:49 -04:00
刘建强 28174970c7 fix: Avoid wild pointers
Pointer command is dangerous if there is no assignment.

Log: Avoid wild pointers
2021-07-28 09:40:37 +08:00
Simon McVittie 5e89318de3 New upstream version 2021.3 2021-07-25 18:38:13 +01:00
Dan Nicholson 81df5c8aba fixup! lib/repo: Add ostree_repo_remote_get_gpg_keys() 2021-07-15 17:03:45 -06:00
Dan Nicholson 814e481fff fixup! bin/remote: Add list-gpg-keys subcommand 2021-07-15 16:25:13 -06:00
Dan Nicholson 30c054b521 fixup! lib/repo: Add ostree_repo_remote_get_gpg_keys() 2021-07-15 16:24:36 -06:00
Dan Nicholson 90a3bda1f8 bin/remote: Include update URLs in list-gpg-keys 2021-07-15 15:50:04 -06:00
Dan Nicholson 27dc5d7d38 lib/repo: Include WKD update URLs in GPG key listing
If the key UID contains a valid email address, include the GPG WKD
update URLs in GVariant returned by ostree_repo_remote_get_gpg_keys().
2021-07-15 15:50:04 -06:00
Dan Nicholson 4fa403aee5 libotutil: Add helper for GPG WKD update URLs
Calculate the advanced and direct update URLs for the key discovery
portion[1] of the OpenPGP Web Key Directory specification, and include
the URLs in the key listing in ostree_repo_remote_get_gpg_keys(). These
URLs can be used to locate updated GPG keys for the remote.

1. https://datatracker.ietf.org/doc/html/draft-koch-openpgp-webkey-service#section-3.1
2021-07-15 15:50:04 -06:00
Dan Nicholson fbff05e28d libotutil: Import implementation of zbase32 encoding
This will be used to implement the PGP Web Key Directory (WKD) URL
generation. This is a slightly cleaned up implementation[1] taken from
the zbase32 author's original implementation[2]. It provides a single
zbase32_encode API to convert a set of bytes to the zbase32 encoding.

I believe this should be acceptable for inclusion in ostree. The license
in the source files is BSD style while the original repo LICENSE file
claims the Creative Commons CC0 1.0 Universal license, which is public
domain.

1. https://github.com/dbnicholson/libbase32/tree/for-ostree
2. https://github.com/zooko/libbase32
2021-07-15 15:50:04 -06:00
Dan Nicholson 74fb0c5f78 bin/remote: Add list-gpg-keys subcommand
This provides a wrapper for the `ostree_repo_remote_get_gpg_keys`
function to show the GPG keys associated with a remote. This is
particularly useful for validating that GPG key updates have been
applied. Tests are added, which checks the
`ostree_repo_remote_get_gpg_keys` API by extension.
2021-07-15 15:50:04 -06:00
Dan Nicholson a50f6d0b9f lib/repo: Add ostree_repo_remote_get_gpg_keys()
This function enumerates the trusted GPG keys for a remote and returns
an array of `GVariant`s describing them. This is useful to see which
keys are collected by ostree for a particular remote. The same
information can be gathered with `gpg`. However, since ostree allows
multiple keyring locations, that's only really useful if you have
knowledge of how ostree collects GPG keyrings.

The format of the variants is documented in
`OSTREE_GPG_KEY_GVARIANT_FORMAT`. This format is primarily a copy of
selected fields within `gpgme_key_t` and its subtypes. The fields are
placed within vardicts rather than using a more efficient tuple of
concrete types. This will allow flexibility if more components of
`gpgme_key_t` are desired in the future.
2021-07-15 15:50:04 -06:00
Dan Nicholson fc073654dc lib/repo: Allow preparing GPG verifier without global keyrings
Currently the verifier decides whether to include the global keyrings
based on whether the specified remote has its own keyring or not. Allow
callers to exclude the global keyrings even when that's not the case.
This will be used in a subsequent commit in order to get the GPG keys
only associated with a remote.
2021-07-15 15:50:04 -06:00
Dan Nicholson c8715c123e lib/repo: Factor out GPG verifier preparation
In order to use the GPG verifier, it needs to be seeded with GPG keys
after instantation. Currently this is only used for verifying data, but
it will also be used for getting a list of trusted GPG keys in a
subsequent commit.
2021-07-15 15:50:04 -06:00
Dan Nicholson dba2cdcbac lib/repo: Factor out GPG verifier key imports
Currently the verifier only imports all the GPG keys when verifying
data, but it would also be useful for inspecting the trusted keys.
2021-07-15 15:50:04 -06:00
Luca BRUNO 38c14b3745
Release 2021.3 2021-07-12 08:38:38 +00:00
Jonathan Lebon 301b52704e ostree-remount: Order before systemd-rfkill.*
The `systemd-rfkill.*` service falls in the category of early things
that need write access to `/var`, so we need to make sure we run before
or it might hit the read-only sysroot.

The long-term fix for this is
https://github.com/ostreedev/ostree/issues/2115.

Closes: https://github.com/coreos/fedora-coreos-tracker/issues/746
2021-06-22 11:22:47 -04:00
Luca BRUNO 70a8f56ce1
lib/commit: respect SOURCE_DATE_EPOCH for commit timestamp
This tweaks `ostree_repo_write_commit` so that it checks for the
envinroment variable `SOURCE_DATE_EPOCH` as a way to override
the current time, which is used as the commit timestamp.

Ref: https://reproducible-builds.org/docs/source-date-epoch/
Ref: https://reproducible-builds.org/specs/source-date-epoch/
2021-06-22 12:15:18 +00:00
Dan Nicholson d9ef76a598 Don't fail build when systemd unit path not defined
In configure the systemd unit path is optional, but in the code it's
assumed to be defined. Add an `#ifdef` that throws an error when it's
not defined like the handling of `HAVE_LIBMOUNT` below it.
2021-06-18 12:35:41 -06:00
Jonathan Lebon c9a318faf4
Merge pull request #2375 from cgwalters/generator-remount
Use generator to enable ostree-remount.service and ostree-finalize-staged.path
2021-06-18 10:25:35 -04:00
Colin Walters 73e3ccc401 Use generator to enable ostree-remount.service and ostree-finalize-staged.path
We struggled for a long time with enablement of our "internal units",
trying to follow the philosophy that units should only be enabled
by explicit preset.

See https://bugzilla.redhat.com/show_bug.cgi?id=1451458
and https://github.com/coreos/rpm-ostree/pull/1482
etc.

And I just saw chat (RH internal on a proprietary system sadly) where
someone hit `ostree-remount.service` not being enabled in CentOS8.

Thinking about this more, I realized we've shipped a systemd generator
for a long time and while its only role until now was to generate `var.mount`,
but by using it to force on our internal units, we don't require
people to deal with presets anymore.

Basically we're inverting things so that "if ostree= is on the kernel
cmdline, then enable our units" and not "enable our units, but have
them use ConditionKernelCmdline=ostree to skip".

Drop the weird gyrations we were doing around `ostree-finalize-staged.path`
too; forking `systemctl start` is just asking for bugs.

So after this, hopefully we won't ever again have to think about
distribution presets and our units.
2021-06-16 09:40:28 -04:00
Christian Kellner f653545833 Fix small typo in ostree-sysroot.c 2021-06-15 23:02:10 +02:00
Colin Walters edf7477ee9 deploy: Warn if we find content in the deployment's /var
This will be ignored, so let's make it very clear
people are doing something wrong.  Motivated by a bug
in a build pipeline that injected `/var/lib/rpm` into an ostree
commit which ended up crashing rpm-ostree because it was an empty db
which it wasn't expecting.

It *also* turns out rpm-ostree is incorrectly dumping content in the
deployment `/var` today, which is another bug.
2021-06-10 07:33:17 -04:00
Dan Nicholson 0cd9dfe815 repo: Use g_new for OstreeRepoAutoLock
GSlice is effectively deprecated and has little to no advantage over
using the system allocator on Linux.
2021-06-07 13:14:05 -06:00
Dan Nicholson 89f4ce2c1d repo: Make locking precondition failures fatal
Use `g_error` and `g_assert*` rather than `g_return*` when checking the
locking preconditions so that failures result in the program
terminating. Since this code is protecting filesystem data, we'd rather
crash than delete or corrupt data unexpectedly.

`g_error` is used when the error is due to the caller requesting an
invalid transition like attempting to pop a lock type that hasn't been
taken. It also provides a semi-useful message about what happened.
2021-06-05 09:15:34 -06:00
Dan Nicholson ccef9784d7 repo: Make locking per-OstreeRepo
Previously each thread maintained its own lock file descriptor
regardless of whether the thread was using the same `OstreeRepo` as
another thread. This was very safe but it made certain multithreaded
procedures difficult. For example, if a main thread took an exclusive
lock and then spawned worker threads, it would deadlock if one of the
worker threads tried to acquire the lock.

This moves the file descriptor from thread local storage to the
`OstreeRepo` structure so that threads using the same `OstreeRepo` can
share the lock. A mutex guards against threads altering the lock state
concurrently.

Fixes: #2344
2021-06-05 09:15:32 -06:00
Dan Nicholson c3ada6fa7a repo: Require lock type in ostree_repo_lock_pop
This simplifies the lock state management considerably since the
previously pushed type doesn't need to be tracked. Instead, 2 counters
are kept to track how many times each lock type has been pushed. When
the number of exclusive locks drops to 0, the lock transitions back to
shared.
2021-06-05 09:07:39 -06:00
Colin Walters 0f36d8c221 repo: Make locking APIs public
Doing anything even somewhat sophisticated requires this;
turns out our own `ostree prune` CLI wants this, e.g.
https://github.com/ostreedev/ostree/issues/2337

Closes: https://github.com/ostreedev/ostree/issues/2286
2021-06-05 09:00:21 -06:00
Timothée Ravier 02527f115e *: rename master to main in tests & examples 2021-05-07 16:55:03 +02:00
Colin Walters 0f3bccf640 pull: Cleanly error out on unknown schemes
Previous to this we'd trip an assertion `abort()` deep in the curl code if e.g.
a user did `ostree remote add foo htttp://...` etc.

Motivated by considering supporting "external remotes" where code outside
ostree does a pull, but we want to reuse the signing verification infrastructure.
2021-04-27 13:11:18 -04:00
Dan Nicholson e660855796 tests: Test without a cache directory by default
Several tests generate summaries and then expect to use the generated
summary immediately. However, this can cause intermittent test failures
when they inadvertantly get a cached summary file. This typically
happens when the test is run on a filesystem that doesn't support user
extended attributes. In that case, the caching code can only use the
last modified time, which only has 1 second granularity. If tests don't
carefully manage the summary modification times or the repo cache then
they are likely subject to races in some test environments.

This introduces an environment variable `OSTREE_SKIP_CACHE` that
prevents the repo from using a cache directory. This is enabled by
default in tests and disabled for tests that are a explicitly trying to
test the caching behavior.

Fixes: #2313
Fixes: #2351
2021-04-19 11:09:44 -06:00
Simon McVittie 3b95643e80 New upstream version 2021.2 2021-04-17 00:55:26 +01:00
Colin Walters 6a72674ec6 Release 2021.2 2021-04-15 13:02:48 -04:00
Simon McVittie 1c67002e63 New upstream version 2021.1 2021-04-14 12:52:43 +01:00
Colin Walters 9b4bffc454 repo: Ensure we set the size for regfile inline
Need to clean up the internal APIs for this.
2021-04-09 22:29:55 +00:00
Colin Walters 6f84aff0ae repo: Add ostree_repo_write_regfile
This API is push rather than pull, which makes it much more
suitable to use cases like parsing a tar file from external
code.

Now, we have a large mess in this area internally because
the original file writing code was pull based, but static
deltas hit the same problem of wanting a push API, so I added
this special `OstreeRepoBareContent` just for writing regular
files from a push API.

Eventually...I'd like to deprecate the pull based API,
and rework things so that for regular files the push API
is the default, and then `write_content_object()` would
be split up into archive/bare cases.

In this world the `ostree_repo_write_content()` API would
then need to hackily bridge pull to push and it'd be
less efficient.

Anyways for now due to this bifurcation, this API only
works on non-archive repositories, but that's fine for
now because that's what I want for the `ostree-ext-container`
bits.
2021-04-09 21:54:44 +00:00
Colin Walters fce69cdf70 repo: Add ostree_repo_write_symlink
Continuation of the addition of `ostree_repo_write_regfile_inline()`.
This will be helpful for ostree-rs-ext and importing from tar, it's
quite inefficient and awkward for small files to end up creating
a whole `GInputStream` and `GFileInfo` and etc. for small files.
2021-04-08 21:10:00 +00:00
Colin Walters 9332955b5f
Merge pull request #2327 from cgwalters/writing-apis
repo: Add ostree_repo_write_regfile_inline
2021-04-08 17:09:11 -04:00
Colin Walters 4e2a14eb0c repo: Add ostree_repo_write_regfile_inline
When working on ostree-ext and importing from tar, it's
quite inefficient and awkward for small files to end up creating
a whole `GInputStream` and `GFileInfo` and etc. for small files.

Plus the gtk-rs binding API to map from `impl Read` to Gio
https://docs.rs/gio/0.9.1/gio/struct.ReadInputStream.html
requires that the input stream is `Send` but the Rust `tar` API
isn't.

This is only 1/3 of the problem; we also need similar APIs
to directly create a symlink, and to stream large objects via
a push-based API.
2021-04-08 14:57:33 +00:00
Colin Walters dfaf314c42 core: Drop unused error handling from object stream helper
I was going to add some new API and I noticed that this function
never returns an error; presumably at one point it did, but
not anymore.  It simplifies the code flow noticeably
to remove that.
2021-04-07 20:01:07 +00:00
Colin Walters b5c21defe9 core: Fix lgtm.com warning about always true `if (bits > 0)`
Since we're not going to change this, let's constant fold the logic
here.
2021-04-07 11:48:35 -04:00
Stefan Berger 81d3017463 rofiles-fuse: Enable support for setting and getting xattrs
Enable support for setting and getting xattrs. Allow modifications
to xattrs only on user.ima xattr.

Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Co-authored-by: Colin Walters <walters@verbum.org>
2021-04-05 17:01:58 -04:00
Colin Walters 57270db9a1 repo: Ensure load_variant_if_exists sets NULL value
Spotted by @jlebon - we usually expect callers to zero, but
let's be explicit.
2021-04-03 16:44:39 -04:00
Colin Walters 0c0e58e06c repo: Fix load_variant_if_exists to return a nullable value
Another introspection fix I hit when trying to use this in Rust.
2021-04-03 16:44:39 -04:00
Felix Krull 47bf29fed3 lib: fix some version tags 2021-03-26 15:59:07 -04:00
Colin Walters e9e4b91120 Release 2021.1 2021-03-23 15:23:55 -04:00
Jonathan Lebon 36a543ceb1 lib/pull: Add some error-prefixing in dirtree scanning
I think these are the paths involved in the error message at
https://github.com/coreos/rpm-ostree/issues/2250.
2021-03-22 16:32:11 -04:00
Kenneth J. Miller cfe313afae Add configure option for unsuffixed GRUB2 commands
GRUB starting with version 2.02 permits the use of the linux and
initrd commands for both EFI boot in *-efi installations, as
well as 32-bit BIOS boot in i386-pc installations.

This makes the use of the -16 and -efi suffixes for BIOS and EFI
boot obsolete on systems with a modern GRUB installation.

The --with-modern-grub configure flag makes ostree use the
unsuffixed linux/initrd commands when generating a GRUB
configuration, while defaulting to the previous behaviour for
users not wanted this option.
2021-03-18 14:31:28 +01:00
Colin Walters 1b28e6041c sysroot: Add _require_booted_deployment() API
This is a common pattern that is replicated both in our code
and in rpm-ostree a lot.  Let's add a canonical API.
2021-03-17 19:55:56 +00:00
Colin Walters 857587615d Add an API+CLI to inject metadata for bootable OSTree commits
I was doing some rpm-ostree work and I wanted to compare two
OSTree commits to see if the kernel has changed.  I think
this should be a lot more natural.

Add `ostree commit --bootable` which calls into a new generic
library API `ostree_commit_metadata_for_bootable()` that
discovers the kernel version and injects it as an `ostree.linux`
metadata key.  And for extra clarity, add an `ostree.bootable`
key.

It's interesting because the "core" OSTree layer is all about
generic files, but this is adding special APIs around bootable
OSTree commits (as opposed to e.g. flatpak as well as
things like rpm-ostree's pkgcache refs).

Eventually, I'd like to ensure everyone is using this and
hard require this metadata key for the `ostree admin deploy`
flow - mainly to prevent accidents.
2021-03-12 19:01:42 +00:00
Phaedrus Leeds 19577522f8 Fix translation of file:// URIs into paths
Currently if a file path contains a special character such as '\', and
that character is encoded into a file:// URI that is passed to
ostree_repo_pull_with_options(), the percent encoding will remain in the
path passed to g_file_new() (in the case of backslash %5C) and the pull
will then fail with a file not found error. This is an important edge
case to handle because by default on many Linux distributions a
filesystem with no label is mounted at a path based on its UUID, and
this is then passed to systemd-escape by Flatpak (when
--enable-auto-sideloading was used at compile time) to create a symbolic
link such as this which contains backslashes:

$ ls -l /run/flatpak/sideload-repos/
total 0
lrwxrwxrwx 1 mwleeds mwleeds 55 Mar  9 14:21
'automount-run-media-mwleeds-29419e8f\x2dc680\x2d4e95\x2d9a31\x2d2cc907d421cb'
-> /run/media/mwleeds/29419e8f-c680-4e95-9a31-2cc907d421cb

And Flatpak then passes libostree a file:// URI containing that path, to
implement sideloading (pulling content from the USB drive).

This results in an error like:

Error: While pulling app/org.videolan.VLC/x86_64/stable from remote
flathub:
/run/flatpak/sideload-repos/automount-run-media-mwleeds-29419e8f%5Cx2dc680%5Cx2d4e95%5Cx2d9a31%5Cx2d2cc907d421cb/.ostree/repo:
opendir(/run/flatpak/sideload-repos/automount-run-media-mwleeds-29419e8f%5Cx2dc680%5Cx2d4e95%5Cx2d9a31%5Cx2d2cc907d421cb/.ostree/repo):
No such file or directory

This patch avoids such errors by using g_file_new_for_uri() instead of
g_file_new_for_path(), so that GLib handles the %-decoding for us.

Bug report by user:
https://community.endlessos.com/t/can-not-install-vlc-from-usb-drive-3-9-3/16353
2021-03-10 10:11:06 -08:00
Phaedrus Leeds 2709da4360 pull: Fix some whitespace and a comment 2021-03-10 10:01:04 -08:00
Philip Withnall 60881b75ec ostree-repo-pull: Fix a leak of the summary data if loading from cache
If the `summary_sig_not_modified` branch is taken above, both
`signatures` and `summary` are loaded from the cache. This makes the
`_ostree_repo_load_cache_summary_if_same_sig()` call below redundant (it
checks `signatures` matches the file it was just loaded from, and then
loads `summary` again) — but that call also currently overwrites
`summary` without clearing the old value.

Fix this by only making that call if `signatures` was retrieved, but the
server said the local `summary` cache was invalid.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2021-03-09 11:47:23 +00:00
OpenShift Merge Robot ba407750b1
Merge pull request #2287 from jlebon/pr/deployment-parsing-comments
lib/sysroot: Add comments and debug statements around sysroot parsing
2021-03-03 14:36:52 -05:00
Jonathan Lebon 02b6197924 lib/sysroot: Add comments and debug statements around sysroot parsing
Was looking at this code more closely today to investigate issues
related to bootlink mismatches (#2283).
2021-03-02 15:36:27 -05:00
Colin Walters 975496d241 deploy: Add subbootversion to journal
To help debug an issue we've seen where `/boot` isn't
in sync with the `/ostree/boot` dir, let's log to the journal
what we're doing.
2021-02-28 14:47:58 +00:00
Colin Walters 093c63cd46 refs: Make ostree_repo_resolve_rev{,_ext}() use (nullable)
We have an `allow_noent` boolean that controls this, but
were missing the `(nullable)` annotation, so the Rust bindings
panic when the ref doesn't exist instead of being `Option<GString>`.
2021-02-19 01:11:43 +00:00
Colin Walters 5a5f54a459 deltas: Fix leak of matches
Found by ASAN.
2021-02-02 21:26:43 +00:00
OpenShift Merge Robot c216a438b8
Merge pull request #2267 from dbnicholson/pull-depth-fixes
Pull depth fixes
2021-01-12 17:34:03 -05:00
Dan Nicholson d7f2955f37 pull: Fix local pull with depth and truncated source history
The local pull path was erroring on any missing commit, but that
prevents a depth pull where the source repo has truncated history. As in
the remote case, this also tries to pull in a tombstone commit if the
source repo supports it.

Fixes: #2266
2021-01-12 14:19:01 -07:00
Dan Nicholson 20047ff1fe pull: Error on depth pull with missing head commit
When pulling with depth, missing parent commits are ignored. However,
the check was applying to any commit, which means that it would succeed
even if the requested commit was missing. This might happen on a
corrupted remote repo or when using ref data from a stale summary.

To achieve this, the semantics of the `commit_to_depth` hash table is
changed slightly to only ever includes parent commits. This makes it
easy to detect when a parent commit is being referenced (although there
is a minor bug there when multiple refs are being pulled) while keeping
references to commits that need their `commitpartial` files cleaned up.
It also means that the table is only populated on depth pulls, which
saves some memory and processing in the common depth=0 case.

Fixes: #2265
2021-01-12 14:19:01 -07:00
OpenShift Merge Robot bdca64340b
Merge pull request #2263 from cgwalters/createat-nullable
repo: Make ostree_repo_create_at take nullable options
2021-01-12 03:38:46 -05:00
Colin Walters 125c83850a repo: Make ostree_repo_create_at take nullable options
Hit this when trying to use the Rust bindings.
2021-01-12 01:20:23 +00:00
Colin Walters 441233b51c repo: Move fsverity bits to ostree-repo-verity.c
This file will get larger when we start doing more with fsverity.
2021-01-11 14:36:40 +00:00
Colin Walters 9a526bbaa5 sysroot: Handle ro /boot but rw /sysroot
The recent change in https://github.com/coreos/fedora-coreos-config/pull/659
broke some of our tests that do `mount -o remount,rw /sysroot` but
leave `/boot` read-only.

We had code for having `/boot` read-only before `/sysroot` but
in practice we had a file descriptor for `/sysroot` that we opened
before the remount that would happen later on.

Clean things up here so that in the library, we also remount
`/boot` at the same time we remount `/sysroot` if either are readonly.

Delete the legacy code for remounting `/boot` rw if we're not in
a mount namespace.  I am fairly confident most users are either
using the `ostree` CLI, or they're using the mount namespace.
2021-01-10 13:49:44 +00:00
Colin Walters a1c0cffeb3 sysroot: Also maintain canonical boot_fd
Just like we hold a fd for `/sysroot`, also do so for `/boot`
instead of opening and closing it in a few places.

This is a preparatory cleanup for further work.
2021-01-10 13:46:11 +00:00
Colin Walters 10556a95b4 main: Unconditionally set up mount namespace
I was being very conservative initially here, but I think it's
really safe to just unconditionally set up the mount namespace.

This avoids having to check twice for a read-only `/sysroot`
(once in the binary and once in the library).
2021-01-10 13:40:52 +00:00
Dan Nicholson 92a484d278 pull: Use GNU coding style 2020-12-18 09:13:38 -07:00
Dan Nicholson 4db2ba0eb1 pull: Allow disabling commit binding verification
In some cases such as backups or mirroring you may want to pull commits
from one repo to another even if there commits that have incorrect
bindings. Fixing the commits in the source repository to have correct
bindings may not be feasible, so provide a pull option to disable
verification.

For Endless we have several repositories that predate collection IDs and
ref bindings. Later these repositories gained collection IDs to support
the features they provide and ref bindings as the ostree tooling was
upgraded. These repositories contain released commits that were valid to
the clients they were targeting at the time. Correcting the bindings is
not really an option as it would mean invalidating the repository
history.
2020-12-17 14:07:08 -07:00
Simon McVittie 2b5dac2d58 New upstream version 2020.8 2020-11-19 14:40:26 +00:00
OpenShift Merge Robot 874f2ca625
Merge pull request #2242 from stb-tester/tarball-import-xattrs
ostree commit --tree=tar: Import xattrs from tarballs
2020-11-17 22:38:32 +01:00
William Manley a88d2f5f7b ostree commit --tree=tar: Import xattrs from tarballs
If you specify an `xattr_callback` the xattrs will still be taken from
there for now.
2020-11-17 16:54:32 +00:00
Luca BRUNO 3e289b1934
Release 2020.8 2020-11-17 10:32:57 +00:00
OpenShift Merge Robot fdd3f7fcdf
Merge pull request #2232 from cgwalters/deploy-docs
deployment: Add a bunch of docs and fix annotations
2020-11-12 19:40:28 +01:00
OpenShift Merge Robot d046631a0f
Merge pull request #2235 from cgwalters/checkout-some-new-style
bin/checkout: Port some to new style
2020-11-12 09:34:44 +01:00
Colin Walters 8fbf2c5b80 deployment: Ensure query_deployments_for returns nullable values
Since that's a common case; hit this while working on rpm-ostree
code using the ostree-rs bindings.
2020-11-11 22:01:39 +00:00
Colin Walters 43913178a7 deployment: Add a bunch of docs and fix annotations
We were missing docs for these, also add some nullability annotations.
Motivated by using these from the Rust bindings.
2020-11-11 22:01:11 +00:00
Colin Walters f7be2a3e4a bin/checkout: Port some to new style
I was reading this code for unrelated reasons and noticed it
was still old style; port most (but not all) to new style.
2020-11-11 21:57:36 +00:00
OpenShift Merge Robot 34390a53b8
Merge pull request #2233 from cgwalters/repo-annotations
sysroot: Fix up some GI nullable annotations
2020-11-09 18:56:24 +00:00
Colin Walters bf8c4c7e32 sysroot: Fix up some GI nullable annotations
Hit `ostree_sysroot_repo()` shouldn't be nullable while using
the ostree Rust bindings.
2020-11-06 20:06:26 +00:00
OpenShift Merge Robot e43d445b5b
Merge pull request #2228 from jlebon/pr/drop-volatile
Drop use of `volatile`
2020-11-03 17:36:14 -05:00
OpenShift Merge Robot f62c7bae2e
Merge pull request #2230 from alexlarsson/pull-dont-save-passed-in-summary
pull: Don't save into cache passed in GByte summaries
2020-11-03 12:35:06 -05:00
Alexander Larsson 52463686af pull: Don't save into cache passed in GByte summaries
The cache shouldn't be affected by the user passing in some other
summary as it may not be the "official one".

I ran into this in flatpak where the passed summary was correct, but
the re-saving of the cache updated the mtime of the cached file which
led to later http If-Modified-Since calls failing to update.
2020-11-03 11:48:33 +01:00
Dan Nicholson 3e527d9447 lib/deltas: Annotate from checksum as nullable
Without this you can't create a scratch delta from GI. While here,
switch the deprecated allow-none annotations to nullable.
2020-11-02 16:42:30 -07:00
Jonathan Lebon f895cf4fd2 Drop use of `volatile`
As detailed in
gitlab.gnome.org/GNOME/glib/-/issues/600#note_877282, volatile
isn't actually needed in these contexts because the atomic operations
already give us strong enough guarantees. In GCC 11, this triggers a
diagnostic due to the volatile qualifier getting dropped anyway.

There is a WIP to do the same in glib:
https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1719

This obsoletes this downstream patch:
https://src.fedoraproject.org/rpms/ostree/c/b8c5a6fb
2020-11-02 14:53:26 -05:00
OpenShift Merge Robot 64e09f46b8
Merge pull request #2226 from alexlarsson/fix-gir-arg
ostree_repo_gpg_sign_data: Fix API doc argument name
2020-10-28 13:31:50 -04:00
Alexander Larsson 2f78441bea ostree_repo_gpg_sign_data: Fix API doc argument name
I got:
src/libostree/ostree-repo.c:5232: Warning: OSTree: ostree_repo_gpg_sign_data: unknown parameter 'out_signature' in documentation comment, should be 'out_signatures'
2020-10-28 15:53:18 +01:00
OpenShift Merge Robot 7bc53f0063
Merge pull request #2224 from stb-tester/bootloader-refactorings
Bootloader probing and construction refactoring
2020-10-28 06:39:57 -04:00
Jonathan Lebon 8717608c7e lib/fetch-curl: Unref timeout source
The timeout timer should always be one-shot, so let's just always
destroy it in the callback. The main context has its own ref on it, so
it won't be freed behind its back.

This *should* fix a leak that was brought up in
https://bugzilla.redhat.com/show_bug.cgi?id=1891761.

Reported-by: Milan Crha <mcrha@redhat.com>
2020-10-27 12:01:01 -04:00
Jonathan Lebon 40edc33ef3 lib/fetcher-curl: Use G_SOURCE_REMOVE instead of FALSE
They're equivalent, though I prefer the former because it's more
descriptive and it makes it really obvious that it's a `GSource`
callback.
2020-10-27 11:57:00 -04:00
William Manley 663c5b41a3 fixup! Refactor `ostree_sysroot_query_bootloader` 2020-10-27 13:24:46 +00:00
William Manley 631528c87b fixup! Refactor: Centralise choosing the appropriate bootloader 2020-10-27 12:35:29 +00:00
William Manley a8dce46b5f Refactor `ostree_sysroot_query_bootloader`
This is more regular, so will make it easier to add more bootloader types
in the future.
2020-10-26 23:51:11 +00:00
William Manley 31acd2ef99 Add support for explicitly requesting any specific bootloader type
...with the `sysroot.bootloader` configuration option.  This can be useful
when converting a system to use `ostree` which doesn't currently have a
bootloader configuration that `ostree` can automatically detect, and is
also useful in combination with the `--sysroot` option when provisioning a
rootfs for systems other than the one you're running `ostree admin deploy`
on.
2020-10-26 23:51:11 +00:00
William Manley 9482ecfe5a Refactor: sysroot.bootloader: Store enum value rather than string
It's easier to extend and it centralises the config parsing.  In other
places we will no longer need to use `g_str_equal` to match these values,
a `switch` statement will be sufficient.
2020-10-26 23:51:11 +00:00
William Manley 062df6ee81 Refactor: Centralise choosing the appropriate bootloader
In preparation for enhancing `_ostree_sysroot_query_bootloader`
2020-10-26 23:51:11 +00:00
William Manley 5e223f2962 ostree_repo_get_bootloader: Document transfer none
I think this may affect bindings too.
2020-10-26 23:51:11 +00:00
Alexander Larsson 654f3d959a ostree pull: Add more g_debug spew around fetching deltas
This is useful to debug what is happening when downloading via deltas.
2020-10-23 13:55:33 +02:00
Alexander Larsson 8cd796f3f1 Add ostree_repo_gpg_sign_data()
This is similar to ostree_sign_data() but for the old gpg code.
Flatpak will need this to reproduce a signed summary.
2020-10-23 13:55:33 +02:00
Alexander Larsson 125ed2b199 pull: Only download summary if we need it for the pull operation
If we have a commit id for all the refs we're pulling, and if we
don't need the summary to list all the refs when mirroring then the
only reason to download the summary is for the list of deltas.

With the new "indexed-deltas" property in the config file (and mirrored
to the summary file) we can detect when we don't need the summary for
deltas and completely avoid downloading it then.
2020-10-23 13:55:33 +02:00
Alexander Larsson 6c8e6539e2 deltas: Set `indexed-deltas` key in the config and summary
Clients can use these during pull and avoid downloading the summary if
needed, or use the indexed-deltas instead of relying on the ones in
the summary which may be left out.
2020-10-23 13:06:46 +02:00
Alexander Larsson 0984ff8471 deltas: Take a shared repo lock while reindexing deltas
This ensures we're not racing with a prune operation that can be removing
the delta indexes we're relying on.
2020-10-23 13:06:46 +02:00
Alexander Larsson df7f07fc6c deltas: Use delta indexes when pulling
If there is no delta index in the summary, try to fetch the
delta index for the commit we're going to and use that to find the
delta (if any).
2020-10-23 13:06:42 +02:00
Alexander Larsson 625606a7ec deltas: Add CLI ops to list and reindex delta-indexes 2020-10-23 12:30:08 +02:00
Alexander Larsson c304703e1d deltas: Make ostree_repo_static_delta_reindex() public
It is useful to be able to trigger this without having to regenerate
the summary. For example, if you are not using summaries, or ar generating
the summaries yourself.
2020-10-23 12:30:08 +02:00
Alexander Larsson 024ef1d756 deltas: Add and document no-deltas-in-summary config option
By default this is FALSE to keep existing clients working.
2020-10-23 12:30:08 +02:00
Alexander Larsson effde3d513 deltas: Update delta indexes when updating summary
When we update the summary file (and its list of deltas) we also update
all delta indexes. The index format is a single `a{sv}` variant identical
to the metadata-part of the summary with (currently) only the
`ostree.static-deltas` key.

Since we expect most delta indexes to change rarely, we avoid
unnecessary writes when reindexing. New indexes are compared to
existing ones and only the changed ones are written to disk.  This
avoids unnecessary write load and mtime changes on the repo server.
2020-10-23 12:30:08 +02:00
Alexander Larsson 8e1f199dd4 deltas: Add ostree_repo_list_static_delta_indexes() function
This lists all the available delta indexes.
2020-10-23 12:30:08 +02:00
Alexander Larsson 87e564eeaa deltas: Add _ostree_get_relative_static_delta_index_path()
This gets the subpath for a delta index file, which is of the form
"delta-indexes/$commit.index", that contains all the deltas going
to the particular commit.
2020-10-23 12:30:08 +02:00
OpenShift Merge Robot f8f6464580
Merge pull request #2205 from pwithnall/etags-and-last-modified
Add support for ETag and Last-Modified headers for summary and summary.sig
2020-10-22 18:20:23 -04:00
Philip Withnall 0974a7faf1 tests: Split RFC 2616 date parsing code out and add tests
This makes it testable, and increases its test coverage too 100% of
lines, as measured by `make coverage`.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2020-10-22 21:03:34 +01:00
Philip Withnall 92215025aa ostree/trivial-httpd: Add Last-Modified/ETag support
This is basic support for the
Last-Modified/ETag/If-Modified-Since/If-None-Match headers. It’s not
high performance, and doesn’t support all of the related caching
features (like the If-Match header, etc.).

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2020-10-22 21:03:34 +01:00
Philip Withnall 0e3add8d40 lib/pull: Hook up HTTP caching headers for summary and summary.sig
As `summary` and `summary.sig` aren’t immutable, HTTP requests to
download them can be optimised by sending the `If-None-Match` and
`If-Modified-Since` headers to avoid unnecessarily re-downloading them
if they haven’t changed since last being checked.

Hook them up to the new support for that in the fetcher.

The `ETag` and `Last-Modified` for each file in the cache are stored as
the `user.etag` xattr and the mtime, respectively. For flatpak, for
example, this affects the cached files in
`~/.local/share/flatpak/repo/tmp/cache/summaries`.

If xattrs aren’t supported, or if the server doesn’t support the caching
headers, the pull behaviour is unchanged from before.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2020-10-22 21:03:34 +01:00
Philip Withnall 407c683524 libostree: Add support for ETag and Last-Modified headers
Add support in the soup and curl fetchers to send the `If-None-Match`
and `If-Modified-Since` request headers, and pass on the `ETag` and
`Last-Modified` response headers.

This currently introduces no functional changes, but once call sites
provide the appropriate integration, this will allow HTTP caching to
happen with requests (typically with metadata requests, where the data
is not immutable due to being content-addressed). That should reduce
bandwidth requirements.

Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
2020-10-22 21:03:34 +01:00
OpenShift Merge Robot 95a6d15145
Merge pull request #2222 from fkrull/patch-v2020.7
lib: add missing GI annotation
2020-10-20 16:48:37 -04:00
Felix Krull 4183b68e70 lib: fix GI parameter tags 2020-10-17 22:14:09 +02:00
Jonathan Lebon dec9eab203 ostree-prepare-root: print st_dev and st_ino as 64-bit ints
This matches what systemd does and should work fine on all platforms.

Possibly resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1888436
2020-10-15 12:26:47 -04:00
Simon McVittie 803ac2144f New upstream version 2020.7 2020-10-14 12:56:10 +01:00
OpenShift Merge Robot 26b98ebc56
Merge pull request #2215 from cgwalters/fd-transfer
deploy: Remove (transfer none) from fd arg
2020-10-14 00:45:13 +02:00
Colin Walters b4e7ab014e deploy: Remove (transfer none) from fd arg
GI complains.  And in general one needs to assume that file
descriptors aren't stolen.
2020-10-13 17:40:11 -04:00
Colin Walters 326b8b13c6 Post-release version bump 2020-10-13 17:38:19 -04:00
Colin Walters 32a3a12973 Release 2020.7 2020-10-13 14:31:26 -04:00
OpenShift Merge Robot a13509ee7d
Merge pull request #2213 from alexlarsson/summary-bugfixes
Various fixes from the indexed-summaries branch
2020-10-08 15:18:34 -04:00
Alexander Larsson 2e9db809b9 signatures: Fix leak in _sign_detached_metadata_append()
This needs to ref_sink the returned variant, as it is used with g_autoptr
in the callers.
2020-10-08 14:10:04 +02:00
Alexander Larsson f821cdb89e fetch_summary_with_options: Fix n-network-retries option parsing
"&u" is not a valid gvariant format string, it should just be "u".
2020-10-08 14:09:16 +02:00
Alexander Larsson 6ed1066ef0 ostree_repo_find_remotes_async: Fix leak of summary
We were creating a GVariant from a GBytes and storing it in an
g_autoptr without ref_sinking it.
2020-10-08 14:08:51 +02:00
Jonathan Lebon b3dc074f5e lib/deploy: Don't leak fd when checksumming dtbs
Likely the root of https://bugzilla.redhat.com/show_bug.cgi?id=1886149.
2020-10-07 14:47:34 -04:00
OpenShift Merge Robot 9f98aa9223
Merge pull request #2198 from cgwalters/no-hardlink-zerosize
checkout: Don't hardlink zero sized files
2020-10-05 22:06:38 +02:00
OpenShift Merge Robot 06a77bfd69
Merge pull request #2155 from jlebon/pr/add-initrds
lib/deploy: Add support for overlay initrds
2020-10-02 16:41:01 -04:00
Colin Walters 558720e7aa checkout: Don't hardlink zero sized files
Alternative to https://github.com/ostreedev/ostree/pull/2197

Python's (usually) zero-sized `__init__.py` files can provoke
us hitting the hardlink limits on some filesystems (`EMLINK`).
At least one Fedora rpm-ostree user hit this.

The benefits of hardlinking here are quite marginal; lots
of hardlinks can behave suboptimally in particular filesystems
like BTRFS too.

This builds on prior code which made this an option, introduced
in 673cacd633
Now we just do it uncondtionally.

Also this provoked a different bug in a very obscure user mode checkout
case; when the "real" permissions were different from the "physical"
permissions, we would still hardlink.  Fix the test case for this.
2020-10-01 16:47:07 -04:00
Philip Withnall 23bdc4e5df ostree/dump: Fix a memory leak
Re-using the `refs` variable for the main list of refs, plus the
iterated lists, meant that the main list was never freed (although all
the iterated ones were freed correctly).

Fix this by using two variables rather than reusing the one.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-10-01 11:06:56 +01:00
Philip Withnall 8d09a1a8ea lib/pull: Read mode and tombstone options from summary file if possible
Otherwise, fall back to downloading and reading them from the `config`
file. See the previous commit for details.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Fixes: #2165
2020-10-01 11:06:56 +01:00
Philip Withnall 206f1d3a13 lib/repo: Add mode and tombstone config options to the summary file
Currently, they are set in the `config` file and cause that to be
downloaded on every pull. Given that the client is already pulling the
`summary` file, it makes sense to avoid an additional network round trip
and cache those options in the `summary` file.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Helps: #2165
2020-10-01 11:06:56 +01:00
Jonathan Lebon 81b13da8e3 lib/deploy: Add support for overlay initrds
In FCOS and RHCOS, the need to configure software in the initramfs has
come up multiple times. Sometimes, using kernel arguments suffices.
Other times, it really must be a configuration file. Rebuilding the
initramfs on the client-side however is a costly operation. Not only
does it add complexity to the update workflow, it also erodes a lot of
the value obtained from using the baked "blessed" initramfs from the
tree itself.

One elegant way to address this is to allow specifying multiple
initramfses. This is supported by most bootloaders (notably GRUB) and
results in each initrd being overlayed on top of each other.

This patch allows libostree clients to leverage this so that they can
avoid regenerating the initramfs entirely. libostree itself is agnostic
as to what kind and how much data overlay initrds contain. It's up to
the clients to enforce such boundaries.

To implement this, we add a new ostree_sysroot_stage_overlay_initrd
which takes a file descriptor and returns a checksum. Then users can
pass these checksums when calling the deploy APIs via the new array
option `overlay_initrds`. We copy these files into `/boot` and add them
to the BLS as another `initrd` entry.
2020-09-30 13:29:32 -04:00
Jonathan Lebon 40fea4c443 lib/deploy: Add deploy/stage APIs with options
And make the `override_kernel_argv` one of those options. This is mostly
a mechanical move here, no functional change otherwise.

Prep for adding a new option.
2020-09-30 13:29:32 -04:00
Jonathan Lebon f7500bb703 lib/bootconfig: Add support for multiple initrd keys
Prep for actually teaching the rest of the codebase about this.

We keep the primary initrd in the `options` hash table for backwards
compatibility.
2020-09-30 13:29:32 -04:00
Jonathan Lebon f04e5d047d lib: Minor versioning related fixes
Fix/add the `Since` marker to the new static delta APIs, and update the
symbol versioning templates/comments.
2020-09-25 15:30:20 -04:00
OpenShift Merge Robot fad95167ce
Merge pull request #2203 from cgwalters/delta-sign-cleanups
delta: Some minor code style fixups
2020-09-25 13:57:33 +00:00
OpenShift Merge Robot e6f673a8ed
Merge pull request #2202 from cgwalters/bootcsum-dtb
deploy: Remove deployment bootcsum assertion
2020-09-24 22:14:31 +00:00
Colin Walters 9198fa040c delta: Some minor code style fixups
- Remove some unused variables
- Switch to declare-and-initialize with others
- Fix some indentation from 4 spaces to 2 (GNU style)
2020-09-24 22:12:56 +00:00
Colin Walters aa2a2783ea deploy: Remove deployment bootcsum assertion
When support for devicetree was added, it created a problem
because old and new ostree versions would compute different
checksums for the "boot data".  The scenario here is:

- Have system with ostree < 2020.4
- Reboot into system with ostree 2020.5
- Try to perform an operation that would retain
  that previous booted deployment (common)

Currently ostree iterates over all the deployments
that will be retained and calls `install_deployment_kernel()`,
even for the booted one (which is a bit silly), but
just to verify that all boot data for the targeted
deployments are installed.

This then re-computes the checksum and we'd trip this
assertion.

In practice though, we don't strictly require them to match;
the only thing that will happen if they don't is that we'll
end up with another copy of the kernel/initramfs - and
that only temporarily until the previous deployment
gets GC'd.

Longer term, I think what we really want to do anyways
is probably closer to like a little ostree repo for `/boot`
so that we can e.g. still hardlink kernels there even if
the initramfs changes, or hardlink both kernel/initramfs
if just the devicetree changes, etc.

Closes: https://github.com/ostreedev/ostree/issues/2154
2020-09-24 21:24:48 +00:00
OpenShift Merge Robot 00df896550
Merge pull request #1985 from fdanis-oss/wip/fda/sign_delta_metadata
Static-delta's superblock signature support
2020-09-24 14:21:16 +00:00
Colin Walters 3441a48c58 checkout: Ensure copies of unreadable usermode checkouts are readable
The extreme special case of "zero mode" files like `/etc/shadow`
comes up again.  What we want is for "user mode" checkouts to
override it to make the file readable; otherwise when operating
as non-root without `CAP_DAC_OVERRIDE` it becomes very difficult
to work with.

Previously, we were hardlinking these files, but then it intersects
with *another* special case around zero sized files, which is
*also* true for `/etc/shadow`.

Trying to avoid hardlinking there unveiled this bug - when
we go to do a copy checkout, we need to override the mode.
2020-09-16 15:14:54 +00:00
Frédéric Danis 2e97f5659f bin/static-delta: Add signature parameters to apply-offline
This allows to check the delta signature before applying it.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis fb1faf17d6 lib/deltas: Check signed delta in execute_offline
Add a new function `ostree_repo_static_delta_execute_offline_with_signature`
which takes a signature engine to verify the delta before applying it.
The `ostree_repo_static_delta_execute_offline` is just a wrapper to this
new function, passing a NULL signature engine.
When this function is called without signature engine, but with a sign
delta, it will only fails if `sign-verify-deltas` is set to true in repo
core options.

This commits move signature existence check and delta signature
verification to share common parts between existing APIs and the new
function.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis bf0c09ffe1 lib/deltas: Support signed delta in dump
This checks if the static delta file is signed or not to be able to
correctly get the superblock to dump.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 0c48423c26 lib/deltas: Support signed delta in execute_offline
This checks if the static delta file is signed or not to be able to
correctly get the superblock to apply.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 512db0435c bin/static-delta: Add command to verify delta signature
Add new "static-delta verify" sub-command.
This supports multiple keys to verify the static-delta file.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 02a19b2c96 lib/deltas: Add signature check API for static-delta superblock
This retrieves the signatures and pass the static delta block as an array
of bytes to ostree_sign_data_verify().

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 92efbc00d8 bin/static-delta: Add support to sign superblock
Add signing ability to "static-delta generate" builtin.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 46667567c5 lib/deltas: Add inline signature for static-delta superblock
While the commits contained in the single static-delta file are signed so
we can check them and operate on trusted data, the superblock isn't signed
in any way, so it end up operating on untrusted data to:
 1. actually find where the trusted data is, and
 2. check whether the update is fit for the current device by looking at
    the collection id stored in the metadata

This commit generates signatures of all static data, and concatenate them
to the existing static delta format, i.e. as a GVariant layout `a{sv}ay`
where
 - a{sv}: signatures
 - ay: existing delta variant

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Phaedrus Leeds c4f26bfdc8 Avoid shadowing local variables
This should help with code readability.

Fixes https://github.com/ostreedev/ostree/issues/2194
2020-09-13 10:08:02 -07:00
Colin Walters 779a901780 commit: Tighten scope of two variables
Prep for adding `-Wshadow` fixes.
2020-09-13 14:58:24 +00:00
Alexander Larsson b7d1a9746b Update the symbols files to match that we're now on 2020.6 2020-09-11 12:35:32 +02:00
Alexander Larsson da853a1783 ostree-repo-pull.c: Extract mirrorlist generation to helper
This code was duplicated in 3 places, so move it to a single place
to clean things up.
2020-09-11 12:03:29 +02:00
Alexander Larsson 155b215cd8 Minor cleanup of _ostree_repo_remote_new_fetcher()
Instead of open coding the extra_headers and append_user_agent
setting everywhere we do this in the constructor.
2020-09-11 12:03:28 +02:00
Alexander Larsson 3957bff0cb Inline repo_remote_fetch_summary
This was only used in one place, and (especially with the simplification
with GMainContextPopDefault) and the one caller doesn't really do
much more than call the helper. Additionally, what little it does (saving
the result in the cache) is inherently tied to how the helper work,
and will become even more so when we support summary indexes.

This is a preparatory cleanup for supporting summary indexes. It
doesn't change any behaviour and passes make check on its own.
2020-09-11 12:03:28 +02:00
Alexander Larsson 4b9e712e82 repo_remote_fetch_summary: Use GMainContextPopDefault
This allows us to drop the "goto out" use and clean up this function.
2020-09-11 12:03:28 +02:00
Alexander Larsson c7df4317bd Add g_autoptr helper for pushing a thread default main context
This happens in a bunch of places, and currently each time it does
we have to use "goto out" style cleanups, which just isn't looking
very nice.
2020-09-11 12:03:28 +02:00
Alexander Larsson f74bc8dd3d fetch_summary_with_options: drop unnecessary "goto out" use 2020-09-11 12:03:28 +02:00
Alexander Larsson 1f1ef4c989 Break out the signature verification code into a helper function
This changes nothing in the behaviour, but we want to later re-use
this when we also verify the summary index.
2020-09-11 12:03:28 +02:00
Alexander Larsson 598adc457f deltas: Break out _ostree_repo_static_delta_superblock_digest() helper
This loads and makes a digest for a delta superblock. The previous
code was used when generating the deltas section in the summary
file. This changes nothing, but is in preparation for using similar
formats in a separate delta index file.
2020-09-11 12:03:28 +02:00
Alexander Larsson 32014d99e6 Add and use ot_checksum_bytes helper
This removes some duplicated code (and will be use even more later).
2020-09-11 12:00:05 +02:00
Alexander Larsson dddb449d2c pull: Actually mmap summary files
The change in cbf1aca1d5c08d2f40832d16670484ba878d95fb actually
only mmaps the signature file, not the summary. This change makes
use mmap both, as well as extract the cache loading into a helper
function that we will later use in more places.
2020-09-11 12:00:02 +02:00
Alexander Larsson 85accb84e8 pull: Break out _ostree_repo_save_cache_summary_file() helper
This is a minor cleanup as its just called twice from
_ostree_repo_cache_summary(). However, later code will need it in more
places.
2020-09-11 11:53:38 +02:00
Alexander Larsson bb2649a8c0 Fix leak when signing
_ostree_detached_metadata_append_gpg_sig() was returning a floating
ref, but all users were using g_autoptr. Fix it by adding a ref-sink.
2020-09-11 11:45:33 +02:00
Alexander Larsson 74bae256fe list-deltas: Don't break on non-subdir entries
ostree_repo_list_static_delta_names() tried to validate that
any second-level directory element was a directory, but there was
a cut-and-paste issue, and it used `dent->d_type` instead
of `sub_dent->d_type`.

This fixes the code, but all old ostree versions will break if
there are non-directories in a subdirectory of the deltas directory
in the repo, so be wary.
2020-09-11 11:45:24 +02:00
Colin Walters a1bd29f245 deploy: Add some error prefixing around xattr setting
Looking at
https://github.com/coreos/coreos-assembler/issues/1703
a user is getting a bare:
`error: fsetxattr: Permission denied`

I don't think it's these code paths since a deploy
isn't happening but on inspection I noticed we didn't
have error prefixing here.
2020-09-09 12:34:44 +00:00
Simon McVittie 12212491d5 New upstream version 2020.6 2020-09-08 10:53:08 +01:00
Jonathan Lebon a7a751b69f ostree-remount: Remount /etc rw if needed
When we remount `/sysroot` as read-only, we also make `/etc` read-only.
This is usually OK because we then remount `/var` read-write, which also
flips `/etc` back to read-write... unless `/var` is a separate
filesystem and not a bind-mount to the stateroot `/var`.

Fix this by just remounting `/etc` read-write in the read-only sysroot
case.

Eventually, I think we should rework this to set everything up the way
we want from the initramfs (#2115). This would also eliminate the window
during which `/etc` is read-only while `ostree-remount` runs.
2020-08-28 14:16:46 -04:00
Jonathan Lebon b3c7b059ea ostree-prepare-root: Fix /etc bind mount
We were bind-mounting the initramfs' `/etc` (to itself) instead of the
target deployment `/etc` (to itself). Since we're already `chdir`'ed
into it, we can just drop the leading slash.
2020-08-28 14:16:29 -04:00
Felix Krull f4d0b17080 lib: mark out parameters as out parameters 2020-08-26 22:32:47 +02:00
Felix Krull d5b8929017 lib: add some missing version tags 2020-08-26 22:32:47 +02:00
Matt Bilker dac2ad288f Fix mkinitcpio with newer systemd versions
- Fixes systemd failing to determine if `/sysroot` is valid because of
  `/etc/os-release` not being available yet.

- Related: #1759
2020-08-25 18:12:55 -04:00
OpenShift Merge Robot 2596a718ce
Merge pull request #2179 from cgwalters/ioctl-fix
linuxfsutil: Pass int to ioctl, not long
2020-08-21 20:13:51 -04:00
Colin Walters 06ed04a816 linuxfsutil: Pass int to ioctl, not long
Otherwise it will fail on big-endian architectures like s390x.
Ref https://bugzilla.redhat.com/show_bug.cgi?id=1867601
2020-08-21 17:41:32 +00:00
Simon McVittie d3fadf14b7 boot: Replace deprecated StandardOutput=syslog with journal, etc.
systemd deprecated this in v246.

Resolves: #2169
Signed-off-by: Simon McVittie <smcv@collabora.com>
2020-08-21 09:58:05 +01:00
Simon McVittie deefe62fe4 New upstream version 2020.5 2020-08-21 00:17:41 +01:00
OpenShift Merge Robot c61ff03304
Merge pull request #2175 from cgwalters/coverity-2020.5
Two small Coverity fixes
2020-08-19 16:46:49 +02:00
OpenShift Merge Robot 56f00586dd
Merge pull request #2176 from cgwalters/pin-str
admin/pin: Enforce that index is a number
2020-08-19 15:45:40 +02:00
Colin Walters 22a445c189 admin/pin: Enforce that index is a number
Validate that we're parsing a number; we want to guard
against typos.

Closes: https://github.com/ostreedev/ostree/issues/2171
2020-08-19 13:11:55 +00:00
Colin Walters 95a7512622 prepare-root: Remove unused variable
Should quiet Coverity.
2020-08-18 23:35:38 +00:00
Colin Walters 1eab48363b pull: Assign idle_src variable before calling unref()
This should pacify Coverity, and also just "reads" better too.
2020-08-18 23:34:57 +00:00
OpenShift Merge Robot 199562fc14
Merge pull request #2149 from stb-tester/boot-self-symlink
sysroot: Support /boot on root or as seperate filesystem for syslinux and u-boot
2020-08-19 01:08:30 +02:00
Jonathan Lebon 10a68cd26b lib/deploy: Clarify comment re. staging API
Don't mention deprecation in the description for
`ostree_sysroot_deploy_tree` since there are legitimate use cases for it
(e.g. to create the first deployment via `ostree admin deploy`).

Instead, make the comment clearly redirect to the staging API when
booted into the sysroot.
2020-08-17 09:48:57 -04:00
Jonathan Lebon 5de3a9759f lib/deploy: Drop unneccessary function arg 2020-08-17 09:48:57 -04:00
Jonathan Lebon e4fb7d3bb1 lib/cleanup: Drop unnecessary GEqualFunc cast 2020-08-17 09:48:57 -04:00
Jonathan Lebon 74bd136286 lib/deploy: Simplify deployment creation
Minor cleanup; we were declaring a superfluous variable.
2020-08-17 09:48:57 -04:00
Jonathan Lebon 61c544df1b lib/deploy: Avoid shadowing variable
There's already a `boot_relpath` variable in the outside scope.
2020-08-17 09:48:57 -04:00
Jonathan Lebon 52a6224606 lib/deploy: Clean up kargs override handling
Tighten up how we handle kargs here so it's more clear. When we call
`sysroot_finalize_deployment`, any karg overrides have already been set
on the bootconfig object of the deployment. So re-setting it here is
redundant and confusing.
2020-08-17 09:48:57 -04:00
João Paulo Rechi Vita 7cf1fb38b0 dracut: Create reproducible images
Without reproducible images, a rebuild of the initrd will create a
different image file (due to things like creation time of the files in
the cpio archive) even if the actual contents in it are exactly the
same, adding an unnecessary download during updates.

Adding 'reproducible=yes' avoids this and creates the same image files
for the same content.
2020-08-13 08:32:18 -07:00
Colin Walters f2773c1b55 Add "transient" unlock
I was thinking a bit more recently about the "live" changes
stuff https://github.com/coreos/rpm-ostree/issues/639
(particularly since https://github.com/coreos/rpm-ostree/pull/2060 )
and I realized reading the last debates in that issue that
there's really a much simpler solution; do exactly the same
thing we do for `ostree admin unlock`, except mount it read-only
by default.

Then, anything that wants to modify it does the same thing
libostree does for `/sysroot` and `/boot` as of recently; create
a new mount namespace and do the modifications there.

The advantages of this are numerous.  First, we already have
all of the code, it's basically just plumbing through a new
entry in the state enumeration and passing `MS_RDONLY` into
the `mount()` system call.

"live" changes here also naturally don't persist, unlike what
we are currently doing in rpm-ostree.
2020-08-07 18:57:56 +00:00
Philip Withnall f5da67d78a pull: Add summary-{,sig-}bytes options to ostree_repo_pull()
These allow the `summary` and `summary.sig` files to be cached at a
higher layer (for example, flatpak) between related pull operations (for
example, within a single flatpak transaction). This avoids
re-downloading `summary.sig` multiple times throughout a transaction,
which increases the transaction’s latency and introduces the possibility
for inconsistency between parts of the transaction if the server changes
its `summary` file part-way through.

In particular, this should speed up flatpak transactions on machines
with high latency network connections, where network round trips have a
high impact on the latency of an overall operation.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-07 16:12:15 +01:00
Philip Withnall bd68c7dfd7 pull: Improve formatting of pull options in documentation
Backticks improve all things.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-07 16:11:44 +01:00
OpenShift Merge Robot 1dedc69a46
Merge pull request #2159 from agners/show-parent
Show commit checksum of parent, if present
2020-08-04 21:54:28 +02:00