Commit Graph

4962 Commits

Author SHA1 Message Date
Philip Withnall 8d09a1a8ea lib/pull: Read mode and tombstone options from summary file if possible
Otherwise, fall back to downloading and reading them from the `config`
file. See the previous commit for details.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Fixes: #2165
2020-10-01 11:06:56 +01:00
Philip Withnall 206f1d3a13 lib/repo: Add mode and tombstone config options to the summary file
Currently, they are set in the `config` file and cause that to be
downloaded on every pull. Given that the client is already pulling the
`summary` file, it makes sense to avoid an additional network round trip
and cache those options in the `summary` file.

Signed-off-by: Philip Withnall <withnall@endlessm.com>

Helps: #2165
2020-10-01 11:06:56 +01:00
Jonathan Lebon 81b13da8e3 lib/deploy: Add support for overlay initrds
In FCOS and RHCOS, the need to configure software in the initramfs has
come up multiple times. Sometimes, using kernel arguments suffices.
Other times, it really must be a configuration file. Rebuilding the
initramfs on the client-side however is a costly operation. Not only
does it add complexity to the update workflow, it also erodes a lot of
the value obtained from using the baked "blessed" initramfs from the
tree itself.

One elegant way to address this is to allow specifying multiple
initramfses. This is supported by most bootloaders (notably GRUB) and
results in each initrd being overlayed on top of each other.

This patch allows libostree clients to leverage this so that they can
avoid regenerating the initramfs entirely. libostree itself is agnostic
as to what kind and how much data overlay initrds contain. It's up to
the clients to enforce such boundaries.

To implement this, we add a new ostree_sysroot_stage_overlay_initrd
which takes a file descriptor and returns a checksum. Then users can
pass these checksums when calling the deploy APIs via the new array
option `overlay_initrds`. We copy these files into `/boot` and add them
to the BLS as another `initrd` entry.
2020-09-30 13:29:32 -04:00
Jonathan Lebon 40fea4c443 lib/deploy: Add deploy/stage APIs with options
And make the `override_kernel_argv` one of those options. This is mostly
a mechanical move here, no functional change otherwise.

Prep for adding a new option.
2020-09-30 13:29:32 -04:00
Jonathan Lebon f7500bb703 lib/bootconfig: Add support for multiple initrd keys
Prep for actually teaching the rest of the codebase about this.

We keep the primary initrd in the `options` hash table for backwards
compatibility.
2020-09-30 13:29:32 -04:00
OpenShift Merge Robot a5f9651dab
Merge pull request #2204 from jlebon/pr/versioning-tweaks
Various minor versioning related tweaks
2020-09-30 17:22:49 +00:00
Jonathan Lebon f04e5d047d lib: Minor versioning related fixes
Fix/add the `Since` marker to the new static delta APIs, and update the
symbol versioning templates/comments.
2020-09-25 15:30:20 -04:00
Jonathan Lebon 75342035d5 Makefile-libostree.am: Uncomment BUILDOPT_IS_DEVEL_BUILD conditional
We shouldn't have to toggle the conditional itself during release
builds. It should only evaluate to true during devel builds.
2020-09-25 15:07:36 -04:00
OpenShift Merge Robot fad95167ce
Merge pull request #2203 from cgwalters/delta-sign-cleanups
delta: Some minor code style fixups
2020-09-25 13:57:33 +00:00
OpenShift Merge Robot e6f673a8ed
Merge pull request #2202 from cgwalters/bootcsum-dtb
deploy: Remove deployment bootcsum assertion
2020-09-24 22:14:31 +00:00
Colin Walters 9198fa040c delta: Some minor code style fixups
- Remove some unused variables
- Switch to declare-and-initialize with others
- Fix some indentation from 4 spaces to 2 (GNU style)
2020-09-24 22:12:56 +00:00
Colin Walters aa2a2783ea deploy: Remove deployment bootcsum assertion
When support for devicetree was added, it created a problem
because old and new ostree versions would compute different
checksums for the "boot data".  The scenario here is:

- Have system with ostree < 2020.4
- Reboot into system with ostree 2020.5
- Try to perform an operation that would retain
  that previous booted deployment (common)

Currently ostree iterates over all the deployments
that will be retained and calls `install_deployment_kernel()`,
even for the booted one (which is a bit silly), but
just to verify that all boot data for the targeted
deployments are installed.

This then re-computes the checksum and we'd trip this
assertion.

In practice though, we don't strictly require them to match;
the only thing that will happen if they don't is that we'll
end up with another copy of the kernel/initramfs - and
that only temporarily until the previous deployment
gets GC'd.

Longer term, I think what we really want to do anyways
is probably closer to like a little ostree repo for `/boot`
so that we can e.g. still hardlink kernels there even if
the initramfs changes, or hardlink both kernel/initramfs
if just the devicetree changes, etc.

Closes: https://github.com/ostreedev/ostree/issues/2154
2020-09-24 21:24:48 +00:00
OpenShift Merge Robot 00df896550
Merge pull request #1985 from fdanis-oss/wip/fda/sign_delta_metadata
Static-delta's superblock signature support
2020-09-24 14:21:16 +00:00
OpenShift Merge Robot 6950a98099
Merge pull request #2199 from cgwalters/no-hardlink-unreadable
checkout: Ensure copies of unreadable usermode checkouts are readable
2020-09-16 17:58:55 +02:00
Colin Walters 3441a48c58 checkout: Ensure copies of unreadable usermode checkouts are readable
The extreme special case of "zero mode" files like `/etc/shadow`
comes up again.  What we want is for "user mode" checkouts to
override it to make the file readable; otherwise when operating
as non-root without `CAP_DAC_OVERRIDE` it becomes very difficult
to work with.

Previously, we were hardlinking these files, but then it intersects
with *another* special case around zero sized files, which is
*also* true for `/etc/shadow`.

Trying to avoid hardlinking there unveiled this bug - when
we go to do a copy checkout, we need to override the mode.
2020-09-16 15:14:54 +00:00
OpenShift Merge Robot ca2f3435be
Merge pull request #2195 from ostreedev/avoid-shadows
Avoid shadowing local variables
2020-09-14 10:43:40 -04:00
Frédéric Danis ecbfe08ec7 tests/delta: Add new tests for applying signed deltas
Add new test to apply offline signed deltas.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 2e97f5659f bin/static-delta: Add signature parameters to apply-offline
This allows to check the delta signature before applying it.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis fb1faf17d6 lib/deltas: Check signed delta in execute_offline
Add a new function `ostree_repo_static_delta_execute_offline_with_signature`
which takes a signature engine to verify the delta before applying it.
The `ostree_repo_static_delta_execute_offline` is just a wrapper to this
new function, passing a NULL signature engine.
When this function is called without signature engine, but with a sign
delta, it will only fails if `sign-verify-deltas` is set to true in repo
core options.

This commits move signature existence check and delta signature
verification to share common parts between existing APIs and the new
function.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 869dbc037e tests/delta: new tests for 'ed25519' signed deltas
Add tests to generate signed deltas and verify them using 'ed25519'
signature engine.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 96bcc25632 tests/libtest.sh: Add skip_without_sign_ed25519() function
Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis c98a993c99 tests/delta: new tests for signed deltas
Add tests to generate signed deltas and verify them using 'dummy'
signature engine.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis bf0c09ffe1 lib/deltas: Support signed delta in dump
This checks if the static delta file is signed or not to be able to
correctly get the superblock to dump.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 0c48423c26 lib/deltas: Support signed delta in execute_offline
This checks if the static delta file is signed or not to be able to
correctly get the superblock to apply.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 512db0435c bin/static-delta: Add command to verify delta signature
Add new "static-delta verify" sub-command.
This supports multiple keys to verify the static-delta file.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 02a19b2c96 lib/deltas: Add signature check API for static-delta superblock
This retrieves the signatures and pass the static delta block as an array
of bytes to ostree_sign_data_verify().

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 92efbc00d8 bin/static-delta: Add support to sign superblock
Add signing ability to "static-delta generate" builtin.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Frédéric Danis 46667567c5 lib/deltas: Add inline signature for static-delta superblock
While the commits contained in the single static-delta file are signed so
we can check them and operate on trusted data, the superblock isn't signed
in any way, so it end up operating on untrusted data to:
 1. actually find where the trusted data is, and
 2. check whether the update is fit for the current device by looking at
    the collection id stored in the metadata

This commit generates signatures of all static data, and concatenate them
to the existing static delta format, i.e. as a GVariant layout `a{sv}ay`
where
 - a{sv}: signatures
 - ay: existing delta variant

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-09-14 09:27:19 +02:00
Phaedrus Leeds c4f26bfdc8 Avoid shadowing local variables
This should help with code readability.

Fixes https://github.com/ostreedev/ostree/issues/2194
2020-09-13 10:08:02 -07:00
OpenShift Merge Robot 9c040c1a73
Merge pull request #2196 from cgwalters/commit-shadowing-fix
commit: Tighten scope of two variables
2020-09-13 13:00:16 -04:00
Colin Walters 779a901780 commit: Tighten scope of two variables
Prep for adding `-Wshadow` fixes.
2020-09-13 14:58:24 +00:00
OpenShift Merge Robot 71acef2d33
Merge pull request #2193 from alexlarsson/preparatory-cleanup
Preparatory cleanup for summary work
2020-09-11 11:07:00 -04:00
Alexander Larsson b7d1a9746b Update the symbols files to match that we're now on 2020.6 2020-09-11 12:35:32 +02:00
Alexander Larsson da853a1783 ostree-repo-pull.c: Extract mirrorlist generation to helper
This code was duplicated in 3 places, so move it to a single place
to clean things up.
2020-09-11 12:03:29 +02:00
Alexander Larsson 155b215cd8 Minor cleanup of _ostree_repo_remote_new_fetcher()
Instead of open coding the extra_headers and append_user_agent
setting everywhere we do this in the constructor.
2020-09-11 12:03:28 +02:00
Alexander Larsson 3957bff0cb Inline repo_remote_fetch_summary
This was only used in one place, and (especially with the simplification
with GMainContextPopDefault) and the one caller doesn't really do
much more than call the helper. Additionally, what little it does (saving
the result in the cache) is inherently tied to how the helper work,
and will become even more so when we support summary indexes.

This is a preparatory cleanup for supporting summary indexes. It
doesn't change any behaviour and passes make check on its own.
2020-09-11 12:03:28 +02:00
Alexander Larsson 4b9e712e82 repo_remote_fetch_summary: Use GMainContextPopDefault
This allows us to drop the "goto out" use and clean up this function.
2020-09-11 12:03:28 +02:00
Alexander Larsson c7df4317bd Add g_autoptr helper for pushing a thread default main context
This happens in a bunch of places, and currently each time it does
we have to use "goto out" style cleanups, which just isn't looking
very nice.
2020-09-11 12:03:28 +02:00
Alexander Larsson f74bc8dd3d fetch_summary_with_options: drop unnecessary "goto out" use 2020-09-11 12:03:28 +02:00
Alexander Larsson 1f1ef4c989 Break out the signature verification code into a helper function
This changes nothing in the behaviour, but we want to later re-use
this when we also verify the summary index.
2020-09-11 12:03:28 +02:00
Alexander Larsson 598adc457f deltas: Break out _ostree_repo_static_delta_superblock_digest() helper
This loads and makes a digest for a delta superblock. The previous
code was used when generating the deltas section in the summary
file. This changes nothing, but is in preparation for using similar
formats in a separate delta index file.
2020-09-11 12:03:28 +02:00
Alexander Larsson 32014d99e6 Add and use ot_checksum_bytes helper
This removes some duplicated code (and will be use even more later).
2020-09-11 12:00:05 +02:00
Alexander Larsson dddb449d2c pull: Actually mmap summary files
The change in cbf1aca1d5c08d2f40832d16670484ba878d95fb actually
only mmaps the signature file, not the summary. This change makes
use mmap both, as well as extract the cache loading into a helper
function that we will later use in more places.
2020-09-11 12:00:02 +02:00
Alexander Larsson 85accb84e8 pull: Break out _ostree_repo_save_cache_summary_file() helper
This is a minor cleanup as its just called twice from
_ostree_repo_cache_summary(). However, later code will need it in more
places.
2020-09-11 11:53:38 +02:00
Alexander Larsson bb2649a8c0 Fix leak when signing
_ostree_detached_metadata_append_gpg_sig() was returning a floating
ref, but all users were using g_autoptr. Fix it by adding a ref-sink.
2020-09-11 11:45:33 +02:00
Alexander Larsson 74bae256fe list-deltas: Don't break on non-subdir entries
ostree_repo_list_static_delta_names() tried to validate that
any second-level directory element was a directory, but there was
a cut-and-paste issue, and it used `dent->d_type` instead
of `sub_dent->d_type`.

This fixes the code, but all old ostree versions will break if
there are non-directories in a subdirectory of the deltas directory
in the repo, so be wary.
2020-09-11 11:45:24 +02:00
OpenShift Merge Robot 69282a3dd3
Merge pull request #2192 from cgwalters/error-prefixing
deploy: Add some error prefixing around xattr setting
2020-09-09 09:25:12 -04:00
Colin Walters a1bd29f245 deploy: Add some error prefixing around xattr setting
Looking at
https://github.com/coreos/coreos-assembler/issues/1703
a user is getting a bare:
`error: fsetxattr: Permission denied`

I don't think it's these code paths since a deploy
isn't happening but on inspection I noticed we didn't
have error prefixing here.
2020-09-09 12:34:44 +00:00
OpenShift Merge Robot 3bac819a5d
Merge pull request #2190 from cgwalters/ci-drop-var-test
ci: Drop var mount test
2020-09-07 19:26:02 -04:00
Colin Walters be0f9e77e5 ci: Drop var mount test
Merged in https://github.com/coreos/fedora-coreos-config/pull/586
2020-09-03 22:14:02 +00:00