Commit Graph

3096 Commits

Author SHA1 Message Date
OpenShift Merge Robot 8a9a496501
Merge pull request #1957 from dbnicholson/commit-sizes
Upstream Endless sizes metadata changes
2020-01-27 08:25:59 -08:00
OpenShift Merge Robot 08e292bc14
Merge pull request #1877 from dbnicholson/specific-gpg-errors
More specific GPG errors
2020-01-26 09:20:51 -08:00
Dan Nicholson 11ad68647a ostree/trivial-httpd: Add log message for autoexit
This is useful when checking if the daemon actually exited since we
don't store the child PID anywhere.
2020-01-24 13:11:54 -07:00
Dan Nicholson fbf5a94e0a ostree/trivial-httpd: Fix --autoexit with --daemonize and --log-file
When --autoexit is used with --daemonize and --log-file, the program
never exits when the root directory is deleted. I believe what happens
is that g_file_new_for_path triggers the glib worker context to be
started to talk to GVfs. Once the program forks, the parent exits and
the thread iterating the worker context is gone. The file monitor then
never receives any events because the inotify helper also runs from the
worker context.

Move the fork earlier just after parsing and validating the command line
arguments. In order to handle setup errors in the child, a pipe is
opened and the parents waits until the child writes a status byte to it.
If the byte is 0, the parent considers the child setup successful and
exits and the child carries on as a daemon. Notably, the child doesn't
reopen stderr to /dev/null until after this so that it can send error
messages there.

Fixes: #1941
2020-01-24 13:11:54 -07:00
Dan Nicholson 0fbfc0b207 lib/gpg: Add more specific OstreeGpgError codes
Currently `ostree_gpg_verify_result_require_valid_signature` always
returns an error that the key used for the signature is missing from the
keyring. However, all that's been determined is that there are no valid
signatures. The error could also be from an expired signature, an
expired key, a revoked key or an invalid signature.

Provide values for these missing errors and return them from
`ostree_gpg_verify_result_require_valid_signature`. The description of
each result is appended to the error message, but since the result can
contain more than one signature but only a single error can be returned,
the status of the last signature is used for the error code. See the
comment for rationale.

Related: flatpak/flatpak#1450
2020-01-24 13:02:17 -07:00
Dan Nicholson c54a3b5daa lib/gpg: Prefer declare-and-initialize style
As noted in
https://github.com/ostreedev/ostree/pull/1872#discussion_r295408768.
2020-01-24 13:02:09 -07:00
Dan Nicholson 97c831dd5f bin/show: Add --print-sizes option to show sizes metadata
Use the new `ostree_commit_get_object_sizes()` API to read the
`ostree.sizes` commit metadata and print a summary.
2020-01-20 20:46:31 -07:00
Dan Nicholson 260bcd1193 core: Add ostree_commit_get_object_sizes API
This function parses the object listing in the `ostree.sizes` metadata
and returns an array of `OstreeCommitSizesEntry` structures.

Unfortunately, for reasons I don't understand, the linker wants to
resolve `_ostree_read_varuint64` from `ostree-core.c` even though it's
not used by `test-checksum.c` at all.
2020-01-20 20:46:29 -07:00
Dan Nicholson fcbb453443 core: Add OstreeCommitSizesEntry type
This will be used when reading out entries in the `ostree.sizes`
metadata. Each entry corresponds to an object in the metadata array.
2020-01-20 20:44:12 -07:00
Dan Nicholson 1bbe674d91 libarchive: Support commit sizes metadata
Call the helper to set the generate_sizes boolean so that object size
data is stored while writing the mtree.
2020-01-20 20:42:27 -07:00
John Hiesey 291e9da258 lib/commit: Include object type in sizes metadata
Append a byte encoding the OSTree object type for each object in the
metadata. This allows the commit metadata to be fetched and then for the
program to see which objects it already has for an accurate calculation
of which objects need to be downloaded.

This slightly breaks the `ostree.sizes` `ay` metadata entries. However,
it's unlikely anyone was asserting the length of the entries since the
array currently ends in 2 variable length integers. As far as I know,
the only users of the sizes metadata are the ostree test suite and
Endless' eos-updater[1]. The former is updated here and the latter
already expects this format.

1. https://github.com/endlessm/eos-updater/
2020-01-20 20:42:27 -07:00
Dan Nicholson a4592678aa tests/sizes: Check duplicate file doesn't add sizes entry
A duplicate file will resolve to the same object, so it shouldn't add
any entries to the sizes metadata.
2020-01-20 20:42:27 -07:00
Dan Nicholson 44fb5e72a1 lib/commit: Make size entries for existing objects
If the object was already in the repo then the sizes metadata entry was
skipped. Move the sizes entry creation after the data has been computed
but before the early return for an existing object.
2020-01-20 20:42:27 -07:00
Dan Nicholson 8ec7d6322f lib/commit: Fix object sizes metadata for multiple commits
The object sizes hash table was only being cleared when the repo was
finalized. That means that performing multiple commits while the repo
was open would reuse all the object sizes metadata for each commit.

Clear the hash table when the sizes metadata is setup and when it's
added to a commit. This still does not fix the issue all the way since
it does nothing to prevent the program from constructing multiple
commits simultaneously. To handle that, the object sizes hash table
should be attached to the MutableTree since that has the commit state.
However, the MutableTree is gone when the commit is actually created.
The hash table would have to be transferred to the root file when
writing the MutableTree. That would be an awkward addition to
OstreeRepoFile, though. Add a FIXME to capture that.
2020-01-20 20:42:27 -07:00
Dan Nicholson 37045b4b46 lib/commit: Only set generate_sizes for archive repos
Rather than checking throughout the code, only set the boolean when
appropriate.
2020-01-20 20:42:27 -07:00
Colin Walters 5af403be0c Support mounting /sysroot (and /boot) read-only
We want to support extending the read-only state to cover `/sysroot`
and `/boot`, since conceptually all of the data there should only
be written via libostree.  Or at least for `/boot` should *mostly*
just be written by ostree.

This change needs to be opt-in though to avoid breaking anyone.

Add a `sysroot/readonly` key to the repository config which instructs
`ostree-remount.service` to ensure `/sysroot` is read-only.  This
requires a bit of a dance because `/sysroot` is actually the same
filesystem as `/`; so we make `/etc` a writable bind mount in this case.

We also need to handle `/var` in the "OSTree default" case of a bind
mount; the systemd generator now looks at the writability state of
`/sysroot` and uses that to determine whether it should have the
`var.mount` unit happen before or after `ostree-remount.service.`

Also add an API to instruct the libostree shared library
that the caller has created a new mount namespace.  This way
we can freely remount read-write.

This approach extends upon in a much better way previous work
we did to support remounting `/boot` read-write.

Closes: https://github.com/ostreedev/ostree/issues/1265
2019-12-11 15:33:57 +00:00
Colin Walters 20daf96880 finalize-staged: Use the core option parsing to load sysroot
Prep for using the default mount namespace handling there that
will land as part of the read-only `/sysroot` and `/boot` work.
See https://github.com/ostreedev/ostree/issues/1265
2019-12-11 14:39:17 +00:00
Colin Walters 978bd19f97 Release 2019.6
Nothing really big here, but let's get a release out
so some bigger things like ro-sysroot, signing, sizes can
bake in master for a bit.
2019-12-09 16:18:01 +00:00
Colin Walters 36039759e2 pull: Add support for basic auth
This has been requested a few times for people delivering
OSTree content and wanting to do access control.
2019-12-02 22:48:33 +00:00
Philip Chimento 54639c03ca libostree: Add ostree_async_progress_copy_state()
This allows copying the state from one OstreeAsyncProgress object to
another, atomically, without invoking the callback. This is needed in
libflatpak, in order to chain OstreeAsyncProgress objects so that you
can still receive progress updates when iterating a different
GMainContext than the one that the OstreeAsyncProgress object was
created under.

See https://github.com/flatpak/flatpak/pull/3211 for the application of
this API.
2019-11-20 12:24:26 -08:00
Philip Chimento 382ad59822 Bump version in symbols file
There were no new symbols in 2019.5 and this version didn't get bumped
when 2019.5 was released.
2019-11-20 11:18:36 -08:00
Robert Fairley aadc4db012 lib/kernel-args: Store kernel args as key/value entries
Define an `OstreeKernelArgsEntry` structure, which holds
both the key and the value. The kargs order array stores
entries for each key/value pair, instead of just the keys.
The hash table is used to locate entries, by storing
entries in a pointer array for each key. The same public
interface is preserved, while maintaining ordering
information of each key/value pair when
appending/replacing/deleting kargs.

Fixes: #1859
2019-11-07 23:39:10 -05:00
Colin Walters 58980a717a lib/keyfile: Treat "group not found" the same as "key not found"
Prep for fsverity, where I want to create a new group
`[fsverity]` in the keyfile that has default values.  We should
treat the absence of a group the same as absence of a key
in these "with defaults" APIs.
2019-11-07 21:15:41 +00:00
Colin Walters 980ca07b03 Release 2019.5 2019-10-30 17:08:00 +00:00
Jonathan Lebon 7ae8da08b9 lib/deploy: Also install HMAC file into /boot
To allow for FIPS mode, we need to also install the HMAC file from
`/usr/lib/modules` to `/boot` alongside the kernel image where the
`fips` dracut module will find it. For details, see:

https://github.com/coreos/fedora-coreos-tracker/issues/302

Note I didn't include the file in the boot checksum since it's itself a
checksum of the kernel, so we don't really gain much here other than
potentially causing an unnecessary bootcsum bump.
2019-10-29 16:45:29 -04:00
OpenShift Merge Robot 476f375cfc
Merge pull request #1953 from uajain/uajain/fix-memory-leak
async-progress: Plug memory leak while destroying GSource
2019-10-29 06:26:17 +01:00
Jonathan Lebon 74936f98d8 lib/pull: Tweak update_timeout logic again
I was hitting `SIGSEGV` when running `cosa build` and narrowed it down
to #1954. What's happening here is that because we're using the default
context, when we unref it in the out path, it may not actually destroy
the `GSource` if it (the context) is still ref'ed elsewhere. So then,
we'd still get events from it if subsequent operations iterated the
context.

This patch is mostly a revert of #1954, except that we still keep a ref
on the `GSource`. That way it is always safe to destroy it afterwards.
(And I've also added a comment to explain this better.)
2019-10-28 15:26:45 -04:00
Umang Jain 78c8c25d64 async-progress: Plug memory leak while destroying GSource
See https://gitlab.gnome.org/GNOME/glib/commit/71973c722
2019-10-25 20:59:15 +05:30
Colin Walters 650d6252af Revert "grub2: Exit gracefully if the configuration has BLS enabled"
This reverts commit 985a141002.
It turned out that some people have old bootloaders, and hence
get the "no entries" problem.  That's much, much much worse
than double entries.
2019-10-24 17:49:53 +00:00
Jonathan Lebon d4a186e80e lib/pull: Avoid calling destroy on unref'ed GSource
We're creating the timer source and then passing ownership to the
context, but because we didn't free the pointer, we would still call
`g_source_destroy` in the exit path. We'd do this right after doing
`unref` on the context too, which would have already destroyed and
unref'ed the source.

Drop that and just restrict the scope of that variable down to make
things more obvious.

Just noticed this after reviewing #1953.
2019-10-23 11:15:02 -04:00
OpenShift Merge Robot 1645416aac
Merge pull request #1947 from cgwalters/more-scan-build-2
More scan build 2
2019-10-18 20:59:06 +02:00
Dan Nicholson 4df90d4012 repo: Stop using deprecated G_GNUC_FUNCTION
In glib 2.62 this has been changed to emitting a warning. Use G_STRFUNC
instead, which has been available for a long time and is already used in
other places in ostree.
2019-10-18 11:10:44 -06:00
Colin Walters fbed380483 lib/checksum-utils: Use g_memdup()
This is clearer and silences a scan-build warning.
2019-10-18 15:07:39 +00:00
Colin Walters 57bb06419a lib: Port variant-builder.c to new style
Seeing `scan-build` warning here, prep for fixing it.
2019-10-18 15:07:39 +00:00
Colin Walters 25c5ae5d08 lib/pull: [scan-build] Silence a dead store warning
This one was actual duplicate code.
2019-10-18 15:07:39 +00:00
Colin Walters bc1980ca38 lib/repo: [scan-build] Quiet a dead store warning
False positive, just add a pacifier.
2019-10-18 15:07:39 +00:00
OpenShift Merge Robot 3b58d89368
Merge pull request #1943 from cgwalters/more-scan-build
More scan build
2019-10-16 20:44:15 +02:00
OpenShift Merge Robot 38729a4f90
Merge pull request #1937 from cgwalters/zipl-backend
bootloader: Add a zipl bootloader backend
2019-10-16 17:21:10 +02:00
Colin Walters c61234a428 bootloader: Add a zipl bootloader backend
zipl is a bit special in that it parses the BLS config files
directly *but* we need to run the command to update the "boot block".

Hence, we're not generating a separate config file like the other
backends.  Instead, extend the bootloader interface with a `post_bls_sync`
method that is run in the same place we swap the `boot/loader` symlink.

We write a "stamp file" in `/boot` that says we need to run this command.
The reason we use stamp file is to prevent the case where the system is
interrupted after BLS file is updated, but before zipl is triggered,
then zipl boot records are not updated.
This opens the door to making things eventually-consistent/reconcilable
by later adding a systemd unit to run `zipl` if we're interrupted via
a systemd unit - I think we should eventually take this approach
everywhere rather than requiring `/boot/loader` to be a symlink.

Author: Colin Walters <walters@verbum.org>
Tested-by: Tuan Hoang <tmhoang@linux.ibm.com>
Co-Authored-By: Tuan Hoang <tmhoang@linux.ibm.com>
2019-10-16 14:22:02 +00:00
Colin Walters 946659aacf prune: [scan-build] Initialize a variable
Another false positive because we only read this if `opt_keep_younger_than` is `TRUE`,
but let's initialize variables on general principle.
2019-10-16 13:48:20 +00:00
Colin Walters a8dc90b02f tree-wide: [scan-build]: Add some asserts that pointers are non-NULL
More "scan-build doesn't understand GError and our out-param conventions"
AKA "these errors would be impossible with Rust's sum type Result<> approach".
2019-10-16 13:44:46 +00:00
Colin Walters 4a38b11159 commit: [scan-build] Remove a dead assignment
The `write_commit()` API defaults to current time, and
this assignment became dead in:
8ba90a3341
2019-10-16 13:38:29 +00:00
Colin Walters 810f24d897 libotutil: Port keyfile-utils.c to new style
I was trying to fix a clang `scan-build` error that jlebon
ended up tracking down in
9344de1ce1

But in the process of tracing through this I found it
way easier to read as "new style" code, so this also ports the
code.

I added a `g_assert()` in there too to help assert that
`g_key_file_get_value` won't leak in the error path.
2019-10-16 13:30:21 +00:00
OpenShift Merge Robot 347e234aaa
Merge pull request #1932 from cgwalters/covscan-fixes-2
libostree: Add an assert to pacify clang-analyzer
2019-10-15 19:56:54 +02:00
Jonathan Lebon 9344de1ce1 src/libotutil: Fix strv memory leak
We were only freeing the array and not the members.

Caught by `clang-analyzer` in:
https://github.com/ostreedev/ostree/pull/1931
2019-10-15 11:59:45 -04:00
Colin Walters 806206fac2 repo: [scan-build]: Mark a variable used
We're just using this to auto-free, quiet the static analysis.
2019-10-15 12:41:27 +00:00
Colin Walters 51d9aa35a9 sysroot: [scan-build] Remove a dead assignment
Just quieting the scan.
2019-10-15 12:41:27 +00:00
Colin Walters f1fdd885ab sysroot: [scan-build]: Remove a dead assignment
Clarify the conditionals here and remove a dead assignment.
2019-10-15 12:41:27 +00:00
Colin Walters 9032182e3c repo: [scan-build] Initialize a variable
Another GLib error convention issue; but eh, we might as
well be conservative and always initialize variables.
2019-10-15 12:41:27 +00:00
Colin Walters aa7795d08d libostree: Add an assert to pacify clang-analyzer
Got this error when trying to rebase libostree in RHEL:

```
Error: CLANG_WARNING: [#def1]
libostree-2019.2/src/libostree/ostree-repo-checkout.c:375:21: warning: Access to field 'disable_xattrs' results in a dereference of a null pointer (loaded from variable 'repo')
```

I think what's happening is it sees us effectively testing
`if (repo == NULL)` via the `while (current_repo)`.  Let's
tell it we're sure it's non-null right after the loop.
2019-10-15 12:40:54 +00:00