We recently discovered `list_objects()` is inefficient with memory.
The more efficient `list_objects_set()` API isn't yet public, but
this fsck code actually just skips over non-commit objects, and
we already have an API to list just those.
I messed this up; the last release should inherit from the previous
release (N-1) and not the previous to that (N-2).
I think (hope) this isn't an ABI break...
Just noticed this when I was going to add a new symbol.
In a prior change we discovered that for bad historical reasons
libostree was returning a mapping "object type+checksum" => "metadata"
but the "metadata" was redundant and pointless.
Optimize the prune API to use a (currently internal) object listing
API which returns a set, not a map. This allows `GHashTable` to
avoid allocating a separate array for the values, neatly cutting
memory usage in half (from ~13MB to ~6MB) on my test case of a
dry-run prune of a FCOS build.
I was looking at https://github.com/ostreedev/ostree/pull/2632
and confused at the usage of
`GVariant *value = g_variant_new ("(b@as)", TRUE, g_variant_new_strv (NULL, 0));`
which looked strange - why the empty strv?
It turns out that this is a historical legacy of the time when
ostree had pack files. And nothing actually cares about the values
of these variants; we should have an API that returns a proper set,
and not a hash.
But...since all of these things have exactly the same value, instead
of allocating lots of redundant copies on the heap, just have
them all hold a refcount on a shared value.
This cuts the heap usage from 20MB to 13MB on a test FCOS repository
build.
It inherently depends on the individual build, and can't
really be an official stable API for introspection users.
I've noticed the value of this flip flop when doing local builds.
I'm fairly certain no one is trying to use it from a higher level
language.
It'd probably make sense to even drop from the official C API,
but I'm trying to be conservative with that.
I was looking at our `.gir` and noticed we had the cmdprivate bits
because the pattern for excluding headers is `-private.h`, which
didn't match `cmdprivate.h`.
(nullable) and (optional) were missing on lookup()'s out parameters,
which caused the rust bindings for the function to not work. Due to the
missing (nullable), it would return a Result<(GString, MutableTree), _>,
not a Result<(Option<GString>, Option<MutableTree>), _>, which led to
panics.
We want to parse a new "bls-append-except-default" key from ostree config. The
key-value pairs specified by this key will be added to the generated
BLS fragments of non-default deployments. They must follow the format
"key1,value1;key2,value2" and so on.
This change will allow us to land GRUB password support in FCOS.
Relevant: https://github.com/coreos/fedora-coreos-tracker/issues/134
I'm aiming to do some more work on the Rust side around `fsck`
like functionality, and this is a useful primitive. There isn't
a great Rust crate for xattrs, and I think it's better to share this
code.
Previously, the reference count was left uninitialized as a result of
bypassing the constructor, and the intended abort-on-error usually
wouldn't have happened.
Fixes: 8a9737a "repo/private: move OstreeRepoAutoTransaction to a boxed type"
Resolves: https://github.com/ostreedev/ostree/issues/2592
Signed-off-by: Simon McVittie <smcv@collabora.com>
This will allow the direct allocation in
ostree_repo_prepare_transaction() to be replaced with a call to this
function, avoiding breaking encapsulation.
Signed-off-by: Simon McVittie <smcv@collabora.com>
Quite a while ago we added staged deployments, which solved
a bunch of issues around the `/etc` merge. However...a persistent
problem since then is that any failures in that process that
happened in the *previous* boot are not very visible.
We ship custom code in `rpm-ostree status` to query the previous
journal. But that has a few problems - one is that on systems
that have been up a while, that failure message may even get
rotated out. And second, some systems may not even have a persistent
journal at all.
A general thing we do in e.g. Fedora CoreOS testing is to check
for systemd unit failures. We do that both in our automated tests,
and we even ship code that displays them on ssh logins. And beyond
that obviously a lot of other projects do the same; it's easy via
`systemctl --failed`.
So to make failures more visible, change our `ostree-finalize-staged.service`
to have an internal wrapper around the process that "catches" any
errors, and copies the error message into a file in `/boot/ostree`.
Then, a new `ostree-boot-complete.service` looks for this file on
startup and re-emits the error message, and fails.
It also deletes the file. The rationale is to avoid *continually*
warning. For example we need to handle the case when an upgrade
process creates a new staged deployment. Now, we could change the
ostree core code to delete the warning file when that happens instead,
but this is trying to be a conservative change.
This should make failures here much more visible as is.
The `archive_entry_symlink()` API can definitely return `NULL`,
reading through the libarchive sources.
I hit this in the wild when using old ostree-ext to try to unpack
a chunked archive.
I didn't try to characterize this more, and sorry no unit test right
now.
Whenever the user has SELinux enabled and has any local
modules/modifications installed, it is necessary to rebuild the policy
in the final deployment, otherwise ostree will leave the binary policy
files unchanged from last deployment as it detects difference against
the base content (in rpm-ostree case this is the RPM content).
To avoid the situation where the policy binaries go stale once any local
customization of the policy is made, try to rebuild the policy as part
of sysroot_finalize_deployment(). Use the special
--rebuild-if-modules-changed switch, which detects if the input module
files have changed relative to last time the policy was built and skips
the most time-consuming part of the rebuild process if modules are
unchanged (thus making this a relatively cheap operation if the user
hasn't made any modifications to the shipped policy).
As suggested by Jonathan Lebon, this uses bubblewrap (via
g_spawn_sync()) to perform the rebuild inside the deployment's
filesystem tree, which also means that ostree will have a runtime
dependency on bubblewrap.
Partially addresses: https://github.com/coreos/fedora-coreos-tracker/issues/701
Signed-off-by: Ondrej Mosnacek <omosnace@redhat.com>
Like every other error return path in this function, jump to the `out`
label on error here. Returning directly will cause leaks.
Spotted by reading the code, not actually necessarily encountered in the
wild.
Signed-off-by: Philip Withnall <pwithnall@endlessos.org>
On OSs that do not consistently merge /usr/bin with /bin, the path to
bash has traditionally been /bin/bash.
Signed-off-by: Simon McVittie <smcv@debian.org>
An indented `#!` is technically meaningless, although many shells will
run text files with the shell if asked to execute them.
Signed-off-by: Simon McVittie <smcv@debian.org>
This prevents writing content into 'bare-split-xattrs` repository,
while carving some space for experimenting via a temporary
`OSTREE_EXP_WRITE_BARE_SPLIT_XATTRS` environment flag.
Jonathan Lebon
Colin Walters
Joseph Marrero Corchado
James Westman
Nikita Dubrovskii
Huijing Hei
Saqib Ali
Luca BRUNO
Simon McVittie
Ondrej Mosnacek
Daniel Kolesa
Philip Withnall
Simon McVittie