Imports one or more GPG keys from a source stream or from the user's
personal keyring into a remote-specific keyring. The keys to import
can optionally be restricted by a list of key IDs.
The imported keys are used to conduct GPG verification when pulling
from the given remote.
An OSTree user noticed that `ostree fsck` would produce `missing
object` errors in the case of interrupted pulls.
It's possible to do e.g. `ostree pull --subpath=/usr/share/rpm ...`,
which gets you just that portion of the commit. The use case for this
was being able to see what changes would appear in an update before
actually downloading all of it.
(I think this would be better covered by static deltas, but those
aren't final yet, and `--subpath` predates it)
Further, `.commitpartial` is used as a successor to the `transaction`
symlink for more precise knowledge in the case where a pull was
interrupted that we needed to resume scanning.
So it makes sense for `ostree fsck` to be aware of it.
rpm-ostree currently uses ostree_repo_checkout_tree(), which as a side
effect will use the uncompressed objects cache by default. This is
rather annoying if you're using rpm-ostree on a server-side
repository, because if you then rsync the repo, you'll be syncing out
the uncompressed objects unless you exclude them.
We added the ability to disable the uncompressed cache in the
repository config to fix this, but it's better to allow application
control over this. The uncompressed cache will in some future version
become opt in as well.
This new API further:
- Drops the `GFile` usage in favor of `openat` APIs
- Improves ergonomics by avoiding callers having to query the source
`GFileInfo` (and carry around a copy of `OSTREE_GIO_FAST_QUERYINFO`)
- Has a more extensible options structure
Per the comment, I rather crudely have the `ostree checkout` builtin
call both APIs to ensure some testing coverage.
However, I'd like to in the future have easier-to-set-up testing code
that calls `libtest.sh` to set up dummy data.
For Anaconda, I needed OSTREE_REPO_REMOTE_CHANGE_ADD_IF_NOT_EXISTS,
with the GFile *sysroot argument to avoid ugly hacks. We want to
write the content provided via "ostreesetup" as a remote to the target
chroot only in the case where it isn't provided as part of the tree
content itself.
This is also potentially useful in idempotent systems management tools
like Ansible.
https://bugzilla.gnome.org/show_bug.cgi?id=741577
We potentially need a lot of argument types for pull. Rather than
have a C function with tons of arguments, let's use a GVariant a{sv}
as a handy extensible (and immutable) bag of properties.
This is prepratory work for adding an option to pull to traverse
history.
https://bugzilla.gnome.org/show_bug.cgi?id=737844
For Anaconda, we have an ugly bootstrapping problem where we need to
add the remote to the repository's config, then do a pull+deploy, then
remove and re-add the config, because /etc/ostree/remotes.d doesn't
exist yet in the target system.
https://bugzilla.gnome.org/show_bug.cgi?id=738698
src/libostree/ostree-repo.c:1759: Warning: OSTree:
ostree_repo_import_object_from: unknown parameter 'checksum' in
documentation comment, should be 'sha256'
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
And use it in pull-local. As one might expect, this is blazingly fast
if they're on the same filesystem.
I'll be using this to "promote" builds between different repositories.
For Fedora and potentially other distributions which use globally
distributed mirrors, metalink is a popular solution to redirect
clients to a dynamic set of mirrors.
In order to make metalink work though, it needs *one* file which can
be checksummed. (Well, potentially we could explode all refs into the
metalink.xml, but that would be a lot more invasive, and a bit weird
as we'd end up checksumming the checksum file).
This commit adds a new command:
$ ostree summary -u
To regenerate the summary file. Can only be run by one process at a
time.
After that's done, the metalink can be generated based on it, and the
client fetch code will parse and load it.
https://bugzilla.gnome.org/show_bug.cgi?id=729585
Changes the pull API to allow pulling only a single directory instead
of the whole deployment. This option is utilized by the check-diff
option in rpm-ostree.
Add a new state directory to hold <checksum>.commitpartial files, so
we know that we've only downloaded partial state.
This patch adds a function that will parse a partial checksum when
resolving a refspec. If the inputted refspec matches a truncated
existing checksum, it will return that checksum to be parsed. If
multiple truncated checksums match the partial refspec, it is not
unique and will return false. This addition is inspired by the same
functionality in Docker, which allows a user to reference a specific
commit without typing the entire checksum.
partial checksums: Add function to abstract comparison
This modifies the list_objects and list_objects_at functions
to take an additional argument for the string that a commit starts
with. If this string arg is not null, it will only list commit
objects beginning with that string. This allows for a new function
ostree_repo_list_commit_objects_starting_with to pass a partial string
and return a list of all matching commits. This improves on the
previous strategy of listing refs because it will list all commit objects,
even ones in past history. This update also includes bugfixes on
error handling and string comparison, and changes the output structure
of resolve_partial_checksum. The new strcuture will no longer return FALSE
without error. Also, the hashtable foreach now uses iter. Also
includes modified test file
There's several use cases for calling into ostree itself to do
mirroring, instead of using bare rsync. For example, it's a bit more
efficient as it doesn't require syncing the objects/ directory.
https://bugzilla.gnome.org/show_bug.cgi?id=728351
The current "transaction" symlink was introduced to fix issues with
interrupted pulls; normally we assume that if we have a metadata
object, we also have all objects to which it refers.
There used to be a "summary" which had all the available refs, but I
deleted it because it wasn't really used, and was still racy despite
the transaction bits.
We still want the pull process to use the transaction link, so don't
delete the APIs, just relax the restriction on object writing, and
introduce a new ostree_repo_set_ref_immediate().
For many OS install scenarios, one runs through an installer which may
come with embedded data, and then the OS is configured post-install to
receive updates.
In this model, it'd be nice to avoid the post-install having to rewrite
the /ostree/repo/config file.
Additionally, it feels weird for admins to interact with "/ostree" -
let's make the system feel more like Unix and have our important
configuration in /etc.
https://bugzilla.gnome.org/show_bug.cgi?id=729343
It's better if this is independent from the OstreeSysroot; for
example, a policy is active in a given deployment root at once, not
for a sysroot globally.
We can also collect SELinux-related API in one place.
Unfortunately at the moment there can be only one instance of this
class per process.
This has a very basic level of functionality (deltas can be generated,
and applied offline). There is only some stubbed out pull code to
fetch them via HTTP.
But, better to commit this now and improve it from a known starting
point, rather than have it languish in a branch.
This will be used by guestmount - it's WAY faster. We only take disks
as a unit, so it's safe. If the process fails halfway through, we
just start over from scratch the next time anyways.
The trees as shipped come with /usr/etc, which should just be labeled
as usr_t. When we do a deployment, we need to relabel the copies of
the files we're making in /etc.
SELinux support is compile and runtime optional.
We can't use #ifdef in the headers, since then g-ir-scanner won't pick
up the functions (unless we included config.h). Let's instead always
have the symbols, but just set an error if we were built without
support for it, just like how pull works.
Several APIs in libostree were moved there from the commandline code,
and have hardcoded g_print() for progress and notifications. This
isn't useful for people who want to write PackageKit backends, custom
GUIs and the like.
From what I can tell, there isn't really a winning precedent in GLib
for progress notifications.
PackageKit has the model where the source has GObject properties that
change as async ops execute, which isn't bad...but I'd like something
a bit more general where say you can have multiple outstanding async
ops and sensibly track their state.
So, OstreeAsyncProgress is basically a threadsafe property bag with a
change notification signal.
Use this new API to move the GSConsole usage (i.e. g_print()) out from
libostree/ and into ostree/.
Add a --generate-sizes option to commit to add size information to the
commit metadata. This will be used by higher level code which wants
to determine the total size necessary for downloading.
This uses gpgv for verification against DATADIR/ostree/pubring.gpg by
default. The keyring can be overridden by specifying OSTREE_GPG_HOME.
Add a unit test for commit signing with gpg key and verifying on pull;
to implement this we ship a test GPG key generated with no password
for Ostree Tester <test@test.com>.
Change all of the existing tests to disable GPG verification.
Add an optional dependency on gpgme to add GPG signatures into the
detached metadata, with the key "ostree.gpgsigs", as an "aay", an
array of signatures (treated as binary data).
The commit command gains a --gpg-sign=<key-id> argument. Also add an
argument --gpg-homedir to set the GPG homedir where we look for
keyrings.
read_commit resolves the ref to a commit, and a lot of consumers want
the resolved commit for their own purposes; this prevents them from
calling resolve_rev themselves.
https://bugzilla.gnome.org/show_bug.cgi?id=707727
We want an OstreeRepoFile to be the way to reference a "filesystem
tree" that's stored in the repo, which is a combination of a DIR_TREE
and a DIR_META. The idea is that once you write an mtree to the repo
using ostree_repo_write_mtree, it becomes serialized and you get an
OstreeRepoFile in return.
Change any APIs that care about DIR_TREE / DIR_META checksums to care
about OstreeRepoFiles instead, which right now is mostly is
ostree_repo_write_commit.
https://bugzilla.gnome.org/show_bug.cgi?id=707727
Previously I thought we'd have to ditch the current commit
format to avoid a{sv} due to
See https://bugzilla.gnome.org/show_bug.cgi?id=673012
But I realized that we don't really have to care about
unpacking/repacking commit objects, so let's just re-expose the
existing metadata a{sv} in commits in the API.
Also, add support for "detached" metadata that can be updated at any
time post-commit. This is specifically designed for GPG signatures.
https://bugzilla.gnome.org/show_bug.cgi?id=707379
Rather than having separate write_ref calls, make clients start a
transaction, add some refs, and then commit it. While this doesn't
make it 100% atomic, it makes it easier for us to use an atomic
model, and it means we don't do as much I/O updating the summary
file and such.
https://bugzilla.gnome.org/show_bug.cgi?id=707644
Matthew Barnes
Giuseppe Scrivano
Colin Walters
Anne LoVerso
Jeremy Whiting
Jasper St. Pierre