Commit Graph

262 Commits

Author SHA1 Message Date
Colin Walters e392820541 core: Do enumerate ff/ objects as well
An embarassing off-by-one here.  I noticed we weren't pruning them.

https://bugzilla.gnome.org/show_bug.cgi?id=733458
2014-07-21 07:25:51 -04:00
Colin Walters 1834a71b1f core: Do enumerate content objects in archive-z2 repositories
Prune has worked fine on bare repositories for some time, but now that
I finally try to delete data on the server side, I notice we weren't
actually enumerating content objects =/

That caused them to not be pruned.

https://bugzilla.gnome.org/show_bug.cgi?id=733458
2014-07-21 07:25:51 -04:00
Colin Walters 72da2e0c36 core: Unify object deletion code with prune
The prune API duplicated logic to delete objects, and furthermore the
core API to delete an object didn't clean up detached metadata.

Fix the duplication by doing the obvious thing: prune should call
_delete.

https://bugzilla.gnome.org/show_bug.cgi?id=733452
2014-07-20 08:57:37 -04:00
Anne LoVerso fd56952d6b ostree-repo-resolve-rev: Add function to accept a partial checksum
This patch adds a function that will parse a partial checksum when
resolving a refspec.  If the inputted refspec matches a truncated
existing checksum, it will return that checksum to be parsed.  If
multiple truncated checksums match the partial refspec, it is not
unique and will return false.  This addition is inspired by the same
functionality in Docker, which allows a user to reference a specific
commit without typing the entire checksum.

partial checksums: Add function to abstract comparison

This modifies the list_objects and list_objects_at functions
to take an additional argument for the string that a commit starts
with.  If this string arg is not null, it will only list commit
objects beginning with that string.  This allows for a new function
ostree_repo_list_commit_objects_starting_with to pass a partial string
and return a list of all matching commits.  This improves on the
previous strategy of listing refs because it will list all commit objects,
even ones in past history.  This update also includes bugfixes on
error handling and string comparison, and changes the output structure
of resolve_partial_checksum. The new strcuture will no longer return FALSE
without error.  Also, the hashtable foreach now uses iter.  Also
includes modified test file
2014-07-18 15:15:44 -04:00
Colin Walters d7067975e7 [staticanalysis]: Fix two uses of uninitialized variables
https://bugzilla.gnome.org/show_bug.cgi?id=732020
2014-06-21 18:08:49 -04:00
Colin Walters 2265ccdeb3 libostree: Add ostree_repo_remote_add() API, port "ostree remote add"
At least one external tool is using the API, and wants to add a
remote, but all of the logic right now is in the tool.  Move it to the
library.

https://bugzilla.gnome.org/show_bug.cgi?id=731984
2014-06-20 15:36:46 -04:00
Colin Walters 51bf1fab0f repo: Support fsync=false configuration
See rationale in the updated docs.  Basically developer use cases as
well as UPS-backed servers.
2014-06-05 17:35:50 -04:00
Colin Walters a4d01976bb repo: Only load /etc/ostree/remotes.d for system repo
They shouldn't be loaded for random test/personal repositories.  Doing
so triggers another bug in that we return them from
ostree_repo_get_config() when then causes clients to write them out
permanently to disk with ostree_repo_write_config().  This caused test
suite failures.
2014-05-26 18:38:35 -04:00
Colin Walters f47a20fb81 Support /etc/ostree/remotes.d
For many OS install scenarios, one runs through an installer which may
come with embedded data, and then the OS is configured post-install to
receive updates.

In this model, it'd be nice to avoid the post-install having to rewrite
the /ostree/repo/config file.

Additionally, it feels weird for admins to interact with "/ostree" -
let's make the system feel more like Unix and have our important
configuration in /etc.

https://bugzilla.gnome.org/show_bug.cgi?id=729343
2014-05-08 18:59:24 -04:00
Colin Walters cbc11f42e5 libostree: Add API to append a GPG signature
This will be used by rpm-ostree which needs to use an external program
to sign commits.
2014-03-14 21:57:48 -04:00
Colin Walters 3ffdef07a4 repo: Split generic GPG commit verification out into helper
This will be used for a future commit which GPG verifies static
deltas.
2014-02-10 09:40:35 -05:00
Colin Walters 2ee2751fd1 repo: Don't set GPG engine executable path
The instructions one finds on the internets are apparently wrong, we
really need to keep the default here, since gpgme uses it to actually
find the helper binary it runs.

This fixes the GPG tests for me on EL7 at least.
2014-02-10 09:12:00 -05:00
Colin Walters 2d6374822b Initial basic static delta code drop
This has a very basic level of functionality (deltas can be generated,
and applied offline).  There is only some stubbed out pull code to
fetch them via HTTP.

But, better to commit this now and improve it from a known starting
point, rather than have it languish in a branch.
2014-02-04 10:31:44 -05:00
Colin Walters 234ae70b4c Add --disable-fsync option to pull-local, and API to repo
This will be used by guestmount - it's WAY faster.  We only take disks
as a unit, so it's safe.  If the process fails halfway through, we
just start over from scratch the next time anyways.
2014-01-31 22:18:13 -05:00
Colin Walters f36cc6a44c repo: Improve GPG error messages
The signing test is failing here on EL7 beta for me - it seems like
gnupg isn't honoring the homedir.
2014-01-30 17:17:36 -05:00
Colin Walters 43fdfa0649 repo: Document ostree_repo_sign_commit(), and add introspection data
The important bit for introspection is (allow-none) on the homedir.
2014-01-20 12:13:37 -05:00
Colin Walters 9adbc165e7 repo: Introspect _sign_commit() and _verify_commit()
We can't use #ifdef in the headers, since then g-ir-scanner won't pick
up the functions (unless we included config.h).  Let's instead always
have the symbols, but just set an error if we were built without
support for it, just like how pull works.
2013-12-23 11:40:57 -05:00
Colin Walters 76d9d92916 repo: Provide a friendlier error message if no signatures are found
I always forget the key to disable it...
2013-12-17 23:00:01 -05:00
Colin Walters a24afd68f0 Move xattr handling into libgsystem, fix sysroot to handle directory ownership/perms
This large patch moves the core xattr logic down into libgsystem,
which allows the gs_shutil_cp_a() API to copy them.  In turn, this
allows us to just use that API instead of rolling our own recursive
copy here.

As noted in the new comment though, one case that we are explicitly
regressing is where the new /etc removes a parent directory that's
needed by a modified file.  This seems unlikely for most vendors now,
but let's do that as a separate bug.

https://bugzilla.gnome.org/show_bug.cgi?id=711058
2013-11-28 23:28:13 -05:00
Colin Walters 73ff642b15 libostree: Remove extra : in comment
g-ir-scanner handles this, but it stuck out.
2013-10-24 20:32:20 -04:00
Colin Walters c65923e642 Add OstreeAsyncProgress, use it for ostree_repo_pull
Several APIs in libostree were moved there from the commandline code,
and have hardcoded g_print() for progress and notifications.  This
isn't useful for people who want to write PackageKit backends, custom
GUIs and the like.

From what I can tell, there isn't really a winning precedent in GLib
for progress notifications.

PackageKit has the model where the source has GObject properties that
change as async ops execute, which isn't bad...but I'd like something
a bit more general where say you can have multiple outstanding async
ops and sensibly track their state.

So, OstreeAsyncProgress is basically a threadsafe property bag with a
change notification signal.

Use this new API to move the GSConsole usage (i.e. g_print()) out from
libostree/ and into ostree/.
2013-10-24 14:27:13 -04:00
Jeremy Whiting f583c4ab0b core: Add size information to commit metadata
Add a --generate-sizes option to commit to add size information to the
commit metadata.  This will be used by higher level code which wants
to determine the total size necessary for downloading.
2013-10-19 11:56:51 -04:00
Colin Walters 7c903eb889 repo: Remove set-but-unused variable
The gpgme examples use this, but from what I can tell we don't really
need to because we don't need detailed results; we only care whether
we signed it at all.
2013-10-15 16:44:19 -04:00
Colin Walters 7b119370a2 repo: Enumerate objects using openat() too, make more efficient
This drops several calls to malloc()/g_object_new() per object
enumeration.  Just a followup to using openat() and friends in other
places.
2013-09-30 08:51:25 -04:00
Jeremy Whiting 7f9eefb62d pull: Verify commits with gpg signatures from detached metadata
This uses gpgv for verification against DATADIR/ostree/pubring.gpg by
default.  The keyring can be overridden by specifying OSTREE_GPG_HOME.

Add a unit test for commit signing with gpg key and verifying on pull;
to implement this we ship a test GPG key generated with no password
for Ostree Tester <test@test.com>.

Change all of the existing tests to disable GPG verification.
2013-09-29 14:49:47 -04:00
Jeremy Whiting 7d5aa74dae core: Use libgpgme to add GPG signatures to detached metadata for commit object
Add an optional dependency on gpgme to add GPG signatures into the
detached metadata, with the key "ostree.gpgsigs", as an "aay", an
array of signatures (treated as binary data).

The commit command gains a --gpg-sign=<key-id> argument.  Also add an
argument --gpg-homedir to set the GPG homedir where we look for
keyrings.
2013-09-28 16:12:35 -04:00
Colin Walters 27b46cb45c libostree: Fix compiler warning 2013-09-10 12:25:13 -04:00
Jasper St. Pierre 223bccf2f6 repo: Make read_commit spit out a resolved commit ref as well
read_commit resolves the ref to a commit, and a lot of consumers want
the resolved commit for their own purposes; this prevents them from
calling resolve_rev themselves.

https://bugzilla.gnome.org/show_bug.cgi?id=707727
2013-09-09 22:58:39 -04:00
Jasper St. Pierre 33db74d0ce repo-file: Base OstreeRepoFile on trees instead of commits
We want an OstreeRepoFile to be the way to represent a filesystem tree
inside an ostree repository. In order to do this, we need to drop the
commit from an OstreeRepoFile, and make that go to callers.

Switch all current users of ostree_repo_file_new_root to
ostree_repo_read_commit, and make the actual constructor private.

https://bugzilla.gnome.org/show_bug.cgi?id=707727
2013-09-09 22:31:59 -04:00
Colin Walters 3ff4b1874c core: Use openat() for reading bare file objects too
...unless we want xattrs, in which case we have to fallback to path
lookup due to lack of llistxattrat().

This looks nicer in strace.
2013-09-08 19:31:44 -04:00
Colin Walters d863ef5838 repo: Delete leftover repo/pending dir
This is a relic from long ago when we were trying to stage objects
before finally committing them all in one go in the pull code.

We're no longer doing that, so stop trying to make the directory.
This also fixes trying to use ostree as non-root to read the
root-owned repo, since we'd fail to create the pending dir.
2013-09-08 15:04:35 -04:00
Colin Walters 9846fb27fd core: Use linkat() for hardlink checkouts too
Clean up how we deal with the uncompressed object cache; we now use
openat()/linkat() and such just like we do for the main objects/.

Use linkat() between the objects and the destination, if possible.

https://bugzilla.gnome.org/show_bug.cgi?id=707733
2013-09-08 14:40:09 -04:00
Colin Walters 75225166da repo: Fix file descriptor leak from previous commit
I keep forgetting mmap doesn't take ownership of the fd.
2013-09-08 10:41:18 -04:00
Colin Walters 5a685ff989 core: Use at-relative lookups for archive-z2 content
We can't quite do it for bare repositories yet because we need to have
a way to go from struct stat -> GFileInfo, and that's buried in gio's
private GLocalFile class.
2013-09-07 10:10:21 -04:00
Colin Walters d2c6e19278 core: Use at-relative lookups for metadata reading
Just use openat() for locating variants, rather than doing the lstat()
+ open().  This also drops several malloc+object allocations from the
lookup path.
2013-09-07 05:03:48 -04:00
Colin Walters b97249d4f9 core: Use at-relative lookup for locating objects too
Add new internal API to both fstatat() and write a pathname for the
given object.  Use it in commit, and also wrapped in the old
GFile-based API.

This is more efficient.
2013-09-07 04:47:40 -04:00
Colin Walters a7c0992d94 core: Add malloc-free API for objects, use *at functions for storing
This is more efficient; we avoid malloc of a number of pathname +
GFile objects, plus the kernel doesn't have to traverse the repo path
again.
2013-09-07 04:18:41 -04:00
Jasper St. Pierre 3dcff0b2c4 Fix build
Before I pushed the "move code to a different file" commit, I rebased,
but forgot to test the build. Sorry.
2013-09-06 20:47:13 -04:00
Jasper St. Pierre 91d3b90c9f Move ref writing to be transaction-based
Rather than having separate write_ref calls, make clients start a
transaction, add some refs, and then commit it. While this doesn't
make it 100% atomic, it makes it easier for us to use an atomic
model, and it means we don't do as much I/O updating the summary
file and such.

https://bugzilla.gnome.org/show_bug.cgi?id=707644
2013-09-06 20:31:12 -04:00
Jasper St. Pierre 5082e1d8e9 repo: Rename "stage" to "write" in the API
An earlier version of this API acted like git in that some objects
would be staged in a temporary directory which would be then committed
in one go by moving files around. The API doesn't match most users
expectations though, as while the stage is nice as a high-level API
it isn't really suited for low-level APIs.

While the stage was removed, the APIs were never renamed. Rename
them now so that they match expectations.

https://bugzilla.gnome.org/show_bug.cgi?id=707644
2013-09-06 20:31:12 -04:00
Jasper St. Pierre c817217ad8 repo: Move commit code to another file
ostree-repo.c is a bit too big, and most of the commit code is
fairly standalone.

https://bugzilla.gnome.org/show_bug.cgi?id=707644
2013-09-06 20:31:11 -04:00
Jasper St. Pierre 89f4817bdf repo: Make prepare_transaction introspectable
We need to document the out pointer as an out pointer.

https://bugzilla.gnome.org/show_bug.cgi?id=707644
2013-09-06 20:28:42 -04:00
Jasper St. Pierre b36e61f813 repo: Make OSTreeCommitModifier introspectable
Callbacks need GDestroyNotifies to be usable from bindings.

https://bugzilla.gnome.org/show_bug.cgi?id=707644
2013-09-06 20:28:42 -04:00
Jasper St. Pierre f5f7fe5e9a repo: Make the ordering consistent between abort/complete_transaction
This just makes the functions nicer to stare at.

https://bugzilla.gnome.org/show_bug.cgi?id=707644
2013-09-06 20:28:42 -04:00
Colin Walters 2b61caa2fe core: Use fd-relative creation for tmp/
Update libgsystem submodule for a bugfix.

This is both more efficient from a kernel perspective, and avoids us
calling gs_file_get_path_cached() on tmp_dir constantly, which
triggered another bug due to lack of locking.
2013-09-06 18:45:16 -04:00
Jasper St. Pierre 02adfc8c03 repo: Rename ostree_repo_check to ostree_repo_open
As it more clearly describes what the function does: load the
repo from disk and initialize it.

At the same time, add a cancellable parameter.

https://bugzilla.gnome.org/show_bug.cgi?id=707582
2013-09-05 14:07:03 -04:00
Jasper St. Pierre 57d285f619 repo: Move the 'init' builtin command to a public API, ostree_repo_create
This continues the goal of making more of ostree accessible by API,
rather than forking out to subprocesses.

https://bugzilla.gnome.org/show_bug.cgi?id=707582
2013-09-05 14:07:03 -04:00
Jasper St. Pierre eb6f017ef8 repo: Use constructed, not constructor
It's simpler to do this after the object is constructed, rather
than in the middle of construction.

https://bugzilla.gnome.org/show_bug.cgi?id=707582
2013-09-05 13:54:36 -04:00
Colin Walters 2534714501 core: Make ostree_write_variant_with_size() private
More work making the file formats and utilities private.
2013-09-04 07:49:41 -04:00
Colin Walters 0d72168961 libostree: Make the format of file objects private
It's now isolated almost entirely to ostree-core.c, except
ostree-repo.c needs to know how to create archive-z2 file headers.  So
give it a private API for that.
2013-09-03 18:43:42 -04:00