Commit Graph

322 Commits

Author SHA1 Message Date
Matthew Barnes ceacc57206 libotutil: Establish a place for GPG utilities
Add ot-gpg-utils.[ch] and move _ostree_gpg_error_to_gio_error() here.
2015-05-01 10:20:34 -04:00
Matthew Barnes 54bf665521 repo: Add ostree_repo_remote_get_gpg_verify()
Trivial function, but it does at least centralize the default value.
2015-04-16 18:13:13 -04:00
Matthew Barnes 20076ff201 repo: Add a "gpg-verify-result" signal
Emitted during a pull operation upon GPG verification (if enabled).
Applications can connect to this signal to output the verification
results if desired.
2015-04-16 18:13:04 -04:00
Matthew Barnes d0770e9993 repo: Improve error handling in sign_data()
Use _ostree_gpg_error_to_gio_error() so the actual GPG error message is
included in the GError.  Then apply an "Unable to blah: " message prefix.
2015-04-16 18:12:58 -04:00
Giuseppe Scrivano 362771aa98 summary: write the contents to a temporary file
do not write directly to the summary file but use a temporary file
first.  It avoids to create an empty file if "ot_util_variant_save"
fails.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-04-14 17:42:25 +02:00
Giuseppe Scrivano 69184e70e9 _ostree_repo_file_replace_contents: make buf const
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-04-14 17:42:24 +02:00
Colin Walters fff8ffdd2f repo: Add a private helper to replace a file, honoring fsync policy
Extracted from discussion in https://github.com/GNOME/ostree/pull/83
2015-04-13 13:28:02 -04:00
Matthew Barnes a25c7fab12 Add ostree_repo_is_writable() 2015-03-25 17:24:05 -04:00
Alexander Larsson 74f4fc2bc8 Fix build with !HAVE_LIBSOUP
Fix up ostree_repo_pull to match the new declaration
2015-03-23 14:26:01 +01:00
Matthew Barnes 9033cf5da1 repo: Reject duplicate signatures when signing commit
Uses OstreeGpgVerifyResult to catch duplicate signatures.

If the commit has already been signed with the given GPG key ID, fail
with a G_IO_ERROR_EXISTS error code.
2015-03-18 11:53:04 -04:00
Matthew Barnes 8d127b9dcb repo: Add ostree_repo_verify_commit_ext()
Similar to ostree_repo_verify_commit(), but returns more verification
details by way of an OstreeGpgVerifyResult object instead of a boolean.
2015-03-18 11:52:47 -04:00
Matthew Barnes 4a2733f9e7 gpg: Add OstreeGpgVerifyResult
Wrappers a referenced gpgme_verify_result_t so detailed verify results
can be examined independently of executing a verify operation.

_ostree_gpg_verifier_check_signature() now returns this object instead
of a single valid/invalid boolean, but the idea is for OstreeRepo to also
return this object for commit signature verification so it can be utilized
at the CLI layer (and possibly by other programs).
2015-03-18 11:52:22 -04:00
Matthew Barnes f47693440d OstreeGpgVerifier: Take the signed data as a GBytes
Similar to c2b01ad.  For some reason I was thinking the commit data
still needed to be written to disk prior to verifying, but it's just
another artifact of spawning gpgv2 (predates using GPGME).

Makes for a nice cleanup in fetch_metadata_to_verify_delta_superblock()
as well.
2015-03-16 16:37:11 -04:00
Giuseppe Scrivano ccbb10d107 ostree-repo.c: fix typo
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-03-16 11:34:48 +01:00
Matthew Barnes 187e8d632e configure.ac: Make gpgme a hard dependency
In anticipation of API enhancements for GPG signature verification, which
would otherwise require a non-functional stub version were GPGME excluded.

GPGME is a pretty lightweight dependency, and the motivation to exclude
it is not clear.
2015-03-11 12:03:33 -04:00
Matthew Barnes c2b01adbf0 OstreeGpgVerifier: Take the signature as a GBytes
The signature data is in memory to begin with, so there's no need to
write it to disk only to immediately read it back.

Also, because the GPGME multi-keyring workaround is somewhat expensive
to setup and teardown, concatenate all signatures into a single GBytes
so _ostree_gpg_verifier_check_signature() is only called once.  We're
currently only looking for one valid signature anyway.
2015-03-06 08:22:44 -05:00
Colin Walters 191d78ebfc repo: Fix assertion to allow NULL options
Spotted by Adam Coldrick.
2015-03-03 21:18:40 -05:00
Colin Walters 488efac728 repo: Port hardlink-scanning code to fd-relative calls
Continuing the migration.
2015-03-03 11:48:41 -05:00
Matthew Barnes c04f9f47e3 core: Add definitions for GPG signature metadata
_OSTREE_METADATA_GPGSIGS_NAME
_OSTREE_METADATA_GPGSIGS_TYPE
2015-03-02 15:12:06 -05:00
Colin Walters 152dcf89b5 repo: detached sigs: Use error prefixing instead of overwriting
Noted when "rpm-ostree compose sign" failed to write to a repo due to
permissions.
2015-02-25 10:39:15 -05:00
Colin Walters 3b3708c312 repo: Port APIs used by prune to fd-relative *at calls
More of the general trend away from GFile * to the faster and more
secure world of *at().
2015-02-22 23:20:12 -05:00
Matthew Barnes 5aa0d51d7a repo: Check for OSTREE_REPO in ostree_repo_new_default()
Convenience feature to avoid having to pass --repo options repeatedly.

Before falling back to the default system repository path, check for a
repository path defined by the OSTREE_REPO environment variable.
2015-02-19 20:44:34 -05:00
Colin Walters 08476ce254 deltas: Prune deltas when the corresponding "to" commit vanishes
We want prune to actually give you back disk space when using deltas.
2015-02-16 10:10:35 -05:00
Colin Walters ee4e393fa1 repo: Store pending objects in prefixed subdirectory
I was hitting a bug in libguestfs/guestmount/FUSE where it blew up
with EINVAL on directories containing lots of files (more than
32000?).  We really want to use prefixed subdirs just like the real
objects/ directory does.

This allows us to share more code between the paths, is more
efficient, etc.
2015-02-15 15:30:19 -05:00
Colin Walters 9cc9804195 Change pull-local to just be a wrapper for pull with file:///
This follows up from the previous commit; now that pull knows how to
do the efficient link() or copy for local files, we can just have
pull-local call into ostree_repo_pull().

As part of this:
 - pull() can also accept a file:/// URI instead
   of a remote name (since pull local supports anonymous pulls)
 - pull() knows an "override-remote-name" option, since pull-local
   supported writing a ref out even if there wasn't a remote with
   that name
2015-02-08 04:43:06 -05:00
Colin Walters be4ad54800 Add an internal API to get a read fd for a content object
This will be used by the static deltas work.
2015-02-05 21:50:26 -05:00
Colin Walters 3b8ed12ab2 repo: Hold an fd "repo_dir_fd" open for the toplevel too
We could just make everything relative to this, but the objects/ and
tmp/ are accessed very often, so I think it's worth holding individual
fds.

This fd can cover everything else: refs, deltas, etc.
2015-02-05 14:15:34 -05:00
Giuseppe Scrivano 6a3959c895 syntax-check: Remove empty lines at the end of file
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-02-02 15:07:56 +01:00
Giuseppe Scrivano 27a45e2edb pull: use a single per-transaction syncfs instead of fsync
Do not write directly to objects/ but maintain pulled files under tmp/
with a "tmpobject-$CHECKSUM.$OBJTYPE" name until they are syncfs'ed to
disk.

Move them under objects/ at ostree_repo_commit_transaction cleanup
time.

Before (test done on a local network):

$ LANG=C sudo time ./ostree --repo=repo pull origin master

0 metadata, 3 content objects fetched; 83820 KiB; 4 delta parts
fetched, transferred in 417 seconds
16.42user 6.73system 6:57.19elapsed 5%CPU (0avgtext+0avgdata
248428maxresident)k
24inputs+794472outputs (0major+233968minor)pagefaults 0swaps

After:

$ LANG=C sudo time ./ostree --repo=repo pull origin master

0 metadata, 3 content objects fetched; 83820 KiB; 4 delta parts
fetched, transferred in 9 seconds
14.70user 2.87system 0:09.99elapsed 175%CPU (0avgtext+0avgdata
256168maxresident)k
0inputs+794472outputs (0major+164333minor)pagefaults 0swaps

https://bugzilla.gnome.org/show_bug.cgi?id=728065

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-01-30 14:03:42 -05:00
Colin Walters f2e4830409 pull: Further extend static delta progress
With deltas, we have an accurate total size, among other things.
2015-01-22 09:07:13 -05:00
Alexander Larsson 5b721a5b08 Allow creating and pulling static deltas starting from "empty"
You create these with something like:
  ostree static-delta generate --empty --to=master

These will be automatically used during pull if no previous revision
exists in the target repo.

These work very much like the normal static deltas except they
are named just by the "to" revision. I.e:

deltas/94/f7d2dc23759dd21f9bd01e6705a8fdf98f90cad3e0109ba3f6c091c1a3774d

for a from-scratch to 94f7d2dc23759dd21f9bd01e6705a8fdf98f90cad3e0109ba3f6c091c1a3774d delta.

https://bugzilla.gnome.org/show_bug.cgi?id=721799
2015-01-14 14:43:32 +01:00
Colin Walters 019635d9c2 repo: Fix bare-user file loads
Regression from 86764dbf00

This function is kind of fiendish now that we have 3 cases, each of
which want to be optimized somewhat to only load what's necessary
(e.g. don't open the file if we don't have an output for stream
requested).

Clean things up so that BARE_USER and BARE are separate conditionals
that share as much as possible, and fix the bug that asserted we
were in BARE mode.

I tested this by running test-basic-user.sh by hand.
2015-01-12 12:43:33 -05:00
Colin Walters 25a5909500 repo: Deduplicate some code in load_file 2015-01-12 12:43:33 -05:00
Matthew Barnes 6ff841d3b0 repo: Report metadata fetch progress separately
Partially resolves https://bugzilla.gnome.org/740276
2015-01-12 11:21:18 -05:00
Colin Walters 026c5c60d3 Use *at() functions for native filesystem commits
This is just an efficiency optimization.  We're getting fairly close
to all of the hot code paths using `*at()`.

Note that we end up maintaining a half-duplicate code path set here,
because we still need to support commits from an arbitrary GFile *,
which in a possible common case is an OSTree commit.

I think it's worth it though.
2015-01-06 22:43:14 -05:00
Colin Walters 86764dbf00 repo: Improve ostree_repo_load_file() to use `*at()` for xattrs
We were already using openat() for the contents, but not the xattrs.
Now that libgsystem 2014.3 has gs_fd_get_all_xattrs(), make use of it.

Clean things up a bit so we only open the fd once.
2015-01-06 16:47:03 -05:00
Colin Walters 1e8e070102 Port to libgsystem errno API, hard depend on 2014.3
This is long overdue to make it to libgsystem.  Update our dependency.
2015-01-04 21:17:11 -05:00
Colin Walters 1bcc7a8e3a Merge branch 'giuseppe/staticdeltas' of https://github.com/giuseppe/ostree 2014-12-19 16:31:31 -05:00
Colin Walters c4efbf6718 core: Deduplicate code converting struct stat -> GFileInfo
We were doing the same thing in a number of places, make a helper
function.
2014-12-19 10:18:59 -05:00
Matthew Barnes 880328ba03 Add ostree_repo_pull_default_console_progress_changed()
Replaces ot_common_pull_progress() in ostree binary, so it can be shared
with rpm-ostree.
2014-12-18 21:31:53 -05:00
Colin Walters ca678224be Static deltas support
https://bugzilla.gnome.org/show_bug.cgi?id=721799
2014-12-18 12:48:47 +01:00
Matthew Barnes 39be27fc93 Add ostree_repo_remote_list()
Lists available remote names.
2014-12-17 16:05:37 -05:00
Colin Walters f6a6e68412 Add more flexible _remote_change() API , expose via 'ostree remote'
For Anaconda, I needed OSTREE_REPO_REMOTE_CHANGE_ADD_IF_NOT_EXISTS,
with the GFile *sysroot argument to avoid ugly hacks.  We want to
write the content provided via "ostreesetup" as a remote to the target
chroot only in the case where it isn't provided as part of the tree
content itself.

This is also potentially useful in idempotent systems management tools
like Ansible.

https://bugzilla.gnome.org/show_bug.cgi?id=741577
2014-12-15 21:28:09 -05:00
Colin Walters 8067e977a7 repo: (cleanup) Use _is_system() rather than inlining it
No reason to duplicate the code.
2014-12-15 13:59:46 -05:00
Matthew Barnes ac882f2128 repo: Add private functions to access remote options
ostree_repo_pull_with_options() needs this, and I'd rather keep the
OstreeRemote struct definition tucked away in ostree-repo.c with its
own internal API.
2014-12-08 12:49:25 -05:00
Matthew Barnes f79896693e repo: Add ostree_repo_remote_get_url()
Peeking at remote details by way of ostree_repo_copy_config() doesn't
work anymore.
2014-12-08 12:49:23 -05:00
Matthew Barnes 17b9e399b8 repo: Add an internal struct to manage remotes
OstreeRemote is a reference-counted struct that encompasses data about a
remote, whether read from a configuration file or created explicitly via
ostree_repo_remote_add().

OstreeRemotes are held in an internal table indexed by remote name.
This solves some problems caused by merging system-wide remote data into
the OstreeRepo's internal config key file.

Also fixes https://bugzilla.gnome.org/show_bug.cgi?id=740911
2014-12-08 12:49:10 -05:00
Alexander Larsson 47c612e5a0 Support for "bare-user" repo format
This format is pretty much the same as the "bare" format, except the
file ownership and xattrs is not stored in the actual filesystem object, but
rather on the side in a user xattr. This means two things:

1) An unprivileged user can store such a repo independent of the types
   of files in it or their xattrs. And you can later (as root)
   reconstruct the real filesystem tree with ownership. Although you
   can't do that using hardlink-sharing. This also means ostree
   fsck does a full verification.

2) Such a repository can be checked out with user-mode (checkout -U)
   as an unprivileged user using hardlinks for space sharing.

Additionally, symlinks are stored as regular files (with the content
being the symlink target) because user xattrs are not supported on
symlinks. We know at checkout time if the file is a symlink because
the original st_mode is stored in the xattr metadata.

https://bugzilla.gnome.org/show_bug.cgi?id=741125
2014-12-08 10:39:39 +01:00
Matthew Barnes 59fd2b1e43 repo: Remove unused remote-cache dir
Unused leftover from when ostree had pack files; minor cleanup.

https://bugzilla.gnome.org/show_bug.cgi?id=739746
2014-11-06 13:31:58 -05:00
Matthew Barnes e4caf3c3cd Fix GFile leak in ostree_repo_constructed() 2014-11-05 15:41:33 -05:00