The tmp directory is lazily created for each fetcher instance, since
it may require superuser permissions and some instances only need
_ostree_fetcher_request_uri_to_membuf() which keeps everything in
memory buffers.
This way two pulls will not use the same tmpdir and accidentally
overwrite each other. However, consecutive OstreeFetchers will reuse
the tmpdirs, so that we can properly resume downloading large objects.
https://bugzilla.gnome.org/show_bug.cgi?id=757611
Concurrent pulls break since we're sharing the staging directory for
all transactions in the repo. This makes us use a per-transaction directory.
However, in order for resumes to work we first look for existing
staging directories and try to aquire an exclusive lock for them. If
we can't find any staging directory or they are all already locked,
then we create a new one.
https://bugzilla.gnome.org/show_bug.cgi?id=757611
This creates a subdirectory of the tmp dir with a selected prefix,
and takes a lockfile to ensure that nobody else is using the same directory.
However, if a directory with the same prefix already exists and is
not locked that is used instead.
The later is useful if you want to support some kind of resumed operation
on the tmpdir.
touch reused dirs
https://bugzilla.gnome.org/show_bug.cgi?id=757611
When a commit is deleted and the repo is configured to use tombstone
commits, create one. Delete the tombstone file only if the commit is
pulled again.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Had a rare situation where I had no libsoup development files, so I
took the opportunity to fix the build errors. Ugly, but works now.
Would be nice if libsoup could be a hard dependency since we rarely
ever test a configuration without it.
xdg-app was hanging for me with v2015.8, but worked with v2015.7.
I narrowed things down to the GMainLoop/context commit, in which
we started pushing a temporary main context for synchronous
requests internally.
That's never really going to work with libsoup - there needs
to be a single main context which works on the socket. Furthermore,
clients couldn't get progress messages that way.
For *other* internal uses where we added APIs that talk to the remote
repo, we cleanly push a temporary main context.
(Note that I kind of snuck in a change here around the GError handling
in pulls that isn't strictly related but came up in testing)
First of all, what we were doing with having GMainLoop in the internal
APIs is wrong. Synchronous APIs should always create their own main
context and not iterate the caller's. Doing the latter creates
potential for evil reentrancy issues. Sync API should block, async
API is for not blocking.
Now that's out of the way, fix the pull code to do the clean
```
while (termination_condition (state))
g_main_context_iteration (mainctx, TRUE);
```
model for looping. This is a lot easier to understand and ultimately
more reliable than having other code call `g_main_loop_quit()`, as the
loop condition is in exactly one place.
We can also remove the idle source which only fired once.
Note we have to add a hack here to discard the synchronous session and
create a new one which we only use async.
https://bugzilla.gnome.org/show_bug.cgi?id=753336
It allows to specify whether GPG verification for the summary file is
enabled for a specific repository.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
The global keyring directory (trusted.gpg.d) is deprecated. Only use it
when a specified remote does NOT have its own keyring, or when verifying
local repository objects.
Note, because mixing in the global keyring directory is now an explicit
choice, OstreeGpgVerifier no longer needs to implement GInitableIface.
If a remote keyring does not already exist, create an empty pubring.gpg
file in the temporary directory prior to importing keys. This prevents
gpg2 from creating a pubring.kbx file in the new keybox format [1]. We
want to stay with the older keyring format since its performances issues
are not relevant here.
[1] https://gnupg.org/faq/whats-new-in-2.1.html#keybox
External daemons like rpm-ostree want push notification any time a
change is made by an external entity. inotify provides notification,
but a problem is there's no easy way to monitor all of the refs.
In the past, there has been discussion of opt-in recursive timestamps:
https://lkml.org/lkml/2013/4/5/307
But in today's world, let's just bump the mtime on the repo itself, as
a central inotify point.
Closes: https://github.com/GNOME/ostree/pull/111
Imports one or more GPG keys from a source stream or from the user's
personal keyring into a remote-specific keyring. The keys to import
can optionally be restricted by a list of key IDs.
The imported keys are used to conduct GPG verification when pulling
from the given remote.
An OSTree user noticed that `ostree fsck` would produce `missing
object` errors in the case of interrupted pulls.
It's possible to do e.g. `ostree pull --subpath=/usr/share/rpm ...`,
which gets you just that portion of the commit. The use case for this
was being able to see what changes would appear in an update before
actually downloading all of it.
(I think this would be better covered by static deltas, but those
aren't final yet, and `--subpath` predates it)
Further, `.commitpartial` is used as a successor to the `transaction`
symlink for more precise knowledge in the case where a pull was
interrupted that we needed to resume scanning.
So it makes sense for `ostree fsck` to be aware of it.
Matthew Barnes
Alexander Larsson
Giuseppe Scrivano
Colin Walters
John Hiesey
Jeff Ortel