If this is true, don't initiate, abort of commit a transaction, instead
it is assumed that the caller initiated the transaction, and that it
will eventually be commited.
This allows you to do multiple pulls or a combination of pulls and
commits in a single transaction.
Closes: #525
Approved by: cgwalters
This pulls in a new compilation flag for wrpseudo compatibility. Also
note we need to add some includes since glnx-libcontainer went away,
and with it some includes for `sys/mount.h` etc.
Closes: #522
Approved by: cgwalters
We were seeing some weird potential memory corruption in this code
when using it for `rpm-ostree-toolbox installer`, which is almost
certainly not its fault, but let's use it as an excuse to port
(mostly) to fd-relative and away from GFile.
Dropping the last GFile use here is a bit tricky as it does have a
nice high level wrapper around inotify.
Closes: #512
Approved by: jlebon
My previous change in https://github.com/ostreedev/ostree/pull/425
actually broke things so we basically used the repository defaults =(
This is a subtle mess since we're only trying to flip things off
for the *new* API.
Clean this up so that the "default repo inheritance" lives only in one
place - in the compat layer for the old checkout API. The new
checkout API defaults to off period, so the repository state is
irrelevant.
Closes: #520
Approved by: jlebon
/var/log is another one of those core directories that should be made
available and properly labeled during early boot before tmpfiles.d
starts up.
Related: https://bugzilla.redhat.com/show_bug.cgi?id=1265295Closes: #513
Approved by: cgwalters
While looking at a slow update issue (which I'm guessing is
unpredictable I/O latency in an OpenStack instance), I noticed
in one of the traces we were inside a fsync here.
Dropping the fsync here is just another of a long series of unwinding
them - we `syncfs()` the sysroot fd and `/boot` and we have a big
`sync()` anyways.
Closes: #508
Approved by: jlebon
More fsync pruning. Since we have a public API for writing the origin
file and it did a fsync before, let's preserve that. But when writing
deployments as part of a full transaction, we rely on the global
`syncfs()`, so add an internal function for origin file writing that
doesn't.
Closes: #509
Approved by: giuseppe
If ostree_sysroot_unload() was called explicitly, then sysroot_fd
would be closed again at finalization time, possibly closing a
random file descriptor belonging to some other part of the
application.
Closes: #507
Approved by: cgwalters
0 was used as an "unset" flag for tmp_dir_fd, which is technically
incorrect. For cache_dir_fd, -1 was used as the sentinal but 0
was checked for, resulting in close(-1).
Closes: #507
Approved by: cgwalters
In https://github.com/ostreedev/ostree/pull/408 we fixed a
bug where we would crash when trying to execute deltas into
an archive repo (which isn't presently supported).
But that was overly aggressive - we obviously *can* execute deltas
when mirroring into a bare repo. This should fix a regression with
the way flatpak uses mirroring to pull from a user repo into the
system.
Closes: #506
Approved by: alexlarsson
This way e.g. flatpak can detect which timestamp it should use.
See `flatpak/common/flatpak-utils.c:flatpak_zero_mtime`.
Closes: #501
Approved by: cgwalters
Since forever, we've been doing two cleanups. In
8ece4d6d51
I thought we were doing just one and wanted to go to zero (if specified),
but I actually just dropped one cleanup.
In https://github.com/projectatomic/rpm-ostree/pull/452
@jlebon pointed out the duplication. Fix this by creating a new internal
deploy wrapper that takes cleanup flags.
(Since we already had the "piecemeal cleanup" API internally, let's
frame it in terms of that, rather than passing down a boolean).
Closes: #500
Approved by: jlebon
If the current repo is already up to date (we have no content to
fetch), it's possible for the fetcher to not request any URIs. So
create and then finalize it quickly.
Finalization involves calling `g_main_loop_quit()` +
`g_thread_wait()`. However, if `g_main_loop_quit()` is run *before*
`g_main_loop_run()`, we'll deadlock because GMainLoop assumes in
`_run()` to start things.
This is a common trap - ideally, GMainLoop would record if `_quit()`
was called before `_run()` or something, but doing that now would
likely break people who are expecting quit() -> run() to restart.
In general, we've moved in various GLib-consuming apps to an
explicit "main context iteration with termination condition" model;
see `pull_termination_condition()` in the pull code.
This fixes this race condition.
I verified that an assertion in `_finalize` that more than
zero URIs were requested was hit in multiple test cases, and this patch
has survived a while of make check loops.
Closes: https://github.com/ostreedev/ostree/issues/496Closes: #499
Approved by: jlebon
This should help avoid address space exhaustion on 32 bit systems,
and in general is obviously going to improve efficiency.
Closes: #498
Approved by: alexlarsson
Quoting Dan Nicholson in
<https://github.com/ostreedev/ostree/pull/330#issuecomment-245499099>
mtime of 0 has been the semantics of ostree deployments from basically
the beginning of the project. We (and others, see
flatpak/flatpak@b5204c9) rely on that fact when generating trees.
In particular, this affects caches that use the mtime of the
associated file or directory to determine if the cache is valid. By
arbitrarily changing the mtime of the files to something else, all
the caches we setup in the build are now invalidated. Preseeding
caches is really important to the user experience as it avoids
having the user wait while they're regenerated on first run.
Now, we could change our build infrastructure to preset all the
mtimes to 1 to match this change, but what does that do for our
existing users who are on an ostree that deploys with mtimes of 0?
We could just revert this change at Endless (and the associated one
in Flatpak), and that would be fine for our users. However, if we
point non-Endless users to our apps, they'll have the great
experience of waiting 10 seconds the first time they launch it while
the fontconfig cache is rebuilt unnecessarily.
Closes: #495
Approved by: jlebon
musl libc's implementation of `realpath` works by opening the path and then
doing a lookup in `/proc/self/fd` to find the canonical path. This fails
if `/proc` is not mounted. This causes problems for us if
`ostree-prepare-root` is `init` as `/proc` won't be mounted.
We have to mount `/proc` anyway for `/proc/cmdline` so this fix just
expands the scope over which `/proc` is mounted to include both our
`realpath` calls.
See also:
* http://www.openwall.com/lists/musl/2016/06/08/2 and
* http://git.musl-libc.org/cgit/musl/tree/src/misc/realpath.c?id=e738b8cbe64b6dd3ed9f47b6d4cd7eb2c422b38dCloses: #485
Approved by: cgwalters
I've seen it fail with musl which needs `/proc` to be mounted for it to
work. The error messages we're rather confusing before. At least this
now points to the right location.
Closes: #485
Approved by: cgwalters
This test previously depended on manually building ostree-prepare-root.
Since 42dab85 we've been able to build static binaries with the usual
autotools build-system. This change reflects the fact that
`ostree-prepare-root` is built into $srcdir rather than `src/switchroot`
where I was building manually.
This test now passes with `./configure --with-static-compiler=gcc` (glibc)
but still fails with `./configure --with-static-compiler=musl-gcc` (musl).
Closes: #485
Approved by: cgwalters
Was failing with error:
src/switchroot/ostree-prepare-root.c:30:20: fatal error: config.h: No such file or directory
compilation terminated.
Reported by and fix provided by @gatispaeglis.
Closes: #485
Approved by: cgwalters
Without this, the manual pages can't actually be regenerated from a dist
tarball, and running make clean will remove all traces of them.
Closes: #486
Approved by: cgwalters
If xsltproc is not installed, then ENABLE_MAN will be false and the
generated man pages won't be distributed. Pass --enable-man to enforce
that the man pages will be generated and distributed.
Closes: #486
Approved by: cgwalters
While converting the mirrorlist code from using GSList to GPtrArray, I
completely missed the fact that there is now a much cleaner way to do
this.
Closes: #484
Approved by: cgwalters
This commit adds mirrorlist support to the fetcher. Users can now
prepend url or/and contenturl by mirrorlist= to interpret the link as a
mirrorlist.
If an object is not found, the fetcher will automatically try the next
mirror in the order given in the list (assuming the order returned by
the server is significant).
Closes: #469
Approved by: cgwalters
This made sense back when we used a main loop even when we needed to
fetch objects synchronously. Nowadays, we no longer actually update
progress before the FETCHING_OBJECTS phase, which is only for async
requests.
This allows us to get rid of fetch_uri_contents_membuf_sync() and to
generalize fetch_uri_contents_utf8_sync() so that it only requires a
fetcher. This will be needed later.
Closes: #469
Approved by: cgwalters
I've found this useful when monitoring multiple logs at the same time to
test the upcoming content & meta URL splitting.
Closes: #469
Approved by: cgwalters
Allow users to pass a --contenturl during `remote add` and store it in
the remote config.
Fish out the contenturl setting from the remote config and use it when
downloading static deltas and objects (except for commit signatures).
The idea here is that items in the trust chain (summary & sigs) can be
fetched from a more secure e.g. TLS-pinned location, while objects
themselves are fetched from another location. Once mirrorlist support is
added, this use-case will become even more advantageous.
Closes: #469
Approved by: cgwalters
OSTree function ostree_repo_pull_with_options starts a
series of operations that makes heavy use of the PullData's
cancellable.
This isn't effective, however, since nowhere in the code
the OtPullData.cancellable field is set. This is visible,
for example, when trying to cancel a Flatpak pull and nothing
happens, because the cancellable is not properly passed
to the pull data.
Fix that by setting the cancellable field of the pull data. It
owns a reference for safety reasons, and unreferences it at the
end of the operation.
ostreedev/ostree#482
Closes: #483
Approved by: cgwalters
This simplifies the build system by removing the need for
`libswitchroot-mountutil.la`.
Original idea by @cgwalters in #477.
Closes: #478
Approved by: cgwalters
The `warn()` libc extension has exactly the same behaviour as our own
`perrorv` function, but is available in (at least) glibc and musl. As an
added bonus the similar function `err()` which will exit with an error
code afterwards.
This implementation is tidier and allows us to get rid of our own
`perrorv`. It paves the way to removing `ostree-mount-util.c` to simplify
the build scripts.
Original idea by @cgwalters in #477.
Closes: #478
Approved by: cgwalters
If the `--with-static-compiler=musl-gcc` configure flag is given.
ostree-prepare-root can be used as init in a system without a populated
/lib. To support this use case we need to link statically as we will be
unable to locate libc.so at run time if it's not installed in /lib.
We support building ostree-prepare-root with a different compiler to the
rest of ostree so we can use musl rather than glibc. This reduces the size
of the executable significantly: from ~700K -> ~30K. We have to use
`_SCRIPTS` here to get autotools to install this as an executable but
without generating rules to make it itself which we have specified
manually.
See https://lists.gnu.org/archive/html/help-gnu-utils/2007-01/msg00007.html
for advice on using autotools in this manner.
Closes: #477
Approved by: cgwalters
POSIX and GNU define conflicting versions of `strerror_r`. The GNU
version returns the string but doesn't necessilary write into buf.
The POSIX version writes into buf and returns the length but doesn't
necessilary append a terminate the string with a NUL if it's too long
to fit in buf.
This commit fixes building ostree-prepare-root with musl libc. The
stripped static build with musl on my machine is 30K vs. 724K with glibc
static and 11K with glibc shared.
Closes: #477
Approved by: cgwalters
We hold a fd open on this, and it's basically now expected
to be immortal. Confer that status.
This was showing up in flatpak crashers, because we'd get
an unexpected errno.
(I didn't test this fixes the crasher, but it's clearly right)
https://bugzilla.redhat.com/show_bug.cgi?id=1347293Closes: #476
Approved by: alexlarsson
I noticed seeing the output of `prune` twice in rpm-ostree, and had
always wondered why. When reading the rpm-ostree code to fix
something else, reasons, I noticed the reason - we were pruning once
here, and then once after rpm-ostree regenerates its "base" refs.
There's no reason to clean twice, so let's add a flag so rpm-ostree
can suppress doing it inside libostree.
Closes: #474
Approved by: giuseppe
It's close-on-exec, not close-on-fork. I was clearly confused when
writing this; it works just fine to reference the fd in the child and
`fchdir()` before exec. So drop the unnecessary duplication.
Just noticed this while reading the code for a random other reason.
Closes: #473
Approved by: giuseppe
Alexander Larsson
Colin Walters
Simon McVittie
Dan Nicholson
Jonathan Lebon
Owen W. Taylor
Giuseppe Scrivano
William Manley
Georges Basile Stavracas Neto