OstreeAsyncProgress currently does some contortions to try and avoid
allocating space for guints and guint64s (on 64-bit platforms), but this
means it uses two GHashTables. A GHashTable allocates 8 buckets even
when empty. Given that the largest usage of OstreeAsyncProgress in
libostree puts 13 uints and 5 uint64s in it, this optimisation does not
save significant (if any) memory.
Instead, change OstreeAsyncProgress to store values internally as
GVariants, and expose this with some new API:
• ostree_async_progress_get_variant()
• ostree_async_progress_set_variant()
Each GVariant is allocated on the heap. As they are immutable, they are
thread-safe once returned by a getter.
The existing API continues to work as before, except in the case where a
key is set/got as both a uint and a uint64 — there will now be a
collision (and a GVariant type checking failure) whereas previously
there was no collision. Nothing in OSTree uses OstreeAsyncProgress this
way though.
The new API can be used to share more complex data via the progress API.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #819
Approved by: cgwalters
Instead of using G_OPTION_ARG_STRING, use G_OPTION_ARG_FILENAME, which
handles filename encoding conversion differently from the locale
conversion which G_OPTION_ARG_STRING. This will fix argument handling on
systems where the filename encoding is not the same as the locale
encoding (which is fairly unlikely since most systems use UTF-8).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #810
Approved by: cgwalters
I happened to glance at the top of my most recent patch and
noticed that I used an `throw_errno()` function in a non-errno place.
I scanned the patch for other instances of this but didn't find one.
Closes: #811
Approved by: jlebon
I was planning to change some of the object loading code in the
future, so here's some porting.
Note that I rewrote `_ostree_repo_has_loose_object()` since it
used an error return across multiple functions.
Honestly I'm not sure about this `TEMP_FAILURE_RETRY()` business...
in reality we're going to end up with a ton of code linked in
process that doesn't do it. Unix sucks =( But I'm keeping
what was there out of consistency.
Closes: #809
Approved by: jlebon
This did a `closedir` in the `goto out` section before, but it
turns out more nicely if we follow the usual pattern of doing
the `open(O_DIRECTORY)` in the callee function and handle `ENOENT`
there.
Closes: #809
Approved by: jlebon
This is a variant of the efforts in https://github.com/ostreedev/ostree/pull/741
Working on `rpm-ostree livefs`, I realized though I needed to just
check out *new* files directly into the live `/etc` (and possibly
delete obsolete files).
The way the current `/etc` merge works is fundamentally different from
that. So my plan currently is to probably do something like:
- Compute diff
- Check out each *new* file individually (as a copy)
- Optionally delete obsolete files
Also, a few other things become more important - in the current deploy code, we
copy all of the files, then relabel them. But we shouldn't expose to *live*
systems the race conditions of doing that, plus we should only relabel files we
checked out.
By converting the deploy's /etc code to use this, we fix the same TODO item
there around atomically having the label set up as we create files. And further,
if we kill the `/var` relabeling which I think is unnecessary since Anaconda
does it, we could delete large chunks of code there.
In the implementation, there are two types of things: regular files, and
symlinks. For regular files, in the `O_TMPFILE` case, we have the ability to
do *everything* atomically (including SELinux labeling) before linking it into
place. So let's just use that. For symlinks, we use `setfscreatecon()`.
Closes: #797
Approved by: jlebon
We really have an astonishing variety of similar functions which write files and
symlinks. I was working on a different PR and the duplication between the
union-mode and add-mode/none-mode checkout functions bothered me.
I realized that the "handle EEXIST" tri-state maps directly to the
`GLnxLinkTmpfileReplaceMode`, so deduping things makes even more sense.
Closes: #801
Approved by: jlebon
I was reading a strace the other day and noticed we were loading the same
`.dirmeta` object many times. Unlike the other object types, `.dirmeta` objects
don't accumulate much over time; there are only so many directory metadata types.
(Without SELinux involved it'd probably be 5-6 I'd guess offhand).
For `fedora-atomic/25/x86_64/docker-host` there are currently 34 `.dirmeta` in
the tree.
But how many times during a checkout did we load those 34 dirmeta objects?
With a quick strace:
```
$ strace -s 2048 -f -o strace.log ostree --repo=repo-build checkout -U fedora-atomic/25/x86_64/docker-host host-test-checkout
$ grep dirmeta strace.log | wc -l
7165
```
After, as you'd expect, we just loaded `34` from disk. We do
6 system calls (`openat+fstat+fstat+read+read+close`) per dirmeta,
so we dropped a total of 42780 system calls - which is about 20% of the total
system calls made.
`perf record` tells me that we're spending ~40 of our time in the kernel during
a checkout, so reducing syscall traffic helps. Though most of that appears to be
in the VFS and XFS layers for `linkat` (which isn't surprising).
So how much did perf improve? Well, on my workstation, I get a lot of
fluctuation in timing, sometimes by 30%, so this was well within the noise. But
it's well worth speeding up checkout, and I think this optimization will shine
more as we improve performance elsewhere.
Closes: #795
Approved by: jlebon
These are leftovers from the packfile code and should have been
deleted in commit: 2a0601efc7
I noticed this now since I wanted to add a new type of caching.
Closes: #795
Approved by: jlebon
I never really liked the term "osname". I feel "stateroot" is a *lot* clearer,
since the osname/stateroot mostly just holds `/var`. Further it avoids the `os`
prefix which is already overloaded.
Some of the existing docs already talked about "operating system state", which
further reinforces this.
There's *lot* more things than this which reference the term "osname", but I
don't want to change *everything* yet in this patch in case we decide to do
something different - this just gets the highlights.
Closes: #794
Approved by: jlebon
This is intended to be used for copying `/usr/etc` → `/etc` for
deployments.
A TODO here is to use `glnx_file_copy_at()` if the repo mode allows
it - then we'd use reflinks if available.
Closes: #804
Approved by: jlebon
There aren't many users of `g_file_enumerator_iterate()` left - those
remaining are usually good candidates for porting. There's some more
porting to do in this file; a mix of trivial and harder. This
one is a good candidate for an individual commit.
Closes: #803
Approved by: jlebon
I plan to make some future changes here, and wanted to do this
first.
Random side note; how about converting the do/while loops for `EINTR` to
`TEMP_FAILURE_RETRY()`? We're very inconsistent about that...
Closes: #792
Approved by: jlebon
This could help others who want to integrate with other init
systems/initramfs.
Commit-message-by: Colin Walters <walters@verbum.org>
Closes: #784
Approved by: cgwalters
`perf record ostree checkout ...` for a bare-user repo was telling
me we were spending a good 13% of our time in the depchain of `ot_lgexattrat()`.
The problem here is that traversing the `/proc` path turns out to be
somewhat expensive - there are LSM (SELinux) checks, etc.
For regular files, opening and just getting the xattr, then closing is still
quite cheap. For symlinks, we'll always need to open anyways.
This appears to shave about ~0.1 seconds off of a checkout of
`fedora-atomic/25/x86_64/docker-host` on my workstation.
Oh, and this was the last user of `ot_lgexattrat()` so we can kill it, which is
nice - the xattr code should really live in libglnx.
Closes: #796
Approved by: jlebon
This is only about 40%, and mostly simpler functions. It's
nice to switch to `g_autoptr(GMatchInfo)` instead of our inline version.
I decided to add more usage of `ot_transfer_out_value()`, though it'd
be nice to try to have a copy of that in libglnx (or possibly glib).
Closes: #791
Approved by: jlebon
Logic error introduced after refactoring; we hoisted the
`is_bare_user_symlink` variable to the top, but its computation
below. But the `is_bare` symlink depended on it.
Closes: https://github.com/ostreedev/ostree/issues/798Closes: #799
Approved by: jlebon
I think the majority of OSTree usage calls pull with refs, not
explicit commits. We even added special "override syntax" with
`@` (e.g. `ostree pull foo@ab12c34`) as a hybrid.
However, some users may want to still pull explicit commits
for whatever reason. The old static delta logic looked at
the previous commit of the ref. However, in https://github.com/ostreedev/ostree/pull/710
we enhanced the logic to look at all local commits.
It's now a lot more natural to teach the delta logic
to support revisions, e.g. `ostree pull someorigin ab12c34`.
This also fixes the problem that before, `--require-static-deltas`
was completely ignored when processing revisions.
This is a nontrivial refactoring of the logic, but the end
result feels a lot more readable to me.
Closes: https://github.com/ostreedev/ostree/issues/783Closes: #787
Approved by: cgwalters
Previously we'd assert and dump core if one used `checkout -H` without
`-U` on a bare-user repo, because we'd hit the bare-user symlink case.
Rework the code to handle this, and add tests. I hit this when I was going to
suggest to someone to use `-H` to ensure they were getting hardlinks.
Closes: #779
Approved by: jlebon
Testing a fetch of `fedora-atomic/.../docker-host` from
an nginx instance over `https://127.0.0.1` using Fedora 25
versions. Average over 3 runs:
Before: ~24.6 seconds
After: ~19 seconds
Speedup: ~30%
Closes: https://github.com/ostreedev/ostree/issues/778Closes: #780
Approved by: jlebon
In particular the 26-variable monster 👹 in `install_deployment_kernel()` is
slain🗡. I didn't touch every function here, trying to keep things gradual.
Closes: #781
Approved by: jlebon
It was reported that in the range request handling, we called `remove_pending()`
twice (once in processing it, and once potentially in the local_error cleanup),
and this could be viewed as a use-after-free. However, right now the range
cleanup and `local_error` being set are mututally exclusive.
Further, the task object already holds a strong reference, so I observed the
refcount was 2. For both of these reasons, there is no use-after-free in
practice.
Reported-By: "Siddharth Sharma" <siddharth@redhat.com>
Closes: #774
Approved by: jlebon
I was planning to change one here, decided to do a conversion
of some of the simpler functions in this file to keep up momentum.
Closes: #776
Approved by: jlebon
I was working on `rpm-ostree livefs` which does some ostree-based
filesystem diffs, and noticed that we were ending up with `/proc`
not being labeled in our base trees.
Reading the selinux-policy source, indeed we have:
```
/proc -d <<none>>
/proc/.* <<none>>
```
This dates pretty far back. We really don't want unlabeled
content in ostree. In this case it's mostly OK since the kernel
will assign a label, but again *everything* should be labeled via
OSTree so that it's all consistent, which will fix `ostree diff`.
Notably, `/proc` is the *only* file path that isn't covered when composing a
Fedora Atomic Host. So I added a hack here to hardcode it (although I'm a bit
uncertain about whether it should really be `proc_t` on disk before systemd
mounts or not).
Out of conservatism, I made this a flag, so if we hit issues down the line, we
could easily change rpm-ostree to stumble on as it did before.
Closes: #768
Approved by: jlebon
Being bitten by lack of PR testing here. There are two bugs:
- First and foremost, I forgot that GObject will call the property setters with
the defaults. This meant we were getting both path="/var/tmp/blah" and fd=-1,
and we were accepting -1 as a fd, which then got converted into AT_FDCWD
which was wrong.
- Since these properties are construct only and mutually exclusive, don't
try to handle one resetting the other. Assert that exactly one of them is set.
Closes: #769
Approved by: jlebon
This is inspired by the [Coccinelle](http://coccinelle.lip6.fr/) usage
in systemd. I also took it a bit further and added infrastructure
to have spatches which should never apply. This acts as a blacklist.
The reason to do the latter is that coccinelle is *way* more powerful than the
regular expresssions we have in `make syntax-check`.
I started with blacklisting `g_error_free()` directly. The reason that's bad is
it leaves a dangling pointer.
Closes: #754
Approved by: jlebon
I happened to read this file and realized there's a lot of cruft left over from
the time when I liked `GFile` and `malloc()`ing like 50 times just to make a
pathname string. Delete it.
Closes: #767
Approved by: jlebon
Actually trying to use this in rpm-ostree, it kept returning successfully when I
didn't expect it to... The first conditional was always succeeding even when I
was asking for a newer minor.
Closes: #766
Approved by: jlebon
I'd like to do this in rpm-ostree at least. Originally I was looking at porting
to `G_DECLARE_FINAL_TYPE` but eh, this is easier for now and won't bump our GLib
dependency which might matter for our embedded users.
For now I just did a few replacements in the `remote` command line. A full port
can come as we do other code cleanups.
This will actually break the flatpak build right now, but
that's easy to fix. And we concluded in e.g.
https://bugs.freedesktop.org/show_bug.cgi?id=95065#c5
it's a bug for downstream projects to do that.
Closes: #756
Approved by: jlebon
I didn't touch everything since at least `commit_loose_object_trusted`
does this:
```
out:
if (G_UNLIKELY (error && *error))
g_prefix_error (error, "Writing object %s.%s: ", checksum, ostree_object_type_to_string (objtype));
```
Which...it'd be interesting to make into an autocleanup. But for now just
keeping up with converting things bit by bit.
Closes: #761
Approved by: jlebon
In [this commit](6ce80f9685)
for some reason I added a `sepolicy` member to the sysroot. I
have no idea why I did that, and it's conceptually wrong
since the policy is specific to a *deployment*.
This bit me when I was working on [a pull request](https://github.com/ostreedev/ostree/pull/763)
elsewhere, since at that point it was `NULL`.
We already pass around the sepolicy in the deployment code, so just stop caching
it.
Closes: #764
Approved by: jlebon
I was playing around in a FAH vagrant box, and hit:
```
Receiving delta parts: 3/4 453.2 kB/s 1.8 MB/145.8 MB
error: opcode set-read-source: No such file object b6e54ba3471b9c116ce6b9bfbf9e55fec60d35cfdb9ae5ae1ee219af02a591b7
```
This is because this host version doesn't yet have
https://github.com/ostreedev/ostree/pull/710
which incidentally fixed this for the case where the OS vendor is using
summary files.
Some organizations may not be using summary files - at least we still try to
support that case. So let's copy the logic very recently added in that commit to
handle the legacy case too.
No new tests since this is a nice-to-have - we really do
expect people to be using summary files now.
Closes: #739
Approved by: jlebon
This adds to file permission masks the same bitmask that will
be applied to file objects in bare-user* repos. This will be
needed in the testsuite to ensure that the things we commit
will be expressable in bare-user-only repos.
Closes: #750
Approved by: cgwalters
This mode is similar to bare-user, but does not store the permission,
ownership (uid/gid) and xattrs in an xattr on the file objects in the
repo. Additionally it stores symlinks as symlinks rather than as
regular files+xattrs, like the bare mode. The later is needed because
we can't store the is-symlink in the xattr.
This means that some metadata is lost, such as the uid. When reading a
repo like this we always report uid, gid as 0, and no xattrs, so
unless this is true in the commit the resulting repository will
not fsck correctly.
However, it the main usecase of the repository is to check out with
--user-mode, then no information is lost, and the repository can
work on filesystems without xattrs (such as tmpfs).
Closes: #750
Approved by: cgwalters
There are a lot of things suboptimal about this approach, but
on the other hand we need to get our CI back up and running.
The basic approach is to - in the test suite, detect if we're on overlayfs. If
so, set a flag in the repo, which gets picked up by a few strategic places in
the core to turn on "ignore xattrs".
I also had to add a variant of this for the sysroot work.
The core problem here is while overlayfs will let us read and
see the SELinux labels, it won't let us write them.
Down the line, we should improve this so that we can selectively ignore e.g.
`security.*` attributes but not `user.*` say.
Closes: https://github.com/ostreedev/ostree/issues/758Closes: #759
Approved by: jlebon
More sophisticated users of libostree like rpm-ostree need control over things
like the system repository. Previously we introduced a "no cleanup" flag to
`ostree_sysroot_simple_write_deployment()`, but that's a high level API that
does filtering on its own.
Since rpm-ostree needs more control, let's expose the bare essentials of the
"sysroot commit" operation with an extensible options structure, where one of
the options is whether or not to do post-transaction repository operations.
Closes: #745
Approved by: jlebon
I happened to be reading this one recently for a reason I forget,
and it's a relatively easy conversion.
Also one not conflicting with any outstanding patches.
Closes: #752
Approved by: jlebon
Use `g_auto()` more sanely with a struct implmenting the "is initialized"
pattern. This is way less ugly for callers, and fixes bugs like
us calling `setfscreatecon()` even if an error occurred beforehand.
Also fold in the logic for "NULL or not loaded" sepolicy into the setup rather
than requiring callers to inline it.
Prep for more users of this function.
Closes: #746
Approved by: jlebon
I'm porting other code away from `GFile`, and while we don't use this
internally, it will let us do so at a later date. I'm averse to changing the
code right now as we don't have good CI coverage of this.
Closes: #746
Approved by: jlebon
The first options are owner_uid/owner_gid, which makes it possible to use diff
on local files where --owner-uid/gid have been passed to commit.
Closes: #740
Approved by: cgwalters
For future work I'm going to tweak how we handle cleanup, and
the private cleanup flags didn't really end up being used - we
only specify "prune repo or not". So fold that into a boolean for now.
The sysroot deploy logic then has a single "do_postclean" boolean, which is all
I want to expose as public API.
Closes: #744
Approved by: jlebon
Add an OpenSSL backend to the checksum input stream, which is where we do a lot
of checksumming (object commit, static deltas).
The raw OpenSSL performance is
[approximately double](https://gist.github.com/cgwalters/169349fd1c06fd4fb4d3a7ce33303222) on
my laptop; not only does OpenSSL have e.g. hand-tuned x86_64 assembly, the
current implementation uses the
[Intel SHA extensions](https://en.wikipedia.org/wiki/Intel_SHA_extensions).
Another reason to do this is I was idly thinking about adding
[Curve25519](https://en.wikipedia.org/wiki/Curve25519) signatures (like e.g.
Alpine does) instead of/in addition to GPG. The rationale for that is
that GPG is pretty heavyweight, both in code footprint and the simple
fact that EC keys are way smaller.
I didn't benchmark ostree with this; we have bigger performance problems
really like the fact we just malloc way too much. But, it's a step
in the right direction I think in combination with the libcurl work
where we're linking to openssl anyways.
Closes: #738
Approved by: jlebon
The current `OstreeChecksumInputStream` is public due to a historical
mistake. I'd like to add an OpenSSL checksum backend, but that's
harder without breaking this API.
Let's ignore it and create a new private version, so it's easier to do the
GLib/OpenSSL abstraction in one place.
Closes: #738
Approved by: jlebon
I just happened to be reading this code; still testing
the waters for the new code style. If we decide to do more
of this it's likely worth trying out a [Coccinelle](http://coccinelle.lip6.fr/)
patch or something.
Closes: #733
Approved by: jlebon
[Previously](https://github.com/ostreedev/ostree/pull/728) we added compile-time
checking for versions, but there are use cases for runtime checking as well,
because in a number of API calls we use `GVariant` as an API extension
mechanism.
Closes: #735
Approved by: jlebon
This goes farther along the path of deprecating it; consumers should
have been ported at this point. Though probably a lot of people
may still use `rpm-ostree-toolbox`.
Closes: #723
Approved by: jlebon
OSTree currently provides no way to inspect the versioning
information at run time, being only available at compile
time through pkg-config.
This is a problem for e.g. Flatpak, that needs to check
whether the 'update-frequency' option is available. Checking
at compile time isn't great since it's not looking for new
symbols, but only if an optional feature is present.
This commit, then, adds a new header that is generated
at compile time, exposing OSTree's versioning information.
Closes: #728
Approved by: cgwalters
It's been almost a month, I think the current git is working well and
not too risky. We have some new API additions which I think the
various consumers of them are going to want.
Closes: #726
Approved by: jlebon
After commit 80b3edc64731a5f0 introducing the option to set
a custom timeout, adapt the ostree program to be able to update
that.
Closes: #725
Approved by: jlebon
When using Flatpak with GNOME Software, it is important to
show the progress of the download and install as close as
possible to the real progress.
However, OSTree forces the frequency to call the async
progress callback to 1 second, which causes an unpleasant
effect on the UI, specially when the download size is so
small that everything happens in less than 1 second.
Fix that by adding making OSTree read a custom 'update-frequency'
option and set the timeout source timeout to that. If
no custom frequency is passed, we assume the default 1
second timeout, maintaining the current behavior.
Closes: #725
Approved by: jlebon
At least in all Linux kernels up to today, one can never `link()` across
devices, so we might as well verify that up front. This will help for a future
patch to add a new type of union-add checkout, since Linux checks for `EEXIST`
before `EXDEV`.
Closes: #714
Approved by: jlebon
I've seen code in a few places that I think on balance is definitely better this
way. Some of our functions have huge variable declaration sections.
This change includes one small example where we could start using declarations
after statements.
A concern I had was - how does this interact with `__attribute__((cleanup))` and
early returns? I tested it, and AFAICS the behavior is what you'd expect - the
cleanup function isn't called if its variable isn't reachable.
Closes: #718
Approved by: jlebon
Add a ostree_raw_file_to_archive_z2_stream_with_options() variant of
ostree_raw_file_to_archive_z2_stream(), to allow a compression-level
option to be passed in and passed through to zlib.
This is useful when building archive-z2 files on the fly for
transmission over a non-bandwidth-limited channel, such as a local
network. In this case, CPU time is more valuable than bandwidth, so we
want a low compression level.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Closes: #721
Approved by: cgwalters
I had to rebuild `glib` with `-fsanitize=address` in order to get a stack trace
to finally get this one. However, *installing* that glib "system wide"
in my container breaks everything (including `rpm-ostree`, `dnf`, `pkg-config` etc.)
that wasn't built with ASAN.
So my test scenario right now is to extract the libs and do e.g.:
```
make && env LD_LIBRARY_PATH=$HOME/src/distgit/fedora/glib2/asan-libs make check TESTS=tests/test-basic.sh
```
Closes: #719
Approved by: jlebon
The previous logic for static deltas was to use as a FROM
revision the current branch tip. However, we want
to support deltas between branches in an automatic
fashion.
If a summary file is available, we already have an
enumerated list of deltas - so the logic introduced
here is to search it, and find the newest commit
we have locally that matches the TO revision target.
This builds on some thoughts from
https://github.com/ostreedev/ostree/pull/151#issuecomment-232390232
Closes: https://github.com/ostreedev/ostree/pull/151Closes: #710
Approved by: giuseppe
Got a report that a Fedora Atomic Host built for ppc64le didn't work with the
`linux16`, it needed `linux`. See the comments for more links.
Closes: #716
Approved by: vathpela
In https://github.com/ostreedev/ostree/pull/408, we disabled the use of
static deltas when mirroring. Later,
https://github.com/ostreedev/ostree/pull/506 loosened this up again so
that we could use static deltas when mirroring into bare{-user} repos.
However, the issue which originally spurrred #408 is even more generic
than that: we want to avoid static deltas for any archive repo, not just
when doing a mirror pull. This patch tightens this up, and also
relocates the decision code to make it easier to read.
Closes: #715
Approved by: cgwalters
The C API (ostree_repo_static_delta_generate) knows what to do
with it, but this parameter was never exposed via command line
tool.
Closes: https://github.com/ostreedev/ostree/issues/695Closes: #703
Approved by: jlebon
Due to the async nature of `GSubprocess` it grabs `SIGCHLD` which
affects other software which might be using libostree, such as
QtOTA.
Closes: https://github.com/ostreedev/ostree/issues/696Closes: #702
Approved by: jlebon
This makes it easier to script downloading updates in the background,
and only do deployments just before rebooting.
Partially addresses https://github.com/ostreedev/ostree/issues/640Closes: #642
Approved by: jlebon
Particularly when HTTP requests fail, I really want a lot more information.
We could theoretically stuff it into the `GError` message field, but
that gets ugly *fast*.
Using the systemd journal allows us to log things in a structured fashion.
Right now e.g. rpm-ostree won't be aware of this additional information,
but I think we could teach it to be down the line.
In the short term, users can learn to find it from `systemctl status rpm-ostreed`
or `journalctl -b -r -u rpm-ostreed`, etc.
One thing I'd like to do next is log successful fetches of e.g. commit objects
as well with more information about the originating server (things like the
final URL if we were redirected, did we use TLS pinning, what was the negotiated
TLS version+cipher, etc).
Closes: #708
Approved by: jlebon
https://github.com/ostreedev/ostree/pull/705 broke the build
on CentOS 7 which only has util-linux 2.23.
When I was thinking about this, I realized that there must really be a way to
make this safe even for older versions. Looking at that version of util-linux,
all we need to do is invert the order of frees so we `mnt_free_table()` *before*
`mnt_free_cache()`, like util-linux does:
https://github.com/karelzak/util-linux/blob/stable/v2.23/sys-utils/eject.c#L1131
We still use the `_unref()` versions if available. I also fixed
the ordering there too for double plus redundant safety.
Closes: #712
Approved by: jlebon
Should fix everything from
<https://kojipkgs.fedoraproject.org//packages/ostree/2017.2/3.fc25/data/logs/x86_64/build.log>
Anything that uses autocleanups should *always* be initialized directly I think,
even if a few lines down we directly assign, since this way it's more robust
against refactoring.
And the `freopen()` warnings are right - IMO we should *always* check return
values.
Closes: #711
Approved by: jlebon
We saw a random ostree SEGV start popping up in our CI environment:
https://github.com/projectatomic/rpm-ostree/pull/641#issuecomment-281870424
Looking at this code more and comparing it to what util-linux does, I noticed we
had a write-after-free, since `mnt_unref_table()` will invoke
`mnt_unref_cache()` on its cache, and that function does:
```
if (cache) {
cache->rfcount--;
```
unconditionally.
Fix this by using `unref()`.
Closes: #705
Approved by: jlebon
I learned today that `docker version` does this and I really like
the idea. While we have the patient open, also add the gitrev
with code taken from https://github.com/projectatomic/rpm-ostree/pull/584Closes: #691
Approved by: giuseppe
It's just simpler, and I'm not sure people are going to care
much about the difference by default.
We already folded in the fallback sizes into the download totals, so folding in
the count makes things consistent; previously you could see e.g.
`3/3 parts, 100MB/150MB` and be confused.
Closes: #678
Approved by: giuseppe
I don't know why I added support for this; it makes no sense really. If we have
large metadata objects something has gone badly wrong.
The delta compiler has always only processed fallbacks for regular
content files.
Dropping support in the fetcher for this will simplify later handling of
fallback progress accounting.
Closes: #678
Approved by: giuseppe
There were a few bugs here.
- We need to keep track of the size of the delta parts we've already processed,
in order to make progress reliable at all in the face of interruptions. Add
a new `fetched-delta-part-size` async progress variable for this.
- The total before disregarded what we'd already downloaded, which was confusing.
Now, a progress percentage is `fetched/total`.
- Correctly handle "unknown bytes/sec" in the progress display.
However, to be fully correct we need to show the fallback objects too. That
would require tracking in the pull code when we fetch an object as a fallback
versus "normally". This would be simpler really if we could assume in a run we
were *only* processing a delta, but currently we don't do that.
Related: https://github.com/ostreedev/ostree/issues/475Closes: #678
Approved by: giuseppe
Doing `g_variant_print (superblock)` is unreadable and not very useful,
since we show the checksums as byte arrays.
However, do show the checksums for fallback objects. This makes it easier to see
which objects are fallbacks (and inspect why).
Closes: #678
Approved by: giuseppe
In https://github.com/ostreedev/ostree/pull/634 we introduced
a subtle regression - the unreadable object was added to the *new*
reachable objects, when it shouldn't have been. Because it
was a *from* object, clients already had it.
This became more obvious now that I'm working on fixing delta
progress - I noticed my deltas were always starting out with 40MB
fetched, which turned out to be a non-world-readable initramfs object.
This code should simply *skip* the unreadable object, and the delta processing
below properly iterates over "new objects", so we'll pick it up from there.
Closes: #678
Approved by: giuseppe
Philip Withnall
Colin Walters
Francesco Giannelli
Daniel J Walsh
Anton Gerasimov
Alexander Larsson
Erik Larsson
Georges Basile Stavracas Neto
André Klitzing
Christian Hergert
Jonathan Lebon
Gatis Paeglis