Follow up to a previous patch that addressed a double-close; I
realized we already had a helper for doing "open dfd iter, do nothing
if we get ENOENT". Raise it to libotuil, and port all consumers.
Closes: #863
Approved by: jlebon
It's just simpler, and I'm not sure people are going to care
much about the difference by default.
We already folded in the fallback sizes into the download totals, so folding in
the count makes things consistent; previously you could see e.g.
`3/3 parts, 100MB/150MB` and be confused.
Closes: #678
Approved by: giuseppe
Doing `g_variant_print (superblock)` is unreadable and not very useful,
since we show the checksums as byte arrays.
However, do show the checksums for fallback objects. This makes it easier to see
which objects are fallbacks (and inspect why).
Closes: #678
Approved by: giuseppe
I was having this thought today about making more of the OS readonly,
and ultimately if we got to the point where all ostree operations are
through the repo and sysroot dfds, we could have rpm-ostree be the
only process holding those fds open, and have a read-only bind mount
on top.
Anyways, we're not there, likely won't be soon, but this gets us
closer to being fully fd relative.
Closes: #628
Approved by: jlebon
This is a follow up to conversation on list - in practice, if we're
backing away from summary signing, then it makes sense to remove the
special casing for checksums in deltas around summary signatures.
This is also related to the recent change to enable GPG checking for
commits in deltas - now we have a more coherent story between the
previous pull path and deltas.
I didn't do any performance checking, and while it's slightly annoying
that we're now doing sha256 on the delta content twice (once for the
part and once per object)...sha256 is pretty fast, I think most users
are I/O bound anyways, and it'd drop even farther if we started using
openssl.
Closes: #612
Approved by: jlebon
If a static delta is generated between 2 commits with the same content,
then the delta will contain 1 part with no checksums. While useless,
this is a valid delta that shouldn't raise an assertion. If the delta
part has no checksums, then there are no objects to recreate and the
processing can be skipped.
Closes: #420
Approved by: cgwalters
I often want to have "idempotent" systems that iterate to a known
state. If after generating a commit, the system is interrupted, I'd
like the next run to still generate a delta. But we don't want to
regenerate if one exists, hence this option.
Closes: #375
Approved by: jlebon
Import `gs_file_enumerator_iterate()` for the next six months or
so...after RHEL 7.3 is released I'm strongly considering hard
requiring 2.46 or so.
Likely at some point we should figure out how to share more "glib
backport" code with NetworkManager at least.
Closes: #341
Approved by: jlebon
We have a lot of "allow_noent" type wrapper functions since
a common pattern is to allow files to not exist, but still
throw cleanly on other issues.
This is another instance of that, and cleans up duplicated error
handling code.
Part of this is prep for moving away from `GFile` consumers.
Closes: #319
Approved by: jlebon
This is similar to changes Krzesimir has been doing recently - we
really don't need the ergonomics of floating refs since we have
autocleanups.
We should continue to change most of our code to sink refs.
Specifically here it was pretty broken that the `_map()` API was
sinking but the other two weren't, and this broke some refactoring I
was trying to do later.
Closes: #317
Approved by: jlebon
I see when analyzing a delta here that due to byteswapping a negative
compression ratio of 540%, 66%, and 28%. Let's arbitrarily pick 20%
as a threshold for detecting byetswapping.
If the average object size is greater than 4GiB, let's assume we're
dealing with opposite endianness. I'm fairly confident no one is
going to be shipping peta- or exa- byte size ostree deltas, period.
Past the gigabyte scale you really want bittorrent or something.
xdg-app passed this a filename directly, and in this case it should be
used as is. This regressed to always look for "superblock" in the same
directory as the passed in filename.
https://bugzilla.gnome.org/show_bug.cgi?id=762617
Now we display stats on the individual parts, such as the blob size
and the number of each type of opcode. Most interesting to me is
things like how many bsdiff opcodes there are vs new objects, etc.
We had code to deal with opening/checksumming/decompressing static
deltas in a few places. I'd like to teach `ostree static-delta show`
how to display more information, and this will allow it to just use
`_ostree_static_delta_part_open()` too.
Right now though, almost all of the details of deltas are private, so
we can't do the "honest thing" and have the command line just use the
shared library.
Eventually some of this should appear in the API, but for now add
command line which is useful for debugging.
This is very useful for the inline-parts case, as you can then include
detached signatures in a single file representing the commit.
It is not as important for the generic pull case, as the detached
metadata is only a single small file. Additionally the detached
metadata is not content referenced and may change after the static
delta file was created, so we need to pull the latest version anyway.
If you pass a diriectory it will look for the "superblock" child, otherwise
it will use the file as the superblock. I need this in xdg-app to be able
to install any filename as a bundle.
In this mode the parts are stored in the metadata of the main delta
superblock file. This can be useful if you want a single-file delta
for easy transport, or for http in the case the delta is very small.
The current layout uses a prefix of two bytes as the initial dir
and a second directory inside that with the superblock. This
updates the list code to handle that.
https://bugzilla.gnome.org/show_bug.cgi?id=721799
This has a very basic level of functionality (deltas can be generated,
and applied offline). There is only some stubbed out pull code to
fetch them via HTTP.
But, better to commit this now and improve it from a known starting
point, rather than have it languish in a branch.
Colin Walters
Jonathan Lebon
Dan Nicholson
Mathnerd314
Alexander Larsson
John Hiesey
Matthew Barnes
Giuseppe Scrivano