g_variant_builder_add() does not replace identical keys in a VARDICT
variant, so signing a commit multiple times results in multiple copies
of "ostree.gpgsigs" in the metadata. And since g_variant_lookup_value()
stops on the first match, subsequent signatures have no net effect.
Instead of GVariantBuilder use GVariantDict, which behaves more like a
hash table.
It avoids gobject-introspection warnings:
src/libostree/bupsplit.h:42: Warning: OSTree: symbol='bupsplit_find_ofs': Unknown namespace for symbol 'bupsplit_find_ofs'
src/libostree/bupsplit.h:43: Warning: OSTree: symbol='bupsplit_selftest': Unknown namespace for symbol 'bupsplit_selftest'
src/libostree/bupsplit.h:33: Warning: OSTree: symbol='BUP_BLOBBITS': Unknown namespace for symbol 'BUP_BLOBBITS'
src/libostree/bupsplit.h:34: Warning: OSTree: symbol='BUP_BLOBSIZE': Unknown namespace for symbol 'BUP_BLOBSIZE'
src/libostree/bupsplit.h:35: Warning: OSTree: symbol='BUP_WINDOWBITS': Unknown namespace for symbol 'BUP_WINDOWBITS'
src/libostree/bupsplit.h:36: Warning: OSTree: symbol='BUP_WINDOWSIZE': Unknown namespace for symbol 'BUP_WINDOWSIZE'
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Checkout was one of the first complex code paths I tried to convert to
*at(). I ended up keeping both, because I hit the "xattrs for a
symlink" problem. Later, Florian gave me a workaround, and we started
using it here, but the GFile * parameters weren't deleted. They're
not used, so do so now.
Starting down the path of not using libgsystem. The main win here
will be code sharing between ostree/rpm-ostree as well as going down
the path of not using GFile * for local files.
Convenience feature to avoid having to pass --repo options repeatedly.
Before falling back to the default system repository path, check for a
repository path defined by the OSTREE_REPO environment variable.
We already set all file mtimes to 0 so that they are constant
over all checkouts, and can be made constant with a known value from
the system where the ostree was created.
However, this was not happening for directories. Zero their mtimes too.
This is important for shipping a fontconfig cache in the ostree;
the fontconfig cache files embed a directory mtime.
The previous diff algorithm was file tree based, and only looked
at modified files that lived at the same path.
However, components like the Linux kernel have versioned
subdirectories, e.g. /usr/lib/modules/$kver/.../ext4.ko. We want to
be able to detect these "modified renames" so that we can compute
diffs (rollsum, bsdiff).
This does an rsync-style prepared delta basically. On my test data,
it shaves ~6MB of uncompressed data. Not a huge amount, but I expect
this to be more useful for things like binaries which embed data, etc.
There's still some silliness here, but there is now only one opcode
open-splice-and-close, that writes a single chunk from the payload.
This is really all we need for metadata, and small content objects are
also fine with this.
We get some deduplication between content objects by creating a
dictionary for (uid,gid,mode) tuples and xattrs.
This still keeps the operation/payload code in, so we could do
rollsums in a future update easily.
I was hitting a bug in libguestfs/guestmount/FUSE where it blew up
with EINVAL on directories containing lots of files (more than
32000?). We really want to use prefixed subdirs just like the real
objects/ directory does.
This allows us to share more code between the paths, is more
efficient, etc.
gnome-continuous uses the ostree_repo_scan_hardlinks() mode to
avoid re-checksumming everything. However, when I ported the commit
code to use openat() and friends, this optimization was lost.
Re add it. The difference is about 15s versus 5 minutes.
This follows up from the previous commit; now that pull knows how to
do the efficient link() or copy for local files, we can just have
pull-local call into ostree_repo_pull().
As part of this:
- pull() can also accept a file:/// URI instead
of a remote name (since pull local supports anonymous pulls)
- pull() knows an "override-remote-name" option, since pull-local
supported writing a ref out even if there wasn't a remote with
that name
It's always been suboptimal to have both pull and pull-local; as we go
beyond the raw object data into things like deltas and summary files,
the logic to perform e.g. mirroring should only be in one place.
This will be used by Pulp's OSTree content plugin at least to perform
promotions.
When doing a pull --mirror from an archive-z2 repository into another
archive-z2 repository, currently we gunzip/checksum/gzip each content
object. The re-gzip process in particular is fairly expensive.
This does assume that the upstream content is trusted and correct.
It'd be nice in the future to do at least a CRC check, if not the full
checksum. (Could we append CRC data to the end of filez objects?)
We could also choose to only do this optimization if fetching over
TLS.
before: 1626 metadata, 20320 content objects fetched; 299634 KiB transferred in 62 seconds
after : 1626 metadata, 20320 content objects fetched; 299634 KiB transferred in 11 seconds
Colin Walters
Matthew Barnes
Giuseppe Scrivano
Daniel Drake