This is an incompatible change to archive-z, thus it is now renamed to
archive-z2 and ostree will no longer parse archive-z.
I noticed in perf that we were spending some time zlib-decompressing
file headers, which is just inefficient. Rather than do this, keep
the headers uncompressed, and just zlib-compress content.
This is where loose content objects are stored as one compressed file,
instead of the two separate ones for regular archive mode. This mode
would be suitable for HTTP servers, beause only one HTTP request is
necessary, and the result would be compressed.
They're not a large efficiency win at the moment, because we don't
do any delta compression.
At the moment, they simply served to compress data, but we will change
the archive mode to do that by default.
This will allow us to use hard links again for user-mode checkouts,
rather than the hackish link cache. It was pretty silly anyways to
have file objects be stored with just a small metadata header
prepended, but uncompressed.
Either they should be hardlinkable, or compressed (in pack files).
Having the archived vs not distinction in the object system wasn't
useful in light of pack files. In fact, we should probably move
towards generating a pack file per commit by default.
Don't expose GChecksum in APIs. Add a new stream class which allows
us to pass an input stream somewhere, but gather a checksum as it's
read.
Move some bits of the internals towards binary csums.
This will allow us to have hardlink checkouts of archives. A key use
case here is an archive repo of an OS (with root-owned files etc.)
where we want to do builds in a user tree.
A positive side effect of doing things this way is that now the SHA256
checksums for a given file should be identical regardless of whether
it's stored in an archive or bare repository.
This moves us closer to consistently passing around a triple of:
(GFileInfo *info, GVariant *xattrs, GInputStream *content)
Which will help the libarchive work.
We never actually dropped into the bits to write metadata as packfiles,
because such a thing doesn't exist.
Also add a GInputStream-based API for writing packfiles.
This commit originally was to port ostree_stat_and_checksum_file() to
GFile*, but I noticed that the checksum code was reading data in host
endianness. Fix that while we're here.
This invalidates all existing repositories.
This necessitated a large set of changes.
We now support an "archive" mode for repositories. In this mode,
files are stored "packed" rather than hard linked. This allows one to
e.g. store an OSTree repository with root-owned files as non-root. It
is also used as the basis for serving repositories via HTTP.
While doing this I realized that GVariant is endianness-dependent; I
decided to just store all data in big endian.
Colin Walters
Stef Walter