We don't want to allow MITM attackers to intercept upgrade requests
and provide clients with older OS versions vulnerable to security
flaws.
Only "ostree admin upgrade" gets this behavior for now - whether we
want to do it for "ostree admin switch" is another question.
This has a very basic level of functionality (deltas can be generated,
and applied offline). There is only some stubbed out pull code to
fetch them via HTTP.
But, better to commit this now and improve it from a known starting
point, rather than have it languish in a branch.
This large patch moves the core xattr logic down into libgsystem,
which allows the gs_shutil_cp_a() API to copy them. In turn, this
allows us to just use that API instead of rolling our own recursive
copy here.
As noted in the new comment though, one case that we are explicitly
regressing is where the new /etc removes a parent directory that's
needed by a modified file. This seems unlikely for most vendors now,
but let's do that as a separate bug.
https://bugzilla.gnome.org/show_bug.cgi?id=711058
I plan to rename all of these APIs to use the term 'loose', so that it
makes more sense after pack files are introduced. External users
should not use them; instead use _load_variant() or _read_commit().
For the cases where we can't hardlink, use at-relative walking of the
path where possible. We still don't have lsetxattrat, so we also need
to deal with pathnames, but that is now only for symlinks.
Again, the advantages of this are a lot less malloc() of pathnames in
ostree, and much less time spent traversing paths inside the kernel.
https://bugzilla.gnome.org/show_bug.cgi?id=707733
Nothing external uses it. We keep ostree_get_xattrs_for_file() public
because it's convenient for external consumers to get xattrs in
exactly the format we desire.
https://bugzilla.gnome.org/show_bug.cgi?id=707733
Clean up how we deal with the uncompressed object cache; we now use
openat()/linkat() and such just like we do for the main objects/.
Use linkat() between the objects and the destination, if possible.
https://bugzilla.gnome.org/show_bug.cgi?id=707733
It's now isolated almost entirely to ostree-core.c, except
ostree-repo.c needs to know how to create archive-z2 file headers. So
give it a private API for that.
We used to have a version of this, but since I'm trying to use
GBytes more, this became a more common operation, and it's annoying
to type out the whole G_VARIANT_TYPE ("ay") each time, and pass
TRUE for trusted.
https://bugzilla.gnome.org/show_bug.cgi?id=706031
If we pass XATTR_REPLACE then the attribute must already exist, which
is not our intent. Passing zero creates the attribute if necessary,
or replaces it when it already exists.
https://bugzilla.gnome.org/show_bug.cgi?id=705893
While the first was useful way back in the day when we were importing
Debian bits and /sbin/init was expecting to find /dev/.initctl as a
named pipe, that's no longer an issue with systemd since it uses
dynamic Unix sockets.
Likewise, character and block devices in /dev are now dynamically
created by the devtmpfs from the kernel.
Less complexity and code here if we just support directories, regular
files, and symbolic links.
Previously we'd open(path, O_NOATIME) and do a series of small read()
calls to just parse the header. I think this will trigger kernel readahead
into the compressed portion, but we don't care about that.
This should be more efficient.
This is an incompatible change to archive-z, thus it is now renamed to
archive-z2 and ostree will no longer parse archive-z.
I noticed in perf that we were spending some time zlib-decompressing
file headers, which is just inefficient. Rather than do this, keep
the headers uncompressed, and just zlib-compress content.
This is where loose content objects are stored as one compressed file,
instead of the two separate ones for regular archive mode. This mode
would be suitable for HTTP servers, beause only one HTTP request is
necessary, and the result would be compressed.
They're not a large efficiency win at the moment, because we don't
do any delta compression.
At the moment, they simply served to compress data, but we will change
the archive mode to do that by default.
This helps make things less confusing when we try to create a file not
owned by our uid, such as running ostree-pull as non-root on a
repository owned by root.
This will allow us to use hard links again for user-mode checkouts,
rather than the hackish link cache. It was pretty silly anyways to
have file objects be stored with just a small metadata header
prepended, but uncompressed.
Either they should be hardlinkable, or compressed (in pack files).
Rather than passing xattr/file_info for all objects, change the API to
assume we're passing the defined object stream for each type. Namely,
for OSTREE_OBJECT_TYPE_FILE, we're now giving the "archive file" data.
This significantly cleans up the code for committing to archive mode
repositories, at the cost of having to (at present) create an
intermediate temporary file when committing to raw repositories.
Having the archived vs not distinction in the object system wasn't
useful in light of pack files. In fact, we should probably move
towards generating a pack file per commit by default.
Don't expose GChecksum in APIs. Add a new stream class which allows
us to pass an input stream somewhere, but gather a checksum as it's
read.
Move some bits of the internals towards binary csums.
Continuing from 16c0cfe9b5, we now have
the problem that we're removing the executable bit, which obviously
breaks things when we chroot in for a build. Fix this by masking over
our bits.
If multiple files have the same hash, we need to ensure we're not
overwriting other tempfiles in the same transaction. Instead
just delete them, since we know they're in the repo.
This will allow us to have hardlink checkouts of archives. A key use
case here is an archive repo of an OS (with root-owned files etc.)
where we want to do builds in a user tree.
A positive side effect of doing things this way is that now the SHA256
checksums for a given file should be identical regardless of whether
it's stored in an archive or bare repository.
It's too confusing that we call the mode "archive" but the actual
files ".packfile". Also, git already has a "packfile" that serves a
totally different purpose.
Colin Walters
Jasper St. Pierre
Tobias Hunger
Stef Walter