commit 08b873457 ("deploy: Remove checksum from generated loader entries")
changed the generated loader entries file names thus making the U-Boot test
case to fail since this test parsed those files.
Fix test-admin-deploy-uboot.sh by looking to the updated files names.
https://bugzilla.gnome.org/show_bug.cgi?id=708511
Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
If we had two deployments with different boot checksums, and were
trying to remove the one that was the same and add a new one (the
normal case), we'd end up assuming due to comparison with 0 that
we only needed to do the fast subbootversion swap.
Fix this by actually putting 1 where we really mean 1.
And update the tests to verify the fix; I have double-verified by
undoing the fix, and noting that the test fails.
https://bugzilla.gnome.org/show_bug.cgi?id=708351
The actual deployment checksum shouldn't be in there, because we may
just swap bootlinks, rendering the name of the old bootloader entry
file invalid. Thankfully nothing actually parsed the names of these
files, so let's just use the index.
OSTree now supports a backend for the U-Boot bootloader,
add a test case for this.
https://bugzilla.gnome.org/show_bug.cgi?id=708069
Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
Currently OSTree supports two bootloader backends: syslinux
and U-Boot; allow to create a stub configuration for both.
https://bugzilla.gnome.org/show_bug.cgi?id=708069
Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
libtest.sh has an setup_os_repository() helper function tha is
used by many tests to setup an OSTree initial repository.
This function creates an syslinux configuration unconditionally
but OSTree supports other bootloader backends besides syslinux.
So, is better to conditionally create a syslinux configuration
only when it is needed.
https://bugzilla.gnome.org/show_bug.cgi?id=708069
Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
Previously I thought we'd have to ditch the current commit
format to avoid a{sv} due to
See https://bugzilla.gnome.org/show_bug.cgi?id=673012
But I realized that we don't really have to care about
unpacking/repacking commit objects, so let's just re-expose the
existing metadata a{sv} in commits in the API.
Also, add support for "detached" metadata that can be updated at any
time post-commit. This is specifically designed for GPG signatures.
https://bugzilla.gnome.org/show_bug.cgi?id=707379
Use a consistent temporary filename to download uri's.
Check for downloaded files before fetching from uri.
Download to hash.part file, then copy/move to hash.done when complete.
Add argument support to setup_fake_remote_repo1 function.
Add test for pull resume.
To implement this, pass --force-range-requests into the trivial-httpd,
which will only serve half of the objects to clients at a time.
https://bugzilla.gnome.org/show_bug.cgi?id=706344
Before, we were writing the "bootversion", which is either 0 or 1, for
all entries. This is completely wrong; the idea of the "version"
field is to compare between entries.
Fix this by writing out the inverted index - internally, index 0 is
the *first* boot entry, so we give it the highest version number, and
index N is the last, so give it version 0.
Then fix the deployment sorting code to correctly reverse the version
number comparison, so we read back the right order.
In practice before this bug didn't matter because "normally" you only
have at most two deployments.
https://bugzilla.gnome.org/show_bug.cgi?id=706546
When running the test-admin-deploy-1.sh unit test,
cat shows the following error:
cat: boot/vmlinuz-3-6.0: No such file or directory
due a trivial typo in the kernel image file name.
Signed-off-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk>
https://bugzilla.gnome.org/show_bug.cgi?id=706371
We'll always have "bare" mode for keeping files-as-hardlinks as root.
But "archive" was my second attempt at a format for non-root file
storage, used by the gnome-ostree buildsystem which runs as non-root.
It was really handy to have a "tar" like mode where I can create
tarballs as a user, that contain files owned by root for example.
The "archive" mode stored content files as two pieces in the
filesystem; ".file" contained metadata, and ".filecontent" was the
actual content, uncompressed. The nice thing about this was that to
check out a tree as non-root, you could just hardlink into the repo.
However, archive was fairly bad for serving via HTTP; it required
*two* HTTP requests per content object, greatly magnifing the already
inefficient fetch process. So "archive-z2" was introduced.
To allow gnome-ostree to still check out trees as a user, the
"uncompressed-object-cache" was introduced, and that's how things have
been working for a while.
So we should just be able to kill this code. Specifically note just
how much better the stage_object() function became.
https://bugzilla.gnome.org/show_bug.cgi?id=706057
While the actual commit object format is presently the same, for a
number of reasons we'd like to change it fairly radically. Among
other things, we need to drop our a{sv} types in objects, to protect
against GVariant changing format.
Since now gnome-ostree now longer uses related objects, and nothing
ever used metadata, just drop them both.
We may revive this later, but commits in their current form aren't
very useful for humans to read, so it doesn't make sense to have a
tool to show a history of useless stuff.
More interesting things are diffs between commits, object statistics,
etc.
Otherwise it's really easy to keep accumulating deployments. Also, we
may want to run this after rebooting, so we're back down to one
operating system.
While the first was useful way back in the day when we were importing
Debian bits and /sbin/init was expecting to find /dev/.initctl as a
named pipe, that's no longer an issue with systemd since it uses
dynamic Unix sockets.
Likewise, character and block devices in /dev are now dynamically
created by the devtmpfs from the kernel.
Less complexity and code here if we just support directories, regular
files, and symbolic links.
Calling it "cleanup" is better since it does more than repo pruning.
We were also doing a prune twice; ot_admin_cleanup() already does one,
so drop the bits to do it in cleanup.c.
These corruption tests could be a lot better...like randomly try
single bit flips, range flips. Better, content-aware fuzzing. But
this is useful for now.
See https://wiki.gnome.org/OSTree/DeploymentModel2
This is a major rework of the on-disk filesystem layout, and the boot
process. OSTree now explicitly supports upgrading kernels, and these
upgrades are also atomic.
The core concept of the new model is the "deployment list", which is
an ordered list of bootable operating system trees. The deployment
list is reflected in the bootloader configuration; which has a kernel
argument that tells the initramfs (dracut) which operating system root
to use.
Invidiual notable changes that come along with this:
1) Operating systems should now come with their etc in usr/etc; OSTree
will perform a 3-way merge at deployment time, and place etc in
the actual root. This avoids the need for a bind mount, and is
just a lot cleaner.
2) OSTree no longer bind mounts /root, /home, and /tmp. It is expected
that the the OS/ has these as symbolic links into /var.
At the moment, OSTree only supports managing syslinux; other
bootloader backends will follow.
A simple HTTP server implementation is so few lines of code when one
is linking to libsoup anyways, so let's just have one here in ostree
that will be used for the test suite.
This allows us to run the archive tests that previously required
apache even in gnome-ostree.
Even if very suboptimally, for now; we copy the files, then copy them
again.
The obvious long term plan is to merge pull-local and pull together,
but truly optimizing that requires the pull code to know how to use
the OstreeRepo APIs when operating on local repositories (as
pull-local does), rather than assuming the remote is an archive-z
fetched over HTTP.
This is useful for the gnome-ostree build system where each build is
one commit, but it's split up into /runtime /devel /debug etc. trees.
Ideally we wouldn't have a /debug subdirectory for "noarch"
components for example.
So add an option to not error out if the given path doesn't exist in
the commit.
This is an incompatible change to archive-z, thus it is now renamed to
archive-z2 and ostree will no longer parse archive-z.
I noticed in perf that we were spending some time zlib-decompressing
file headers, which is just inefficient. Rather than do this, keep
the headers uncompressed, and just zlib-compress content.
This is where loose content objects are stored as one compressed file,
instead of the two separate ones for regular archive mode. This mode
would be suitable for HTTP servers, beause only one HTTP request is
necessary, and the result would be compressed.
They're not a large efficiency win at the moment, because we don't
do any delta compression.
At the moment, they simply served to compress data, but we will change
the archive mode to do that by default.
This can be a large performance win in certain circumstances:
* Cold buffer cache (we don't block the whole process)
* Requiring a copy instead of hardlink
This will be useful for ostbuild; a user can create their own archive
mode repository which transparently inherits objects from the
root-owned one in /ostree.
This is a convenient way to have a lookaside directory of hard links,
which can greatly speed up checkouts. In the future we probably want
to push this down into the repository.
Having the archived vs not distinction in the object system wasn't
useful in light of pack files. In fact, we should probably move
towards generating a pack file per commit by default.
The builder wants the ability to mark a given file as e.g. setuid. To
implement this, the repo now has a callback-based API when importing a
directory to modify or remove items.
The commit tool accepts a "statoverride" file as input which looks like:
+mode /path/to/file
The tar files we're making of artifacts don't include parent
directories. Now we could change the builder to make them, but we can
also just autocreate them on import. Mode 0755 with no xattrs seems
OK here.
It's pretty trivial to map a previously existing commit tree into a
mutable tree too. While we're here change the command line arguments
for commit so that we can now properly overlay any combination of
directory, commit, or tarfile.
ostbuild will generate two artifacts: foo-runtime.tar.gz and
foo-devel.tar.gz in the general case. When committing to the devel
tree, it'd be lame (i.e. slower and not atomic) to have to commit
twice.
We really want the ability to take a .tar.gz and directly import
it into a repository, without creating a temporary filesystem tree.
First, doing it this way is significantly faster. Also, this allows
us to handle importing tar files with e.g. uid 0 files into packed
repositories as non-root, which is very useful for tests and builds.
Javier Martinez Canillas
Colin Walters
Jeremy Whiting
Stef Walter