Commit Graph

800 Commits

Author SHA1 Message Date
Colin Walters aac52cb9f7 main: Move GCancellable to toplevel
This is just cleaner, and makes the builtins slightly more of a
library, which in turn makes it easier to turn them into real API.
2013-07-16 18:13:39 -04:00
Colin Walters 272274f0af prune: Don't fail if a to-be-pruned object doesn't exist
Previously, if a prune was interrupted, further attempts would fail.
It's just better to silently continue here.
2013-07-16 10:01:58 -04:00
Colin Walters 62a896350b Drop support for fifos and devices
While the first was useful way back in the day when we were importing
Debian bits and /sbin/init was expecting to find /dev/.initctl as a
named pipe, that's no longer an issue with systemd since it uses
dynamic Unix sockets.

Likewise, character and block devices in /dev are now dynamically
created by the devtmpfs from the kernel.

Less complexity and code here if we just support directories, regular
files, and symbolic links.
2013-07-16 09:35:44 -04:00
Colin Walters 8724565291 core: Drop a leftover unnecessary #define _GNU_SOURCE
We now use AC_USE_SYSTEM_EXTENSIONS.
2013-07-16 09:22:42 -04:00
Colin Walters 444234e7db core: Use gsystem API for generating temporary names
Drops duplicated code.
2013-07-16 08:51:53 -04:00
Colin Walters acdf1ca4fe core: Use gs_file_create()
This allows us to drop a chmod() invocation, since the regular file
path was the only thing that didn't handle mode.
2013-07-16 08:51:53 -04:00
Colin Walters ed55998f36 Update libgsystem 2013-07-16 08:51:53 -04:00
Colin Walters fb93b95807 admin: Rename prune -> cleanup, avoid doing repo prune twice
Calling it "cleanup" is better since it does more than repo pruning.

We were also doing a prune twice; ot_admin_cleanup() already does one,
so drop the bits to do it in cleanup.c.
2013-07-15 16:13:12 -04:00
Colin Walters d5f1ce4e17 admin: Initialize sysroot to / by default, not garbage from the stack
Oops.
2013-07-11 17:29:04 -04:00
Colin Walters 60f5183298 admin status: Print out origin too
This is useful data.
2013-07-11 17:07:14 -04:00
Colin Walters e95935a3f6 main: Also make ostree admin foo --help work
A followup for the previous commit.
2013-07-10 18:42:58 -04:00
Colin Walters b7460cf0b6 main: Only parse options in subcommands
This makes e.g. "ostree commit --help" work, which is really good.
2013-07-10 18:23:52 -04:00
Colin Walters be2bebaf3e main: Don't look for /sysroot/ostree too
No longer necessary now that we expect OS/ trees to have
/ostree -> /sysroot/ostree.
2013-07-10 18:13:55 -04:00
Colin Walters 124416d832 Use AC_USE_SYSTEM_EXTENSIONS instead of #define _GNU_SOURCE
It's less copy/paste, works everywhere, etc.

Also fix some missing #include "config.h".
2013-07-10 13:25:35 -04:00
Colin Walters 8cb348315e main: Print "error: " prefix on errors
It's just clearer that something went wrong.
2013-07-10 13:25:15 -04:00
Colin Walters 32d2cd57c1 admin: Add a warm fuzzy message to deploy about transactions
Since it sounds cool and stuff.
2013-07-10 02:30:26 -04:00
Colin Walters 12c76188c0 Delete leftover kernel update hooks 2013-07-09 20:38:05 -04:00
Colin Walters 89181c4e8e Build without libsoup again 2013-07-09 20:35:15 -04:00
Colin Walters 88cca89317 ostree: Some header file cleanups 2013-07-09 20:30:00 -04:00
Colin Walters 45c7536697 cmd: Drop "ostree admin install" and curl fetcher
It isn't useful at the moment, since the deploy stuff all changed.  It
will make sense to bring back later, but for now let's not carry
broken untested code.
2013-07-09 20:18:29 -04:00
Colin Walters 2535f32c56 libostree: Move pull code into here
More library work.
2013-07-09 20:14:53 -04:00
Colin Walters 5dd0d5da40 libostree: Move prune into OstreeRepo namespace
More library work.
2013-07-09 20:05:31 -04:00
Colin Walters 2aa0251b1b libostree: Rename ostree-traverse.h into OstreeRepo namespace
This operates on the repo, it might as well be methods on it.
2013-07-09 19:59:39 -04:00
Colin Walters cb6b69616c libostree: Split off -refs.c
Continuing to break up ostree-repo.c.
2013-07-09 19:49:00 -04:00
Colin Walters 3b6a94d272 libostree: Split off libarchive code 2013-07-09 19:11:37 -04:00
Colin Walters dbffd31017 libostree: Start splitting up the monstrous ostree-repo.c
Checkout now lives in its own file.
2013-07-09 19:02:38 -04:00
Colin Walters 5ad0671cd2 Switch to #pragma once for headers
It's just less tedious, and we're GCC/LLVM specific anyways.
2013-07-09 18:53:22 -04:00
Colin Walters 7f48c77204 pull: Explicitly use thread-default main context
As preparation for pushing our own temporary context.
2013-07-09 18:46:00 -04:00
Colin Walters 13e33cd76b admin: Use internal pull API instead of subprocess
So we're doing more of the library thing.
2013-07-09 18:34:05 -04:00
Colin Walters 4c2a8bc210 pull: Destroy queue watching GSource
When being more of a library, we can't leave garbage attached to the
default main context.
2013-07-09 18:33:30 -04:00
Colin Walters b18e21be1d core: Move pull logic into an API
The general trend should be becoming more of a shared library with
command line wrappers.
2013-07-09 17:58:03 -04:00
Colin Walters a181310a49 pull: Add "tls-permissive" config option to disable SSL/TLS certificate checks
Like GIT_SSL_NO_VERIFY=true, available for the same reasons.
2013-07-09 15:40:20 -04:00
Colin Walters 7092e88eac pull: Verify SSL/TLS certificates against system CA file by default
We also need a mechanism like GIT_SSL_NO_VERIFY...probably an option
in the config.
2013-07-09 15:02:48 -04:00
Colin Walters b5b2aca592 prepare-root: Don't be strict about the sysroot mount name here
While the systemd integration effectively requires /sysroot, it will
help people trying to use OSTree with other initramfs systems
(e.g. initramfs-tools) if we don't hardcode that requirement in this
tool.
2013-07-09 12:04:02 -04:00
Colin Walters 43d69ac8a3 tests: Drop numeric prefix
Theoretically it's useful to have layers of tests, but in practice
it's just annoying to assign numbers.
2013-07-09 10:41:07 -04:00
Colin Walters 56f8584fe4 repo: Store transaction file, use it to optimize for pull resumes
If pull is interrupted, we may have downloaded an arbitrary subset of
the requested objects.  Previously, we handled this by scanning for
all objects each time.

However, there's an easy optimization - this patch creates a lock file
in the repo.  If we don't see that file when starting a pull, we know
we don't need to stat() every file; presence of a dirtree object for
example implies the existence of everything it references.
2013-07-08 09:05:34 -04:00
Colin Walters 305cd02e0c Drop obsolete GRUB2 and kernel update hooks
We only support syslinux at the moment; grub2 should learn to parse
the bootloader spec.
2013-07-07 21:56:54 -04:00
Colin Walters 53dcb3a68d Remove stub daemon code
This really shouldn't have lived so long...it does nothing now, and we
should probably just be exposing an API for packagekit anyways.
2013-07-07 18:14:53 -04:00
Colin Walters 26cef497a6 Remove built in "triggers"
Originally, the idea was that clients would replicate "OS/tree"s from
a build server, but we'd run things like "ldconfig" on the client.
This was to allow adding e.g. the nVidia binary driver.

However, the triggers were the only thing in the system at the moment
that really had expected knowledge of the *contents* of the OS, like
the location of binaries.

For now, it's architecturally cleaner if we move the burden of
triggers to the tree builder (e.g. gnome-ostree or RPM).  Eventually
we may want OSTree to assist with this type of thing (perhaps
something like RPM %ghost), but this is the right thing to do now.
2013-07-07 14:37:59 -04:00
Colin Walters f6bca20551 admin: Fix various compiler warnings 2013-07-07 14:30:01 -04:00
Colin Walters c66148160c admin: Also delete unsed boot directories
My VM filled up /boot.  Oops.
2013-07-07 13:45:18 -04:00
Colin Walters 1fa1443bae admin: Split up the monstrous ot-admin-functions.c
Now util, cleanup, and functions.
2013-07-07 12:42:02 -04:00
Colin Walters 4b170d656c Switch to libgsystem local allocation macros
And drop our compatibility wrapper.
2013-07-07 12:27:44 -04:00
Colin Walters bb6eedfb25 [INCOMPATIBLE CHANGE] Implement new deployment model
See https://wiki.gnome.org/OSTree/DeploymentModel2

This is a major rework of the on-disk filesystem layout, and the boot
process.  OSTree now explicitly supports upgrading kernels, and these
upgrades are also atomic.

The core concept of the new model is the "deployment list", which is
an ordered list of bootable operating system trees.  The deployment
list is reflected in the bootloader configuration; which has a kernel
argument that tells the initramfs (dracut) which operating system root
to use.

Invidiual notable changes that come along with this:

1) Operating systems should now come with their etc in usr/etc; OSTree
   will perform a 3-way merge at deployment time, and place etc in
   the actual root.  This avoids the need for a bind mount, and is
   just a lot cleaner.
2) OSTree no longer bind mounts /root, /home, and /tmp.  It is expected
   that the the OS/ has these as symbolic links into /var.

At the moment, OSTree only supports managing syslinux; other
bootloader backends will follow.
2013-07-07 11:31:26 -04:00
Colin Walters ecb3f0de03 deploy: Error out if we see usr/etc
This version of the code doesn't know what to do with new-model trees,
so just abort if we see that.  We'll likely never care about upgrades
from 1.0 to 2.0.
2013-07-05 17:29:07 -04:00
Colin Walters e8cba55a34 core: Allow '.' in refspecs too
For version numbers.
2013-07-05 16:51:47 -04:00
Colin Walters ec21dc4242 Add "trivial-httpd" builtin, use it in tests
A simple HTTP server implementation is so few lines of code when one
is linking to libsoup anyways, so let's just have one here in ostree
that will be used for the test suite.

This allows us to run the archive tests that previously required
apache even in gnome-ostree.
2013-07-05 16:28:40 -04:00
Colin Walters a93f2b8d16 pull: Make fetcher work for file:/// URIs too
Even if very suboptimally, for now; we copy the files, then copy them
again.

The obvious long term plan is to merge pull-local and pull together,
but truly optimizing that requires the pull code to know how to use
the OstreeRepo APIs when operating on local repositories (as
pull-local does), rather than assuming the remote is an archive-z
fetched over HTTP.
2013-07-01 22:43:17 -04:00
Colin Walters dc0f3c3dcb Add support for deleting refs
The internal API will be used by admin, and "ostree refs --delete"
is handy for interactive management.
2013-07-01 15:41:27 -04:00
Colin Walters 5b3fca8426 Add "refs" builtin
This is just useful to look at before pruning, etc.
2013-06-29 14:51:08 -04:00
Colin Walters 613f57007c Extract prune logic into an internal API
This will be used by ostree admin deploy.
2013-06-29 13:49:34 -04:00
Colin Walters a167bb7342 pull-local: Support --remote argument
For offline upgrades, pull-local can now write the refs into a
specific remote, rather than using the local heads.
2013-06-29 13:00:42 -04:00
Colin Walters 1ba852ebaa core: Add "refspec" which is remote:refname
This allows an unambiguous reference; otherwise, it was too easy to
have confusion between local heads and remotes.
2013-06-29 12:51:29 -04:00
Colin Walters e3dc0c91df gioutils: Many new utility functions
Reading symbolic links becomes a much more common thing now.
2013-06-29 12:51:29 -04:00
Colin Walters f6946a368f prepare-root: Only bind mount /home, /tmp, /root if they are directories
What we expect for new systems is for these to be symbolic links:
/home -> /sysroot/home
etc.
2013-06-23 17:56:14 -04:00
Colin Walters 7bb20b0533 core: Port more callsites to gs_file_enumerator_iterate()
See previous commit.
2013-06-14 19:45:40 -04:00
Colin Walters fbd9f17e39 core: Update libgsystem, port some uses of GFileEnumerator
This makes the code *so* much nicer.
2013-06-14 19:14:35 -04:00
Colin Walters c57e63a515 pull-local: Fix race condition pointed out by Benjamin
We could drop into g_main_loop_run() after the worker
thread had called g_main_loop_quit().

Fix this by following the pattern suggested by Ryan of a while loop
around our termination condition, and g_main_context_iteration().
2013-06-08 14:15:43 -04:00
Colin Walters 3e1e26352f pull-local: Fix race condition
https://bugzilla.gnome.org/show_bug.cgi?id=701861
2013-06-08 12:58:44 -04:00
Colin Walters 7e882cc2cf dracut: Add ostree-remount
Linux creates a copy of the soure mount flags when creating a bind
mount; if the source is read-only, then the bind mount is.

The problem is that systemd will remount the rootfs read/write, but
each mount (/home, /var etc.) will still be read-only.  We need to
remount every bind mount except for /usr to read-write too.

This only "worked" with the old ostree-switch-root because it
effectively force mounted the rootfs read-write always, ignoring the
"ro" flag.
2013-06-04 15:59:52 -04:00
Colin Walters 26baee6c3b dracut: Also ensure we're using the systemd module
I can't work out how this happens otherwise in Fedora...
2013-06-03 20:49:07 -04:00
Colin Walters 6496abacbd Add --with-dracut
This installs a Dracut module which parses the ostree= kernel command
line argument, and if given, sets up the OS/ at /sysroot, which
systemd's switch-root then moves into.  This only works if dracut is
configured to use systemd itself.
2013-06-03 15:28:09 -04:00
Colin Walters 35df7a7eb4 ostree-prepare-root: Don't require an extraneous argument 2013-06-02 22:19:24 -04:00
Colin Walters 8eaaea795c ostree-prepare-root: Parse /proc/cmdline, make RPRIVATE, set up /sysroot
This does everything we need except for the actual switch-root.
2013-06-02 21:41:08 -04:00
Colin Walters 1e080b9c73 ostree-prepare-root: New binary, used for systemd-in-initramfs setups
Rather than attempting to hack up the "switch-root" functionality of
systemd, this binary allows us to simply prepare the root before we
switch into it.
2013-06-02 15:39:20 -04:00
Colin Walters 16157c61af Drop support for generating initramfs images client side using dracut
This is the responsibility of whatever is generating filesystem trees.
2013-05-31 13:09:27 -04:00
Colin Walters ba5fc5cbcc Fold ostree-pull into main binary
The rationale for the separation was always kind of weak; I want to
refactor the command line argument parsing, and it was complicating
things.
2013-05-30 19:26:48 -04:00
Colin Walters 86e504ba1d pull: Exit cleanly with an error on failure to fetch refs 2013-05-27 09:29:33 -04:00
Colin Walters fab160562d upgrade: Actually prune deployments again
That was a rather critical logic error...
2013-05-27 09:29:05 -04:00
Colin Walters 0852362efa upgrade: Actually deploy upgraded tree again
I need a brown paper bag for this one...
2013-05-16 15:38:17 -04:00
Colin Walters f9484e9bab admin: Add -r option to upgrade to initiate a reboot if tree changes
$ ostree admin upgrade -r

is convenient to fire off inside a VM and come back to it later.
2013-05-16 08:29:20 -04:00
Colin Walters 18af230e17 upgrade: Don't require specification of osname
Since this is a highly visible command, let's have a sane default and
upgrade the currently running OS.
2013-05-15 08:18:53 -04:00
Colin Walters 82f444020e Add a --version argument that also outputs features
And use it in the tests to skip libarchive tests when the build
doesn't support it.
2013-05-01 19:15:06 -04:00
Colin Walters c60c70e9a9 checkout: Add --allow-noent option
This is useful for the gnome-ostree build system where each build is
one commit, but it's split up into /runtime /devel /debug etc. trees.
Ideally we wouldn't have a /debug subdirectory for "noarch"
components for example.

So add an option to not error out if the given path doesn't exist in
the commit.
2013-05-01 12:15:02 -04:00
Colin Walters f4327cc6a0 repo: Fix race condition in async checkout
When multiple threads need to uncompress an object, there was
a race condition where thread A could get EEXIST, unlink,
then thread B calls linkat(), then thread A tries to link() but
fails.

We can just loop in this case.
2013-04-30 19:34:20 -04:00
Colin Walters 8e4b296a96 libotutil: Drop is-hidden from fast queryinfo
Since now we look at .hidden files and bang on the mainloop for every
file lookup if that's specified =(
2013-04-29 22:08:56 -04:00
Colin Walters c110680f5d libgsystem: Update to latest 2013-04-29 14:04:48 -04:00
Colin Walters 8c694622b1 deploy: Swap current symlink only after updating the kernel
While this still isn't fully atomic (that depends on the bootloader),
this better ensures that the deployed kernel is booted with the
intended tree.  For example, if we get ENOSPC when writing out the
kernel, we won't have swapped the symlink.
2013-04-26 18:15:51 -04:00
Colin Walters 52a3369709 libgsystem: Update 2013-04-22 12:27:47 -04:00
Colin Walters b06c4a2081 pull-deploy: Tweak documentation to show using it for switching trees 2013-04-21 10:39:27 -04:00
Colin Walters 5565735bcb admin: Use linkcopy_sync_data() for initramfs instead of rename()
It's possible (likely even) that /tmp is on a separate filesystem; in
that case, a raw rename() is going to fail.

Saw this on the ostree.gnome.org builder.
2013-04-15 12:50:58 -04:00
Colin Walters a01783dafd prune: Use g_format_size() so we get a human-readable number
We don't really care about the precise bytes.
2013-04-08 16:56:05 -04:00
Colin Walters 4271f84755 waitablequeue: Only use API from GLib 2.34
As intended.
2013-04-03 00:50:16 -04:00
Colin Walters c563d07cb6 switchroot: Suppress compiler warning for asprintf
We're not going to run OOM here, if we do, we're totally screwed
anyways.
2013-04-01 21:52:24 -04:00
Colin Walters bac4d7a0d2 pull: Rework threading communication model
Previously, I've observed bugs where we either:
1) Exit too early, leaving undownloaded objects
2) Hang while downloading

This rewrite hopefully fixes both.
2013-04-01 18:56:38 -04:00
Colin Walters 5d1b0ec1b3 core: Fix object enumeration for archive-z2 repositories
This makes fsck work again, among other things.
2013-04-01 11:43:03 -04:00
Colin Walters bde7e5c719 pull: Drop support for original archive repositories
For future refactoring here, it's just too complex to maintain support
for both of them, and the advantages of archive-z2 outweigh archive
too much.
2013-03-29 17:22:56 -04:00
Colin Walters 7b2fc8df36 admin: Assume we're using /ostree if there's a toplevel /usr
Mainly so admin install works without an existing /ostree again.
2013-03-18 18:11:00 -04:00
Colin Walters e4faf0d98e admin: Support installing pregenerated initramfs images from tree
Us running dracut is a hack, basically.  We should default to
generating this on the server.
2013-03-16 17:45:15 -04:00
Colin Walters ae3945210c config-diff: Properly print files relative to root
Just less ugly.
2013-03-12 08:48:05 -04:00
Colin Walters d6d2253f5a admin: Fix lookup of active root
This should hopefully *really* fix deleting running deployments

https://bugzilla.gnome.org/show_bug.cgi?id=694762
2013-03-07 16:46:41 -05:00
Colin Walters 30d46f9a02 admin: Don't delete the running deployment
https://bugzilla.gnome.org/show_bug.cgi?id=694762
2013-03-04 22:52:08 -05:00
Colin Walters dba406bb8f deploy: Don't run triggers if /usr/share/ostree/triggers-run exists
This is preparatory work for a future gnome-ostree commit where we'll
default to running triggers on the build server side.
2013-03-04 11:52:43 -05:00
Colin Walters a016d744ea Update libgsystem 2013-02-26 12:19:05 -05:00
Colin Walters 202c5e35c9 core: Use default /sysroot/ostree/repo or just /ostree/repo as appropriate
The old logic was just weird, special casing root, and we can easily
autodetect the case where we need /ostree/repo.
2013-02-24 10:00:23 -05:00
Colin Walters 99fe7b66b3 pull-local: Don't call g_hash_table_init() on a NULL hash
Not sure why this wasn't crashing before.
2013-02-23 11:15:19 -05:00
Colin Walters a821420711 core: Add API to convert csum -> checksum without malloc()
Just doing some profiling, this was in the top malloc() callers.
2013-02-23 11:02:56 -05:00
Colin Walters a305c79f7a pull-local: Fix uninitialized variable ref 2013-02-21 13:05:14 -05:00
Colin Walters 8521438273 pull-local: Output status even in non-tty case
This way I can see progress when running builddisks.
2013-02-21 08:54:13 -05:00
Colin Walters 420e5f706a admin: Use /sysroot/ostree automatically
When booted inside an ostree system, we want /sysroot/ostree, not
just /ostree by default.
2013-02-19 11:02:33 -05:00