Commit Graph

843 Commits

Author SHA1 Message Date
Jeremy Whiting 71f6f10cd2 trivial-httpd: Add --force-range-requests option to force range requests
This will be used to test resuming interrupted downloads for
ostree-pull.

With this option, if a whole file is asked for, only half of the file
is given.  Then the client should retry with a range request, and
we'll give them the other half.

https://bugzilla.gnome.org/show_bug.cgi?id=705925
2013-08-14 17:33:23 -04:00
Colin Walters 61773f6ca4 core: Add a comment header about the format
This should probably be a design document or something, but this is
useful for now.
2013-08-14 17:35:46 +02:00
Stef Walter 1904e3e0f4 Don't pass non-checksums to ostree_repo_has_object()
The 'ostree show' command passed non-checksum arguments
to the ostree_repo_has_object() function which led to
an assertion.

https://bugzilla.gnome.org/show_bug.cgi?id=705967
2013-08-14 16:08:18 +02:00
Colin Walters 8c31e6fda7 tests: Add xattr test case
Since it was broken up until now for multiple attributes =(
2013-08-14 15:07:14 +02:00
Stef Walter 860cc11160 Fix crasher when running 'ostree cat'
Cause was a buffer read overrun.

https://bugzilla.gnome.org/show_bug.cgi?id=705964
2013-08-14 14:53:17 +02:00
Stef Walter 97947373ee Add some verbose log output when xattr functions fail
If any of the system xattr functions fail, clearly write out the
reason in the verbose log output.

https://bugzilla.gnome.org/show_bug.cgi?id=705893
2013-08-14 07:50:49 +02:00
Stef Walter 3f9c150b77 Don't use XATTR_REPLACE with lsetxattr()
If we pass XATTR_REPLACE then the attribute must already exist, which
is not our intent. Passing zero creates the attribute if necessary,
or replaces it when it already exists.

https://bugzilla.gnome.org/show_bug.cgi?id=705893
2013-08-13 15:42:54 +02:00
Stef Walter c246c4194f Properly separate sorted xattr names
We expect to be handling a string delimited by \0 characters, as
returned by llistxattr(). So stick to that behavior here.

https://bugzilla.gnome.org/show_bug.cgi?id=705893
2013-08-13 15:42:54 +02:00
Stef Walter 0111ec1307 Add some verbose logging to 'ostree commit'
This is so we can see near which file we fail when 'ostree commit'
falls over.

https://bugzilla.gnome.org/show_bug.cgi?id=705905
2013-08-13 15:38:51 +02:00
Stef Walter 462bcd86de Add --verbose and -v arguments to ostree for details
Use the GLib debug mechanism to handle verbose output.

https://bugzilla.gnome.org/show_bug.cgi?id=705905
2013-08-13 15:37:12 +02:00
Stef Walter 1f8c7a2524 Better parsing for global ostree options
* Specifying global options after the command for a more natural:
   # ostree commit --repo=/path/to/repo ...
 * Support asking for --help without --repo
   # ostree commit --help
 * Support short form of -h
 * Support specifying --repo without equals sign
   # ostree --repo /path/to/repo commit ...
 * Support global --help and -h
   # ostree --help
 * Ditto for ostree admin sub commands
 * Removed some leaky code

https://bugzilla.gnome.org/show_bug.cgi?id=705903
2013-08-13 14:28:36 +02:00
Colin Walters 44012dbdd9 Update libgsystem
With fd leak fix.
2013-08-07 10:36:52 +02:00
Colin Walters 71c4ea67f7 core: Clean up temporary file creation
Use new libgsystem API, don't expose the repo's tmpdir.
2013-08-06 23:58:29 +02:00
Colin Walters 9e480cd6a9 core: Finish making object path API private
This allows us to more easily change the internals later.
2013-07-31 03:48:33 -04:00
Colin Walters 0dec1b287c core: Reduce API surface for file access
This function was only used by internals, make it private.
2013-07-31 00:35:53 -04:00
Colin Walters 8d5b8dd740 admin: Clean up builtin prototype
Pass through GCancellable, and just use GFile *sysroot, since that's
all OtAdminBuiltinOpts was.
2013-07-29 18:44:41 -04:00
Colin Walters 760b866104 main: Assume / for ls if no path given
Since it just is kind of a sane default.
2013-07-27 20:12:07 -04:00
Colin Walters 7c5c3f2af8 core: Fix all introspection warnings
Still lots more docs to write and API to cleanup, but this is better.
2013-07-27 10:13:30 -04:00
Colin Walters a5d43bb959 Install a shared library
This required a fair bit of surgery because previously ostree.h
included otutil.h, but that's supposed to be a private library.
2013-07-26 19:25:07 -04:00
Colin Walters 74b2c7aab3 core: Work around libguestfs/FUSE issue with setuid binaries
For some reason, the setuid bits are being stripped.  This
workaround is enough for now.
2013-07-26 14:48:15 -04:00
Colin Walters 5aea9db3f2 Add API to retrieve statistics from transactions, use it in commit
It's just interesting to see =) This is also kind of prototyping out
some more "structured" output.
2013-07-25 19:23:25 -04:00
Colin Walters 8aa48a5f9c fsck: Fix crash on repositories with < 10 objects 2013-07-25 12:06:21 -04:00
Colin Walters 2bcb024824 pull: Tweak status line again, do what git does
Almost, we don't have bytes transferred.  And we lie about percentages
since as metadata comes in the total number of objects to fetch goes
up.
2013-07-24 18:20:19 -04:00
Colin Walters 89430bbe71 main: Fix previous commit to find repo in cwd over /ostree/repo
The tests when run inside gnome-ostree broke, and this is better
behavior.
2013-07-24 14:46:53 -04:00
Colin Walters 187c75eae5 main: Support using repo from current directory
$ cd repo
$ ostree ls foo /
...

Can be a lot more convenient than typing --repo=repo a lot.
2013-07-24 14:05:12 -04:00
Colin Walters c9b61cbfee Drop support for related objects and metadata in commit objects
While the actual commit object format is presently the same, for a
number of reasons we'd like to change it fairly radically.  Among
other things, we need to drop our a{sv} types in objects, to protect
against GVariant changing format.

Since now gnome-ostree now longer uses related objects, and nothing
ever used metadata, just drop them both.
2013-07-24 12:59:27 -04:00
Colin Walters 1e7c0bb70b main: Fix an uninitialized variable 2013-07-24 09:36:51 -04:00
Colin Walters 688d8f176b pull: Always scan for commit object, even if ref is unchanged
If the admin encounters corruption and does:

$ ostree admin fsck --delete

We want them to be able to recover the objects easily from the
network; with this patch, they do:

$ ln -s dummyvalue /ostree/repo/transaction
$ ostree refs --delete remotename:branchname
$ ostree pull remotename

This patch avoids the need for the refs --delete; we might as well
force scan the commit, and with this patch we still print that it
changed.
2013-07-23 19:36:15 -04:00
Colin Walters 3ea62abe75 main: ls: Behave like standard Unix "ls" with directories
Default to listing directory contents, add -d/--dironly to suppress
this.
2013-07-23 18:41:03 -04:00
Colin Walters 4de1d51266 main: Squash a compiler warning 2013-07-23 18:34:34 -04:00
Colin Walters 7b056e3c50 main: Consistently use "opt_" as a prefix for option local variables
This makes it easier to find where we're using global state, which is
useful when refactoring code into libostree.
2013-07-23 18:33:29 -04:00
Colin Walters b6ab480272 main: commit: Drop broken --parent option
No idea why we had this, it didn't do anything in the current code.
Noticed while doing other refactoring.
2013-07-23 18:24:52 -04:00
Colin Walters 3b9da094d8 main: Drop log builtin
We may revive this later, but commits in their current form aren't
very useful for humans to read, so it doesn't make sense to have a
tool to show a history of useless stuff.

More interesting things are diffs between commits, object statistics,
etc.
2013-07-23 18:19:14 -04:00
Colin Walters ea914e7e49 main: pull: Reduce the status line length a bit
Drop "objects" where it's obvious [ to me anyways =) ], and drop the
bytes transferred since our math was off, and the HTTP status kind of
shows that.
2013-07-23 17:29:46 -04:00
Colin Walters 3d7bff2d41 admin: Add an "undeploy" command
Otherwise it's really easy to keep accumulating deployments.  Also, we
may want to run this after rebooting, so we're back down to one
operating system.
2013-07-23 09:19:24 -04:00
Colin Walters 67823beb1f core: Don't strip setuid bits when creating files
This ugly regression occurred because I overlooked the fact that our
chown() invocation would strip off setuid.
2013-07-18 14:23:55 -04:00
Colin Walters 3e2d2f3e9e main: show: Don't strip mode, let's see the whole thing
Just useful for debugging.
2013-07-18 14:23:12 -04:00
Colin Walters f0c5a5f334 main: Add --delete option to fsck
This is useful for clearing out all corrupted objects locally.
2013-07-18 12:09:44 -04:00
Colin Walters c8801ae489 main: Clean up fsck code: honor --quiet, warn (but continue) on missing objects
When we make fsck --delete work again, it will be convenient to
continue on missing objects.
2013-07-18 09:05:58 -04:00
Colin Walters a4f5ad8542 Update libgsystem 2013-07-17 23:00:03 -04:00
Colin Walters 41cb37a858 main: Make "ostree show" actually useful for files
We already have "ostree cat" for files, here we want to see a dump of
the metadata.
2013-07-17 21:10:04 -04:00
Colin Walters 1b3ef57cf4 diff: Add --stats option
This a neat way to see how many objects are shared between two
commits.
2013-07-17 19:20:53 -04:00
Colin Walters a3b6401788 main: Add --link-checkout-speedup option to commit
And disable devino scan by default.  For the gnome-ostree build case,
our commits are from "make install DESTDIR=", so they won't be
hardlinks into the repo.  In that case, particularly as the repository
size grows, the cost of building up the devino -> checksum mapping
becomes a problem.

The compose step will use this option though.
2013-07-17 19:20:53 -04:00
Colin Walters aac52cb9f7 main: Move GCancellable to toplevel
This is just cleaner, and makes the builtins slightly more of a
library, which in turn makes it easier to turn them into real API.
2013-07-16 18:13:39 -04:00
Colin Walters 272274f0af prune: Don't fail if a to-be-pruned object doesn't exist
Previously, if a prune was interrupted, further attempts would fail.
It's just better to silently continue here.
2013-07-16 10:01:58 -04:00
Colin Walters 62a896350b Drop support for fifos and devices
While the first was useful way back in the day when we were importing
Debian bits and /sbin/init was expecting to find /dev/.initctl as a
named pipe, that's no longer an issue with systemd since it uses
dynamic Unix sockets.

Likewise, character and block devices in /dev are now dynamically
created by the devtmpfs from the kernel.

Less complexity and code here if we just support directories, regular
files, and symbolic links.
2013-07-16 09:35:44 -04:00
Colin Walters 8724565291 core: Drop a leftover unnecessary #define _GNU_SOURCE
We now use AC_USE_SYSTEM_EXTENSIONS.
2013-07-16 09:22:42 -04:00
Colin Walters 444234e7db core: Use gsystem API for generating temporary names
Drops duplicated code.
2013-07-16 08:51:53 -04:00
Colin Walters acdf1ca4fe core: Use gs_file_create()
This allows us to drop a chmod() invocation, since the regular file
path was the only thing that didn't handle mode.
2013-07-16 08:51:53 -04:00
Colin Walters ed55998f36 Update libgsystem 2013-07-16 08:51:53 -04:00
Colin Walters fb93b95807 admin: Rename prune -> cleanup, avoid doing repo prune twice
Calling it "cleanup" is better since it does more than repo pruning.

We were also doing a prune twice; ot_admin_cleanup() already does one,
so drop the bits to do it in cleanup.c.
2013-07-15 16:13:12 -04:00
Colin Walters d5f1ce4e17 admin: Initialize sysroot to / by default, not garbage from the stack
Oops.
2013-07-11 17:29:04 -04:00
Colin Walters 60f5183298 admin status: Print out origin too
This is useful data.
2013-07-11 17:07:14 -04:00
Colin Walters e95935a3f6 main: Also make ostree admin foo --help work
A followup for the previous commit.
2013-07-10 18:42:58 -04:00
Colin Walters b7460cf0b6 main: Only parse options in subcommands
This makes e.g. "ostree commit --help" work, which is really good.
2013-07-10 18:23:52 -04:00
Colin Walters be2bebaf3e main: Don't look for /sysroot/ostree too
No longer necessary now that we expect OS/ trees to have
/ostree -> /sysroot/ostree.
2013-07-10 18:13:55 -04:00
Colin Walters 124416d832 Use AC_USE_SYSTEM_EXTENSIONS instead of #define _GNU_SOURCE
It's less copy/paste, works everywhere, etc.

Also fix some missing #include "config.h".
2013-07-10 13:25:35 -04:00
Colin Walters 8cb348315e main: Print "error: " prefix on errors
It's just clearer that something went wrong.
2013-07-10 13:25:15 -04:00
Colin Walters 32d2cd57c1 admin: Add a warm fuzzy message to deploy about transactions
Since it sounds cool and stuff.
2013-07-10 02:30:26 -04:00
Colin Walters 12c76188c0 Delete leftover kernel update hooks 2013-07-09 20:38:05 -04:00
Colin Walters 89181c4e8e Build without libsoup again 2013-07-09 20:35:15 -04:00
Colin Walters 88cca89317 ostree: Some header file cleanups 2013-07-09 20:30:00 -04:00
Colin Walters 45c7536697 cmd: Drop "ostree admin install" and curl fetcher
It isn't useful at the moment, since the deploy stuff all changed.  It
will make sense to bring back later, but for now let's not carry
broken untested code.
2013-07-09 20:18:29 -04:00
Colin Walters 2535f32c56 libostree: Move pull code into here
More library work.
2013-07-09 20:14:53 -04:00
Colin Walters 5dd0d5da40 libostree: Move prune into OstreeRepo namespace
More library work.
2013-07-09 20:05:31 -04:00
Colin Walters 2aa0251b1b libostree: Rename ostree-traverse.h into OstreeRepo namespace
This operates on the repo, it might as well be methods on it.
2013-07-09 19:59:39 -04:00
Colin Walters cb6b69616c libostree: Split off -refs.c
Continuing to break up ostree-repo.c.
2013-07-09 19:49:00 -04:00
Colin Walters 3b6a94d272 libostree: Split off libarchive code 2013-07-09 19:11:37 -04:00
Colin Walters dbffd31017 libostree: Start splitting up the monstrous ostree-repo.c
Checkout now lives in its own file.
2013-07-09 19:02:38 -04:00
Colin Walters 5ad0671cd2 Switch to #pragma once for headers
It's just less tedious, and we're GCC/LLVM specific anyways.
2013-07-09 18:53:22 -04:00
Colin Walters 7f48c77204 pull: Explicitly use thread-default main context
As preparation for pushing our own temporary context.
2013-07-09 18:46:00 -04:00
Colin Walters 13e33cd76b admin: Use internal pull API instead of subprocess
So we're doing more of the library thing.
2013-07-09 18:34:05 -04:00
Colin Walters 4c2a8bc210 pull: Destroy queue watching GSource
When being more of a library, we can't leave garbage attached to the
default main context.
2013-07-09 18:33:30 -04:00
Colin Walters b18e21be1d core: Move pull logic into an API
The general trend should be becoming more of a shared library with
command line wrappers.
2013-07-09 17:58:03 -04:00
Colin Walters a181310a49 pull: Add "tls-permissive" config option to disable SSL/TLS certificate checks
Like GIT_SSL_NO_VERIFY=true, available for the same reasons.
2013-07-09 15:40:20 -04:00
Colin Walters 7092e88eac pull: Verify SSL/TLS certificates against system CA file by default
We also need a mechanism like GIT_SSL_NO_VERIFY...probably an option
in the config.
2013-07-09 15:02:48 -04:00
Colin Walters b5b2aca592 prepare-root: Don't be strict about the sysroot mount name here
While the systemd integration effectively requires /sysroot, it will
help people trying to use OSTree with other initramfs systems
(e.g. initramfs-tools) if we don't hardcode that requirement in this
tool.
2013-07-09 12:04:02 -04:00
Colin Walters 43d69ac8a3 tests: Drop numeric prefix
Theoretically it's useful to have layers of tests, but in practice
it's just annoying to assign numbers.
2013-07-09 10:41:07 -04:00
Colin Walters 56f8584fe4 repo: Store transaction file, use it to optimize for pull resumes
If pull is interrupted, we may have downloaded an arbitrary subset of
the requested objects.  Previously, we handled this by scanning for
all objects each time.

However, there's an easy optimization - this patch creates a lock file
in the repo.  If we don't see that file when starting a pull, we know
we don't need to stat() every file; presence of a dirtree object for
example implies the existence of everything it references.
2013-07-08 09:05:34 -04:00
Colin Walters 305cd02e0c Drop obsolete GRUB2 and kernel update hooks
We only support syslinux at the moment; grub2 should learn to parse
the bootloader spec.
2013-07-07 21:56:54 -04:00
Colin Walters 53dcb3a68d Remove stub daemon code
This really shouldn't have lived so long...it does nothing now, and we
should probably just be exposing an API for packagekit anyways.
2013-07-07 18:14:53 -04:00
Colin Walters 26cef497a6 Remove built in "triggers"
Originally, the idea was that clients would replicate "OS/tree"s from
a build server, but we'd run things like "ldconfig" on the client.
This was to allow adding e.g. the nVidia binary driver.

However, the triggers were the only thing in the system at the moment
that really had expected knowledge of the *contents* of the OS, like
the location of binaries.

For now, it's architecturally cleaner if we move the burden of
triggers to the tree builder (e.g. gnome-ostree or RPM).  Eventually
we may want OSTree to assist with this type of thing (perhaps
something like RPM %ghost), but this is the right thing to do now.
2013-07-07 14:37:59 -04:00
Colin Walters f6bca20551 admin: Fix various compiler warnings 2013-07-07 14:30:01 -04:00
Colin Walters c66148160c admin: Also delete unsed boot directories
My VM filled up /boot.  Oops.
2013-07-07 13:45:18 -04:00
Colin Walters 1fa1443bae admin: Split up the monstrous ot-admin-functions.c
Now util, cleanup, and functions.
2013-07-07 12:42:02 -04:00
Colin Walters 4b170d656c Switch to libgsystem local allocation macros
And drop our compatibility wrapper.
2013-07-07 12:27:44 -04:00
Colin Walters bb6eedfb25 [INCOMPATIBLE CHANGE] Implement new deployment model
See https://wiki.gnome.org/OSTree/DeploymentModel2

This is a major rework of the on-disk filesystem layout, and the boot
process.  OSTree now explicitly supports upgrading kernels, and these
upgrades are also atomic.

The core concept of the new model is the "deployment list", which is
an ordered list of bootable operating system trees.  The deployment
list is reflected in the bootloader configuration; which has a kernel
argument that tells the initramfs (dracut) which operating system root
to use.

Invidiual notable changes that come along with this:

1) Operating systems should now come with their etc in usr/etc; OSTree
   will perform a 3-way merge at deployment time, and place etc in
   the actual root.  This avoids the need for a bind mount, and is
   just a lot cleaner.
2) OSTree no longer bind mounts /root, /home, and /tmp.  It is expected
   that the the OS/ has these as symbolic links into /var.

At the moment, OSTree only supports managing syslinux; other
bootloader backends will follow.
2013-07-07 11:31:26 -04:00
Colin Walters ecb3f0de03 deploy: Error out if we see usr/etc
This version of the code doesn't know what to do with new-model trees,
so just abort if we see that.  We'll likely never care about upgrades
from 1.0 to 2.0.
2013-07-05 17:29:07 -04:00
Colin Walters e8cba55a34 core: Allow '.' in refspecs too
For version numbers.
2013-07-05 16:51:47 -04:00
Colin Walters ec21dc4242 Add "trivial-httpd" builtin, use it in tests
A simple HTTP server implementation is so few lines of code when one
is linking to libsoup anyways, so let's just have one here in ostree
that will be used for the test suite.

This allows us to run the archive tests that previously required
apache even in gnome-ostree.
2013-07-05 16:28:40 -04:00
Colin Walters a93f2b8d16 pull: Make fetcher work for file:/// URIs too
Even if very suboptimally, for now; we copy the files, then copy them
again.

The obvious long term plan is to merge pull-local and pull together,
but truly optimizing that requires the pull code to know how to use
the OstreeRepo APIs when operating on local repositories (as
pull-local does), rather than assuming the remote is an archive-z
fetched over HTTP.
2013-07-01 22:43:17 -04:00
Colin Walters dc0f3c3dcb Add support for deleting refs
The internal API will be used by admin, and "ostree refs --delete"
is handy for interactive management.
2013-07-01 15:41:27 -04:00
Colin Walters 5b3fca8426 Add "refs" builtin
This is just useful to look at before pruning, etc.
2013-06-29 14:51:08 -04:00
Colin Walters 613f57007c Extract prune logic into an internal API
This will be used by ostree admin deploy.
2013-06-29 13:49:34 -04:00
Colin Walters a167bb7342 pull-local: Support --remote argument
For offline upgrades, pull-local can now write the refs into a
specific remote, rather than using the local heads.
2013-06-29 13:00:42 -04:00
Colin Walters 1ba852ebaa core: Add "refspec" which is remote:refname
This allows an unambiguous reference; otherwise, it was too easy to
have confusion between local heads and remotes.
2013-06-29 12:51:29 -04:00
Colin Walters e3dc0c91df gioutils: Many new utility functions
Reading symbolic links becomes a much more common thing now.
2013-06-29 12:51:29 -04:00
Colin Walters f6946a368f prepare-root: Only bind mount /home, /tmp, /root if they are directories
What we expect for new systems is for these to be symbolic links:
/home -> /sysroot/home
etc.
2013-06-23 17:56:14 -04:00
Colin Walters 7bb20b0533 core: Port more callsites to gs_file_enumerator_iterate()
See previous commit.
2013-06-14 19:45:40 -04:00
Colin Walters fbd9f17e39 core: Update libgsystem, port some uses of GFileEnumerator
This makes the code *so* much nicer.
2013-06-14 19:14:35 -04:00