bac4d7a0d2
pull: Rework threading communication model
...
Previously, I've observed bugs where we either:
1) Exit too early, leaving undownloaded objects
2) Hang while downloading
This rewrite hopefully fixes both.
2013-04-01 18:56:38 -04:00
5d1b0ec1b3
core: Fix object enumeration for archive-z2 repositories
...
This makes fsck work again, among other things.
2013-04-01 11:43:03 -04:00
bde7e5c719
pull: Drop support for original archive repositories
...
For future refactoring here, it's just too complex to maintain support
for both of them, and the advantages of archive-z2 outweigh archive
too much.
2013-03-29 17:22:56 -04:00
7b2fc8df36
admin: Assume we're using /ostree if there's a toplevel /usr
...
Mainly so admin install works without an existing /ostree again.
2013-03-18 18:11:00 -04:00
e4faf0d98e
admin: Support installing pregenerated initramfs images from tree
...
Us running dracut is a hack, basically. We should default to
generating this on the server.
2013-03-16 17:45:15 -04:00
ae3945210c
config-diff: Properly print files relative to root
...
Just less ugly.
2013-03-12 08:48:05 -04:00
d6d2253f5a
admin: Fix lookup of active root
...
This should hopefully *really* fix deleting running deployments
https://bugzilla.gnome.org/show_bug.cgi?id=694762
2013-03-07 16:46:41 -05:00
30d46f9a02
admin: Don't delete the running deployment
...
https://bugzilla.gnome.org/show_bug.cgi?id=694762
2013-03-04 22:52:08 -05:00
dba406bb8f
deploy: Don't run triggers if /usr/share/ostree/triggers-run exists
...
This is preparatory work for a future gnome-ostree commit where we'll
default to running triggers on the build server side.
2013-03-04 11:52:43 -05:00
a016d744ea
Update libgsystem
2013-02-26 12:19:05 -05:00
202c5e35c9
core: Use default /sysroot/ostree/repo or just /ostree/repo as appropriate
...
The old logic was just weird, special casing root, and we can easily
autodetect the case where we need /ostree/repo.
2013-02-24 10:00:23 -05:00
99fe7b66b3
pull-local: Don't call g_hash_table_init() on a NULL hash
...
Not sure why this wasn't crashing before.
2013-02-23 11:15:19 -05:00
a821420711
core: Add API to convert csum -> checksum without malloc()
...
Just doing some profiling, this was in the top malloc() callers.
2013-02-23 11:02:56 -05:00
a305c79f7a
pull-local: Fix uninitialized variable ref
2013-02-21 13:05:14 -05:00
8521438273
pull-local: Output status even in non-tty case
...
This way I can see progress when running builddisks.
2013-02-21 08:54:13 -05:00
420e5f706a
admin: Use /sysroot/ostree automatically
...
When booted inside an ostree system, we want /sysroot/ostree, not
just /ostree by default.
2013-02-19 11:02:33 -05:00
c9f56564a3
switchroot: Use MS_MOVE to / rather than chroot
...
This fixes a number of bugs; the two biggest are:
1) On shutdown systemd can correctly umount /
2) We can use linux-user-chroot inside the system, as is
necessary to do upgrades
See http://lists.freedesktop.org/archives/systemd-devel/2012-September/006703.html
2013-02-11 20:46:53 -05:00
1d98f8865c
pull-deploy: Allow switching trees too
...
$ ostree admin pull-deploy gnome-ostree trees/gnomeos-3.8-x86_64-devel
Now works to switch your current tree to -devel from the current
runtime.
2013-02-01 10:56:06 -05:00
42e3f1f4ab
pull-local: Fix crasher in previous commit when not connected to tty
2013-01-28 15:44:13 -05:00
aa5e11af12
pull-local: Port to GSConsole
2013-01-28 15:35:47 -05:00
13f36aa224
pull-local: Support pulling checksums too
...
This matches what pull allows.
2013-01-28 15:32:06 -05:00
5424652a7e
libgsystem: Update to the latest
2013-01-25 15:19:34 -05:00
64934202e4
admin: Don't overwrite previous ref unless it's different
...
We shouldn't overwrite deploy/<osname>/previous unless we actually
have something new.
This fixes the bug where two consecutive:
$ ostree admin upgrade foo
$ ostree admin upgrade foo
would end up pruning the data for foo/previous from the repo.
2013-01-24 17:32:59 -05:00
59c98d0a04
pull: Drop dead code to pad output strings too
...
This is in GSConsole now.
2013-01-24 15:38:06 -05:00
6fde0b61a8
Port ostree-pull to GSConsole
...
As part of this, we need to ensure our streams are inherited across
the chain of child processes.
2013-01-24 15:22:16 -05:00
fafb256f57
Update to the latest libgsystem
2013-01-23 15:20:19 -05:00
13aefbc096
admin install: Ensure /ostree is initialized first
...
https://bugzilla.gnome.org/show_bug.cgi?id=692008
2013-01-18 08:38:09 -05:00
341923b928
admin: Ensure /var/log/journal exists
...
This is admittedly a hack, since the story is that services should be
handling /var on their own. But on the other hand the current systemd
story is that admins should create it to enable it. Possibly a better
fix is --enable-journal-always or something for systemd.
2013-01-17 22:08:21 -05:00
50ca6b672b
update-kernel: Change dracut to use tmpdir on host, fix deploy path lookup
...
We had a bit of legacy code that looked for "current" if the deploy
path was unspecified; this needs to be deploy/OSNAME/current.
Secondly, we need to point dracut at a tmpdir outside of the root when
doing FUSE, because directories with mode 0700 and owned by root:root
aren't writable.
Also update libgsystem to the latest.
2013-01-09 04:32:51 -05:00
77fa143bf3
admin: Add --no-bootloader option to update-kernel
...
Will be used by the QA scripts, since we need to inspect the kernel
configuration and generate a correct grub conf from that, rather than
trying to have update-bootloader reuse an existing config, since there
won't be one initially.
2013-01-08 13:57:24 -05:00
ecc0d1e07f
Update to the latest libgsystem
2013-01-08 07:04:29 -05:00
1dcf796868
admin update-kernel: Drop support for booting from host kernel
...
Now that gnome-ostree has a kernel, this functionality is no longer
necessary. It was always a hack.
2013-01-06 12:37:45 -05:00
6b868d84d8
admin update-kernel: Ensure initramfs has correct owner/mode
...
It was 0600, and owned by the creating process (in the fuse case,
non-root!). We want it 0644/root/root.
2013-01-06 06:43:49 -05:00
9795f4a67e
admin: Propagate --boot-dir to subcommands
...
Really, I should be using internal functions here...
2013-01-06 06:12:42 -05:00
ccfdf22bb8
core: Drop dead prototypes from header
...
Followup to previous commit.
2013-01-04 19:47:56 -05:00
cd5fce713c
core: Reuse more libgsystem API in utils
...
More code drain to libgsystem.
2013-01-04 19:46:02 -05:00
c9a840c659
core: Ensure kernel and initramfs are put into place with fdatasync()
...
Via a new libgsystem API.
2013-01-04 19:32:26 -05:00
47bd290065
core: Drain fdatasync() API into libgsystem
...
Just code cleanup.
2013-01-04 17:25:32 -05:00
2db497fd3f
admin: Allow specifying --boot-dir
...
This is intended for use by QA tools that want to manipulate a
loopback-mounted disk image or the like.
2013-01-02 22:44:06 -05:00
6c9c7b0173
admin: restore current-etc symlink
...
It is useful now like it was before, and it is used by the grub
update script.
https://bugzilla.gnome.org/show_bug.cgi?id=690743
2012-12-26 19:42:46 +01:00
9bd4d35c2b
pull: Fix a race condition causing an early exit
...
This is a little less magical than what we had before.
2012-12-23 17:22:08 -05:00
3c67c209db
pull: Use a basic exponential moving average for bytes/sec
...
Still should look what other downloading programs do, but this
is better then what we had before.
2012-12-23 14:29:15 -05:00
0ab1f78ec8
admin: Add new run-triggers command
...
In some cases we want the ability to run triggers independently of
checking out a tree. For example, due to kernel limitations which
impact the gnome-ostree build system, we may need to run triggers on
first boot via systemd.
Secondarily, if the user installs a system extension which adds a new
shared library to /usr/lib for example, the system will need to run
the triggers again.
Also, I think I want to take triggers out of the core and put them in
ostree admin anyways.
2012-12-22 14:52:33 -05:00
3832544ac4
admin: Rework /ostree/deploy to support multiple independent operating systems
...
The real vision of OSTree is to "multiple versions of multiple
operating systems". Up until now, it's worked to install gnome-ostree
inside a host distribution, but several things don't work quite right
if you try to do completely different systems.
In the new model, there's the concept of an "osname" which encompasses
a few properties:
1) Its own /var
2) A set of trees deployed in /ostree/deploy/OSNAME/
3) Its own "current" and "previous" links.
Now it no longer really makes sense to boot with "ostree=current".
Instead, you specify e.g. "ostree=gnome/current".
This is an incompatible change to the deployment code - you will need
to run init-os gnome and redeploy.
All "ostree admin" subcommands now take an OSNAME argument.
2012-12-21 13:47:15 -05:00
16d312e82f
admin: Support installing a kernel from the tree, default to it
...
The "protocol" here is rather lame; we just look for
/boot/vmlinuz-RELEASE and /lib/modules/RELEASE. But good enough for
now.
2012-12-21 12:20:55 -05:00
19df2d372d
core: Use new libgsystem GSSubprocess API
...
It's less screwing around with GPtrArray in most cases.
2012-12-21 11:52:28 -05:00
90ccc7b2d2
core: Pick better names for tmpfiles if no prefix given
...
Just makes it easier to track down where files came from.
2012-12-21 11:30:58 -05:00
3d41d1d2db
OtWorkerQueue: Don't exit until we've fulfilled initial requests
...
This counts as an implicit "hold" on the first items, which we must
complete. Otherwise there are race conditions where the queue may
signal idle when in fact the main process is still working.
2012-12-21 09:36:38 -05:00
fddc11ca74
Update to latest libgsystem
...
Has a bugfix change to gs_subprocess_new().
2012-12-20 21:14:31 -05:00
ad956ff9e2
core: Allow creating tempfile without associated output stream
...
Just slightly more efficient, and we'll use this for spawning curl to
a tempfile.
2012-12-20 21:14:27 -05:00
a97211e3d0
core: Clean up tmpfile creation code
...
I have been seeing random failures here...not entirely sure why yet,
but this code is cleaner.
2012-12-13 15:12:51 -05:00
380b54d689
Update libgsystem
...
Just keeping up.
2012-12-13 15:12:33 -05:00
b637ea7674
admin init-fs: New builtin
...
Should be used when initializing a new root filesystem for a "pure
OSTree" system; for example, what "ostbuild privhelper-deploy-qemu"
does when creating a filesystem image loopback.
2012-12-09 17:01:51 -05:00
a2acaedfdc
admin: Process newly added config files recursively
...
Previously we just failed out if there was a new directory in /etc
added by the user (or the system).
2012-12-07 18:29:03 -05:00
227d22adca
Update to the latest libgsystem
2012-12-03 12:05:41 -05:00
a9a9baeb2a
Update to latest libgsystem
...
Has a rather important bugfix.
2012-12-02 23:02:39 -05:00
fc7f05940c
core: Use mmap() with O_NOATIME if merely reading archive-z2 entries
...
Previously we'd open(path, O_NOATIME) and do a series of small read()
calls to just parse the header. I think this will trigger kernel readahead
into the compressed portion, but we don't care about that.
This should be more efficient.
2012-12-02 15:20:17 -05:00
7fcebed4e8
cleanup: Rebase spawn API on top of libgsystem GSSubprocess
...
We didn't need most the parameters anyways, so this is fewer lines of
code. It also helps me test GSubprocess.
2012-12-01 16:53:06 -05:00
b423ea2041
Update to the latest libgsystem
2012-11-29 18:16:45 -05:00
6dda8da6c5
Switch to using libgsystem shutil
...
More code moved into libgsystem.
2012-11-29 17:11:24 -05:00
1ac307e6b8
Switch to libgsystem file reading API
...
More draining common utilities into libgsystem.
2012-11-29 16:58:39 -05:00
f3d765c903
Switch to libgsystem directory creation API
...
More code being dropped into a common place.
2012-11-29 16:40:17 -05:00
6d81120455
otutil: Move checksum related bits into checksum-utils.[ch]
...
This paves the way for slimming ot-gio-utils.[ch] into libgsystem.
2012-11-29 16:28:24 -05:00
5761733218
Drop some bits from libotutil that are now merged in libgsystem
2012-11-29 16:24:49 -05:00
d1ddfc973c
libotutil: Drop dead prototype
...
Was never implemented.
2012-11-29 15:56:17 -05:00
f2b1be6a1e
deploy: With --no-kernel, do copy host kernel modules
...
This is used by the qemu scripts, and there we *do* need the kernel
modules inside the target system.
2012-11-21 12:38:29 -05:00
05e7b6d596
pull-local: Make multithreaded
...
We were blocking for easily 1/10 or 1/5 of a second in fdatasync(),
which drastically slows down the whole process.
This threading isn't quite as good as the ostree-pull command, but it
lets us avoid the dependency on libsoup everywhere, and it's simpler.
2012-11-19 12:08:41 -05:00
de1ce843f1
core: Drop old GLIB_CHECK_VERSION(2.32) bits
...
We hard require 2.34 now; if it doesn't exist on the system, that's
what embedded dependencies are for.
2012-11-18 15:44:21 -05:00
3f4d223361
admin prune: New builtin for cleaning up deployments and repo
...
After a while of pull-deploy cycles, you start to accumulate a lot of
them. While the deployment read-only part is hardlinked, the -etc
space adds up.
Additionally, the repository itself just gets large.
The new command "ostree admin prune" deletes everything except the
"current" and "previous" deployments.
2012-11-16 17:41:46 -05:00
05895780a3
deploy: Write refs into repo for current/previous deployments
...
We don't want this data to be gc'd by default by a prune.
2012-11-16 17:14:23 -05:00
ffd774e2e6
prune: Report how much disk space we freed
...
Just nicer to see...
2012-11-16 15:45:05 -05:00
7861b98673
prune: Add option to traverse refs only
...
The previous code (unintentionally) only traversed from refs; so data
only reachable from previous commits would be deleted. That shouldn't
be the default, but we do want to offer it as an option.
So add a --refs-only option.
2012-11-16 12:50:13 -05:00
51b1dd7cbd
traverse: Skip already traversed commits
...
This is just more efficient, since e.g. "ostree prune" may invoke
ostree_traverse_commit() multiple times for the same commit.
2012-11-16 12:35:52 -05:00
4169d628c5
prune: Default to deleting loose objects, code cleanup
...
We're getting closer to matching 'git gc'.
2012-11-16 10:58:35 -05:00
ee533d6ab9
admin: code cleanup
...
Refactor internal functions so they can be used by a later patch.
2012-11-16 10:40:49 -05:00
cda5b28bab
ot-main: Ignore lt- prefixes in binary names
...
libtool is the best
https://bugzilla.gnome.org/show_bug.cgi?id=682438
2012-10-24 12:20:15 -04:00
cc4df4f3c2
admin: Add new pull-deploy command
...
Fetch the latest for the current tree, and deploy it in one go.
2012-10-22 20:41:49 -04:00
f405737164
libgsystem: Update
2012-10-22 18:24:31 -04:00
a0449ed246
Make ostree admin --help output more useful
...
At a minimum, it should list the available subcommands. This is
still not perfect, since there is no way to get at the help output
of the subcommands - getting that right needs more refactoring.
Signed-off-by: Colin Walters <walters@verbum.org>
2012-10-22 18:10:56 -04:00
5a51bee024
core: quiet a compiler warning
2012-10-17 20:54:52 -04:00
0c3e901dae
core: Check out files before directories
...
Otherwise we pretty easily hit maximum FD limits =(
2012-10-15 20:03:13 -04:00
bdbf76b6eb
core: Ensure correct file data is synced in non-overwrite case
2012-10-15 18:44:55 -04:00
1642310486
core: Use linkat() rather than link() in threaded checkouts
...
This seems to work around a likely Linux kernel VFS bug, where I
randomly see ENOENT on link() when we *definitely* called mkdir() at
an earlier point in time.
2012-10-15 10:49:48 -04:00
d6956bfc19
core: And one more race in multithreaded mkdir -p
...
Sigh =(
2012-10-15 10:49:48 -04:00
c0a6b5da71
core: Fix devino speedup for archive-z
2012-10-14 17:19:30 -04:00
0d04738801
core: Rework archive-z mode to have header be uncompressed
...
This is an incompatible change to archive-z, thus it is now renamed to
archive-z2 and ostree will no longer parse archive-z.
I noticed in perf that we were spending some time zlib-decompressing
file headers, which is just inefficient. Rather than do this, keep
the headers uncompressed, and just zlib-compress content.
2012-10-14 17:10:57 -04:00
2b90e987bf
core: Call fdatasync() before we rename()
...
Just noticed this from strace, doesn't really matter, but looks nicer.
2012-10-14 15:34:33 -04:00
b0b409f82e
checkout: Drop uncompressed object GC for now
...
It's racy when multiple processes are involved, and needs a bit more
thought.
2012-10-13 19:22:29 -04:00
5b512d4c78
core: Fix case where we hardlinked successfully but thought we didn't
...
We need to return hardlink_supported even if we had to do an unlink
first.
2012-10-13 18:41:47 -04:00
ad26c7a80b
core: A few more error-prefixings
2012-10-13 18:02:39 -04:00
c159188b81
core: One more minor tweak to recursive directory creation
...
We would fail in the case where we were trying to create a
subdirectory of /; this didn't actually happen, just noticed
via code inspection.
2012-10-13 17:54:19 -04:00
a284075731
core: Prefix some error messages
...
To help debug why I still have this race condition...
2012-10-13 17:46:24 -04:00
6b09f90122
core: Threadsafe recursive direction creation is harder than it appears
2012-10-13 10:47:03 -04:00
b4d408f483
core: Fix regression in previous commit
2012-10-12 15:25:25 -04:00
4436ed34d8
core: Make mkdir -p function safer
...
Recursing here is just a more obvious way to do it, rather than
relying on the semantics of g_file_make_directory_with_parents().
2012-10-12 15:10:39 -04:00
238da603b8
core: Ensure file data is synced to disk when checking out via non-hardlinks
...
Otherwise we aren't crash-safe.
2012-10-12 13:09:10 -04:00
e9dd22dc86
repo: Avoid race condition in threaded checkout with symbolic links
...
We were inconsistently putting symbolic links into the uncompressed
object cache, which could cause the threaded checkout to get confused.
2012-10-12 11:24:06 -04:00
ee73c0fa34
core: Add dynamic uncompressed object cache for archive-z
...
This gives us something closer to the advantages of archive and
archive-z when using the latter. Concretely we get deduplication
among multiple checkouts, along with the "devino" hash table trick
during commits to avoid checksumming content again.
This is enabled by default.
2012-10-11 18:33:03 -04:00
3c0fe01152
pull: Print elapsed time
2012-10-05 17:20:40 -04:00
52a0b7bdb1
core: Only do devino scan on commit (speeds up pull)
...
When fetching data remotely, there's no point to the devino scan
because we're not going to be committing local files.
Only do it for "commit".
2012-10-05 17:12:39 -04:00
Colin Walters
Giovanni Campagna
Jasper St. Pierre
Matthias Clasen