packaging/ostree
1
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,005 Commits 2 Branches 0 Tags
16d312e82f08f27bfa4d4f3dd4b0404ceab731de
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Colin Walters 16d312e82f admin: Support installing a kernel from the tree, default to it 
The "protocol" here is rather lame; we just look for
/boot/vmlinuz-RELEASE and /lib/modules/RELEASE.  But good enough for
now.
 2012-12-21 12:20:55 -05:00
Colin Walters 19df2d372d core: Use new libgsystem GSSubprocess API 
It's less screwing around with GPtrArray in most cases.
 2012-12-21 11:52:28 -05:00
Colin Walters b3d8c8c0e5 build: Set libsoup min/max defines 
So we don't get warnings about soup_requester being deprecated.
 2012-12-21 11:33:05 -05:00
Colin Walters 90ccc7b2d2 core: Pick better names for tmpfiles if no prefix given 
Just makes it easier to track down where files came from.
 2012-12-21 11:30:58 -05:00
Colin Walters 3d41d1d2db OtWorkerQueue: Don't exit until we've fulfilled initial requests 
This counts as an implicit "hold" on the first items, which we must
complete.  Otherwise there are race conditions where the queue may
signal idle when in fact the main process is still working.
 2012-12-21 09:36:38 -05:00
Colin Walters fddc11ca74 Update to latest libgsystem 
Has a bugfix change to gs_subprocess_new().
 2012-12-20 21:14:31 -05:00
Colin Walters ad956ff9e2 core: Allow creating tempfile without associated output stream 
Just slightly more efficient, and we'll use this for spawning curl to
a tempfile.
 2012-12-20 21:14:27 -05:00
Colin Walters a97211e3d0 core: Clean up tmpfile creation code 
I have been seeing random failures here...not entirely sure why yet,
but this code is cleaner.
 2012-12-13 15:12:51 -05:00
Colin Walters 380b54d689 Update libgsystem 
Just keeping up.
 2012-12-13 15:12:33 -05:00
Colin Walters b637ea7674 admin init-fs: New builtin 
Should be used when initializing a new root filesystem for a "pure
OSTree" system; for example, what "ostbuild privhelper-deploy-qemu"
does when creating a filesystem image loopback.
 2012-12-09 17:01:51 -05:00
Colin Walters a2acaedfdc admin: Process newly added config files recursively 
Previously we just failed out if there was a new directory in /etc
added by the user (or the system).
 2012-12-07 18:29:03 -05:00
Colin Walters 227d22adca Update to the latest libgsystem 2012-12-03 12:05:41 -05:00
Colin Walters a9a9baeb2a Update to latest libgsystem 
Has a rather important bugfix.
 2012-12-02 23:02:39 -05:00
Colin Walters fc7f05940c core: Use mmap() with O_NOATIME if merely reading archive-z2 entries 
Previously we'd open(path, O_NOATIME) and do a series of small read()
calls to just parse the header.  I think this will trigger kernel readahead
into the compressed portion, but we don't care about that.

This should be more efficient.
 2012-12-02 15:20:17 -05:00
Colin Walters 7fcebed4e8 cleanup: Rebase spawn API on top of libgsystem GSSubprocess 
We didn't need most the parameters anyways, so this is fewer lines of
code.  It also helps me test GSubprocess.
 2012-12-01 16:53:06 -05:00
Colin Walters b423ea2041 Update to the latest libgsystem 2012-11-29 18:16:45 -05:00
Colin Walters 6dda8da6c5 Switch to using libgsystem shutil 
More code moved into libgsystem.
 2012-11-29 17:11:24 -05:00
Colin Walters 1ac307e6b8 Switch to libgsystem file reading API 
More draining common utilities into libgsystem.
 2012-11-29 16:58:39 -05:00
Colin Walters f3d765c903 Switch to libgsystem directory creation API 
More code being dropped into a common place.
 2012-11-29 16:40:17 -05:00
Colin Walters 6d81120455 otutil: Move checksum related bits into checksum-utils.[ch] 
This paves the way for slimming ot-gio-utils.[ch] into libgsystem.
 2012-11-29 16:28:24 -05:00
Colin Walters 5761733218 Drop some bits from libotutil that are now merged in libgsystem 2012-11-29 16:24:49 -05:00
Colin Walters d1ddfc973c libotutil: Drop dead prototype 
Was never implemented.
 2012-11-29 15:56:17 -05:00
Colin Walters f2b1be6a1e deploy: With --no-kernel, do copy host kernel modules 
This is used by the qemu scripts, and there we *do* need the kernel
modules inside the target system.
 2012-11-21 12:38:29 -05:00
Colin Walters 05e7b6d596 pull-local: Make multithreaded 
We were blocking for easily 1/10 or 1/5 of a second in fdatasync(),
which drastically slows down the whole process.

This threading isn't quite as good as the ostree-pull command, but it
lets us avoid the dependency on libsoup everywhere, and it's simpler.
 2012-11-19 12:08:41 -05:00
Colin Walters de1ce843f1 core: Drop old GLIB_CHECK_VERSION(2.32) bits 
We hard require 2.34 now; if it doesn't exist on the system, that's
what embedded dependencies are for.
 2012-11-18 15:44:21 -05:00
Colin Walters 3f4d223361 admin prune: New builtin for cleaning up deployments and repo 
After a while of pull-deploy cycles, you start to accumulate a lot of
them.  While the deployment read-only part is hardlinked, the -etc
space adds up.

Additionally, the repository itself just gets large.

The new command "ostree admin prune" deletes everything except the
"current" and "previous" deployments.
 2012-11-16 17:41:46 -05:00
Colin Walters 05895780a3 deploy: Write refs into repo for current/previous deployments 
We don't want this data to be gc'd by default by a prune.
 2012-11-16 17:14:23 -05:00
Colin Walters ffd774e2e6 prune: Report how much disk space we freed 
Just nicer to see...
 2012-11-16 15:45:05 -05:00
Colin Walters 7861b98673 prune: Add option to traverse refs only 
The previous code (unintentionally) only traversed from refs; so data
only reachable from previous commits would be deleted.  That shouldn't
be the default, but we do want to offer it as an option.

So add a --refs-only option.
 2012-11-16 12:50:13 -05:00
Colin Walters 51b1dd7cbd traverse: Skip already traversed commits 
This is just more efficient, since e.g. "ostree prune" may invoke
ostree_traverse_commit() multiple times for the same commit.
 2012-11-16 12:35:52 -05:00
Colin Walters 4169d628c5 prune: Default to deleting loose objects, code cleanup 
We're getting closer to matching 'git gc'.
 2012-11-16 10:58:35 -05:00
Colin Walters ee533d6ab9 admin: code cleanup 
Refactor internal functions so they can be used by a later patch.
 2012-11-16 10:40:49 -05:00
Colin Walters 8f624ab51b ostree.spec.in: Update from Fedora review comments 
See https://bugzilla.redhat.com/show_bug.cgi?id=819951
 2012-11-16 10:11:39 -05:00
Colin Walters 2769ff36ff Release 2012.12 2012-10-31 22:02:35 -04:00
Colin Walters cc4df4f3c2 admin: Add new pull-deploy command 
Fetch the latest for the current tree, and deploy it in one go.
 2012-10-22 20:41:49 -04:00
Colin Walters f405737164 libgsystem: Update 2012-10-22 18:24:31 -04:00
Colin Walters 5a51bee024 core: quiet a compiler warning 2012-10-17 20:54:52 -04:00
Colin Walters 0c3e901dae core: Check out files before directories 
Otherwise we pretty easily hit maximum FD limits =(
 2012-10-15 20:03:13 -04:00
Colin Walters bdbf76b6eb core: Ensure correct file data is synced in non-overwrite case 2012-10-15 18:44:55 -04:00
Colin Walters 1642310486 core: Use linkat() rather than link() in threaded checkouts 
This seems to work around a likely Linux kernel VFS bug, where I
randomly see ENOENT on link() when we *definitely* called mkdir() at
an earlier point in time.
 2012-10-15 10:49:48 -04:00
Colin Walters d6956bfc19 core: And one more race in multithreaded mkdir -p 
Sigh =(
 2012-10-15 10:49:48 -04:00
Colin Walters c0a6b5da71 core: Fix devino speedup for archive-z 2012-10-14 17:19:30 -04:00
Colin Walters 0d04738801 core: Rework archive-z mode to have header be uncompressed 
This is an incompatible change to archive-z, thus it is now renamed to
archive-z2 and ostree will no longer parse archive-z.

I noticed in perf that we were spending some time zlib-decompressing
file headers, which is just inefficient.  Rather than do this, keep
the headers uncompressed, and just zlib-compress content.
 2012-10-14 17:10:57 -04:00
Colin Walters 2b90e987bf core: Call fdatasync() before we rename() 
Just noticed this from strace, doesn't really matter, but looks nicer.
 2012-10-14 15:34:33 -04:00
Colin Walters b0b409f82e checkout: Drop uncompressed object GC for now 
It's racy when multiple processes are involved, and needs a bit more
thought.
 2012-10-13 19:22:29 -04:00
Colin Walters 5b512d4c78 core: Fix case where we hardlinked successfully but thought we didn't 
We need to return hardlink_supported even if we had to do an unlink
first.
 2012-10-13 18:41:47 -04:00
Colin Walters ad26c7a80b core: A few more error-prefixings 2012-10-13 18:02:39 -04:00
Colin Walters c159188b81 core: One more minor tweak to recursive directory creation 
We would fail in the case where we were trying to create a
subdirectory of /; this didn't actually happen, just noticed
via code inspection.
 2012-10-13 17:54:19 -04:00
Colin Walters a284075731 core: Prefix some error messages 
To help debug why I still have this race condition...
 2012-10-13 17:46:24 -04:00
Colin Walters 6b09f90122 core: Threadsafe recursive direction creation is harder than it appears 2012-10-13 10:47:03 -04:00
Colin Walters b4d408f483 core: Fix regression in previous commit 2012-10-12 15:25:25 -04:00
Colin Walters 4436ed34d8 core: Make mkdir -p function safer 
Recursing here is just a more obvious way to do it, rather than
relying on the semantics of g_file_make_directory_with_parents().
 2012-10-12 15:10:39 -04:00
Colin Walters 238da603b8 core: Ensure file data is synced to disk when checking out via non-hardlinks 
Otherwise we aren't crash-safe.
 2012-10-12 13:09:10 -04:00
Colin Walters e9dd22dc86 repo: Avoid race condition in threaded checkout with symbolic links 
We were inconsistently putting symbolic links into the uncompressed
object cache, which could cause the threaded checkout to get confused.
 2012-10-12 11:24:06 -04:00
Colin Walters ee73c0fa34 core: Add dynamic uncompressed object cache for archive-z 
This gives us something closer to the advantages of archive and
archive-z when using the latter.  Concretely we get deduplication
among multiple checkouts, along with the "devino" hash table trick
during commits to avoid checksumming content again.

This is enabled by default.
 2012-10-11 18:33:03 -04:00
Colin Walters 3c0fe01152 pull: Print elapsed time 2012-10-05 17:20:40 -04:00
Colin Walters 52a0b7bdb1 core: Only do devino scan on commit (speeds up pull) 
When fetching data remotely, there's no point to the devino scan
because we're not going to be committing local files.

Only do it for "commit".
 2012-10-05 17:12:39 -04:00
Colin Walters efe27f1b05 pull: Ensure worker queue finishes if we unhold an empty queue too 2012-10-04 20:32:14 -04:00
Colin Walters 9618232f4d pull: Stage content asynchronously 
For similar reasons as metadata, this avoids having the main thread
blocked in fdatasync(), and even better - we can achieve much higher
parallelism if we have multiple threads blocked on fdatasync().
 2012-10-04 20:00:00 -04:00
Colin Walters 5b8e833351 pull: Stage metadata objects asynchronously 
This avoids the main thread being blocked on fdatasync(); also as a
bonus we checksum metadata in a separate thread too.
 2012-10-04 18:24:37 -04:00
Colin Walters f1b4db15a2 pull: Ensure queued filemeta requests don't starve everything else 
We need this hack for "archive mode" repositories; otherwise,
what ends up happening is that we get 10000+ requests pending
for .filemeta files, which we can't process until we also get
the .filecontent.

Note this hack is unneccessary when fetching from archive-z
repositories.
 2012-10-04 18:23:18 -04:00
Colin Walters 4e51701bea pull: Merge metadata and content fetch phases 
This is a notable speedup when the metadata scanner is working, and we
don't have a lot of traffic; we can pull down data at the same time.
 2012-10-04 15:37:23 -04:00
Colin Walters 5c1dc390ae pull: Asynchronous metadata fetch 
Create a worker thread for processing metadata, reserving the main
thread for HTTP requests.

This can create a very significant efficiency win for large pull
requests since we are much more likely to keep a full pipeline open.

The status display is also nicer now.
 2012-10-03 22:35:33 -04:00
Colin Walters 5f25762122 admin: Install grub2 config file if we detect /etc/grub.d 
Decouple this from the kernel postinst one, since it's possible to
have one but not the other.
 2012-10-01 19:34:25 -04:00
Colin Walters 8153799121 admin: Prepare /var/log/dracut.log when creating initramfs 
Otherwise dracut complains.
 2012-10-01 10:25:36 -04:00
Colin Walters 758d6132e3 Release 2012.11 2012-09-25 22:25:06 -04:00
Colin Walters 3e9c358628 Update libgsystem 
This ensures we're disting the README, which makes the autogen.sh do
the right thing.
 2012-09-25 22:25:06 -04:00
Colin Walters 044881b60f build: Add one header file missing from dist 2012-09-25 19:13:05 -04:00
Colin Walters 8922871733 build: Add autogen.sh, the COPYING.{GPL,LGPL} files to EXTRA_DIST 
We want them in tarballs too.
 2012-09-25 18:44:08 -04:00
Colin Walters 503535da48 Release 2012.10 2012-09-25 18:08:18 -04:00
Colin Walters 7559759a91 packages: Two spec file fixes 2012-09-25 18:08:18 -04:00
Colin Walters 0aab024602 autogen: Only do submodule update if we don't have libgsystem 
Otherwise we'll fail due to not being a git repository in the
embedded-dependency tarball case.
 2012-09-24 22:20:36 -04:00
Colin Walters a864e2cc19 autogen.sh: Use set -e 
Otherwise we blindly continue if autoreconf fails, which is pretty
damn lame.
 2012-09-24 08:30:54 -04:00
Colin Walters 26e487bacf core: use open(..., O_NOATIME) for reading loose objects 
More efficient, and atime shouldn't matter for these.
 2012-09-24 07:36:42 -04:00
Colin Walters 7aa620109e Add tests missing from previous commit 2012-09-23 19:25:09 -04:00
Colin Walters 40ce43036f Add an archive-z repository mode 
This is where loose content objects are stored as one compressed file,
instead of the two separate ones for regular archive mode.  This mode
would be suitable for HTTP servers, beause only one HTTP request is
necessary, and the result would be compressed.
 2012-09-23 19:23:45 -04:00
Colin Walters 35a1ff51d0 pull: Fix two minor memory leaks 2012-09-23 17:32:11 -04:00
Colin Walters 34c49f0411 core: Clean up staging API and internals 
Cleanly separate metadata/content APIs, rather than defaulting to
raw streams.  This helps most use cases.

Also, drop support for staging content without knowing the total
length.  This complicated the code, and for things like streaming
HTTP, we should be able to figure this out from Content-Length.
 2012-09-23 16:44:12 -04:00
Colin Walters 2a0601efc7 core: Drop packfiles as they are now 
They're not a large efficiency win at the moment, because we don't
do any delta compression.

At the moment, they simply served to compress data, but we will change
the archive mode to do that by default.
 2012-09-23 16:02:03 -04:00
Colin Walters aa9aaa9a62 tests: Drop tests of removed --atomic-retarget option 2012-09-23 15:56:28 -04:00
Colin Walters a8764bf996 diff: Fix case when used on native files 2012-09-23 15:55:02 -04:00
Colin Walters 08be599d50 packaging: Update spec file per Fedora review 
See https://bugzilla.redhat.com/show_bug.cgi?id=819951
 2012-09-20 20:12:01 -04:00
Colin Walters 1a274aecd6 packages: Update ostree.spec for file changes 2012-09-20 19:52:13 -04:00
Colin Walters c746f3b37c packages: Tweak srpm command 
This spec should be usable for both embedded dependencies and not.
 2012-09-20 18:00:06 -04:00
Colin Walters a6e9e54891 admin: Initialize /var more fully 
This helps out gnome-ostree, because gdm needs /var/lib for example.
 2012-09-18 08:58:07 -04:00
Colin Walters 59ddbcc500 pull: Check the remote repo type 
We can only pull from archive repositories right now.  This will also
be useful for adding compressed archives later.
 2012-09-15 12:41:25 -04:00
Colin Walters 31153913ff Extract keyfile helpers into libotutil 
Will be used by ostree-pull too soon.
 2012-09-15 11:20:56 -04:00
Colin Walters 117b9c109e ostadmin: Fix up deploy config logic again 
Clearly we need tests here...
 2012-09-09 11:42:17 -04:00
Colin Walters 2f8b3517f4 admin: Ensure we pass options to subcommands 
Otherwise ostree admin --ostree-dir=/foo deploy --no-kernel bar
doesn't work.
 2012-09-09 11:31:22 -04:00
Colin Walters 7831ae81c2 update-kernel: Fix free -> g_object_unref 2012-09-09 10:42:13 -04:00
Colin Walters fa5485c6c4 Move "ostadmin" => "ostree admin" 
This helps us avoid polluting the global binary namespace.
 2012-09-08 19:34:10 -04:00
Colin Walters 2ddbeb1f71 Make /etc a writable mount 
First, move deployments to /ostree/deploy.  Having them in the
toplevel clutters the filesystem layout too much.

When we deploy a tree like /ostree/deploy/NAME, there is now also a
writable directory /ostree/deploy/NAME-etc.  This is mounted as
read-write inside the system.

On an initial install, that directory is copied from
/ostree/deploy/NAME/etc.  On subsequent deployments, we find any
changes made in the current deployment's /etc, and apply that set of
changes to the new deployment's /etc.

See https://live.gnome.org/OSTree/EverythingInEtcIsABug
 2012-09-06 08:17:23 -04:00
Colin Walters 9ba968a5f4 otutil: Add a few more helper functions 
To be used by wip/etc-writable branch.
 2012-09-03 10:43:42 -07:00
Colin Walters cf31942a69 core: Refactor diff to be an internal API 
For future use by other code.
 2012-08-31 17:39:04 -07:00
Colin Walters 38d62d79a2 ostadmin: Also ensure /ostree/modules exists 
We depend on host kernel for now.
 2012-08-30 17:42:36 -07:00
Colin Walters c690416b72 build: Add dist-snapshot 
Take the current git HEAD and make a tarball from it; should be useful
for putting snapshots into Poky.
 2012-08-28 10:14:11 -04:00
Colin Walters 60368fae76 Use correct libgsystem 
Accidentally pushed a local patch.
 2012-08-28 09:58:07 -04:00
Colin Walters 2527f81abc Merge remote-tracking branch 'aperezdc/fixes' 2012-08-28 09:45:39 -04:00
Colin Walters a17a8e1f4e Update libgsystem 2012-08-27 18:48:14 -04:00
Colin Walters 4d2f770748 core: Use O_NOATIME to open metadata 
We really don't need atime for metadata, it's just a speed hit.
 2012-08-27 16:07:39 -04:00