bae584c64a
core: Associate branches with remotes, move trigger runs into checkout
...
Also add --atomic-retarget option to checkout. This does the magical
symlink dance to do atomic swaps between trees.
2012-04-03 23:46:34 -04:00
3f23ac156a
core: Clean up stale packfiles
2012-04-03 23:46:34 -04:00
8792007bc1
core: Validate structure of objects in fsck
2012-04-03 08:52:58 -04:00
9c7a47434d
core: Drop "staged" state for objects, refactor pull to split metadata/data
...
Previously we had the "staged" state to ensure we didn't add a commit
object without the associated dirtree, etc. However it's
easier/better to just ensure in the pull command that we have all
referenced objects.
Also change pull to download metadata first. This will allow adding
a progress bar later.
2012-04-02 19:52:22 -04:00
d8173a5125
core: Move commit/dirtree traversal into library
...
So it can more easily be reused by other builtins.
2012-04-02 18:32:40 -04:00
a0b7d94cb6
core: Fix a few memory leaks
2012-04-02 18:32:40 -04:00
ea4d2be39c
core: Return input stream for raw repositories too
...
This bug is more obvious when fsck is rebased on top of
ostree_repo_load_file().
2012-04-02 18:32:40 -04:00
b971ce0dfa
core: Only depend on GLib 2.30
2012-04-02 13:42:09 -04:00
80bdfd7f42
core: Add pack files
...
This concept is also directly inspired by git. At present, our
implementation is quite similar, except we don't have delta
compression.
2012-03-31 11:51:47 -04:00
70afd6011f
core: Add helper functions to convert (checksum, objtype) pair to/from string
2012-03-14 09:37:00 -04:00
db93c43255
core: Fix "cat" crashing when it should have returned ENOENT
2012-03-08 12:10:58 -05:00
76bc35186e
core: Add --union mode to checkout
...
This is another step towards ostbuild using this instead of the
"compose" builtin.
2012-03-06 11:59:06 -05:00
83fb6d56e1
core: Don't call unlink() on directory if it already exists
...
If we fail to make a directory because it exists, we shouldn't call
unlink() on it.
2012-03-06 11:59:06 -05:00
1513f29495
core: Pull diff functionality out into "diff" builtin
...
There's no good reason for this to be in core when it's only in use by
the diff builtin.
2012-03-06 11:59:06 -05:00
1f7d776a18
core: Improve checkout API
...
Expose the lower-level functionality in libostree, change checkout
builtin to be a higher level driver. This will allow us to more
easily improve the "checkout" builtin..
2012-03-06 11:59:06 -05:00
f44b8aca44
core: Add ostree-types.h
...
This allows us to have circular references between the headers.
2012-03-06 11:59:06 -05:00
eb7c3d01a3
core: Add 'cat' builtin
2012-03-06 11:59:06 -05:00
b783ebc189
core: Add API to validate rev
2012-02-27 08:37:37 -05:00
810a3dac86
core: Write a refs/summary file for archive repositories
...
This will allow us to retrieve all refs over HTTP easily.
2012-02-27 06:20:42 -05:00
d3a73225b0
core: Add API to enumerate all refs
2012-02-27 05:58:42 -05:00
5329f1706b
core: Further fix fallout from archive mode files
...
Continuing from 16c0cfe9b5
2012-02-24 17:50:44 -05:00
51916a7002
core: Look up refs like origin/foo correctly again
...
We want to support both "bare" lookups where "foo" can be local, or in
any remote, as well as prefixed ones for a specific remote.
This fixes ostree-pull noticing that nothing has changed.
2012-02-24 16:08:34 -05:00
16c0cfe9b5
core: Fix regression when committing symlinks in archive mode
...
Commit 802959f398
2012-02-24 10:05:26 -05:00
14cda593d5
core: Correctly look up refs in remotes
2012-02-23 23:52:59 -05:00
b2ad353b42
core: Store "pending" objects explicitly, fix pull to use them
...
Before we were creating randomly-named temporary files in repo/tmp
when downloading via pull, but that means if the download process is
interrupted, we have to redownload everything again.
Let's still keep the concept of a "transaction" where files are
stored in the repository as atomically as possible (i.e. we
do a bunch of rename() calls), but now we also have an explicit
"tmp/pending/objects" directory that contains named objects.
This allows us to then skip redownloading things that are pending.
2012-02-23 18:19:00 -05:00
802959f398
core: Ensure .archive-content files are mode 0644
...
We don't want to inherit the permissions of the actual file, otherwise
trying to fetch e.g. /etc/passwd when being served via Apache httpd
will break.
2012-02-23 12:34:43 -05:00
e8dbaa3c07
core: Add generic "commit filter" API, use it to implement statoverride
...
The builder wants the ability to mark a given file as e.g. setuid. To
implement this, the repo now has a callback-based API when importing a
directory to modify or remove items.
The commit tool accepts a "statoverride" file as input which looks like:
+mode /path/to/file
2012-01-22 13:27:38 -05:00
39fe050f04
core: Add standard::size to info we get from archives
...
This makes "ostree ls" show the file size.
2012-01-19 09:54:09 -05:00
ebe35dbefe
core: Ensure we delete duplicate files in a transaction
...
If multiple files have the same hash, we need to ensure we're not
overwriting other tempfiles in the same transaction. Instead
just delete them, since we know they're in the repo.
2012-01-15 12:25:52 -05:00
9910c0ddf4
core: Create a directory for each transaction
...
This helps ensure that we can more easily identify stale tempfiles
left over if we crash in the middle of a transaction.
2012-01-15 11:03:01 -05:00
40226c2769
core: Move triggers into separate binary: ostree-run-triggers
...
I'm trying to keep ostree as being closer to just being the versioning
filesystem, so let's split out the triggers into a different binary
(although still namespaced ostree-).
2012-01-13 15:08:27 -05:00
62821024b9
core: While walking mtrees during commit, ensure the referenced path is resolved
...
This gives us a clean error instead of an assertion.
2012-01-11 13:42:59 -05:00
5119cbdb7c
core: Add --no-xattrs option for commit
...
Will be used by ostbuild.
2012-01-05 19:10:26 -05:00
3e59cc1305
core: Add --tar-autocreate-parents option for commit
...
The tar files we're making of artifacts don't include parent
directories. Now we could change the builder to make them, but we can
also just autocreate them on import. Mode 0755 with no xattrs seems
OK here.
2011-12-23 06:49:04 -05:00
4bbc99fb1d
core: Fix build with libarchive disabled
2011-12-22 19:52:30 -05:00
dab4611263
core: Add --skip-if-unchanged option for commit
...
There's not much point for OS builds to have "empty" commits.
2011-12-22 18:47:30 -05:00
3292dcb6d6
core: Allow / in ref names like git does
...
Using / is nice basically.
2011-12-22 15:59:47 -05:00
9bcd4ff8aa
core: Use global random object
...
We were pointlessly creating a new rand (and thus opening /dev/random)
on every temp file creation, but still using the global rand.
2011-12-22 11:23:16 -05:00
d25f1bf73d
core: Use GFile for repo constructor API, and a bit more internally
...
Also, ensure that the repo directory GFile is absolute - this avoids
a getcwd() syscall every time we construct a GFile object.
2011-12-22 11:04:08 -05:00
e9fd921afe
core: Change compose to operate purely in-memory
...
This is *significantly* faster than checking out each branch into the
real filesystem, then importing it again.
2011-12-21 21:42:13 -05:00
add55849ab
core: Allow specifying a previous commit tree as a basis for a new commit
...
It's pretty trivial to map a previously existing commit tree into a
mutable tree too. While we're here change the command line arguments
for commit so that we can now properly overlay any combination of
directory, commit, or tarfile.
2011-12-21 20:10:10 -05:00
c8377c7c5c
core: Expose more details of commit via API
...
Rather than offering high level "commit directory", instead perform
operations on a mtree. Commits are treated more like regular objects.
Change the commit builtin to drive this all at a lower level.
2011-12-21 18:58:44 -05:00
59281d5901
core: Add omitted mutable tree files
2011-12-21 18:58:27 -05:00
72257bab2b
core: Rename "store" public APIs to "stage" since that's what they do
2011-12-21 17:13:04 -05:00
06548f137e
core: Make OstreeRepoFile "public"
...
* "ls" used it
* It's not like our internal API is 100% stable anyways
* The _ is ugly
2011-12-21 13:41:38 -05:00
2b7a83d1cc
core: Add new "mutable tree" class, use it for tar file import
...
The tar import code forced the resuscitation of a hackish "FileTree"
data type for representing an in-memory tree. Split this out
into an OstreeMutableTree class for future use by any other in-memory
tree construction.
2011-12-21 13:34:10 -05:00
14d682305b
core: Remove dead code from OstreeRepoFile
...
Originally I thought it would make sense for OstreeRepoFile to be
mutable, and it probably does, but we should do it in a different way.
2011-12-21 11:01:41 -05:00
bdfde03b61
core: Support committing multiple tarballs in the same transaction
...
ostbuild will generate two artifacts: foo-runtime.tar.gz and
foo-devel.tar.gz in the general case. When committing to the devel
tree, it'd be lame (i.e. slower and not atomic) to have to commit
twice.
2011-12-20 17:21:15 -05:00
11d57d63e3
core: Support --owner-uid and --owner-gid options for commit
...
This allows us to more easily import user-built tarballs into
a root-owned OSTree repo.
2011-12-18 17:36:46 -05:00
134283afbf
core: In user mode checkouts of archives, use hard links
...
Now that we've done all the gyrations to separate content from
metadata in archives, we can just hard link when doing user checkouts.
2011-12-15 16:12:31 -05:00
e014b2d9c2
core: Fix a few memory leaks
2011-12-15 15:17:27 -05:00
9a71ab187d
core: INCOMPATIBLE CHANGE: Split archive files in two parts (meta and content)
...
This will allow us to have hardlink checkouts of archives. A key use
case here is an archive repo of an OS (with root-owned files etc.)
where we want to do builds in a user tree.
A positive side effect of doing things this way is that now the SHA256
checksums for a given file should be identical regardless of whether
it's stored in an archive or bare repository.
2011-12-15 13:14:06 -05:00
ecbffd4915
core: Don't insert null byte in temporary file names
2011-12-15 13:10:41 -05:00
8ab2296a70
core: Add utility functions to convert a OstreeObjectType to/from a string
2011-12-15 13:09:51 -05:00
b8e8b58585
core: INCOMPATIBLE CHANGE: Name repo files with their type (e.g. .dirmeta)
...
This makes inspection easier. Internally the code gets simpler because
metadata and files are more unified; there is just one object type.
2011-12-12 12:29:09 -05:00
dc4164993b
core: Initialize variables in fsck
...
Otherwise we segfault on an unhandled filename.
2011-12-08 18:23:25 -05:00
f6117de8d3
core: s/pack/archived/
...
This completes the rename from the previous commit.
2011-12-08 17:45:37 -05:00
d6ceb91c35
core: Make ot_transfer_out_value() take a & for the second argument
...
As Ray Strode argued, it's confusing to have something that looks like
a function be magical. And OT_TRANSFER_OUT_VALUE is uglier.
2011-12-08 17:31:45 -05:00
35fa68a74c
core: INCOMPATIBLE CHANGE: Rename .packfile to .archive
...
It's too confusing that we call the mode "archive" but the actual
files ".packfile". Also, git already has a "packfile" that serves a
totally different purpose.
2011-12-08 17:15:59 -05:00
30c53a967c
core: Switch is_archive to an enumeration
...
This is in preparation for adding a third mode.
2011-12-08 17:05:16 -05:00
7932811fbb
core: Delete some dead code
2011-12-08 14:55:41 -05:00
dccd106731
core: Clean up checkout code
...
Take a GFile * for destination. Also, we only need one recursive
function, not two.
2011-12-08 10:28:44 -05:00
d70950e891
core: Add -U option for checkout to skip lchown() and xattrs
...
This is useful for checking out an OS filesystem tree as a regular
user, and not caring about /dev and stuff.
2011-12-06 21:38:04 -05:00
a94e0b8bf8
core: Ensure we chown created directories too
2011-12-06 20:16:03 -05:00
c32141c2bf
core: Remove some useless wrapping of packfile unpacking in checkout
2011-12-06 20:09:09 -05:00
ce7345d51e
core: Major cleanup to internal import API
2011-12-05 14:21:18 -05:00
b8000c00d5
core: Further unify code for importing from local FS and libarchive
...
Note this change makes it so we no longer call link() from an import
filesystem tree to the repository. This is a Good Thing really; it
makes local FS commits slower, but also less prone to corruption.
2011-12-05 11:20:14 -05:00
267bf8ef4a
core: Ensure we initialize checksum in FIFO case too
2011-12-05 11:15:12 -05:00
9d39517554
core: Add _from_input variant of checksum API
2011-12-05 10:28:42 -05:00
78f435d245
core: Fix handling of hardlinks for tar commits
...
We were creating files with the wrong name. Add a test.
2011-12-04 14:38:30 -05:00
17cc772cf3
core: Support for committing from any libarchive-supported format
...
We really want the ability to take a .tar.gz and directly import
it into a repository, without creating a temporary filesystem tree.
First, doing it this way is significantly faster. Also, this allows
us to handle importing tar files with e.g. uid 0 files into packed
repositories as non-root, which is very useful for tests and builds.
2011-12-04 14:08:55 -05:00
f9315e8f82
core: Fix a memory leak in checkout
2011-12-04 14:08:55 -05:00
45addeadf0
core: Handle NULL xattrs in packfile generation, throw error on invalid mode
2011-12-04 13:12:06 -05:00
ae3a5e3c07
core: Clean up filename utility API
...
Remove more unused functions, and change pathname splitting to handle
more cases like duplicate //, and to throw an error on .. as a filename.
2011-12-04 12:12:34 -05:00
1f6d49cdb9
core: Initialize checksums for non-regular files, handle NULL xattrs
2011-12-04 12:11:29 -05:00
0003e101f6
core: New ot_transfer_out_value() macro
...
This makes out values considerably less typing.
2011-12-02 18:17:06 -05:00
3ab69101f7
core: Fix packed file creation
...
We were calculating the wrong checksum, and also mistakenly
not using packed files in archive mode for non-regular files.
2011-12-02 17:49:57 -05:00
b5ce760678
core: Don't use g_file_delete(), it calls into g_vfs_get_default()
...
For some (probably crazy) reason. Skipping this and calling
unlink() directly makes 'strace' MUCH smaller.
2011-12-02 14:32:25 -05:00
0d4df5b3fe
core: Avoid reading regular file data twice when archiving
2011-12-02 14:23:01 -05:00
d1950da1a0
core: Further unify API to create files
...
We now have just one place which writes to the filesystem. Wrap a
temporary file allocation API on top of that.
2011-12-02 13:42:25 -05:00
7fc625c967
core: Move pack file parsing into core, add ostree_create_file_from_input() API
...
This moves us closer to consistently passing around a triple of:
(GFileInfo *info, GVariant *xattrs, GInputStream *content)
Which will help the libarchive work.
2011-12-02 10:11:09 -05:00
f98e2a2ddb
core: Tweak dir metadata importing code for libarchive work
...
This will allow us to share a bit more code.
2011-12-01 19:45:00 -05:00
177d845f5b
core: Modify pack file importer to also optionally checksum
...
For future work on importing from an archive.
2011-12-01 19:25:35 -05:00
23d01d97dd
core: Propagate GCancellable a bit from the import APIs
2011-11-30 22:20:25 -05:00
702c38739e
core: Drop some dead code from packfile writing, expose GInputStream API too
...
We never actually dropped into the bits to write metadata as packfiles,
because such a thing doesn't exist.
Also add a GInputStream-based API for writing packfiles.
2011-11-30 22:15:05 -05:00
556662b24c
core: Extract stage_and_checksum() internal API
...
This will be used for staging both metadata and data consistently
before actually importing it.
2011-11-30 21:20:15 -05:00
99bf19314e
core: Add ot_gfile_create_tmp() to utils, use it
2011-11-30 20:32:06 -05:00
247e027125
repo: Don't call g_hash_table_destroy on a NULL hash
2011-11-30 20:23:54 -05:00
84562f161f
core: Use ot_clear_checksum()
2011-11-29 14:06:54 -05:00
7e32bc6cd7
core: Make commit always operate on directory contents, not file list
...
This simplifies things significantly, at some cost in flexibility.
We'll later add the ability to e.g. filter out files by regular
expression.
2011-11-28 12:57:25 -05:00
b296dc4efc
core: Some tweaks to ls output
2011-11-28 10:17:49 -05:00
2f1490f766
core: Fix a compiler warning
2011-11-27 20:15:07 -05:00
87547827c8
core: Add ls builtin
2011-11-27 20:10:48 -05:00
b0b0ffcd61
core: Add asynchronous checksum API, use it in checksum builtin
2011-11-27 17:21:04 -05:00
3c025e526f
core: Ad ot_clear_gvariant(), use it
...
This saves us a conditional.
2011-11-22 10:20:10 -05:00
aa69aae94e
core: Move merge dirs code into gio utils
2011-11-21 15:04:07 -05:00
b7a45b5815
core: Also chown() after unpacking objects
2011-11-18 18:50:29 -05:00
7d63ad5ae8
core: Switch to GFile* for unpacking and storing objects
2011-11-18 16:14:01 -05:00
c66474750b
core: Expose API to get repository tmpdir
2011-11-18 16:01:28 -05:00
f6ec479f17
core: Rework object iteration API to be GFile based and pass more data
...
It makes more sense to have e.g. the details of .packfile naming
inside the repo, and pass the expected checksum and type.
2011-11-18 15:21:32 -05:00
8621eb85b1
core: Skip an unnecessary unlink() call
2011-11-18 09:41:36 -05:00
7276e2df43
core: Have single "overwrite" flag instead of ignore_exists+force pair
...
The default is always ignore_exists. Also port the internals here
to use more GIO code, and stop using *at syscall variants since they're
only useful if used 100%.
2011-11-18 09:37:25 -05:00
860c3cf502
core: Convert public get_object_path() to GFile*
2011-11-18 09:04:25 -05:00
9cf15ed151
core: More cleanup of GIO utility API
2011-11-18 08:43:47 -05:00
231f7ed76c
core: Use ot_gfile_ prefix for new local GFile creation function
2011-11-18 08:10:23 -05:00
7f64d5cec7
core: Make checksum API also operate on directories
2011-11-18 06:34:54 -05:00
b8cef545d1
core: CHANGE CHECKSUM ALGORITHM, port checksum API to GFile
...
This commit originally was to port ostree_stat_and_checksum_file() to
GFile*, but I noticed that the checksum code was reading data in host
endianness. Fix that while we're here.
This invalidates all existing repositories.
2011-11-17 19:32:01 -05:00
2c2b9c0e00
core: Convert metadata parsing API to GFile
2011-11-17 18:36:57 -05:00
218684786d
core: Use GFile for xattr API
2011-11-17 18:33:26 -05:00
a8735778f0
core: Use ot_gfile_get_path_cached() pervasively
...
This requires no free() calls and is just better.
2011-11-17 18:23:36 -05:00
8e9b9c59d6
core: Fix pull to actually download files too
2011-11-17 10:40:55 -05:00
ed2be6306d
core: Should write remote refs in refs/<remote>/<branch>
...
Not just refs/<branch>, otherwise multiple remotes conflict.
2011-11-17 10:40:01 -05:00
db9801bff2
core: Allow resolving e.g. "origin/master" as a rev
...
Also:
* Disallow .. in revs to avoid looking outside where we should be
* Allow resolving empty revs
2011-11-17 10:36:04 -05:00
aa865bbb83
core: Validate file names read from directory variants
...
In a future where we pull data from remote servers, we don't want
to allow path uplinks.
2011-11-16 23:23:30 -05:00
4a26be1bef
core: diff: Correctly handle files changing type
2011-11-16 23:09:32 -05:00
c33db03b4c
core: Flesh out diff a bit more
...
Now correctly notices changes to directory metadata (not just
contents).
2011-11-16 22:51:09 -05:00
115626c264
core: Fix a minor memory leak
2011-11-16 22:51:09 -05:00
9fb390664a
core: Implement diff command
2011-11-16 18:16:41 -05:00
d196a47043
core: We actually want rdev, not dev for devices
2011-11-15 15:15:46 -05:00
18f0b537a4
build: Move sources into src/ again
...
This is necessary if we want to build when srcdir == builddir,
otherwise we blow up because "ostree" is a source directory and a
binary.
2011-11-14 15:39:38 -05:00
21c7ff74b6
De-recursify source tree
2011-11-02 15:45:32 -04:00
2b00cf3810
core: add ^ for rev-parse
2011-11-02 13:22:13 -04:00
bab31db7ee
core: Squash some memory leaks
2011-11-01 12:20:33 -04:00
a897642315
For archive checkouts, actually unpack the packfiles
2011-11-01 10:59:07 -04:00
8769f4afd8
Switch to big-endian storage for all variant data
...
This ensures the saved archives are architecture-independent.
2011-10-31 22:41:50 -04:00
a160a2a5fa
Add support for pulling from remote archives
...
This necessitated a large set of changes.
We now support an "archive" mode for repositories. In this mode,
files are stored "packed" rather than hard linked. This allows one to
e.g. store an OSTree repository with root-owned files as non-root. It
is also used as the basis for serving repositories via HTTP.
While doing this I realized that GVariant is endianness-dependent; I
decided to just store all data in big endian.
2011-10-31 20:28:47 -04:00
80fcb3b818
checkout: Use ot_util_new_file_for_path
2011-10-31 20:22:34 -04:00
ee4edf114b
Don't default to "master", require a branch
...
It doesn't really make sense to have a default branch, since we expect
people to have multiple roots. Thus, require a branch
specification always.
2011-10-27 21:42:44 -04:00
7ca1c3d2e7
core: remove HEAD file, use branches instead
...
HEAD in git describes a working copy, and we don't have those.
Instead, default to a "master" branch. This also lets us support
multiple branches.
2011-10-27 14:32:05 -04:00
ac99188c23
core: Add a config file with repo version
...
This should let us expand later.
2011-10-26 22:21:00 -04:00
d0b3a07cf1
core: Fix checksum for symlinks walking off into uninitialized memory
2011-10-26 18:33:33 -04:00
03943c659f
Drop glib requrirement to 2.28
...
This lets it build on more currently-released systems like Fedora 15.
2011-10-26 14:12:13 -04:00
4d86ee0a1d
Pile of work on parallel-debian
2011-10-24 22:02:16 -04:00
4d39070efd
Fix up triggers and gnomeos-make-image.sh
...
Now both do something.
2011-10-20 17:24:21 -04:00
a2880a6ddd
src: Squash some unused variables
2011-10-20 16:30:34 -04:00
9d8522dfa1
Some initial work on triggers
2011-10-20 16:12:02 -04:00
b7d45436a0
Don't use g_variant_new_from_fixed_array(), it was only added in 2.31
...
g_variant_new_from_data() works, and actually allows us to avoid
a memdup()/free().
2011-10-20 10:20:51 -04:00
aa25334286
repo: Only open regular files
...
Otherwise we'll try to open device files with predictably bad
consequences.
2011-10-19 18:06:06 -04:00
961b1c80db
Do "bare" repositories by default
...
Mixing the repository and checkouts is discouraged.
2011-10-19 17:45:00 -04:00
b27df6fd72
"Hacktree" is now known as "OSTree"
...
It just sounds better.
2011-10-18 14:44:48 -04:00
Colin Walters