c77908bf51
libotutil: Make use of GBytes in ot_variant_read()
...
This is just cleaner; we avoid using GObject data, etc.
https://bugzilla.gnome.org/show_bug.cgi?id=706031
2013-08-14 19:55:34 -04:00
76cd7ae4ea
libotutil: Add API to create an "ay" GVariant from GBytes
...
We used to have a version of this, but since I'm trying to use
GBytes more, this became a more common operation, and it's annoying
to type out the whole G_VARIANT_TYPE ("ay") each time, and pass
TRUE for trusted.
https://bugzilla.gnome.org/show_bug.cgi?id=706031
2013-08-14 19:23:33 -04:00
462bcd86de
Add --verbose and -v arguments to ostree for details
...
Use the GLib debug mechanism to handle verbose output.
https://bugzilla.gnome.org/show_bug.cgi?id=705905
2013-08-13 15:37:12 +02:00
a5d43bb959
Install a shared library
...
This required a fair bit of surgery because previously ostree.h
included otutil.h, but that's supposed to be a private library.
2013-07-26 19:25:07 -04:00
3b9da094d8
main: Drop log builtin
...
We may revive this later, but commits in their current form aren't
very useful for humans to read, so it doesn't make sense to have a
tool to show a history of useless stuff.
More interesting things are diffs between commits, object statistics,
etc.
2013-07-23 18:19:14 -04:00
124416d832
Use AC_USE_SYSTEM_EXTENSIONS instead of #define _GNU_SOURCE
...
It's less copy/paste, works everywhere, etc.
Also fix some missing #include "config.h".
2013-07-10 13:25:35 -04:00
5ad0671cd2
Switch to #pragma once for headers
...
It's just less tedious, and we're GCC/LLVM specific anyways.
2013-07-09 18:53:22 -04:00
4b170d656c
Switch to libgsystem local allocation macros
...
And drop our compatibility wrapper.
2013-07-07 12:27:44 -04:00
e3dc0c91df
gioutils: Many new utility functions
...
Reading symbolic links becomes a much more common thing now.
2013-06-29 12:51:29 -04:00
8e4b296a96
libotutil: Drop is-hidden from fast queryinfo
...
Since now we look at .hidden files and bang on the mainloop for every
file lookup if that's specified =(
2013-04-29 22:08:56 -04:00
4271f84755
waitablequeue: Only use API from GLib 2.34
...
As intended.
2013-04-03 00:50:16 -04:00
bac4d7a0d2
pull: Rework threading communication model
...
Previously, I've observed bugs where we either:
1) Exit too early, leaving undownloaded objects
2) Hang while downloading
This rewrite hopefully fixes both.
2013-04-01 18:56:38 -04:00
fafb256f57
Update to the latest libgsystem
2013-01-23 15:20:19 -05:00
ccfdf22bb8
core: Drop dead prototypes from header
...
Followup to previous commit.
2013-01-04 19:47:56 -05:00
cd5fce713c
core: Reuse more libgsystem API in utils
...
More code drain to libgsystem.
2013-01-04 19:46:02 -05:00
47bd290065
core: Drain fdatasync() API into libgsystem
...
Just code cleanup.
2013-01-04 17:25:32 -05:00
9bd4d35c2b
pull: Fix a race condition causing an early exit
...
This is a little less magical than what we had before.
2012-12-23 17:22:08 -05:00
19df2d372d
core: Use new libgsystem GSSubprocess API
...
It's less screwing around with GPtrArray in most cases.
2012-12-21 11:52:28 -05:00
3d41d1d2db
OtWorkerQueue: Don't exit until we've fulfilled initial requests
...
This counts as an implicit "hold" on the first items, which we must
complete. Otherwise there are race conditions where the queue may
signal idle when in fact the main process is still working.
2012-12-21 09:36:38 -05:00
fddc11ca74
Update to latest libgsystem
...
Has a bugfix change to gs_subprocess_new().
2012-12-20 21:14:31 -05:00
227d22adca
Update to the latest libgsystem
2012-12-03 12:05:41 -05:00
7fcebed4e8
cleanup: Rebase spawn API on top of libgsystem GSSubprocess
...
We didn't need most the parameters anyways, so this is fewer lines of
code. It also helps me test GSubprocess.
2012-12-01 16:53:06 -05:00
6dda8da6c5
Switch to using libgsystem shutil
...
More code moved into libgsystem.
2012-11-29 17:11:24 -05:00
1ac307e6b8
Switch to libgsystem file reading API
...
More draining common utilities into libgsystem.
2012-11-29 16:58:39 -05:00
f3d765c903
Switch to libgsystem directory creation API
...
More code being dropped into a common place.
2012-11-29 16:40:17 -05:00
6d81120455
otutil: Move checksum related bits into checksum-utils.[ch]
...
This paves the way for slimming ot-gio-utils.[ch] into libgsystem.
2012-11-29 16:28:24 -05:00
5761733218
Drop some bits from libotutil that are now merged in libgsystem
2012-11-29 16:24:49 -05:00
d1ddfc973c
libotutil: Drop dead prototype
...
Was never implemented.
2012-11-29 15:56:17 -05:00
05e7b6d596
pull-local: Make multithreaded
...
We were blocking for easily 1/10 or 1/5 of a second in fdatasync(),
which drastically slows down the whole process.
This threading isn't quite as good as the ostree-pull command, but it
lets us avoid the dependency on libsoup everywhere, and it's simpler.
2012-11-19 12:08:41 -05:00
de1ce843f1
core: Drop old GLIB_CHECK_VERSION(2.32) bits
...
We hard require 2.34 now; if it doesn't exist on the system, that's
what embedded dependencies are for.
2012-11-18 15:44:21 -05:00
d6956bfc19
core: And one more race in multithreaded mkdir -p
...
Sigh =(
2012-10-15 10:49:48 -04:00
c159188b81
core: One more minor tweak to recursive directory creation
...
We would fail in the case where we were trying to create a
subdirectory of /; this didn't actually happen, just noticed
via code inspection.
2012-10-13 17:54:19 -04:00
6b09f90122
core: Threadsafe recursive direction creation is harder than it appears
2012-10-13 10:47:03 -04:00
b4d408f483
core: Fix regression in previous commit
2012-10-12 15:25:25 -04:00
4436ed34d8
core: Make mkdir -p function safer
...
Recursing here is just a more obvious way to do it, rather than
relying on the semantics of g_file_make_directory_with_parents().
2012-10-12 15:10:39 -04:00
ee73c0fa34
core: Add dynamic uncompressed object cache for archive-z
...
This gives us something closer to the advantages of archive and
archive-z when using the latter. Concretely we get deduplication
among multiple checkouts, along with the "devino" hash table trick
during commits to avoid checksumming content again.
This is enabled by default.
2012-10-11 18:33:03 -04:00
efe27f1b05
pull: Ensure worker queue finishes if we unhold an empty queue too
2012-10-04 20:32:14 -04:00
5c1dc390ae
pull: Asynchronous metadata fetch
...
Create a worker thread for processing metadata, reserving the main
thread for HTTP requests.
This can create a very significant efficiency win for large pull
requests since we are much more likely to keep a full pipeline open.
The status display is also nicer now.
2012-10-03 22:35:33 -04:00
31153913ff
Extract keyfile helpers into libotutil
...
Will be used by ostree-pull too soon.
2012-09-15 11:20:56 -04:00
9ba968a5f4
otutil: Add a few more helper functions
...
To be used by wip/etc-writable branch.
2012-09-03 10:43:42 -07:00
4d2f770748
core: Use O_NOATIME to open metadata
...
We really don't need atime for metadata, it's just a speed hit.
2012-08-27 16:07:39 -04:00
2396608754
repo: Call fdatasync() before adding objects to the repo
...
I run builds on my laptop, but it also crashes about 1/4 of the time
while suspending. It's definitely undesrirable to get e.g. empty
.dirtree objects because they corrupt builds. Concretely, I was
getting empty contents committed for xorg-util-macros.
Now, we used to write out temporary files using g_file_replace() which
does a fsync() during close, but then switched to a more "manual"
g_file_append_to().
We could switch back to g_file_replace(), but the problem is, we don't
want to call fsync() on temporary files in the case where we already
have the object. Attempting to add an object we already have is a
*very* common case.
This is both the old and new code sequence for the case where an
object is already stored:
open(temp, O_WRONLY)
write() write() write()
close()
lstat(objects/3a/9fe332...) = 0
unlink(temp)
In the *new* code, here's the case where an object *isn't* stored:
open(temp, O_WRONLY)
write() write() write()
close()
lstat(objects/3a/9fe332...) = -1
open(temp, O_RDONLY)
fdatasync()
close()
rename(temp, objects/3a/9fe332)
Compare with the *old* code path for when an object isn't stored:
open(temp, O_WRONLY)
write() write() write()
close()
lstat(objects/3a/9fe332...) = -1
link(temp, objects/3a/9fe332)
unlink(temp)
The problem with this is we really need to fdatasync(). Also doing
just rename() instead of the weird link()/unlink() helps us express to
the filesystem that we want atomic semantics. For example, BTRFS has
special handling for rename().
2012-08-27 15:35:40 -04:00
a6c19aa00c
ostadmin: Change command line for qemu deploy helper
...
The qemu helper really wants to copy kernel modules, but not
update the system bootloader. Allow it to reuse ostadmin for
this.
Note that our previous path of shelling out to "cp -al" broke because
it refused to make cross-device links.
# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
# On branch master
# Your branch is ahead of 'origin/master' by 1 commit.
#
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# modified: src/libotutil/ot-gio-utils.c
# modified: src/libotutil/ot-gio-utils.h
# modified: src/ostadmin/ot-admin-builtin-deploy.c
#
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# embedded-dependencies/glib/
# embedded-dependencies/libsoup/
2012-08-05 16:39:57 -04:00
0d86702115
core: Drop ot_clear_hashtable for g_clear_pointer()
2012-07-17 08:18:43 -04:00
43a2fec6a8
core: Drop ot_clear_ptrarray in favor of g_clear_pointer
2012-07-17 08:16:54 -04:00
2ced4abf11
core: Drop ot_clear_gvariant() in favor of g_clear_pointer
2012-07-17 08:14:33 -04:00
57fcafd1ab
core: Remove ot_clear_checksum() in favor of g_clear_pointer()
2012-07-17 08:08:57 -04:00
c8d7fc216d
core: Use g_spawn_check_exit_status()
2012-07-17 08:06:20 -04:00
da43fb6db5
Hard require GLib 2.34
...
Anyone wanting to build against an earlier version can use the
embedded-dependencies system.
2012-07-15 18:22:41 -04:00
464f4a81c9
Add libgsystem as git external
...
Don't replace ot_lfree and stuff yet though...to much code churn.
2012-07-15 11:59:05 -04:00
664e9b404d
core: drop dead/unused code
...
The daemon is a nice idea but needs a lot more thought. Let's just
delete the unused code for now and come back to it later.
2012-06-30 13:49:25 -04:00
311493338e
core: Drop ot_gfile_new_for_path
...
Not necessary anymore since we switched to GIO_USE_VFS=local in the
binaries.
2012-06-30 13:42:24 -04:00
e485bace01
pull: Download and checksum asynchronously
...
This is quite a noticeable speedup when downloading loose objects.
2012-06-05 22:38:17 -04:00
f95090095e
core: Map metadata from repository as trusted
...
ot_util_variant_map() assumed untrusted; add a boolean, and use it for
the repository internals.
2012-05-18 16:30:44 -04:00
50cc2356f4
core: Really fix saving variants with REPLACE_DESTINATION
2012-05-14 20:49:00 -04:00
102180c5f2
core: Replace pack superindex via rename() rather than truncating
...
This helps allow cloning repositories via "cp -al".
2012-05-14 11:22:18 -04:00
18b6f5fd0c
ostadmin: Initial code
2012-05-07 20:43:03 -04:00
38dd179017
core: Compile again on GLib 2.28
2012-05-05 11:45:19 -04:00
ff3867c052
core: Increase type safety of some local alloc functions
2012-05-03 16:05:53 -04:00
38ef75e6e0
core: Re-unify on one OSTREE_OBJECT_TYPE_FILE
...
Having the archived vs not distinction in the object system wasn't
useful in light of pack files. In fact, we should probably move
towards generating a pack file per commit by default.
2012-04-11 22:27:59 -04:00
f15c184584
core: Clean up checksumming
...
Don't expose GChecksum in APIs. Add a new stream class which allows
us to pass an input stream somewhere, but gather a checksum as it's
read.
Move some bits of the internals towards binary csums.
2012-04-11 22:04:06 -04:00
87f45052af
core: Port libotutil to local-alloc
2012-04-09 15:57:46 -04:00
ca08ad6c5e
core: Add macros for local allocation
...
This is GCC-specific, but it makes the code significantly
cleaner.
2012-04-09 15:56:28 -04:00
b971ce0dfa
core: Only depend on GLib 2.30
2012-04-02 13:42:09 -04:00
68cce01556
core: Add stream->variant helper, as well as clear macros for ptrarray/hashtable
2012-03-31 10:32:26 -04:00
b221b0fdd6
core: Add a few more GIO utility functions
2012-03-31 10:32:02 -04:00
f484665bfe
libotuil: Fix missing #define in header
2012-03-06 11:59:06 -05:00
39fe050f04
core: Add standard::size to info we get from archives
...
This makes "ostree ls" show the file size.
2012-01-19 09:54:09 -05:00
786ce3b560
core: Preemptively error out on paths longer than PATH_MAX
...
Just noticed this while thinking like an attacker.
2011-12-22 16:50:01 -05:00
0018b6d75f
core: Drop unix::is-mountpoint from fast queryinfo flags
...
If this is present, we call access() and stat() on the parent
directory.
2011-12-22 11:07:55 -05:00
b8e8b58585
core: INCOMPATIBLE CHANGE: Name repo files with their type (e.g. .dirmeta)
...
This makes inspection easier. Internally the code gets simpler because
metadata and files are more unified; there is just one object type.
2011-12-12 12:29:09 -05:00
d6ceb91c35
core: Make ot_transfer_out_value() take a & for the second argument
...
As Ray Strode argued, it's confusing to have something that looks like
a function be magical. And OT_TRANSFER_OUT_VALUE is uglier.
2011-12-08 17:31:45 -05:00
56e0f231b5
core: Correctly map S_ISDIR to G_FILE_TYPE_DIRECTORY
2011-12-04 13:12:06 -05:00
ae3a5e3c07
core: Clean up filename utility API
...
Remove more unused functions, and change pathname splitting to handle
more cases like duplicate //, and to throw an error on .. as a filename.
2011-12-04 12:12:34 -05:00
7545405b73
core: Delete some dead utility code
2011-12-02 19:54:53 -05:00
0003e101f6
core: New ot_transfer_out_value() macro
...
This makes out values considerably less typing.
2011-12-02 18:17:06 -05:00
32c8166b25
core: Ensure we clean up leftover directories during merge
...
(Not sure why this regressed)
2011-12-02 17:58:45 -05:00
d1950da1a0
core: Further unify API to create files
...
We now have just one place which writes to the filesystem. Wrap a
temporary file allocation API on top of that.
2011-12-02 13:42:25 -05:00
c7235182a4
core: Use g_io_error_from_errno() to ensure we have more useful error codes
2011-12-02 11:40:06 -05:00
7fc625c967
core: Move pack file parsing into core, add ostree_create_file_from_input() API
...
This moves us closer to consistently passing around a triple of:
(GFileInfo *info, GVariant *xattrs, GInputStream *content)
Which will help the libarchive work.
2011-12-02 10:11:09 -05:00
bb0867aeb8
core: Add ot_gfile_create_tmp_symlink()
...
In some cases we'll need the ability to create a symbolic link, then
actually import it.
2011-12-01 19:25:02 -05:00
556662b24c
core: Extract stage_and_checksum() internal API
...
This will be used for staging both metadata and data consistently
before actually importing it.
2011-11-30 21:20:15 -05:00
99bf19314e
core: Add ot_gfile_create_tmp() to utils, use it
2011-11-30 20:32:06 -05:00
70574c3c06
otutil: New utility functions
2011-11-29 13:42:05 -05:00
fd58e16a8d
core: Minor code cleanup
2011-11-25 12:01:05 -05:00
3c025e526f
core: Ad ot_clear_gvariant(), use it
...
This saves us a conditional.
2011-11-22 10:20:10 -05:00
aa69aae94e
core: Move merge dirs code into gio utils
2011-11-21 15:04:07 -05:00
9cf15ed151
core: More cleanup of GIO utility API
2011-11-18 08:43:47 -05:00
231f7ed76c
core: Use ot_gfile_ prefix for new local GFile creation function
2011-11-18 08:10:23 -05:00
6591e586ed
core: Remove last user of g_file_new()
2011-11-18 08:02:19 -05:00
a8735778f0
core: Use ot_gfile_get_path_cached() pervasively
...
This requires no free() calls and is just better.
2011-11-17 18:23:36 -05:00
aa865bbb83
core: Validate file names read from directory variants
...
In a future where we pull data from remote servers, we don't want
to allow path uplinks.
2011-11-16 23:23:30 -05:00
18f0b537a4
build: Move sources into src/ again
...
This is necessary if we want to build when srcdir == builddir,
otherwise we blow up because "ostree" is a source directory and a
binary.
2011-11-14 15:39:38 -05:00
21c7ff74b6
De-recursify source tree
2011-11-02 15:45:32 -04:00
2b00cf3810
core: add ^ for rev-parse
2011-11-02 13:22:13 -04:00
7ca1c3d2e7
core: remove HEAD file, use branches instead
...
HEAD in git describes a working copy, and we don't have those.
Instead, default to a "master" branch. This also lets us support
multiple branches.
2011-10-27 14:32:05 -04:00
03943c659f
Drop glib requrirement to 2.28
...
This lets it build on more currently-released systems like Fedora 15.
2011-10-26 14:12:13 -04:00
b27df6fd72
"Hacktree" is now known as "OSTree"
...
It just sounds better.
2011-10-18 14:44:48 -04:00
Colin Walters
Stef Walter