Commit Graph

2263 Commits

Author SHA1 Message Date
Matthew Barnes af30fc764a fetcher: Add "config-flags" construct-only property
A lot of effort here just to avoid touching SoupSession directly in
ostree_fetcher_new().  The reason will become apparent in subsequent
commits.

Note this introduces generated enum/flags GTypes using glib-mkenums.
I could have just made the property type as plain integer, but doing
properties right will henceforth be easier now that the automake-fu
is established.
2015-12-14 09:41:29 -05:00
Alexander Larsson 96eed95720 repo: Allocate a tmpdir for each OstreeFetcher to isolate concurrent downloads
This way two pulls will not use the same tmpdir and accidentally
overwrite each other. However, consecutive OstreeFetchers will reuse
the tmpdirs, so that we can properly resume downloading large objects.

https://bugzilla.gnome.org/show_bug.cgi?id=757611
2015-12-14 08:39:11 +01:00
Alexander Larsson f771461b4a repo: Use per-transaction staging dir
Concurrent pulls break since we're sharing the staging directory for
all transactions in the repo. This makes us use a per-transaction directory.

However, in order for resumes to work we first look for existing
staging directories and try to aquire an exclusive lock for them. If
we can't find any staging directory or they are all already locked,
then we create a new one.

https://bugzilla.gnome.org/show_bug.cgi?id=757611
2015-12-14 08:38:51 +01:00
Alexander Larsson be19c88861 repo: Add _ostree_repo_allocate_tmpdir helper
This creates a subdirectory of the tmp dir with a selected prefix,
and takes a lockfile to ensure that nobody else is using the same directory.
However, if a directory with the same prefix already exists and is
not locked that is used instead.

The later is useful if you want to support some kind of resumed operation
on the tmpdir.

touch reused dirs

https://bugzilla.gnome.org/show_bug.cgi?id=757611
2015-12-14 08:37:55 +01:00
Alexander Larsson 211ca6795c Update to latest libglnx 2015-12-14 08:37:17 +01:00
Colin Walters 73d77a2193 build: Also add a configure check for YACC/bison
So we error out more nicely if not found.
2015-12-07 10:39:37 -05:00
Colin Walters 76f4507557 build: Delete generated parse-datetime.c file, use AM_V_GEN
Bison is a well known external dependency, so just require it.
Including the generated content in git means it may or may not
be regenerated based randomly on timestamps, etc.

Also use `$(AM_V_GEN)` so we get prettier output.
2015-12-07 10:33:23 -05:00
Colin Walters 8ba90a3341 cmdline: Fatally error if the timestamp in a commit is invalid
Previously we were just ignoring this, which hid a bug in
an earlier commit that generated them.

Also change the `commit` program to use both APIs - this
involves extra code, but not too much.

This way, reverting the fix with this on top caused the test suite to
fail.  Adding an active test for this would need a custom test program
using the C API, or adding a cmdline flag to the client, neither of
which quite seemed worth it.
2015-12-04 11:26:07 -05:00
Giuseppe Scrivano f88a9733ce tests: add missing ${CMD_PREFIX}
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-12-04 11:10:51 -05:00
Giuseppe Scrivano 335ea3f339 parse-datetime: use the module from gnulib
Use the parse-datetime module from gnulib, and adapt it to not require
other modules as portability is not really an issue for us.

DATE can be specified in different formats, such as: "-1 week", "last
monday", "1 week ago".

Include the generated .c file in the repository so to not add another
dependency to Bison.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-12-04 11:10:46 -05:00
Colin Walters 6335dadb49 Release 2015.11 2015-12-04 09:53:03 -05:00
Colin Walters ed9f5ce031 gpg-verifier: Fix compiler warning 2015-12-04 09:51:14 -05:00
Matthew Barnes 94bc9765d5 repo: Fix backwards timestamp in ostree_repo_write_commit()
ostree_repo_write_commit_with_time() converts the timestamp to
big-endian byte order.

ostree_repo_write_commit() was also doing this when calling
ostree_repo_write_commit_with_time(), resulting in a corrupted
commit object (timestamp bytes were backwards).

Recent regression in 14ffd7022a
2015-12-03 12:43:52 -05:00
Matthew Barnes a02174b0ba remote: Print full refspec in "ostree remote refs"
Just to make copy-and-paste a little easier, as I often use this command
immediately before rebasing.

 e.g.

 # ostree remote refs fedora-atomic
 fedora-atomic:fedora-atomic/f23/x86_64/docker-host
 fedora-atomic:fedora-atomic/f23/x86_64/testing/docker-host
 ^^^^^^^^^^^^^^ (this part is new)

 # rpm-ostree rebase fedora-atomic:fedora-atomic/f23/x86_64/testing/docker-host
2015-12-02 23:13:44 -05:00
Matthew Barnes 581b7d6183 fetcher: Remove "total_requests" counter
Incremented, but not used for anything.
2015-12-01 12:34:34 -05:00
Matthew Barnes 97efe12ac6 fetcher: Remove "sending_messages" hash table
Vestige of ostree_fetcher_query_state_text(), removed last year.
2015-12-01 12:34:28 -05:00
Matthew Barnes 70e121c159 glnx: Update from master
More autocleanup backports, this time GFileOutputStream.
2015-11-23 14:49:06 -05:00
Matthew Barnes 74d8e5f159 trivial-httpd: Avoid SoupBuffer when there's no content
This was supposed to be merged into the previous commit, but I
messed up.
2015-11-23 14:30:31 -05:00
Matthew Barnes df75fc232a repo: Never delete .commitmeta files
Do not delete a .commitmeta file after removing the last metadata entry.
This way a client will pull the empty .commitmeta file and overwrite old
metadata as expected.

https://bugzilla.gnome.org/750459
2015-11-23 14:22:45 -05:00
Matthew Barnes 5307af5a7a repo: Validate checksums have correct length
ostree_checksum_bytes_peek() can return NULL if the checksum has an
incorrect length (most likely from disk corruption) but most callers
are not prepared to handle this and would likely crash.

Use ostree_checksum_bytes_peek_validate() instead, which sets a
GError on an invalid checksum.
2015-11-17 20:35:54 -05:00
Giuseppe Scrivano 45cee1bd70 tests: prefix invocation of ostree with where missing
And add a syntax rule to avoid this in future.

Fixed by:

sed -i -e 's|^ostree |${CMD_PREFIX} ostree |g' tests/*.sh

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-16 11:07:55 +01:00
Giuseppe Scrivano 11bc9e58c1 tests: add tests for prune --keep-younger-than=DATE
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-16 11:07:55 +01:00
Giuseppe Scrivano 68f70d8e74 commit: add --timestamp=TIMESTAMP
It allows to override the timestamp of the commit.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-16 11:07:55 +01:00
Giuseppe Scrivano 14ffd7022a libostree: add new API ostree_repo_write_commit_with_time
It extends ostree_repo_write_commit as it permits to override the
commit timestamp.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-16 11:07:55 +01:00
Giuseppe Scrivano b393e5a2e9 ostree: do not print the usage on each G_IO_ERROR_NOT_SUPPORTED
It may have a different meaning, and the usage screen is not helpful.

Print the usage screen only when the command is not found.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-16 11:07:55 +01:00
Giuseppe Scrivano f506644830 pull: make slightly clearer when failing for missing xattrs support
It is not clear why pull fails on tmpfs.  The additional message might
be helpful.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-16 11:07:55 +01:00
Giuseppe Scrivano f56688da71 prune: add --keep-younger-than=DATE
The format used for DATE is "%Y-%m-%d %H:%M:%S %z"

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-16 11:07:38 +01:00
Giuseppe Scrivano daa6f150c4 tests: add test for ostree prune --delete-commit
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-16 10:57:33 +01:00
Giuseppe Scrivano 0dee70bcd6 fsck: create a tombstone when the parent is missing
Change the previous logic that a tombstone commit was created when
a partialcommit is found.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-16 10:57:33 +01:00
Giuseppe Scrivano 0aa836c205 prune: add --delete-commit
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-16 10:57:33 +01:00
Matthew Barnes a19a112d9d libostree: Fix a couple compiler warnings 2015-11-12 14:32:07 -05:00
Alexander Larsson 1c056eb282 deltas: Support including detached metadata in static deltas
This is very useful for the inline-parts case, as you can then include
detached signatures in a single file representing the commit.

It is not as important for the generic pull case, as the detached
metadata is only a single small file. Additionally the detached
metadata is not content referenced and may change after the static
delta file was created, so we need to pull the latest version anyway.
2015-11-10 08:56:16 +01:00
Alexander Larsson 598afd5030 pull: Verify checksums from static deltas unless gpg signed summary
Otherwise untrusted repos can lie about the commit ids.
2015-11-10 08:56:15 +01:00
Alexander Larsson ec56fea821 deltas: Support passing filename to ostree_repo_static_delta_execute_offline
If you pass a diriectory it will look for the "superblock" child, otherwise
it will use the file as the superblock. I need this in xdg-app to be able
to install any filename as a bundle.
2015-11-10 08:56:15 +01:00
Alexander Larsson dac57b6410 deltas: Support passing filename to delta generator 2015-11-10 08:56:15 +01:00
Alexander Larsson b22b859dc5 deltas: Make min-fallback-size 0 actually disable fallbacks
This is what the docs specify.
2015-11-10 08:56:14 +01:00
Alexander Larsson 78f14555c8 deltas: Verify checksums in apply-offline unless skip_validate is TRUE 2015-11-10 08:56:14 +01:00
Alexander Larsson 7a4fb1b2f1 Add _ostree_repo_open|commit_untrusted_content_bare
Also renames OstreeRepoTrustedContentBareCommit to
OstreeRepoContentBareCommit so that it can be used by both.

This will be needed when we introduce checksum verification of objects
in static deltas.
2015-11-10 08:56:14 +01:00
Alexander Larsson 3fa3e12e02 static-delta apply-offline: Don't skip validation
This makes no sense, at least by default.
2015-11-10 08:56:13 +01:00
Alexander Larsson e418acb31b delta: Ensure the from commit exists when applying static delta 2015-11-10 08:56:13 +01:00
Alexander Larsson eaa678eaa4 deltas: Make apply-offline only read the parts once
No need to read() the file for the checksum if we then directly mmap it.
Instead we just mmap it initially and checksum from that.
2015-11-10 08:56:13 +01:00
Alexander Larsson ee70ffbbcb Add tests for inline static deltas 2015-11-10 08:56:13 +01:00
Alexander Larsson ccb77d3bd8 static-deltas generate: Add --inline option to CLI tool 2015-11-10 08:56:12 +01:00
Alexander Larsson 11a79220e2 static deltas: Add support for inline-parts
In this mode the parts are stored in the metadata of the main delta
superblock file.  This can be useful if you want a single-file delta
for easy transport, or for http in the case the delta is very small.
2015-11-10 08:56:12 +01:00
Alexander Larsson 30768a979f generate-static-delta: Support min-fallback-size 0 to disable fallbacks 2015-11-10 08:56:12 +01:00
Colin Walters efdb4d8f44 docs: Note not to put private keys in /usr/share/ostree
https://bugzilla.gnome.org/show_bug.cgi?id=757524
2015-11-07 21:15:20 -05:00
Giuseppe Scrivano 438e21e858 tests: add tests for prune and tombstones commits
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-03 09:53:38 +01:00
Giuseppe Scrivano 29c425e742 fsck: add argument --add-tombstones
It is used to create tombstones for missing commits

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-03 09:53:38 +01:00
Giuseppe Scrivano 646fe139e6 repo: create a tombstone commit when deleting a commit
When a commit is deleted and the repo is configured to use tombstone
commits, create one.  Delete the tombstone file only if the commit is
pulled again.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-03 09:53:38 +01:00
Giuseppe Scrivano 76c5cc07db pull: add support for tombstone commits
Add a new object type: OSTREE_OBJECT_TYPE_TOMBSTONE_COMMIT that is
used when a commit was intentionally removed.

If the remote repository doesn't use tombstone commits, do not fail on
a missing commit (change 0b795785dd).

When the remote repository uses tombstones, if a commit cannot be
found, check if the tombstone file is present and fail if it is not
present.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2015-11-03 09:53:38 +01:00