Commit Graph

518 Commits

Author SHA1 Message Date
Colin Walters 6a5e66b152 pull: Support --mirror option
There's several use cases for calling into ostree itself to do
mirroring, instead of using bare rsync.  For example, it's a bit more
efficient as it doesn't require syncing the objects/ directory.

https://bugzilla.gnome.org/show_bug.cgi?id=728351
2014-06-16 17:14:13 -04:00
Colin Walters e31daf448a libostree: Silently ignore EPERM when setting EXT2_IMMUTABLE_FL
In the case of running ostree as non-root on a regular filesystem (not
tmpfs which doesn't support immutable), we should just silently do
nothing if we encounter EPERM.  Cache the result to avoid spam in
strace.

https://bugzilla.gnome.org/show_bug.cgi?id=728006
2014-06-09 14:44:17 -04:00
Colin Walters 24c64d6b4f libostree: Make OstreeFetcher explicitly private
Per previous commit.  We're not attempting to export an HTTP to
clients, the high level API is ostree_repo_pull().

https://bugzilla.gnome.org/show_bug.cgi?id=731369
2014-06-09 14:04:53 -04:00
Colin Walters 4ce93db595 libostree: Make OstreeTlsCertInteraction explicitly private
Similar to previous commit, this should be private.

https://bugzilla.gnome.org/show_bug.cgi?id=731369
2014-06-09 14:04:53 -04:00
Colin Walters 9f734494ff libostree: Make OstreeLibarchiveInputStream explicitly private
We weren't installing the headers, but at the moment all symbols
starting with ostree_ were being exported.  Fix that by prefixing
non-static symbols with '_'.

https://bugzilla.gnome.org/show_bug.cgi?id=731369
2014-06-09 14:04:53 -04:00
Colin Walters 51bf1fab0f repo: Support fsync=false configuration
See rationale in the updated docs.  Basically developer use cases as
well as UPS-backed servers.
2014-06-05 17:35:50 -04:00
Vadim Rutkovsky ef00572999 Honor disable_fsync during checkout also
Finally, fsync to ensure all entries are on disk, unless disabled.
 We support disabling this for cases like server-side buildroot
 construction where we don't need to be robust against power loss
2014-06-04 16:14:46 +02:00
Colin Walters b4d21e9406 deploy: Set the immutable bit on the deployment root
This prevents people from creating new directories there and expecting
them to be persisted.  The OSTree model has all local state to be in
/etc and /var.

This introduces a compile-time dependency on libe2fsprogs.

We're only doing this for the root directory at the moment.

https://bugzilla.gnome.org/show_bug.cgi?id=728006
2014-06-03 13:41:48 -04:00
Colin Walters 47610b45c2 Limit metadata to 10 MiB
If fetching GPG-signed commits over plain HTTP, a MitM attacker can
fill up the drive of targets by simply returning an enormous stream
for the commit object.

Related to this, an attacker can also cause OSTree to perform large
memory allocations by returning enormous GVariants in the metadata.

This helps close that attack by limiting all metadata objects to 10
MiB, so the initial fetch will be truncated.

But now the attack is only slightly more difficult as the attacker
will have to return a correctly formed commit object, then return a
large stream of < 10 MiB dirmeta/dirtree objects.

https://bugzilla.gnome.org/show_bug.cgi?id=725921
2014-05-27 14:15:27 -04:00
Colin Walters 6002356747 fetcher: Unref pending result when completing
Otherwise we were just leaking it.

https://bugzilla.gnome.org/show_bug.cgi?id=725921
2014-05-27 12:01:14 -04:00
Colin Walters f1ccaa581e repo: Don't require a txn for writing
The current "transaction" symlink was introduced to fix issues with
interrupted pulls; normally we assume that if we have a metadata
object, we also have all objects to which it refers.

There used to be a "summary" which had all the available refs, but I
deleted it because it wasn't really used, and was still racy despite
the transaction bits.

We still want the pull process to use the transaction link, so don't
delete the APIs, just relax the restriction on object writing, and
introduce a new ostree_repo_set_ref_immediate().
2014-05-26 18:49:17 -04:00
Colin Walters a4d01976bb repo: Only load /etc/ostree/remotes.d for system repo
They shouldn't be loaded for random test/personal repositories.  Doing
so triggers another bug in that we return them from
ostree_repo_get_config() when then causes clients to write them out
permanently to disk with ostree_repo_write_config().  This caused test
suite failures.
2014-05-26 18:38:35 -04:00
Colin Walters 62c2a204c9 trivial: commit: Fix docstring typos 2014-05-26 17:57:36 -04:00
Colin Walters 78726244a0 deploy: Convert remaining g_print() to systemd journal messages
These bits should be logged more sanely.
2014-05-24 13:19:29 -04:00
Colin Walters bf37a9c741 deploy: Remove deployment root print
It doesn't look very professional; in the future though we should have
a progress bar here or something.
2014-05-24 12:30:56 -04:00
Colin Walters f47a20fb81 Support /etc/ostree/remotes.d
For many OS install scenarios, one runs through an installer which may
come with embedded data, and then the OS is configured post-install to
receive updates.

In this model, it'd be nice to avoid the post-install having to rewrite
the /ostree/repo/config file.

Additionally, it feels weird for admins to interact with "/ostree" -
let's make the system feel more like Unix and have our important
configuration in /etc.

https://bugzilla.gnome.org/show_bug.cgi?id=729343
2014-05-08 18:59:24 -04:00
Colin Walters 25ad4a9f65 pull: Add tls-client-cert-{path,key} (if we have new enough libsoup)
This is an actually working version of client-side certificates.
Depends on:
See: https://bugzilla.gnome.org/show_bug.cgi?id=334021

We detect whether libsoup is new enough for this.

https://bugzilla.gnome.org/show_bug.cgi?id=729356
2014-05-01 17:13:13 -04:00
Colin Walters 64b4ec6ef0 Revert "pull: Add tls-client-cert-{path,key}"
This reverts commit 94f9ee7bce.
Doesn't actually work, see
https://bugzilla.gnome.org/show_bug.cgi?id=334021
2014-05-01 10:23:11 -04:00
Colin Walters 94f9ee7bce pull: Add tls-client-cert-{path,key}
These can be used to present a client certificate when making requests
to a repository.
2014-05-01 07:57:06 -04:00
Colin Walters 7cc4a20f3b pull: Correctly handle error state when fetching optional data
For the static deltas work, we're using the already-extant internal
API to perform a HTTP fetch for optional data - static deltas are
optional.

Except that we didn't correctly unset the error if we were doing an
optional fetch and the data wasn't found.
2014-04-29 10:59:57 -04:00
Colin Walters 626fc8519b pull: Stay in mainloop if we're synchronously fetching URI
The static deltas work will be doing some synchronous fetching even
after refs are downloaded.
2014-04-29 10:59:57 -04:00
Colin Walters 006e8f0073 deltas: Link to liblzma, add internal API to use it
For future static deltas work, we'll be linking to liblzma.  Since
it's fairly widespread, let's just make it a hard dependency.
2014-04-29 10:59:57 -04:00
Colin Walters f040c02048 libostree: Add _finish() API to async progress
Since OstreeAsyncProgress queues to the mainloop, we might "lose" the
last message.  Give callers a way to force a flush.
2014-04-29 10:59:57 -04:00
Colin Walters 959db9f0de core: Add _STRING variants of GVariant object formats
For the static deltas work, we're going to embed a commit in the delta
superblock, so we need a format string without the G_VARIANT_TYPE().
2014-04-29 08:51:00 -04:00
Colin Walters 22ab7b3e79 core: Use posix_fallocate() when writing objects
This helps ensure the filesystem allocates space efficiently.
2014-04-28 19:42:12 -04:00
Colin Walters 2c5d726076 deploy: fsync() copy of /etc
This unfortunately requires reimplementing gs_shutil_cp_a(), except
while we're here, we also use the *at calls.
2014-04-11 18:43:58 -04:00
Colin Walters b423d9e9cf deploy: fsync() kernel/initramfs and bootloader config parent directories
Ensure they've hit disk.
2014-04-11 18:43:27 -04:00
Colin Walters 646c8be8dc deploy: fdatasync() bootloader configuration files
Yet more data we're writing out that needs to be sync'd.
2014-04-11 18:43:27 -04:00
Colin Walters 053dd72649 deploy: Ensure .origin file is fsync()ed on disk
Along with its parent directory.
2014-04-11 18:43:27 -04:00
Colin Walters 0b8495f09a deploy: Also fsync parent directory of modified config files 2014-04-11 18:43:13 -04:00
Colin Walters 38feca0e4d deploy: Ensure that any modified config files are fsync()d
It really wouldn't do for one to be missing one's ssh keys for
example...
2014-04-11 18:43:13 -04:00
Colin Walters 1b16553600 checkout: fsync() directory on checkouts
We want to be really sure that our deployment roots have hit the disk.
2014-04-11 18:43:13 -04:00
Colin Walters 7acad0a2e5 deploy: Ensure that all directories we create are fsync()d
There's two halves to this; first, when we create an hierarchy, we
need to call fsync().  Second, we need to fsync again anytime after
we've modified a directory.
2014-04-11 18:43:13 -04:00
Colin Walters d27c78eab5 pull: Display download progress of individual objects as we get it
It was kind of annoying at least for rpm-ostree upgrades since /boot
happens to be first and we eat a 21MB initramfs with no download
progress.

https://bugzilla.gnome.org/show_bug.cgi?id=726348
2014-04-11 01:31:14 -04:00
Colin Walters 829e0382ab bootloader: fdatasync() bootloader configuration
Let's be a bit more conservative here and actually fdatasync() the
configurations we're generating.

I'm seeing an issue at the moment where syslinux isn't finding the
config sometimes, and while I don't think this is the issue, let's try
it.
2014-04-03 22:32:30 -04:00
Colin Walters cc8bacf8b4 bootloaders: Always write out bootloader config file
There was an attempted optimization to only write if changed, but this
is broken - we always write the bootloader config into a new
directory.

In theory we should only be writing if it changed, but let's not do a
broken optimization.
2014-04-03 21:08:03 -04:00
Colin Walters ac40de5773 deploy: Add a debug message for which bootloader is used
Debugging something...
2014-04-03 19:39:08 -04:00
Colin Walters 890d685c79 deploy: Add an assertion that deployment directory exists
I may be seeing something going wrong in rpm-ostree, just adding this
assertion for my own sanity.
2014-04-03 18:29:07 -04:00
Colin Walters edc8665206 fetcher: Honor http_proxy environment variable
https://bugzilla.gnome.org/show_bug.cgi?id=706809
2014-04-01 19:12:40 -04:00
Colin Walters 7bd00e3005 checkout: Always do chmod even in _MODE_USER
The previous commit here changed things so that we do mkdir(x, 0700),
then fchmod later only if we created the directory.

However the logic was incorrect; we still need to chmod even in
MODE_USER if we created the directory.
2014-04-01 13:07:17 -04:00
Colin Walters 5bca569506 upgrade: Support --allow-downgrade again
This was unintentially dropped with the OstreeSysrootUpgrader rebase.
2014-03-31 22:27:56 -04:00
Colin Walters 913337c557 sysroot-upgrader: Check merge deployment, not ref for differences
Otherwise this broke atomicity; we could fetch/store the ref, then
crash, and then not upgrade the next time we tried upgrading.

The correct model is: the tree has changed if the new ref is different
from the merge deployment.
2014-03-31 14:41:13 -04:00
Colin Walters 8d13e5bb74 sysroot: Be more conservative with bootlink optimization
Trying to implement "rpm-ostree rollback", in the case where we have 2
deployments with the same bootconfig that we're reordering, we need to
write bootconfig, not just swap the bootlinks.
2014-03-28 21:06:23 -04:00
Colin Walters 11ecf9267e libostree/upgrader: Don't pull if there is no remote
In this case we're just reading the local repo.
2014-03-27 07:36:50 -04:00
Colin Walters 9389b7961c libostree/upgrader: Add an API to retrieve an origin description
This will be used by "rpm-ostree upgrade".
2014-03-25 16:46:39 -04:00
Colin Walters 90883674a9 libostree/upgrader: Throw an error if osname is empty
This shouldn't happen; I'm just adding the check to force a rebuild.
2014-03-25 15:47:20 -04:00
Colin Walters 7baa600e23 Add an OstreeSysrootUpgrader API
This moves some utility code from the ostree tool into the shared
library, which will make it easier to consume by external tools.
2014-03-24 18:08:22 -04:00
Colin Walters ffb9d34671 Move basic commit API into ostree_sysroot_simple_write_deployment()
The admin commands had this shared in tool common, but we want to
encourage external programs to do this as well.
2014-03-23 11:36:09 -04:00
Colin Walters a80872afa0 pull: Don't print, just use progress callback
More work on removing g_print() from the library.
2014-03-21 14:52:56 -04:00
Colin Walters 4309b49866 libostree: Add ostree_sysroot_deployment_set_kargs()
It turns out people sometimes want to be able to change the kernel
arguments.  Add a convenient API to do so for the current deployment.

This will be used by Anaconda.
2014-03-19 09:49:55 -04:00