Commit Graph
100 Commits
Author SHA1 Message Date
Colin Walters 6002356747 fetcher: Unref pending result when completing
Otherwise we were just leaking it.

https://bugzilla.gnome.org/show_bug.cgi?id=725921
2014-05-27 12:01:14 -04:00
Colin Walters f1ccaa581e repo: Don't require a txn for writing
The current "transaction" symlink was introduced to fix issues with
interrupted pulls; normally we assume that if we have a metadata
object, we also have all objects to which it refers.

There used to be a "summary" which had all the available refs, but I
deleted it because it wasn't really used, and was still racy despite
the transaction bits.

We still want the pull process to use the transaction link, so don't
delete the APIs, just relax the restriction on object writing, and
introduce a new ostree_repo_set_ref_immediate().
2014-05-26 18:49:17 -04:00
Colin Walters a4d01976bb repo: Only load /etc/ostree/remotes.d for system repo
They shouldn't be loaded for random test/personal repositories.  Doing
so triggers another bug in that we return them from
ostree_repo_get_config() when then causes clients to write them out
permanently to disk with ostree_repo_write_config().  This caused test
suite failures.
2014-05-26 18:38:35 -04:00
Colin Walters 62c2a204c9 trivial: commit: Fix docstring typos 2014-05-26 17:57:36 -04:00
Colin Walters 78726244a0 deploy: Convert remaining g_print() to systemd journal messages
These bits should be logged more sanely.
2014-05-24 13:19:29 -04:00
Colin Walters bf37a9c741 deploy: Remove deployment root print
It doesn't look very professional; in the future though we should have
a progress bar here or something.
2014-05-24 12:30:56 -04:00
Colin Walters fc11c8673f trivial: TODO: Add link sizes/progress bar 2014-05-23 07:59:35 -04:00
Colin Walters 1a6a8dcb83 trivial: TODO: Add link for http unprivileged 2014-05-23 07:57:23 -04:00
Colin Walters ac56501dfa packaging: BR libgsystem 2014-05-22 22:51:21 -04:00
Colin Walters 19abd8277f selinux-ensure-labeled: Support no arguments to just traverse deployment root
This makes it easy to use for the case where rpm-ostree-toolbox is
injecting systemd services into the deployment root, and we don't
actually need to traverse the whole FS.
2014-05-22 22:51:21 -04:00
Colin Walters a864190a04 admin switch: Allow switching just remote names
This is a followup to the previous commit; for the installation media
case we want to keep the current origin ref, and only switch remotes.
2014-05-15 08:58:44 -04:00
Colin Walters 24e1bf2552 admin switch: Support switching remotes as well
Say I have an installation from CDROM; the remote name may be
"installmedia" or something like that.  We want to allow also
switching remotes.
2014-05-12 17:31:34 -04:00
Colin Walters 7ccfc93f71 build: Add missing DESTDIR
Hooray for read-only bind mounts and building as non-root.
2014-05-09 09:07:00 -04:00
Colin Walters f47a20fb81 Support /etc/ostree/remotes.d
For many OS install scenarios, one runs through an installer which may
come with embedded data, and then the OS is configured post-install to
receive updates.

In this model, it'd be nice to avoid the post-install having to rewrite
the /ostree/repo/config file.

Additionally, it feels weird for admins to interact with "/ostree" -
let's make the system feel more like Unix and have our important
configuration in /etc.

https://bugzilla.gnome.org/show_bug.cgi?id=729343
2014-05-08 18:59:24 -04:00
Colin Walters 25ad4a9f65 pull: Add tls-client-cert-{path,key} (if we have new enough libsoup)
This is an actually working version of client-side certificates.
Depends on:
See: https://bugzilla.gnome.org/show_bug.cgi?id=334021

We detect whether libsoup is new enough for this.

https://bugzilla.gnome.org/show_bug.cgi?id=729356
2014-05-01 17:13:13 -04:00
Colin Walters 64b4ec6ef0 Revert "pull: Add tls-client-cert-{path,key}"
This reverts commit 94f9ee7bce.
Doesn't actually work, see
https://bugzilla.gnome.org/show_bug.cgi?id=334021
2014-05-01 10:23:11 -04:00
Colin Walters 94f9ee7bce pull: Add tls-client-cert-{path,key}
These can be used to present a client certificate when making requests
to a repository.
2014-05-01 07:57:06 -04:00
Colin Walters 7cc4a20f3b pull: Correctly handle error state when fetching optional data
For the static deltas work, we're using the already-extant internal
API to perform a HTTP fetch for optional data - static deltas are
optional.

Except that we didn't correctly unset the error if we were doing an
optional fetch and the data wasn't found.
2014-04-29 10:59:57 -04:00
Colin Walters 626fc8519b pull: Stay in mainloop if we're synchronously fetching URI
The static deltas work will be doing some synchronous fetching even
after refs are downloaded.
2014-04-29 10:59:57 -04:00
Colin Walters 006e8f0073 deltas: Link to liblzma, add internal API to use it
For future static deltas work, we'll be linking to liblzma.  Since
it's fairly widespread, let's just make it a hard dependency.
2014-04-29 10:59:57 -04:00
Colin Walters f040c02048 libostree: Add _finish() API to async progress
Since OstreeAsyncProgress queues to the mainloop, we might "lose" the
last message.  Give callers a way to force a flush.
2014-04-29 10:59:57 -04:00
Colin Walters 349083194d otutil: Correctly add ref to bytes when creating GVariant
This one went undiscovered for a while because it turned out we
weren't using it...
2014-04-29 08:53:28 -04:00
Colin Walters 959db9f0de core: Add _STRING variants of GVariant object formats
For the static deltas work, we're going to embed a commit in the delta
superblock, so we need a format string without the G_VARIANT_TYPE().
2014-04-29 08:51:00 -04:00
Colin Walters 021c4ee9d3 show: Fix segfault if we can't find an object
We need to use the original rev here.
2014-04-29 07:42:30 -04:00
Colin Walters 5346e0615b libotutil: Fix two bugs in usage of posix_fallocate()
* It's invalid to call it with a size of 0, so don't do that.

* posix_* apparently don't set errno.  So capture the return value and
  use that.
2014-04-29 07:40:25 -04:00
Colin Walters 22ab7b3e79 core: Use posix_fallocate() when writing objects
This helps ensure the filesystem allocates space efficiently.
2014-04-28 19:42:12 -04:00
Colin Walters 32663a5a50 build: Remove --enable-embedded-dependencies
This is going to bitrot too much; we'll figure out a better way to do
this when someone if a RHEL6 port ever happens.  Probably Software
Collections.
2014-04-22 09:08:35 -04:00
Colin Walters 7cb7d8f741 Release 2014.4 2014-04-13 10:50:27 -07:00
Colin Walters 2c5d726076 deploy: fsync() copy of /etc
This unfortunately requires reimplementing gs_shutil_cp_a(), except
while we're here, we also use the *at calls.
2014-04-11 18:43:58 -04:00
Colin Walters b423d9e9cf deploy: fsync() kernel/initramfs and bootloader config parent directories
Ensure they've hit disk.
2014-04-11 18:43:27 -04:00
Colin Walters 646c8be8dc deploy: fdatasync() bootloader configuration files
Yet more data we're writing out that needs to be sync'd.
2014-04-11 18:43:27 -04:00
Colin Walters 18aaa49724 libotutil: Make use of dirfd-relative API in ot_gfile_replace_contents_fsync()
It's just more efficient.
2014-04-11 18:43:27 -04:00
Colin Walters 053dd72649 deploy: Ensure .origin file is fsync()ed on disk
Along with its parent directory.
2014-04-11 18:43:27 -04:00
Colin Walters 0b8495f09a deploy: Also fsync parent directory of modified config files 2014-04-11 18:43:13 -04:00
Colin Walters 38feca0e4d deploy: Ensure that any modified config files are fsync()d
It really wouldn't do for one to be missing one's ssh keys for
example...
2014-04-11 18:43:13 -04:00
Colin Walters 1b16553600 checkout: fsync() directory on checkouts
We want to be really sure that our deployment roots have hit the disk.
2014-04-11 18:43:13 -04:00
Colin Walters 7acad0a2e5 deploy: Ensure that all directories we create are fsync()d
There's two halves to this; first, when we create an hierarchy, we
need to call fsync().  Second, we need to fsync again anytime after
we've modified a directory.
2014-04-11 18:43:13 -04:00
Colin Walters b19aea441a libotutil: Add API to create directory hierarchy recursively *and* fsync
To be really sure that any directory entries have hit disk we need to
call fsync() on the directory fd.  This API allows us to conveniently
create a directory hierarchy, fsyncing all of it along the way.
2014-04-11 18:43:13 -04:00
Colin Walters d27c78eab5 pull: Display download progress of individual objects as we get it
It was kind of annoying at least for rpm-ostree upgrades since /boot
happens to be first and we eat a 21MB initramfs with no download
progress.

https://bugzilla.gnome.org/show_bug.cgi?id=726348
2014-04-11 01:31:14 -04:00
Colin Walters d4d4ef552d libotutil: Fix mistaken return of TRUE in error path
Gah, must have been a refactoring bug.
2014-04-09 01:51:43 -04:00
Colin Walters 1bdabda5f3 Use external libgsystem 2014.2
It's been split off for a while, let's kill the code duplication.

Among other things, this fixes the systemd detection for the journal
logging.
2014-04-04 16:52:37 -04:00
Colin Walters 4bd8b5c850 switch: Always allow chronological downgrades
Ideally we'd have something a bit more strict, but...without
downloading the parentage, this is all we can do at the moment.
2014-04-04 16:51:37 -04:00
Colin Walters 76a976817f deploy: Call fsync() on parent directory before/after symlink swap
Let's be conservative here and try hard to ensure the symlink has the
correct content on disk.
2014-04-03 22:39:41 -04:00
Colin Walters 829e0382ab bootloader: fdatasync() bootloader configuration
Let's be a bit more conservative here and actually fdatasync() the
configurations we're generating.

I'm seeing an issue at the moment where syslinux isn't finding the
config sometimes, and while I don't think this is the issue, let's try
it.
2014-04-03 22:32:30 -04:00
Colin Walters cc8bacf8b4 bootloaders: Always write out bootloader config file
There was an attempted optimization to only write if changed, but this
is broken - we always write the bootloader config into a new
directory.

In theory we should only be writing if it changed, but let's not do a
broken optimization.
2014-04-03 21:08:03 -04:00
Colin Walters ac40de5773 deploy: Add a debug message for which bootloader is used
Debugging something...
2014-04-03 19:39:08 -04:00
Colin Walters 890d685c79 deploy: Add an assertion that deployment directory exists
I may be seeing something going wrong in rpm-ostree, just adding this
assertion for my own sanity.
2014-04-03 18:29:07 -04:00
Colin Walters 49d736c265 tool: End status line after pull
Otherwise we potentially get overlapped output.
2014-04-03 08:04:32 -04:00
Colin Walters edc8665206 fetcher: Honor http_proxy environment variable
https://bugzilla.gnome.org/show_bug.cgi?id=706809
2014-04-01 19:12:40 -04:00
Colin Walters 7bd00e3005 checkout: Always do chmod even in _MODE_USER
The previous commit here changed things so that we do mkdir(x, 0700),
then fchmod later only if we created the directory.

However the logic was incorrect; we still need to chmod even in
MODE_USER if we created the directory.
2014-04-01 13:07:17 -04:00
Colin Walters 956b18ace7 tools: Fix unintialized variables 2014-04-01 10:00:18 -04:00
Colin Walters 5bca569506 upgrade: Support --allow-downgrade again
This was unintentially dropped with the OstreeSysrootUpgrader rebase.
2014-03-31 22:27:56 -04:00
Colin Walters 913337c557 sysroot-upgrader: Check merge deployment, not ref for differences
Otherwise this broke atomicity; we could fetch/store the ref, then
crash, and then not upgrade the next time we tried upgrading.

The correct model is: the tree has changed if the new ref is different
from the merge deployment.
2014-03-31 14:41:13 -04:00
Colin Walters 1fd01077a5 ostree(1): Document GPG verification, link to ostree.repo(5) 2014-03-30 19:37:17 -04:00
Colin Walters e7e6215ae3 doc: Add ostree.repo and ostree.repo-config manual pages
In particular I wanted to document gpg-verify.
2014-03-30 19:32:51 -04:00
Colin Walters 8d13e5bb74 sysroot: Be more conservative with bootlink optimization
Trying to implement "rpm-ostree rollback", in the case where we have 2
deployments with the same bootconfig that we're reordering, we need to
write bootconfig, not just swap the bootlinks.
2014-03-28 21:06:23 -04:00
Colin Walters ad46467f2b tool: End status line in switch/upgrade too 2014-03-27 07:45:13 -04:00
Colin Walters 11ecf9267e libostree/upgrader: Don't pull if there is no remote
In this case we're just reading the local repo.
2014-03-27 07:36:50 -04:00
Colin Walters 9389b7961c libostree/upgrader: Add an API to retrieve an origin description
This will be used by "rpm-ostree upgrade".
2014-03-25 16:46:39 -04:00
Colin Walters 90883674a9 libostree/upgrader: Throw an error if osname is empty
This shouldn't happen; I'm just adding the check to force a rebuild.
2014-03-25 15:47:20 -04:00
Colin Walters 7baa600e23 Add an OstreeSysrootUpgrader API
This moves some utility code from the ostree tool into the shared
library, which will make it easier to consume by external tools.
2014-03-24 18:08:22 -04:00
Colin Walters ffb9d34671 Move basic commit API into ostree_sysroot_simple_write_deployment()
The admin commands had this shared in tool common, but we want to
encourage external programs to do this as well.
2014-03-23 11:36:09 -04:00
Colin Walters 6c463f0b20 Release 2014.3 2014-03-21 14:54:03 -04:00
Colin Walters a80872afa0 pull: Don't print, just use progress callback
More work on removing g_print() from the library.
2014-03-21 14:52:56 -04:00
Colin Walters 44c386c8f9 pull: End status line on error as well
This way we don't append the error to the current line.
2014-03-19 09:49:55 -04:00
Colin Walters b68f8f95f0 core: Add "admin instutil set-kargs"
This will be used by Anaconda as a convenience command to set the
bootloader arguments.
2014-03-19 09:49:55 -04:00
Colin Walters 4309b49866 libostree: Add ostree_sysroot_deployment_set_kargs()
It turns out people sometimes want to be able to change the kernel
arguments.  Add a convenient API to do so for the current deployment.

This will be used by Anaconda.
2014-03-19 09:49:55 -04:00
Colin Walters 9fed088d6e libostree: Fix ostree_deployment_clone() to also clone bootconfig
This way one can easily mutate it for a new deployment list.
2014-03-19 09:49:55 -04:00
Colin Walters 606918e763 libostree: Add ostree_bootconfig_parser_clone()
This will be necessary to fix ostree_deployment_clone(), but is
potentially useful on its own for other consumers.
2014-03-19 09:49:55 -04:00
Colin Walters 24b1e9c0ac Add "ostree admin instutil", move selinux-ensure-labeled there
There are going to be a few utilities that are only useful for
installers and disk image creation tools.  Let's not expose them all
at the toplevel; instead, hide them under "instutil".
2014-03-19 09:49:55 -04:00
Colin Walters a10ddca1da test-sysroot: Use GSystem to spawn subprocess
I was getting a weird hang in the installed tests with the shell as a
zombie process, not reaped by the parent, which was just stuck in
select() on the output pipes.  The thing is we don't actually want to
capture stdout/stderr, we just want to inherit.

GSystem.Subprocess makes that possible, so let's just use it now that
it's a proper installed library.
2014-03-19 09:15:38 -04:00
Colin Walters efd3a75daa Revert "Disable test-sysroot.test"
This reverts commit 73868a96d1.
2014-03-19 09:15:38 -04:00
Colin Walters fbc232d72b TODO: update 2014-03-15 19:05:52 -04:00
Colin Walters ba4f7ac7be Remove custom SELinux policy
This was a temporary hack until the requisite bits landed upstream in
the Fedora SELinux policy.
2014-03-15 13:16:14 -04:00
Colin Walters b6b7d37c9d libostree: Fix crash if output is not a tty
This was a recent regression.
2014-03-14 21:59:36 -04:00
Colin Walters cbc11f42e5 libostree: Add API to append a GPG signature
This will be used by rpm-ostree which needs to use an external program
to sign commits.
2014-03-14 21:57:48 -04:00
Colin Walters 4995352fe0 libostree: Add a better error if we fail to read keyring directory
I had accidentally put it in the -devel package and not noticed.
2014-03-14 14:42:46 -04:00
Colin Walters e11de9357c admin: selinux-ensure-labeled: new builtin
Code like rpm-ostree generates disk images directly.  In order to
ensure SELinux labeling is correct, it currently has a helper program
that runs over the deployment root, then over the whole disk and to
only set a label if none exist.

In order to make it easier to write installers such as Anaconda
without having them depend on rpm-ostree (or whatever other
build-server side program), pull in the helper code here.
2014-03-13 08:21:45 -04:00
Colin Walters 1f7f7382b1 deploy: Less usage of g_print
No need to spam the console here.
2014-03-11 13:16:38 -04:00
Colin Walters bf918f1586 libostree: Remove g_print() from bootloader code
No need to be so chatty.
2014-03-11 13:15:57 -04:00
Colin Walters 5c839f0267 pull: Drop some g_print(), replace others with async progress
We shouldn't g_print() from a library, particularly when the
expectation is that the client has an async progress set up.

This should fix the pull output extending the status line.
2014-03-11 09:36:54 -04:00
Colin Walters c2123bfc71 pull: Ensure temporary data that appears corrupted is deleted
If a MITM attacker (or just network corruption) causes a temporary
downloaded object in tmp/ to be corrupted, we'll end up
continually trying to commit it, and fail.

Fix this unlinking the temp file immediately after opening it.  This
will ensure that if we exit due to an error (or crash), the kernel
will clean up the space for us.

https://bugzilla.gnome.org/show_bug.cgi?id=725924
2014-03-07 19:36:55 -05:00
Colin Walters 19d36c540a packaging: Update internal spec file 2014-03-01 15:46:03 -05:00
Colin Walters cca4cf3e62 build: Add --enable-selinux-custom-policy
Don't use this.

It's just for me, and only temporarily until this stuff all lands in
the Fedora (and ideally upstream) selinux-policy.
2014-03-01 15:10:40 -05:00
Colin Walters 5bda4b4347 Release 2014.2 2014-03-01 13:55:55 -05:00
Colin Walters 79fa7ca692 Add /run/ostree-booted
The idea with this is that things like yum should be able to look for
it and determine whether or not they should assume that they can
change things on the system.

https://bugzilla.gnome.org/show_bug.cgi?id=725380
2014-02-28 03:49:25 -05:00
Colin Walters 4f04a55a5d upgrade/switch: Fix status line being overwritten with pull progress 2014-02-27 16:27:26 -05:00
Colin Walters d8852d099a checkout: Use fd-relative open of newly created directory
We were walking the full path again on our directories, no need to do
that.
2014-02-27 11:49:49 -05:00
Colin Walters b6704f8346 checkout: Only fchown/fchmod directories after we're done populating them
See https://mail.gnome.org/archives/ostree-list/2014-February/msg00020.html
2014-02-27 11:19:33 -05:00
Colin Walters 683a719b7e Update libgsystem, use it to set dirfd-relative xattrs on symlinks
This is a bit more efficient in that we're not walking full paths, and
it helps avoid security/reliability issues if an attacker (or just a
misbehaving process) has the ability to mutate paths in the middle.
2014-02-26 16:34:37 -05:00
Colin Walters 0eb8c37ecb upgrade: Properly set origin_refspec variable for resolve/printing
1) We were ignoring the remote, which is broken
2) We were printing NULL later on
2014-02-26 15:13:12 -05:00
Colin Walters c85682cb85 switch: Don't check whether revision matches
It's quite possible that say "buildmaster" and "smoketested" are the
same revision - but we should allow switching between them.
2014-02-26 15:12:08 -05:00
Colin Walters b762c2f8f1 pull: Remove explicit threading
Mixing async and threads has proved to be too much for my little mind.
It has race conditions that I've tried repeatedly to fix, but failed.

The threading here was scanning metadata objects - and there are
two parts to that:

1) Physically loading them from disk
2) Parsing them

Now #1 has been partially addressed by avoiding a storm of lstat() if
we're starting from a known working state.  If pull gets interrupted,
then we do need to rescan all objects.  Also, we can address this with
local metadata packfiles.

The other potentially slow bit is that we recurse across the metadata,
blocking the main thread.  We could ameliorate that in the future by
scheduling metadata parsing as idle "chunks".

Anyways, let's move the needle back to reliability, and readd speed
more carefully.

https://bugzilla.gnome.org/show_bug.cgi?id=706456
2014-02-21 15:04:23 -05:00
Colin Walters f2e0162846 upgrade: Refuse chronologically older commits unless --allow-downgrade
We don't want to allow MITM attackers to intercept upgrade requests
and provide clients with older OS versions vulnerable to security
flaws.

Only "ostree admin upgrade" gets this behavior for now - whether we
want to do it for "ostree admin switch" is another question.
2014-02-20 18:25:56 -05:00
Colin Walters 9fc9008af8 repo: Fix crash without SELinux policy enabled during commit 2014-02-20 14:19:49 -05:00
Colin Walters 071916196c build: Fix build without SELinux 2014-02-20 10:12:53 -05:00
Colin Walters 3337334be5 libostree: Split off SELinux OstreeSePolicy class
It's better if this is independent from the OstreeSysroot; for
example, a policy is active in a given deployment root at once, not
for a sysroot globally.

We can also collect SELinux-related API in one place.

Unfortunately at the moment there can be only one instance of this
class per process.
2014-02-19 08:43:45 -05:00
Colin Walters cc49096044 Add internal SELinux policy overrides
In the future, this will likely include an ostree_t domain.  For now,
this is just a few additional allow rules.
2014-02-19 08:40:19 -05:00
Colin Walters 53bd648dac manual-tests: New directory with custom test scripts
This is just a demo script.
2014-02-14 18:16:37 -05:00
Colin Walters 371cebf258 build: make "sudo make install" over existing install work 2014-02-13 10:57:51 -05:00