Commit Graph

2216 Commits

Author SHA1 Message Date
Colin Walters 3441a48c58 checkout: Ensure copies of unreadable usermode checkouts are readable
The extreme special case of "zero mode" files like `/etc/shadow`
comes up again.  What we want is for "user mode" checkouts to
override it to make the file readable; otherwise when operating
as non-root without `CAP_DAC_OVERRIDE` it becomes very difficult
to work with.

Previously, we were hardlinking these files, but then it intersects
with *another* special case around zero sized files, which is
*also* true for `/etc/shadow`.

Trying to avoid hardlinking there unveiled this bug - when
we go to do a copy checkout, we need to override the mode.
2020-09-16 15:14:54 +00:00
Phaedrus Leeds c4f26bfdc8 Avoid shadowing local variables
This should help with code readability.

Fixes https://github.com/ostreedev/ostree/issues/2194
2020-09-13 10:08:02 -07:00
Colin Walters 779a901780 commit: Tighten scope of two variables
Prep for adding `-Wshadow` fixes.
2020-09-13 14:58:24 +00:00
Alexander Larsson b7d1a9746b Update the symbols files to match that we're now on 2020.6 2020-09-11 12:35:32 +02:00
Alexander Larsson da853a1783 ostree-repo-pull.c: Extract mirrorlist generation to helper
This code was duplicated in 3 places, so move it to a single place
to clean things up.
2020-09-11 12:03:29 +02:00
Alexander Larsson 155b215cd8 Minor cleanup of _ostree_repo_remote_new_fetcher()
Instead of open coding the extra_headers and append_user_agent
setting everywhere we do this in the constructor.
2020-09-11 12:03:28 +02:00
Alexander Larsson 3957bff0cb Inline repo_remote_fetch_summary
This was only used in one place, and (especially with the simplification
with GMainContextPopDefault) and the one caller doesn't really do
much more than call the helper. Additionally, what little it does (saving
the result in the cache) is inherently tied to how the helper work,
and will become even more so when we support summary indexes.

This is a preparatory cleanup for supporting summary indexes. It
doesn't change any behaviour and passes make check on its own.
2020-09-11 12:03:28 +02:00
Alexander Larsson 4b9e712e82 repo_remote_fetch_summary: Use GMainContextPopDefault
This allows us to drop the "goto out" use and clean up this function.
2020-09-11 12:03:28 +02:00
Alexander Larsson f74bc8dd3d fetch_summary_with_options: drop unnecessary "goto out" use 2020-09-11 12:03:28 +02:00
Alexander Larsson 1f1ef4c989 Break out the signature verification code into a helper function
This changes nothing in the behaviour, but we want to later re-use
this when we also verify the summary index.
2020-09-11 12:03:28 +02:00
Alexander Larsson 598adc457f deltas: Break out _ostree_repo_static_delta_superblock_digest() helper
This loads and makes a digest for a delta superblock. The previous
code was used when generating the deltas section in the summary
file. This changes nothing, but is in preparation for using similar
formats in a separate delta index file.
2020-09-11 12:03:28 +02:00
Alexander Larsson 32014d99e6 Add and use ot_checksum_bytes helper
This removes some duplicated code (and will be use even more later).
2020-09-11 12:00:05 +02:00
Alexander Larsson dddb449d2c pull: Actually mmap summary files
The change in cbf1aca1d5c08d2f40832d16670484ba878d95fb actually
only mmaps the signature file, not the summary. This change makes
use mmap both, as well as extract the cache loading into a helper
function that we will later use in more places.
2020-09-11 12:00:02 +02:00
Alexander Larsson 85accb84e8 pull: Break out _ostree_repo_save_cache_summary_file() helper
This is a minor cleanup as its just called twice from
_ostree_repo_cache_summary(). However, later code will need it in more
places.
2020-09-11 11:53:38 +02:00
Alexander Larsson bb2649a8c0 Fix leak when signing
_ostree_detached_metadata_append_gpg_sig() was returning a floating
ref, but all users were using g_autoptr. Fix it by adding a ref-sink.
2020-09-11 11:45:33 +02:00
Alexander Larsson 74bae256fe list-deltas: Don't break on non-subdir entries
ostree_repo_list_static_delta_names() tried to validate that
any second-level directory element was a directory, but there was
a cut-and-paste issue, and it used `dent->d_type` instead
of `sub_dent->d_type`.

This fixes the code, but all old ostree versions will break if
there are non-directories in a subdirectory of the deltas directory
in the repo, so be wary.
2020-09-11 11:45:24 +02:00
Colin Walters a1bd29f245 deploy: Add some error prefixing around xattr setting
Looking at
https://github.com/coreos/coreos-assembler/issues/1703
a user is getting a bare:
`error: fsetxattr: Permission denied`

I don't think it's these code paths since a deploy
isn't happening but on inspection I noticed we didn't
have error prefixing here.
2020-09-09 12:34:44 +00:00
Felix Krull f4d0b17080 lib: mark out parameters as out parameters 2020-08-26 22:32:47 +02:00
Felix Krull d5b8929017 lib: add some missing version tags 2020-08-26 22:32:47 +02:00
Colin Walters 06ed04a816 linuxfsutil: Pass int to ioctl, not long
Otherwise it will fail on big-endian architectures like s390x.
Ref https://bugzilla.redhat.com/show_bug.cgi?id=1867601
2020-08-21 17:41:32 +00:00
OpenShift Merge Robot c61ff03304
Merge pull request #2175 from cgwalters/coverity-2020.5
Two small Coverity fixes
2020-08-19 16:46:49 +02:00
Colin Walters 1eab48363b pull: Assign idle_src variable before calling unref()
This should pacify Coverity, and also just "reads" better too.
2020-08-18 23:34:57 +00:00
OpenShift Merge Robot 199562fc14
Merge pull request #2149 from stb-tester/boot-self-symlink
sysroot: Support /boot on root or as seperate filesystem for syslinux and u-boot
2020-08-19 01:08:30 +02:00
Jonathan Lebon 10a68cd26b lib/deploy: Clarify comment re. staging API
Don't mention deprecation in the description for
`ostree_sysroot_deploy_tree` since there are legitimate use cases for it
(e.g. to create the first deployment via `ostree admin deploy`).

Instead, make the comment clearly redirect to the staging API when
booted into the sysroot.
2020-08-17 09:48:57 -04:00
Jonathan Lebon 5de3a9759f lib/deploy: Drop unneccessary function arg 2020-08-17 09:48:57 -04:00
Jonathan Lebon e4fb7d3bb1 lib/cleanup: Drop unnecessary GEqualFunc cast 2020-08-17 09:48:57 -04:00
Jonathan Lebon 74bd136286 lib/deploy: Simplify deployment creation
Minor cleanup; we were declaring a superfluous variable.
2020-08-17 09:48:57 -04:00
Jonathan Lebon 61c544df1b lib/deploy: Avoid shadowing variable
There's already a `boot_relpath` variable in the outside scope.
2020-08-17 09:48:57 -04:00
Jonathan Lebon 52a6224606 lib/deploy: Clean up kargs override handling
Tighten up how we handle kargs here so it's more clear. When we call
`sysroot_finalize_deployment`, any karg overrides have already been set
on the bootconfig object of the deployment. So re-setting it here is
redundant and confusing.
2020-08-17 09:48:57 -04:00
Colin Walters f2773c1b55 Add "transient" unlock
I was thinking a bit more recently about the "live" changes
stuff https://github.com/coreos/rpm-ostree/issues/639
(particularly since https://github.com/coreos/rpm-ostree/pull/2060 )
and I realized reading the last debates in that issue that
there's really a much simpler solution; do exactly the same
thing we do for `ostree admin unlock`, except mount it read-only
by default.

Then, anything that wants to modify it does the same thing
libostree does for `/sysroot` and `/boot` as of recently; create
a new mount namespace and do the modifications there.

The advantages of this are numerous.  First, we already have
all of the code, it's basically just plumbing through a new
entry in the state enumeration and passing `MS_RDONLY` into
the `mount()` system call.

"live" changes here also naturally don't persist, unlike what
we are currently doing in rpm-ostree.
2020-08-07 18:57:56 +00:00
Philip Withnall f5da67d78a pull: Add summary-{,sig-}bytes options to ostree_repo_pull()
These allow the `summary` and `summary.sig` files to be cached at a
higher layer (for example, flatpak) between related pull operations (for
example, within a single flatpak transaction). This avoids
re-downloading `summary.sig` multiple times throughout a transaction,
which increases the transaction’s latency and introduces the possibility
for inconsistency between parts of the transaction if the server changes
its `summary` file part-way through.

In particular, this should speed up flatpak transactions on machines
with high latency network connections, where network round trips have a
high impact on the latency of an overall operation.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-07 16:12:15 +01:00
Philip Withnall bd68c7dfd7 pull: Improve formatting of pull options in documentation
Backticks improve all things.

Signed-off-by: Philip Withnall <withnall@endlessm.com>
2020-08-07 16:11:44 +01:00
Colin Walters 0d91206a62 Post-release version bump 2020-07-22 14:09:26 +00:00
Colin Walters 901747f985 Release 2020.4
A lot of stuff here, new signing API is the biggest.  Let's
get a release out.
2020-07-21 21:48:51 +00:00
OpenShift Merge Robot 12a9161a79
Merge pull request #2152 from cgwalters/pull-fdatasync
pull: Add --per-object-fsync
2020-07-20 13:55:26 -04:00
Colin Walters a615d35762 pull: Add --per-object-fsync
This is the opposite of
https://github.com/ostreedev/ostree/issues/1184

Motivated by OpenShift seeing etcd performance issues during
OS updates: https://github.com/openshift/machine-config-operator/issues/1897

Basically, if we switch to invoking `fsync()` as we go, it makes
ostree performance worse (in my tests, 31s to write 2G versus 7s if we
delay sync) but it avoids *huge* outliers in `fsync()` time for etcd.
2020-07-18 14:59:01 +00:00
Colin Walters 5b75358357 pull: Also append bytes written
This is very useful information that we get from the transaction
stats.  Append it to the final display if we're not inheriting
the transaction.
2020-07-17 17:03:08 +00:00
William Manley 0ced9fde76 sysroot: Support /boot on root or as seperate filesystem for syslinux and u-boot
We use a similar trick to having a `sysroot -> .` symlink on the real root
here to support both /boot on root as well as on a separate filesystem.  No
matter how it's mounted `/boot/xyz` will always refer to the file you'd
expect.

This is nicer than my previous attempts at this because there's no
configuration nor auto-detection required.
2020-07-15 20:37:49 +01:00
Frédéric Danis 892e9acf25 lib/deltas: convert ostree_repo_static_delta_generate to new style
The "new style" code generally avoids `goto err` because it conflicts
with `__attribute__((cleanup))`.  This fixes a compiler warning.

Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
2020-07-03 13:35:39 +00:00
Jonathan Lebon 1d755f62af lib/repo: Handle EACCES for POSIX locking
If `glnx_make_lock_file` falls back to `flock`, on NFS this uses POSIX
locks (`F_SETLK`). As such, we need to be able to handle `EACCES` as
well as `EAGAIN` (see `fnctl(2)`).

I think this is what coreos-ostree-importer has been hitting, which runs
on RHEL7 in the Fedora infra and does locking over an NFS share where
multiple apps could concurrently pull things into the repo.
2020-06-30 15:09:41 -04:00
OpenShift Merge Robot fd8ecdf047
Merge pull request #2131 from cgwalters/sign-success
signapi: Change API to also return a success message
2020-06-24 18:24:41 -04:00
OpenShift Merge Robot adcd261596
Merge pull request #2001 from agners/multi-device-tree
deploy: support devicetree directory
2020-06-24 03:02:18 -04:00
OpenShift Merge Robot 9067c1ca49
Merge pull request #2136 from cgwalters/doc-timestamp
core: Add documentation for ostree_commit_get_timestamp()
2020-06-19 13:57:33 -04:00
Colin Walters 935f879542 sysroot: Remove unimplemented ostree_sysroot_lock_with_mount_namespace
This came in with 5af403be0c but
was never implemented.

I noticed this now because the Rust ostree bindings generate a
wrapper for it which the linker tries to use.
2020-06-18 21:19:43 +00:00
Colin Walters c87a4c71bc core: Add documentation for ostree_commit_get_timestamp()
Working on some rpm-ostree bits and was going to pass
this to the `chrono` crate and I forgot the format, went
to look at the docs.  Oops.
2020-06-18 20:57:53 +00:00
OpenShift Merge Robot 33c95812bb
Merge pull request #2132 from cgwalters/remove-unused-symbol
libostree-devel.sym: Remove nonexistent stub symbol
2020-06-17 16:18:26 -04:00
Colin Walters 9336837194 libostree-devel.sym: Remove nonexistent stub symbol
This should have been removed when we added symbols to this list.
2020-06-17 19:38:26 +00:00
Denis Pynkin ede93dc2ef sign/ed25519: fix return value if no correct keys in file
Fix the return value if file doesn't contains correct public key(s).

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-06-17 19:45:05 +03:00
Denis Pynkin ce73876389 sign/ed25519: fix the abort in case of incorrect public key
We need to check the size of public key before trying to use it.

Signed-off-by: Denis Pynkin <denis.pynkin@collabora.com>
2020-06-17 19:44:56 +03:00
Colin Walters 36258036ae signapi: Change API to also return a success message
This is the dual of 1f3c8c5b3d
where we output more detail when signapi fails to validate.

Extend the API to return a string for success, which we output
to stdout.

This will help the test suite *and* end users validate that the expected
thing is happening.

In order to make this cleaner, split the "verified commit" set
in the pull code into GPG and signapi verified sets, and have
the signapi verified set contain the verification string.

We're not doing anything with the verification string in the
pull code *yet* but I plan to add something like
`ostree pull --verbose` which would finally print this.
2020-06-17 00:33:47 +00:00