Commit Graph

4522 Commits

Author SHA1 Message Date
Dan Nicholson b825083549 tests/gpghome: Create revocation certificates for keys
These can then be imported during a test to revoke a key without trying
to go through the gpg --generate-revocation dialog. Note that these need
to go in a subdirectory of the homedir since `gpgkeypath` will try to
import every regular file in the homedir.
2020-01-24 13:02:09 -07:00
Dan Nicholson 63414e85c3 tests/libtest: Make temporary gpghome private
gpg prints a warning about unsafe permissions if the homedir is group or
world readable. This is just noise in the test logs, so appease it by
making the homedir 700.
2020-01-24 13:02:09 -07:00
Dan Nicholson 7f04c5d764 tests/libtest: Record long GPG key IDs and fingerprints
Use long GPG key IDs as it's safer and matches the format used by gpg
and gpgme. Add the associated fingerprints since these are needed by gpg
when manipulating keys.
2020-01-24 13:02:09 -07:00
Dan Nicholson c54a3b5daa lib/gpg: Prefer declare-and-initialize style
As noted in
https://github.com/ostreedev/ostree/pull/1872#discussion_r295408768.
2020-01-24 13:02:09 -07:00
OpenShift Merge Robot 8cc81126a1
Merge pull request #1993 from dbnicholson/f29-rpmostree-fixes
ci/rpmostree: Bump to 2019.4
2020-01-22 15:01:25 +01:00
Dan Nicholson a6994459c1 ci/rpmostree: Bump to 2019.4
The vmcheck tests in 2019.3 fail because of an SSH control socket issue
on overlayfs. This is fixed in 2019.4[1]. That has some other changes
such as using Python 3 in tests. The package dependencies have been
synced from the rpm-ostree CI for that.

Unfortunately, this is no longer a totally representative test of f29
since it has 2019.3 in updates. But that's the price you pay for
exercising someone else's CI from your own CI.

1. c89f81c138

Fixes: #1994
2020-01-21 20:33:58 -07:00
OpenShift Merge Robot 859dd00bfa
Merge pull request #1991 from dbnicholson/test-fixes
tests/core: Really pick C.UTF-8 locale
2020-01-21 21:13:02 +01:00
Dan Nicholson 5135a1e58a tests/core: Really pick C.UTF-8 locale
The case-ignoring regex `^(C|en_US)` will match any locale that starts
with `c`. On my system this is `ca_AD.utf8`, which breaks the test
suite. Instead, use a single regex that includes the joining `.` rather
than 2 separate regexes. This also changes `head` to use the `-n`
option, which has been preferred for at least 10 years in the coreutils
version and is supported by busybox as well.
2020-01-21 10:29:41 -07:00
Dan Nicholson 97c831dd5f bin/show: Add --print-sizes option to show sizes metadata
Use the new `ostree_commit_get_object_sizes()` API to read the
`ostree.sizes` commit metadata and print a summary.
2020-01-20 20:46:31 -07:00
Dan Nicholson 260bcd1193 core: Add ostree_commit_get_object_sizes API
This function parses the object listing in the `ostree.sizes` metadata
and returns an array of `OstreeCommitSizesEntry` structures.

Unfortunately, for reasons I don't understand, the linker wants to
resolve `_ostree_read_varuint64` from `ostree-core.c` even though it's
not used by `test-checksum.c` at all.
2020-01-20 20:46:29 -07:00
Dan Nicholson fcbb453443 core: Add OstreeCommitSizesEntry type
This will be used when reading out entries in the `ostree.sizes`
metadata. Each entry corresponds to an object in the metadata array.
2020-01-20 20:44:12 -07:00
Dan Nicholson 1bbe674d91 libarchive: Support commit sizes metadata
Call the helper to set the generate_sizes boolean so that object size
data is stored while writing the mtree.
2020-01-20 20:42:27 -07:00
John Hiesey 291e9da258 lib/commit: Include object type in sizes metadata
Append a byte encoding the OSTree object type for each object in the
metadata. This allows the commit metadata to be fetched and then for the
program to see which objects it already has for an accurate calculation
of which objects need to be downloaded.

This slightly breaks the `ostree.sizes` `ay` metadata entries. However,
it's unlikely anyone was asserting the length of the entries since the
array currently ends in 2 variable length integers. As far as I know,
the only users of the sizes metadata are the ostree test suite and
Endless' eos-updater[1]. The former is updated here and the latter
already expects this format.

1. https://github.com/endlessm/eos-updater/
2020-01-20 20:42:27 -07:00
Dan Nicholson a4592678aa tests/sizes: Check duplicate file doesn't add sizes entry
A duplicate file will resolve to the same object, so it shouldn't add
any entries to the sizes metadata.
2020-01-20 20:42:27 -07:00
Dan Nicholson 4f1b991246 tests/sizes: Test that sizes metadata is not reused
Ensure that the object sizes hash table is cleared after a commit and
not only when the repo is closed.
2020-01-20 20:42:27 -07:00
Dan Nicholson 1ea719b76b tests/sizes: Test sizes metadata with existing objects
Repeat the commit to make sure that the files are enumerated again for
the size metadata.
2020-01-20 20:42:27 -07:00
Dan Nicholson 44fb5e72a1 lib/commit: Make size entries for existing objects
If the object was already in the repo then the sizes metadata entry was
skipped. Move the sizes entry creation after the data has been computed
but before the early return for an existing object.
2020-01-20 20:42:27 -07:00
Dan Nicholson 8ec7d6322f lib/commit: Fix object sizes metadata for multiple commits
The object sizes hash table was only being cleared when the repo was
finalized. That means that performing multiple commits while the repo
was open would reuse all the object sizes metadata for each commit.

Clear the hash table when the sizes metadata is setup and when it's
added to a commit. This still does not fix the issue all the way since
it does nothing to prevent the program from constructing multiple
commits simultaneously. To handle that, the object sizes hash table
should be attached to the MutableTree since that has the commit state.
However, the MutableTree is gone when the commit is actually created.
The hash table would have to be transferred to the root file when
writing the MutableTree. That would be an awkward addition to
OstreeRepoFile, though. Add a FIXME to capture that.
2020-01-20 20:42:27 -07:00
Dan Nicholson 694b741a36 tests/sizes: Improve metadata validation
Ensure all 3 of the checksum, compressed size and uncompressed size are
correct. For repeatable objects, skip xattrs and use canonical
permissions for the commit. For the sizes, read a varint rather than
assuming they will be a single byte. To work around bugs in gjs with
byte array unpacking, manually build the array byte by byte. Split out
some helper functions to use in subsequent tests.
2020-01-20 20:42:27 -07:00
Dan Nicholson 37045b4b46 lib/commit: Only set generate_sizes for archive repos
Rather than checking throughout the code, only set the boolean when
appropriate.
2020-01-20 20:42:27 -07:00
OpenShift Merge Robot 20d84f40fa
Merge pull request #1987 from akiernan/us-switchroot-tests
Skip /var test if running with systemd and libmount
2020-01-16 16:47:41 +01:00
OpenShift Merge Robot c0d760bb0c
Merge pull request #1988 from akiernan/us-switchroot-installed
test-switchroot.sh: Find ostree-prepare-root in installed tests
2020-01-14 20:30:33 +01:00
Alex Kiernan d61183ce43 fixup! test-switchroot.sh: Find ostree-prepare-root in installed tests 2019-12-30 13:28:35 +00:00
Alex Kiernan e4db245bec test-switchroot.sh: Find ostree-prepare-root in installed tests
When running with installed tests, ostree-prepare-root (probably)
exists in /usr/lib. Add heuristics to look for it based on the directory
we're running from.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-12-29 14:24:30 +00:00
Alex Kiernan 87ccb400a2 tests: Skip /var test if running with systemd and libmount
If running with systemd and libmount then /var mounting is deferred for
systemd. Skip the relevant tests in this case as it will always fail.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-12-29 14:18:36 +00:00
Alex Kiernan 5c62a7e4d0 build: Expose systemd in OSTREE_FEATURES
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-12-29 13:27:29 +00:00
Alex Kiernan f6867358e2 test-switchroot.sh: Exclude /proc from file list
Since we're not interested in any file inside /proc, exclude it from the
file listing in our fake root thus avoiding failures when processes die
during our execution and find(1) can't then look inside those
directories.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-12-29 12:37:10 +00:00
Simon McVittie 4a3cae5aba New upstream version 2019.6 2019-12-14 10:47:26 +00:00
OpenShift Merge Robot b9a95afacc
Merge pull request #1767 from cgwalters/sysroot-mnt-namespace
Support mounting /sysroot (and /boot) read-only
2019-12-11 21:34:36 +01:00
Colin Walters 5af403be0c Support mounting /sysroot (and /boot) read-only
We want to support extending the read-only state to cover `/sysroot`
and `/boot`, since conceptually all of the data there should only
be written via libostree.  Or at least for `/boot` should *mostly*
just be written by ostree.

This change needs to be opt-in though to avoid breaking anyone.

Add a `sysroot/readonly` key to the repository config which instructs
`ostree-remount.service` to ensure `/sysroot` is read-only.  This
requires a bit of a dance because `/sysroot` is actually the same
filesystem as `/`; so we make `/etc` a writable bind mount in this case.

We also need to handle `/var` in the "OSTree default" case of a bind
mount; the systemd generator now looks at the writability state of
`/sysroot` and uses that to determine whether it should have the
`var.mount` unit happen before or after `ostree-remount.service.`

Also add an API to instruct the libostree shared library
that the caller has created a new mount namespace.  This way
we can freely remount read-write.

This approach extends upon in a much better way previous work
we did to support remounting `/boot` read-write.

Closes: https://github.com/ostreedev/ostree/issues/1265
2019-12-11 15:33:57 +00:00
Colin Walters 20daf96880 finalize-staged: Use the core option parsing to load sysroot
Prep for using the default mount namespace handling there that
will land as part of the read-only `/sysroot` and `/boot` work.
See https://github.com/ostreedev/ostree/issues/1265
2019-12-11 14:39:17 +00:00
OpenShift Merge Robot 8f6da76f0f
Merge pull request #1981 from cgwalters/release-2019.6
Release 2019.6
2019-12-09 14:26:55 -05:00
Colin Walters c820a6133b Post-release version bump 2019-12-09 16:18:28 +00:00
Colin Walters 978bd19f97 Release 2019.6
Nothing really big here, but let's get a release out
so some bigger things like ro-sysroot, signing, sizes can
bake in master for a bit.
2019-12-09 16:18:01 +00:00
OpenShift Merge Robot 89382d77b3
Merge pull request #1221 from cgwalters/pull-basic-auth
pull: Add support for basic auth
2019-12-03 06:41:08 -08:00
Colin Walters 36039759e2 pull: Add support for basic auth
This has been requested a few times for people delivering
OSTree content and wanting to do access control.
2019-12-02 22:48:33 +00:00
OpenShift Merge Robot c2651e759d
Merge pull request #1980 from ssssam/sam/readme-buildstream-tweak
README.md: Tweak text about BuildStream
2019-12-02 14:10:18 +01:00
Sam Thursfield a4e2b97e65 README.md: Tweak text about BuildStream
BuildStream no longer uses libostree internally (see
https://gitlab.com/BuildStream/buildstream/issues/387). It still
has first class support for interoperating with libostree repos.
2019-12-01 22:28:18 +01:00
OpenShift Merge Robot 437d2183a4
Merge pull request #1978 from agners/fix-build-race
Avoid race condition when building outside of source tree
2019-11-26 13:52:25 +01:00
Stefan Agner 77b27c07df Avoid race condition when building outside of source tree
When building outside of source tree it can happen that src/ostree/
does not exist (yet) when bison is called. This leads to an build
error like so:
  bison: src/ostree/parse-datetime.c: cannot open: No such file or directory

Make sure that src/ostree/ exists when parse-datetime.c is built.
2019-11-26 09:22:30 +01:00
OpenShift Merge Robot 1dff567f2c
Merge pull request #1976 from cgwalters/repo-finder-test
tests/repo-finder: Run realpath() on /tmp
2019-11-25 15:26:31 +01:00
Colin Walters 16bb294384 tests/repo-finder: Run realpath() on /tmp
This fixes running this test case inside
https://github.com/cgwalters/coretoolbox
2019-11-23 18:04:58 +00:00
OpenShift Merge Robot 7a7838b7a7
Merge pull request #1968 from ptomato/ostree-async-progress-copy-state
libostree: Add ostree_async_progress_copy_state()
2019-11-21 15:14:04 +01:00
Philip Chimento 54639c03ca libostree: Add ostree_async_progress_copy_state()
This allows copying the state from one OstreeAsyncProgress object to
another, atomically, without invoking the callback. This is needed in
libflatpak, in order to chain OstreeAsyncProgress objects so that you
can still receive progress updates when iterating a different
GMainContext than the one that the OstreeAsyncProgress object was
created under.

See https://github.com/flatpak/flatpak/pull/3211 for the application of
this API.
2019-11-20 12:24:26 -08:00
Philip Chimento 382ad59822 Bump version in symbols file
There were no new symbols in 2019.5 and this version didn't get bumped
when 2019.5 was released.
2019-11-20 11:18:36 -08:00
OpenShift Merge Robot 6ba5c57d59
Merge pull request #1909 from rfairley/rfairley-kargs-order-entries
lib/kernel-args: Store kernel args as key/value entries
2019-11-08 23:54:12 +01:00
OpenShift Merge Robot eb05ace3ed
Merge pull request #1971 from akiernan/us-tests-dir
build: create tests directory for split builds
2019-11-08 17:08:15 +01:00
OpenShift Merge Robot 4a992caaad
Merge pull request #1972 from cgwalters/bump-libglnx-20191108
Bump libglnx
2019-11-08 16:26:47 +01:00
Colin Walters 1e0761849c Bump libglnx
This has a few fixes, mainly I want to get this in
as prep for fs-verity.

Update submodule: libglnx

```
Alex Kiernan (1):
      macros: Add TEMP_FAILURE_RETRY for musl

Alexander Larsson (1):
      Add glnx_open_anonymous_tmpfile_full() allowing you to specify the directory

Colin Walters (8):
      Merge branch 'shutil-rm-rf-errprefix' into 'master'
      Merge branch 'us-temp-failure-retry' into 'master'
      Merge branch 'anonymous-tmpfile-dir' into 'master'
      Merge branch 'meson-older-compilers' into 'master'
      fdio: Add glnx_tmpfile_reopen_rdonly()
      Merge branch 'reopen-rdonly' into 'master'
      build-sys: Add libglnx-testlib.c to Automake
      Merge branch 'testlib-automake' into 'master'

Jonathan Lebon (1):
      Merge branch 'uchar' into 'master'

Simon McVittie (5):
      missing: Remove unused <uchar.h>
      Run the fdio test in its own temporary directory
      meson: Define HAVE_DECL_FOO to 0 if foo isn't declared
      Make the Meson build work on older compilers
      CI: Target a Fedora stable release

Will Thompson (3):
      Add meson.build files
      Document using this as a Meson subproject
      Add GitLab CI

```
2019-11-08 13:36:23 +00:00
Alex Kiernan f07838fa08 build: create tests directory for split builds
When `--disable-dependency-tracking` is in effect with separate build
directory, the tests directory isn't created as a result of the
dependency generation, which leads to a build race for the tests
directory being created and failures:

  Making all in .
  make[2]: Entering directory 'TOPDIR/build/tmp/work/riscv64-yoe-linux-musl/ostree/2019.5-r0/build'
  (echo '[Test]' > tests/test-local-pull-depth.sh.test.tmp; \
  echo 'Type=session' >> tests/test-local-pull-depth.sh.test.tmp; \
  echo 'Exec=env G_TEST_SRCDIR=/usr/libexec/installed-tests/libostree G_TEST_BUILDDIR=/usr/libexec/installed-tests/libostree /usr/libexec/installed-tests/libostree/test-local-pull-depth.sh' >> tests/test-local-pull-depth.sh.test.tmp; \
  mv tests/test-local-pull-depth.sh.test.tmp tests/test-local-pull-depth.sh.test)
  /bin/sh: tests/test-local-pull-depth.sh.test.tmp: No such file or directory
  /bin/sh: line 1: tests/test-local-pull-depth.sh.test.tmp: No such file or directory
  /bin/sh: line 2: tests/test-local-pull-depth.sh.test.tmp: No such file or directory
  mv: cannot stat 'tests/test-local-pull-depth.sh.test.tmp': No such file or directory
  make[2]: *** [Makefile:9282: tests/test-local-pull-depth.sh.test] Error 1

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
2019-11-08 06:51:36 +00:00