Commit Graph
100 Commits
Author SHA1 Message Date
Colin Walters e69321176d gitignore: Add ci-build/
Generated by our scripts.
2022-05-31 11:42:42 -04:00
Colin Walters 199ef821c8 ci: Add a codestyle.sh
This matches what we do in rpm-ostree.  Move the `glnx_fd_close`
bits out of `make syntax-check` in preparation for dropping that.
2022-05-31 11:42:42 -04:00
Colin Walters aea20704b5 ci: Drop an unused files
Cruft from previous CI flows.
2022-05-31 11:42:42 -04:00
Colin Walters 2838f42eec ci: Drop libsoup build in jenkins
GH actions is fast zero cost system that covers these build matrix
things well.  Let's keep our Fedora CI system doing more of the
qemu heavy lifting.
2022-05-31 10:44:41 -04:00
Colin Walters 2363d3ad98 Merge pull request #2605 from saqibali-2k/pr/bls-append
ostree-systroot-deploy: parse bls-append-except-default key
2022-05-25 11:33:10 -04:00
Colin Walters ea0be27bd3 Merge pull request #2612 from HuijingHei/deploy-karg-delete
Teach --karg-delete option to ostree admin deploy
2022-05-24 11:18:54 -04:00
Colin Walters fe4ca42760 Merge pull request #2580 from nikita-dubrovskii/sdboot_partition
s390x: generate sd-boot at its own partition
2022-05-23 08:41:35 -04:00
Colin Walters 1c201d6be4 ci: Move codestyle and min build to GH actions
Part of general momentum to leave heavy lifting (e.g. VM tests)
to Jenkins and Prow and use the cheaper/faster GH actions for
plain builds.
2022-05-17 08:11:09 -04:00
Colin Walters 2106f12637 Merge pull request #2608 from cgwalters/fanalyzer-fixes
ci: Drop gcc `-fanalyzer` by default
2022-05-16 15:55:35 -04:00
Colin Walters 87cf46bbd9 ci: Drop all runAsUser: 0 in builds
This triggers the new git warning; I think this is only
here because long ago we used coreos-assembler as a buildroot,
but that's long since fixed.
2022-05-16 12:37:15 -04:00
Colin Walters 41b9f8ff97 ci: Drop gcc -fanalyzer by default
There's a pile of new warnings in GCC 12 from F36, I dug into
them but most of them seem inscrutable - I am not seeing the bugs.

For now to unblock further work, stop using this flag.
2022-05-16 11:00:08 -04:00
Colin Walters 6a055f802f sysroot: Strengthen several g_return_if_fail into g_assert()
Similar to prior changes; motivated by testing out an updated
`gcc -fanalyzer`.
2022-05-16 11:00:08 -04:00
Colin Walters e27fed45fd sysroot: Quiet one gcc -fanalyzer warning
I don't think this can happen in practice, but add an assertion
to verify it.
2022-05-15 16:03:01 -04:00
Colin Walters 9f8d026f82 ci: use cargo-deny
Copied from https://github.com/ostreedev/ostree-rs-ext/pull/291

Part of unifying our CI.
2022-05-10 17:14:01 -04:00
Colin Walters 891c7dffa7 Merge pull request #2575 from cgwalters/merge-ostree-rs
Merge ostree-rs into this repository
2022-05-09 16:41:08 -04:00
Colin Walters 73d3c16a9d Merge pull request #2601 from cgwalters/expose-get-xattrs
Add APIs to get xattrs from disk
2022-05-09 15:00:53 -04:00
Colin Walters 4a9a2f0b44 Merge pull request #2602 from cgwalters/fix-docs-ci
ci: Work around GH actions `container:` + git + security bug
2022-05-09 15:00:00 -04:00
Colin Walters 60404565e5 rust-bindings: Remove some unused CI/test bits
We're not using Vagrant or Gitlab, and our container flow is
different.
2022-05-09 14:53:40 -04:00
Colin Walters e9141e97c1 ci: Move rust-bindings CI to toplevel
It should replace our stub one.
2022-05-09 14:52:46 -04:00
Colin Walters ee2c31badd tests/inst: Add .gitignore
Need this now that it is it's own workspace.
2022-05-09 14:52:26 -04:00
Colin Walters 9db45cad5a ci: Work around GH actions container: + git + security bug
See https://github.com/actions/checkout/issues/760
2022-05-09 14:46:41 -04:00
Colin Walters 558d966420 tests/inst: Fix install rules for ostree-rs merger
`tests/inst` became its own workspace.
2022-05-09 12:57:40 -04:00
Colin Walters f79b2cea91 Add APIs to get xattrs from disk
I'm aiming to do some more work on the Rust side around `fsck`
like functionality, and this is a useful primitive.  There isn't
a great Rust crate for xattrs, and I think it's better to share this
code.
2022-05-09 12:33:38 -04:00
Colin Walters d72f1684a0 cfg.mk: Don't even look at rust-bindings/
It's really tempting to remove `make syntax-check`, it has very
very rarely found any real problems.

But anyways, just exclude all the binding code because it trips
up random problems we simply don't care about like mentions of
`O_NDELAY` in the `GLib-2.0.gir`.
2022-05-06 16:46:46 -04:00
Colin Walters ffb3b2bc7a docs/ima: Also link the SUSE docs
They have a lot of stuff there.
2022-05-06 14:43:58 -04:00
Colin Walters 252060906b build-sys: Adjust for merge of ostree-rs
Fix up the paths for the crates now that the Rust bindings are in
`rust/`.

We can't today include the test suite because it depends on `ostree-rs-ext`
which would make everything circular.

(Building that now requires a separate `cd tests/inst && cargo build`)
2022-05-06 12:53:57 -04:00
Colin Walters a5ef4cd5cc Add a repo() accessor to TransactionGuard
I want to write APIs that *require* the caller to have set up
an ostree transaction.  It's natural to require passing a guard
to do so.  But then we want an accessor for the repo.
2022-05-06 12:53:57 -04:00
Colin Walters 1541c5eb2e lib: Run cargo fmt
Prep for merge into ostree, where we want to run `cargo fmt` checks
in CI.
2022-05-06 12:53:57 -04:00
Colin Walters 887f5b09be repo: Add query_file API
The underlying `ostree_repo_load_file()` API has the caller pass
`NULL` for output arguments it doesn't want.  This isn't sanely
bindable in Rust - what the generator does is always request
all values, but maps them all to `Option<T>`.

The main cases are where a user wants either metadata, or both
metadata and content.  This API gives just metadata; it's a
bit more efficient as we don't need to open the file, and doesn't
require the caller to `unwrap()`.
2022-05-06 12:53:57 -04:00
Colin Walters 1199ae9bbc Add manual bindings for MutableTree reading
I'm trying to debug a problem in ostree-rs-ext, and it's
handy to be able to do `dbg!(mtree.copy_files())`.
2022-05-06 12:53:57 -04:00
Colin Walters 46b4a12b23 Release 0.13.6 2022-05-06 12:53:57 -04:00
Colin Walters ecbe3ba0f8 Fast-track fix for ostree_gpg_verify_result_get_all()
This cherry picks just the changes from
https://github.com/ostreedev/ostree/pull/2537

We don't need to wait to respin a new ostree release just
for this.
2022-05-06 12:53:57 -04:00
Colin Walters dde1a7b43f Release 0.13.5 2022-05-06 12:53:57 -04:00
Colin Walters 9250effa8f Add COMMIT_META_CONTAINER_CMD constant
Today we hardcode `/bin/bash` in
https://github.com/coreos/coreos-assembler/blob/2088d24884771093101d95f915c921505128ef76/src/cmd-build#L405

But that breaks the concept of a bidirectional bridge between
container image and ostree commit because this little bit of
knowledge is encoded at the buildsystem side.

This metadata key is intended to be written into an ostree commit,
and then we will use it automatically in `container encapsulate`.

The "source of truth" for this key will hence be able live in the same
place that's generating the ostree commit.

The more "proper" place for this is probably alongside the other
constants in the libostree core C code.  But that's tedious and
slow to release.  And Rust is the future.  And we've been slowly
adding more "core ostree" functionality here.
2022-05-06 12:53:57 -04:00
Colin Walters 19224a411a repo: Add two more cap-std APIs
Followup to the previous PR.  I realized now with `io_lifetimes`
we can offer a safe `dfd_borrow()` that *borrows* the file descriptor
for the repository.  (In contrast to the current `.dfd()` that returns
the raw version)

Building on that, add another API that re-acquires a `Dir` instance.
(In the future in theory we could optimize this more by knowing
 whether or not the repo was constructed via cap-std, and perhaps
 in theory synthesize a `&Dir` reference, but I don't think we
 need that now)
2022-05-06 12:53:57 -04:00
Colin Walters 6940896c4e Add a cap-std-apis feature with open/create
I'm trying to make more use of `cap-std` in our stack, and
this will be a key enabling API.

Actually a notable side benefit of this is that we don't need
to teach the ostree C code itself to use `openat2`, we inherit
cap-std's setup.

All of the internal ostree code using the prior `openat()` should
continue to work.

I only did basic sanity checking of this; there may be bugs
in other APIs.
2022-05-06 12:53:57 -04:00
Colin Walters 4f7eea6aa7 Release 0.13.4 2022-05-06 12:53:57 -04:00
Colin Walters d1731d0ea8 repo: Add an API to read and parse directory metadata
The fact that the uid/gid/mode are big endian bit me when I was
trying to parse this "by hand" in ostree-rs-ext.

Let's add a footgun-free API for this.

(And yeah, we should probably do the same for the other variant types)
2022-05-06 12:53:57 -04:00
Colin Walters 440d872f68 repo: Add require_rev method
The `resolve_rev` C method should really have been
`resolve_rev_optional` from the start - it is more obviously wrong
in Rust because the input parameter `allows_noent` controls
whether the returned `Option` can ever be `None`.

I debated adding this to the C bindings, and may still do so,
but eh it's faster to write + ship in Rust, and the future of ostree is
Rust anyways.
2022-05-06 12:53:57 -04:00
Colin Walters faaf0457fd (cargo-release) start next development iteration 0.13.4-alpha.0 2022-05-06 12:53:57 -04:00
Colin Walters 34147475b5 (cargo-release) version 0.13.3 2022-05-06 12:53:57 -04:00
Colin Walters 0432bd48b9 Bump ostree-sys version 2022-05-06 12:53:57 -04:00
Colin Walters 99ac68cb31 Update to 2021.5 2022-05-06 12:53:57 -04:00
Colin Walters 349933ab16 (cargo-release) version 0.13.3-alpha.1 2022-05-06 12:53:57 -04:00
Colin Walters c3141df56d (cargo-release) version 0.13.2 2022-05-06 12:53:57 -04:00
Colin Walters f8852ca945 repo: Add auto_transaction and TransactionGuard
This gives auto-cancelling semantics on `Drop`, plus a nicer
`.commit()` method on the transaction.

Matches the currently private `_OstreeRepoAutoTransaction` in the C
library.
2022-05-06 12:53:57 -04:00
Colin Walters 69950574f7 (cargo-release) start next development iteration 0.13.2-alpha.0 2022-05-06 12:53:57 -04:00
Colin Walters 507787161c (cargo-release) version 0.13.1 2022-05-06 12:53:57 -04:00
Colin Walters 955f0ddb9d repo: Expose dfd_as_file()
The `dfd()` API returns just an integer.  Add a safe API that
makes a copy of the fd.  What we really want here is `BorrowedFd` from
https://github.com/rust-lang/rfcs/blob/master/text/3128-io-safety.md
but that isn't here yet.
2022-05-06 12:53:57 -04:00
Colin Walters e33767cc2a Make SePolicy have Send
It's safe to send between threads, and I want to do so
in ostree-rs-ext to send to a tokio worker thread.
2022-05-06 12:53:57 -04:00
Colin Walters 7b47de7a14 lib: Reexport libc::AT_FDCWD
Useful with `Repo::open_at()`.

Right now ostree-rs-ext pulls in libc for this and `fgetxattr`,
but the latter should go into nix.
2022-05-06 12:53:57 -04:00
Colin Walters 2bfbfe3c6f sys: Release 0.9.0
Should have been bumped in the previous commit.
2022-05-06 12:53:57 -04:00
Colin Walters 09ef16fdbf Release 0.13
Just an update to support libostree v2021.4, but bumping semver
because a few APIs (correctly) gained `Option<T>`.
2022-05-06 12:53:57 -04:00
Colin Walters b17f3b37f0 Release 0.12.3
No major changes, just exporting the `ffi` bits.
2022-05-06 12:53:57 -04:00
Colin Walters 0f7a1d9c0c Cargo.toml: Bump to glib 0.14.4
Not strictly required for this repo, but it has the new variant
bindings we want in ostree-rs-ext.
2022-05-06 12:53:57 -04:00
Colin Walters f9a91bfabd lib: Export ffi too
Matching how gtk-rs does it.  Right now rpm-ostree does depend
on interacting with `ostree-sys` via the cxxrs bits.
2022-05-06 12:53:57 -04:00
Colin Walters 9a5f14ce68 Release 0.12.2
```
Colin Walters (8):
      Add 2021.3 feature
      Re-export glib, gio
      Deny unused results, warn on missing docs (except auto/)
      Add new GLib 0.14 variant types for metadata types
      Fix build with --features=v2021_3, use in CI by default
      Add more documentation for --features=v2021_3
      Use glib-sys via re-exported `glib::ffi` (and similar for gio)
      Release 0.12.2

Luca BRUNO (5):
      cargo: fix version in features chain
      lib: fix 'dox' feature
      cargo: point to docs.rs and clean up
      ostree: release 0.12.1
      cargo: bump ostree-sys to 0.8.1
```
2022-05-06 12:53:57 -04:00
Colin Walters 9b57bda607 Use glib-sys via re-exported glib::ffi (and similar for gio)
In general only `-sys` crates should depend on other `-sys`
crates.  IOW for us, `ostree-sys` depends on `glib-sys`.

By using the re-export, we avoid needing to keep a version lock
between `glib` and `glib-sys` in our main crate.  And similar
is true of our higher level reverse dependencies (e.g. `ostree-rs-ext`).

Also weaken our dependency to `0.14` as that's clearer.
2022-05-06 12:53:57 -04:00
Colin Walters 709b35bf11 Fix build with --features=v2021_3, use in CI by default
It's a huge trap for us not to build with the latest ostree feature
on, I didn't have my IDE configured for it, and CI didn't have
it on.

The previous bump to glib 0.14 broke the Sign code.
2022-05-06 12:53:57 -04:00
Colin Walters abec2a9e34 Add more documentation for --features=v2021_3
My previous pass was at the default feature level.
2022-05-06 12:53:57 -04:00
Colin Walters 83c829eaad Add new GLib 0.14 variant types for metadata types
This way it's more convenient for downstream crates like ostree-rs-ext
to convert loaded variants.

TODO: Can we add a feature for the `gvariant` crate and expose via
that too?
2022-05-06 12:53:57 -04:00
Colin Walters 48e0d334b8 Deny unused results, warn on missing docs (except auto/)
And add basic docs for our manually implemented functions.
2022-05-06 12:53:57 -04:00
Colin Walters deedffde06 Re-export glib, gio
Re-export our dependencies.  See https://gtk-rs.org/blog/2021/06/22/new-release.html
"Dependencies are re-exported".  Users will need e.g. `gio::File`, so this avoids
them needing to update matching versions.

Closes: https://github.com/ostreedev/ostree-rs/issues/12
2022-05-06 12:53:56 -04:00
Colin Walters 6303229c4e Add 2021.3 feature
(We should add doing this as a SOP for ostree releases)
2022-05-06 12:53:56 -04:00
Colin Walters 04a42dff1e Bump versions
The glib 0.14 change is semver incompatible.
2022-05-06 12:53:56 -04:00
Colin Walters 8a5ac02822 Update to glib 0.14
An intimidating spam of compiler errors at the start, but the
biggest was handling the new convention of `ostree_sys::` => `ffi::`.

This will require a semver bump of course.
2022-05-06 12:53:56 -04:00
Colin Walters f276c040bd ci: Fix buildroot to use new official image
Which is maintained and has updated rust.
2022-05-06 12:53:56 -04:00
Colin Walters ae189bec80 Mark src/auto/* as generated 2022-05-06 12:53:56 -04:00
Colin Walters bd843b2eae Bump sys version requirement 2022-05-06 12:53:56 -04:00
Colin Walters 42110ce01b Bump versions 2022-05-06 12:53:56 -04:00
Colin Walters 7d15179670 Update to 2021.2
Sync to https://github.com/ostreedev/ostree/releases/tag/v2021.2
2022-05-06 12:53:56 -04:00
Colin Walters 9bb0dd3c4d ci: Add GH action to build 2022-05-06 12:53:56 -04:00
Colin Walters b5496f70f6 repo: Add concurrency=send
Ultimately a repo is just a file descriptor wrapper with some
cached data, etc.  We can send it between threads, much like how
`gio::File` is `Send`.

Motivated by trying to write to a repo from a separate thread
in https://github.com/cgwalters/ostree-container
2022-05-06 12:53:56 -04:00
Colin Walters 3583143cf0 configure: post-release version bump 2022-05-04 13:51:45 -04:00
Colin Walters 0554beb4c2 Release 2022.3 2022-05-04 13:50:06 -04:00
Colin Walters d3d3e4ea13 Add an ostree-boot-complete.service to propagate staging failures
Quite a while ago we added staged deployments, which solved
a bunch of issues around the `/etc` merge.  However...a persistent
problem since then is that any failures in that process that
happened in the *previous* boot are not very visible.

We ship custom code in `rpm-ostree status` to query the previous
journal.  But that has a few problems - one is that on systems
that have been up a while, that failure message may even get
rotated out.  And second, some systems may not even have a persistent
journal at all.

A general thing we do in e.g. Fedora CoreOS testing is to check
for systemd unit failures.  We do that both in our automated tests,
and we even ship code that displays them on ssh logins.  And beyond
that obviously a lot of other projects do the same; it's easy via
`systemctl --failed`.

So to make failures more visible, change our `ostree-finalize-staged.service`
to have an internal wrapper around the process that "catches" any
errors, and copies the error message into a file in `/boot/ostree`.

Then, a new `ostree-boot-complete.service` looks for this file on
startup and re-emits the error message, and fails.

It also deletes the file.  The rationale is to avoid *continually*
warning.  For example we need to handle the case when an upgrade
process creates a new staged deployment.  Now, we could change the
ostree core code to delete the warning file when that happens instead,
but this is trying to be a conservative change.

This should make failures here much more visible as is.
2022-04-26 13:02:46 -04:00
Colin Walters 98587a72db Merge pull request #2588 from cgwalters/ostree-and-ima
docs: Add new IMA document
2022-04-21 18:13:07 -04:00
Colin WaltersandJonathan Lebon b070e0f54a Update docs/ima.md
Co-authored-by: Jonathan Lebon <jonathan@jlebon.com>
2022-04-21 16:24:57 -04:00
Colin Walters 614d30acf3 docs: Add new IMA document
Now that the fixed code for `ima-sign` landed in
https://github.com/ostreedev/ostree-rs-ext/pull/283
2022-04-21 12:04:33 -04:00
Colin Walters b92dbc49f0 ci: Mask zincati for synthetic update
Our CI started falling over because coreos-assembler checks
for units stuck activating as of https://github.com/coreos/coreos-assembler/pull/2810

Really need to centralize the code for this and fix the root
problem, but...not today.

xref https://github.com/coreos/coreos-assembler/pull/2814
2022-04-20 20:23:10 -04:00
Colin Walters d75b934c59 Merge pull request #2583 from cgwalters/curl-check-setopt
fetcher/curl: Consistently check return value `curl_easy_setopt`
2022-04-20 20:22:12 -04:00
Colin Walters 4fd0d6b4c1 fetcher/curl: Consistently check return value curl_easy_setopt
Static analyzers don't like when we only check it sometimes.

And we definitely want to know if any of these are failing.
2022-04-20 15:16:22 -04:00
Colin Walters 688f0c4eea Merge pull request #2578 from lucab/ups/move-fixture
tests: move fixture to fix installed tests
2022-04-12 11:47:07 -04:00
Colin Walters f21944da1c Merge pull request #2577 from dbnicholson/man-html
Publish man pages
2022-04-09 15:46:25 -04:00
Colin Walters 2346d5f4d5 libarchive: Handle archive_entry_symlink() returning NULL
The `archive_entry_symlink()` API can definitely return `NULL`,
reading through the libarchive sources.

I hit this in the wild when using old ostree-ext to try to unpack
a chunked archive.

I didn't try to characterize this more, and sorry no unit test right
now.
2022-04-04 10:25:35 -04:00
Colin Walters fdfb353f19 Merge pull request #2569 from WOnder93/finalize-deployment-selinux-policy
deploy: Try to rebuild policy in new deployment if needed
2022-03-28 20:55:07 -04:00
Colin Walters c58a4fe661 deploy: Be a bit more verbose about SELinux bits
Let's log when we don't find the expected CLI argument which
will help debug things.
2022-03-28 17:46:59 -04:00
Colin Walters 86741ad18a Merge pull request #2573 from jtojnar/single-bin-coreutils-extensions
tests/test-cli-extensions: Fix with single-binary coreutils
2022-03-28 11:11:55 -04:00
Colin Walters afdc84b97b Update to sh-inline 0.2
Syncing up with the latest.
2022-03-14 09:54:48 -04:00
Colin Walters e6f92f76fa ci: Update docs workflow to use fcos-buildroot
This way we can also use `./ci/installdeps.sh` which avoids yet
another list of dependencies.
2022-03-14 09:27:07 -04:00
Colin Walters 089b821098 ci: Disable Ubuntu LTS
It doesn't have a new enough glib.
2022-03-14 09:21:15 -04:00
Colin Walters 6ba254ab11 Merge pull request #2551 from q66/guri
lib: bump glib requirement to 2.66 and port to GUri
2022-03-14 08:52:51 -04:00
Colin Walters 75ae283f23 Update to rand 0.8
Part of general crate updates.
2022-03-11 15:34:59 -05:00
Colin Walters fba7efb6da Update to ostree-ext 0.6
Part of general crate updates.
2022-03-11 15:33:02 -05:00
Colin Walters 34d1bcc68a Update to nix 0.23
Part of general crate updates.
2022-03-11 15:32:27 -05:00
Colin Walters b6d1119f91 tests: Stop using inventory crate
I was reading this thread
https://internals.rust-lang.org/t/from-life-before-main-to-common-life-in-main/16006/30
and that reminded me about this code, which it turns out actually
doesn't compile with my default local cargo config:
```
$ cat ~/.cargo/config
[target.x86_64-unknown-linux-gnu]
rustflags = ["-Ctarget-cpu=native", "-C", "link-arg=-fuse-ld=lld"]

[profile.release]
incremental = true
$ cargo b
...
error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "/var/srv/walters/src/github/ostreedev/ostree/target/debug/deps/ostree_test-4ca8e730f9dc6ffc.10325uqlhkyr5uol.rcgu.o" "/var/srv/walte"
  = note: ld.lld: error: undefined symbol: __start_linkme_NONDESTRUCTIVE_TESTS
          >>> referenced by 22nn09lfsklfqvyy
          >>>               /var/srv/walters/src/github/ostreedev/ostree/target/debug/deps/ostree_test-4ca8e730f9dc6ffc.22nn09lfsklfqvyy.rcgu.o:(ostree_tes)

```

For now let's just go back to having a static list of functions.
We don't have *too* many of those.
2022-03-11 14:53:29 -05:00
Colin Walters 0d020a7145 tmpfiles: Create /run/ostree
This is referenced by https://github.com/ostreedev/ostree-rs-ext/blob/9645cee4f29786ba51ae9d62a52eeef9230146fd/lib/src/globals.rs#L16
specifically used for the (container image) pull secret in
`/run/ostree/auth.json`.

Let's pre-create the directory so users don't have to.

Motivated by https://github.com/openshift/machine-config-operator/pull/3007#discussion_r824172564
2022-03-11 13:08:23 -05:00
Colin Walters 566fa59753 Merge pull request #2561 from pwithnall/pull-leak
ostree-repo-pull: Take correct out path on error
2022-03-08 11:22:42 -05:00
Colin Walters 775ff6e687 Merge pull request #2560 from smcv/sigpipe
test-prune: Read to the end of cut(1) output
2022-03-08 10:29:11 -05:00
Colin Walters 21a083db69 Merge pull request #2559 from smcv/shebang
Fix shebang in s390x-se-luks-gencpio
2022-03-08 09:46:07 -05:00