The semantics of multiple process locking are covered by
test-concurrency.py, but the semantics of the repository locking from a
single process aren't handled there.
This checks how the repository locking is handled from a single thread
with one OstreeRepo, a single thread with multiple OstreeRepos, and
multiple threads sharing an OstreeRepo.
If there's a locking issue in this test, then it's likely not going to
resolve after a few seconds of serializing access. Lower the default 30
second lock timeout to 5 seconds to prevent the test from hanging
unnecessarily.
Use `g_error` and `g_assert*` rather than `g_return*` when checking the
locking preconditions so that failures result in the program
terminating. Since this code is protecting filesystem data, we'd rather
crash than delete or corrupt data unexpectedly.
`g_error` is used when the error is due to the caller requesting an
invalid transition like attempting to pop a lock type that hasn't been
taken. It also provides a semi-useful message about what happened.
Previously each thread maintained its own lock file descriptor
regardless of whether the thread was using the same `OstreeRepo` as
another thread. This was very safe but it made certain multithreaded
procedures difficult. For example, if a main thread took an exclusive
lock and then spawned worker threads, it would deadlock if one of the
worker threads tried to acquire the lock.
This moves the file descriptor from thread local storage to the
`OstreeRepo` structure so that threads using the same `OstreeRepo` can
share the lock. A mutex guards against threads altering the lock state
concurrently.
Fixes: #2344
This simplifies the lock state management considerably since the
previously pushed type doesn't need to be tracked. Instead, 2 counters
are kept to track how many times each lock type has been pushed. When
the number of exclusive locks drops to 0, the lock transitions back to
shared.
In gnupg 2.3.0[1], if a primary key is expired and a subkey does not
have an expiration or its expiration is older than the primary key, the
subkey's expiration will be reported as the primary's. Previously a
subkey without an expiration would not report one regardless of the
primary key's expiration.
This caused a regression in a test setting an expiration on a primary
key. The test was checking that the subkey was not expired by asserting
that there was no `Key expired` line in the signature verification
output. With gnupg 2.3.0+, it will show as expired, causing the test to
fail.
Remove the assertion since it's not consistent across gnupg versions. In
practice we don't care whether the subkey is considered expired or not
as long as the signature verification fails when the primary key is
expired.
1. https://dev.gnupg.org/T3343Fixes: #2359
The ostree repo has read permissions set for workflows, which prevents
the documentation job from pushing the built docs to the gh-pages
branch. Raise the job's permissions to write for repo contents to allow
that.
Make a copy of `apidoc/html` to `docs/reference` and then tell Jekyll to
include it verbatim. This will include the gtk-doc API docs on the
static site. A link is added to the main index.
A script is added to do the copy (a symlink won't do) and is setup to
run before Jekyll in the GitHub workflow. Ideally this would be a local
Jekyll plugin to make the process automatic, but the github-pages gem
doesn't allow that.
This uses the Jekyll Actions GitHub action to push the rendered docs to
the gh-pages branch rather than GitHub's automated docs flow. That will
allow greater control over how the docs are generated. Pushing to the
gh-pages branch only happens on pushes to main. For pull requests, the
docs are only built.
This mimics the GitHub Pages environment so that you can build and serve
the site locally for testing. It's will also be required later for using
Jekyll Actions[1] instead of the automated GitHub Pages flow.
1. https://github.com/marketplace/actions/jekyll-actions
This returns a 404 since the site is already generated from the docs
directory. Furthermore, the `CONTRIBUTING.md` markdown file isn't in the
generated site, just the HTML.
Instead, use jekyll's `link` tag to create the link. Unfortunately,
before jekyll 4.0 (github-pages uses 3.9), you have to prepend the base
URL.
Several tests generate summaries and then expect to use the generated
summary immediately. However, this can cause intermittent test failures
when they inadvertantly get a cached summary file. This typically
happens when the test is run on a filesystem that doesn't support user
extended attributes. In that case, the caching code can only use the
last modified time, which only has 1 second granularity. If tests don't
carefully manage the summary modification times or the repo cache then
they are likely subject to races in some test environments.
This introduces an environment variable `OSTREE_SKIP_CACHE` that
prevents the repo from using a cache directory. This is enabled by
default in tests and disabled for tests that are a explicitly trying to
test the caching behavior.
Fixes: #2313Fixes: #2351
Although people are likely not deploying on 32 bit x86 anymore,
deploying on 32 bit armv7 is still often used. Add back an i386 build on
debian to try to catch 32 bit bugs in CI.
Currently the Ubuntu builds are stuck waiting for an answer on what
timezone to use. That could be fixed, but generally the way to do these
types of installs is to set the DEBIAN_FRONTEND environment variable to
`noninteractive` so that debconf just chooses a default. This is what
debootstrap does, for instance. I tested installing tzdata on a local
focal container this way and it just chooses `Etc/UTC` as the timezone.
If you don't configure with --enable-gtk-doc, these won't be included in
the generated .gitignore. If you later configure with --enable-gtk-doc
in the same checkout, they'll show up as untracked files. Include them
in GITIGNOREFILES to ensure they're always added regardless of configure
options.
Like the top level .gitignore, there's no reason to track this if it's
being generated automatically. It only produces spurious diff changes
that you almost certainly don't want.
The local pull path was erroring on any missing commit, but that
prevents a depth pull where the source repo has truncated history. As in
the remote case, this also tries to pull in a tombstone commit if the
source repo supports it.
Fixes: #2266
When pulling with depth, missing parent commits are ignored. However,
the check was applying to any commit, which means that it would succeed
even if the requested commit was missing. This might happen on a
corrupted remote repo or when using ref data from a stale summary.
To achieve this, the semantics of the `commit_to_depth` hash table is
changed slightly to only ever includes parent commits. This makes it
easy to detect when a parent commit is being referenced (although there
is a minor bug there when multiple refs are being pulled) while keeping
references to commits that need their `commitpartial` files cleaned up.
It also means that the table is only populated on depth pulls, which
saves some memory and processing in the common depth=0 case.
Fixes: #2265
In some cases such as backups or mirroring you may want to pull commits
from one repo to another even if there commits that have incorrect
bindings. Fixing the commits in the source repository to have correct
bindings may not be feasible, so provide a pull option to disable
verification.
For Endless we have several repositories that predate collection IDs and
ref bindings. Later these repositories gained collection IDs to support
the features they provide and ref bindings as the ostree tooling was
upgraded. These repositories contain released commits that were valid to
the clients they were targeting at the time. Correcting the bindings is
not really an option as it would mean invalidating the repository
history.
Dan Nicholson