There's quite a lot of changes in this release since 2016.5, but one
thing I want to call out explicitly is that we fixed a race condition
in the HTTP pull code that could cause hangs or crashes that mostly
occurred only when doing "large" pulls (thousands of object requests).
If this occurs, client systems can work around it by cancelling and
retrying the pull.
Other high level changes:
Jonathan Lebon:
ostree-repo-libarchive.c: major refactor
This laid more groundwork for rpm-ostree to import RPM (CPIO) archives
sanely. If you are integrating with an existing package management
format, this should help a lot.
Mathnerd314 (11):
repo: use OSTREE_TIMESTAMP (=1) for checked-out files
Is a quite interesting change - we previously used a timestamp of
zero, but Nix has apparently done better with `1` for various reasons.
This is a pretty subtle change, so be on the lookout for regressions.
Thanks to all contributors!
Alexander Larsson (13):
ostree export: Add --subpath support
ostree export: Add --prefix option
repo: Allow loading files staged in the transaction
repo: Fix temporary directory allocation
repo: Handle parent repos with different remote configuration
tests: Fix build with old glib
Build on older versions of glib
libotutil: Fix leak of VariantMapData
pull: Fix leak of debug temporary strings
pull: Fix leak of FetchObjectData in some cases
pull: Fix leaked uri in request_static_delta_superblock_sync
pull: Fix leak of refs_to_fetch
tests: Exit valgrind tests if a leak is detected
Colin Walters (50):
core: Remove ostree_checksum_update_meta that was not implemented
lib: Remove ostree_repo_file_make_empty_tree
Rename test-abi to test-symbols, start verifying symbols are documented
libtest: Skip FUSE if CAP_SYS_ADMIN not in bounding set
repo: Simplify internal has_object() lookup code
deploy: Start hardlinking kernel/initramfs on single-part again
build: Introduce --with-dracut=yesbutnoconf
libglnx porting: Migrate from GSConsole
repo: Fix reading repos on read-only media
repo: Add OSTREE_REPO_TEST_ERROR=pre-commit env var
repo: Add OSTREE_BOOTID override envvar for debugging
repo: Clean up staging directory for previous boot IDs
repo: Make repo/tmp expiry configurable via tmp-expiry-seconds
lib: Add more filename validations (no ., .. or /) in commit logic
libglnx porting: Migrate to glnx_stream_fstat()
build: Use --std=gnu99
trivial-httpd: Add __attribute__((format))
Make enum generation private by default, export stub symbol
build: Don't run glib-mkenums if there's nothing to do
README: x/xdg-app/flatpak/
libglnx porting: Port "load utf8 file" contents API callers
repo: Port uncompressed cache GC to libglnx
libglnx porting: Port callers of gs_file_unlink to just unlink()
libglnx porting: Port away from gs_file_ensure_directory()
sysroot: Correct error handling path in previous libglnx port
core: Port to new libglnx tempname API
lib: Change ot_util_variant_map helpers to consistently sink refs
lib: Drop GFile variant mapping API for fd-relative
libglnx porting: Port away from gs_file_map_noatime()
glnx porting: Port away from gs_file_get_basename_cached()
tests: Use strict mode by default for C tests
tests/test-pull-c: New test that runs through the pull API via C
glnx porting: Drop last uses of gs_file_get_basename_cached()
libglnx porting: Drop last use of gs_transfer_out_value()
glnx porting: Drop use of gs_file_openat_noatime
lib: Add `_ALLOW_NOENT` flag to internal variant mapping API
lib: Add a helper for mmap->bytes with openat(), use it in repo
repo: (minor) Drop a GFile usage in signature creation
lib: Fix a leak in remote parsing
lib: Add OSTREE_SUPPRESS_SYNCFS environment variable
tests: Modernize valgrind infrastructure
lib: Port some manual close() cleanups to be glnx_fd_close
manual: Link to mender.io
Add "archive" as an alias for "archive-z2"
repo: Avoid a possible divide by zero in progress
manual: Discuss mirroring
build: Fix libreaddir-rand to honor global CFLAGS
tests: Support OT_SKIP_READDIR_RAND
pull: Ensure we always process queue only from main thread
Release 2016.6
Gatis Paeglis (3):
Remove unused variables from install_deployment_kernel
Fix --enable_rofiles_fuse=no build
Fix build when have_libsoup_client_certs=no
Giuseppe Scrivano (1):
man: add missing options for commit
Jonathan Lebon (7):
ostree-repo-libarchive.c: major refactor
ot-builtin-commit.c: add --skip-list option
tests/test-libarchive.sh: add more test
tests/test-libarchive-import.c: add tests
OstreeRepo: add extensions directory
ostree_repo_write_commit: add missing docstring arg
configure.ac: properly check for bison
Krzesimir Nowak (26):
lib: Fix a typo
trivial-httpd: Remove useless code
lib: Fix an introspection annotation
tests, lib: Remove unused variables
lib: Add URL override pull option
pull: Add a --url option
trivial-httpd: Add a --log-file option
tests: Add a test for ostree pull --url
sysroot: Plug some leaks
core: Improve variant format docs
repo: Plug a leak
repo: Allow using options for fetching summary
libostree: Variant-related leak plugs and fixes
libostree: Fix bracket-missing warnings
libostree: Add g_autoptr() support for the public types
build: Remind about bumping related versions
build: Allow ostree to use g_autoptr on its types internally
build: Install new autocleanups header
libotutil: Plug a leak
core: Add a function creating an archive-z2 content stream
repo: Factor out the check of gpg result to a separate function
repo: Add a missing Returns paragraph
repo: Factor out setting up a GPG verifier
repo: Add functions for verifying any data in repository
summary: Fix a typo
repo: Fix build on 32-bit systems
Mathnerd314 (11):
commit: accept NULL subject argument
commit: Add --editor / -e command line argument
commit: Display subject/body in commit editor
commit: Don't require a subject
log: Display "(no subject)" for commits without a subject
commit: Update manpage to reflect --editor
repo: use OSTREE_TIMESTAMP (=1) for checked-out files
pull: Move libsoup-related code from ostree-repo.c to ostree-repo-pull.c
pull: Move pull and pull_onedir back into ostree-repo.c
docs: Get API docs working again
related-projects: Update with more information and projects
Micah Abbott (1):
docs: fix swapped link syntax
Sam Spilsbury (2):
Makefile: Move to AM_DISTCHECK_CONFIGURE_FLAGS
repo: Display estimated time remaining when pulling
Simon McVittie (12):
libtest: show files' contents when assertions about them fail
test-parent: this test requires user xattrs
test-sysroot.js: set "strict mode" when sourcing libtest.sh
Link libreaddir-rand to libdl
autogen.sh: rely on VPATH for submodules' Makefile-*.am.inc
build: distribute libostree.sym in tarballs
build: distribute test scripts in tarballs
build: distribute TAP helper scripts in tarballs
build: distribute more test files
libtest: make gpghome in tmpdir writeable
libostreetest: include libtest.sh from srcdir, not builddir
build: re-create tests/ostree-symlink-stamp if Makefile changes
Yu Qi Zhang (2):
refs: add "ostree refs --create" and unit tests
libglnx porting: delete temp files on failure of file creation
Git-EVTag-v0-SHA512: 216ec7ea61eeabb0404207b01387b7de85276b2fbf09e39928f4800eddac06904d690dbf66260282a1cc4127f15a68feb0a8da31921d7d943856569cb81b1d6f
-----BEGIN PGP SIGNATURE-----
iQEcBAABCgAGBQJXaCX8AAoJENxF/VkhwT8LKN8H/0G8eTKx/H+BXn72Ok/dR4Gh
P+EAsqvASavYq5y8eg5ZdAoIy8tttQdO9zX/RZIiU/QrI+iwtngL9xmjYh+fsVEo
T9YgTXbwSI6k3g4zV6BPcwPN3/PBd4esGgDnf9uEv6RjDPpuXhyPYyOnW8Rr1xth
vC4/gBEbKRlCeRCiB1E0hiYjELxu2DyESlBZfNQ0lPvxDhrp/ZlvyR3/IkQWFhao
GVJDaCot+hvdNdclHu1izHa8qfVoAMJokLXD6Y3SiQxZtokUBgNJ8EwbEJ+A0YRx
Ojl8jKwz1zmxtMz2M4qLbVKjzBecAj6t+WSbLW6x0MGZs1q235LBy0lSJIv+8SA=
=RTt1
-----END PGP SIGNATURE-----
Merge tag 'v2016.6' into upstream/latest
Release 2016.6
Add the functionality to use the same name for refs in local and remote
repos. This helps users keep track of local refs of remote origin, much
like local and remote git branches.
Previously, when a local ref is specified, resolve_refspec would fall
back to searching through remote repos if the ref is not found locally.
This function now takes an extra flag to specify whether it should
search through remote repos. Additionally, ostree_repo_resove_rev_ext
was added to call resolve_refspec with fallback_remote being false, so
refs --create would no longer complain when trying to create a local
ref of the same name as a remote one.
Fix remote repo parsing not being handled correctly on refs --create.
Closes: #363
Approved by: jlebon
In flatpak i was using a parent repo, and it failed to update
with ENOENT when dispatching an set-read-source opcode, because the
object it referenced was in the parent repo.
This fixes that by making _ostree_repo_read_bare_fd look
at parent_repo.
Closes: #362
Approved by: cgwalters
We noticed that once a ref folder is created, there is no existing
command that can remove it. For example, once "foo/bar" is created,
even if the user deletes foo or all the refs under foo, the folder
will persist.
Now when the user attempts to create a ref "foo" either through commit
or refs --create, if a folder "foo" exists but is empty of refs, the
folder is removed and the new ref "foo" is created.
New unit tests in tests-ref.sh verify this functionality.
Closes: #354
Approved by: cgwalters
Import `gs_file_enumerator_iterate()` for the next six months or
so...after RHEL 7.3 is released I'm strongly considering hard
requiring 2.46 or so.
Likely at some point we should figure out how to share more "glib
backport" code with NetworkManager at least.
Closes: #341
Approved by: jlebon
This was the last caller of libgsystem that isn't
`gs_file_get_path_cached()`. I think the use case ostree has where
the same code can be called via command line and via a shared library
*and* via a daemon is rather unusual, so let's just copy the code for
logging from libgsystem into here.
For example rpm-ostree hard depends on a daemon mode, so it'll just
use `sd_journal` directly.
Closes: #341
Approved by: jlebon
I was easily reproducing a hang on pulls with thousands of requests on
current git master. The initial symptom seemed to be that there are
multiple code paths where we don't invoke
`session_thread_process_pending_queue()`. We really need to do
that any time we remove something from the outstanding queue,
to ensure it gets filled again.
A further issue is that we were tying the lifecycle of the pending
object to the `GTask`, but the task could be unref'd from the main
thread (via a `GSource` on the main thread), and that introduced
threadsafety issues, because the hash table and other data suddenly
could be concurrently modified.
Both of these need to be fixed together. First, we introduce
`Arc<Pending>`, and ensure that both the main and worker threads hold
references.
Second, we ensure that we re-process the queue *immediately* whenever
a task is done, inside the worker thread, rather than doing it
incidentally via an unref. This architecture is quite similar to what
the outside pull code is doing.
Closes: #350
Approved by: jlebon
For some reason I'm really not inclined to debug right now, `libtool`
is eating the `-fsanitize=address` argument when linking
`libreaddir-rand.so`, which causes failures since we're trying to
`LD_PRELOAD`.
Closes: #351
Approved by: jlebon
It's unfortunate that in automake one has to explicitly include the globa
`$(AM_CFLAGS)` if one sets `CFLAGS`, and similarly for other variables.
I'm trying to use `-fsanitize=address`, and not including it was
causing linker failures.
We also weren't inheriting the global warnings etc., so I had to fix a
decl-after-statement.
Closes: #351
Approved by: jlebon
The previous code was subject to a divide by zero if less than a
second had passed. Rework it so we only do the divide if more than a
second has passed.
Closes: #349
Approved by: Mathnerd314
We noticed the temp files being left over in ostree when (mistakenly)
trying to create refs with names in use by folders. This fix removes
temp files created by glnx_file_replace_contents_at on failure.
Closes: #348
Approved by: cgwalters
I find the "-z2" is really a long ago relic of the past when I changed
the format. We no longer have anything to do with the original
`archive`, so let's start allowing people to type `--mode=archive`
which just looks saner.
At some point later I'll update the docs too, but it'll be an annoying
transition period as we'll have to say "On older OSTree, use -z2" etc.
Closes: #346
Approved by: giuseppe
Added the ability to create a ref (much like a git tag) for an
existing commit through "ostree refs EXISTING --create=NEWREF".
Previously the only way to create a new ref was by creating a new commit,
but refs --create allows multiple refs to point to the same commit.
The command will fail if:
- None/more than one existing ref is specified
- The specified EXISTING tag does not exist, or was not specified
- The specified NEWREF already exists, or is the name of a folder
Add unit tests in tests-ref.sh to verify above functionality
Closes: #340
Approved by: jlebon
Just noticed this while reading some code, we didn't have many manual
`out: close()` bits left, this pushes us over the edge to autocleanup
almost everywhere.
Closes: #332
Approved by: jlebon
The filename of the real ostree executable could be either .libs/ostree
or .libs/lt-ostree.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #337
Approved by: cgwalters
Otherwise, during "make distcheck" it will be read-only (because it
is a copy of the read-only ${srcdir} with permissions preserved), and
deletion will fail during cleanup.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #337
Approved by: cgwalters
This avoids hard-coding the autogen-time ${srcdir} into our build
system, and might fix "make distcheck".
We don't need to specify ${srcdir} explicitly in the various variables,
because Automake always uses make's VPATH feature to look in both
${builddir} and ${srcdir} at build time.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #337
Approved by: cgwalters
It uses dlsym(). There's no point in being extra-portable here
because OSTree only targets Linux anyway.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #336
Approved by: cgwalters
As with the C tests in commit 08580118, this makes sure the test
fails as soon as something goes wrong.
Signed-off-by: Simon McVittie <smcv@debian.org>
Closes: #335
Approved by: giuseppe
The recent memleak fixes motivated me to look at the bitrotted code to
run invocations of `ostree` in the test suite underneath valgrind.
There are a few things here. First, update suppressions file from
libhif, since I recently worked on it.
When running *uninstalled* as we now support, we need
`libtool --mode=execute` in the mix so it expands out to
the uninstalled binary and we don't valgrind the intermediate shell.
However, it's harder than that because we chdir into a tmpdir,
which defeats the libtool logic. AFAICS, the only fix for this
is to determine the realbin path before we chdir, and then unfortunately
we need to change every use of `ostree` to `${OSTREE}` =(
Then this immediately breaks for me on RHEL7 because my ancient
copy of `valgrind-3.10.0-16.el7.x86_64` is unaware of syscall 306, i.e.
`syncfs`.
But let's do this first before I dive into that.
Closes: #292
Approved by: krnowak
As the docs say, `g_regex_match()` still allocates a match even if it
returns `FALSE`. Using `g_autoptr` is just plain better.
Closes: #292
Approved by: krnowak
In particular, NixOS has changed somewhat, and Conda is worth
looking at. Also it seems reasonable to mention rpm-ostree /
Gnome Continuous.
Closes: #331
Approved by: cgwalters
Simon McVittie
Colin Walters
Alexander Larsson
Yu Qi Zhang
Mathnerd314
Krzesimir Nowak