When we're only pulling a subset of the refs available in the remote, it
doesn't make sense to copy the remote's summary (which may not be valid
for the local repo). This makes the check here match the one done
several lines above when we decide whether to error out if there's no
remote summary available.
This extends the fix in https://github.com/ostreedev/ostree/pull/935 for
the case of collection-refs.
Also, add a unit test for this issue, based on the existing one in
pull-test.sh.
This will be useful in the unit test added by the next commit. It just
passes OSTREE_REPO_PULL_FLAGS_MIRROR to the call to
ostree_repo_pull_from_remotes_async().
Add G_IO_ERROR_PARTIAL_INPUT to the list of error codes caused by
transient networking errors which lead us to retry the request. When
attempting to install the spotify flatpak you often get the error
message "Connection terminated unexpectedly" and the download of the deb
file fails. In this case, libsoup is setting G_IO_ERROR_PARTIAL_INPUT
and sometimes a subsequent download attempt is successful, so we should
treat it as transient.
Ideally we would behave as wget does in this case and retry the download
picking up where we left off in the file rather than starting over, but
that would require changes to libsoup I think.
Sadly this patch does not fix the flatpak installation of spotify in the
face of such errors, because flatpak doesn't use libostree to download
extra data, but presumably it's possible we could encounter such an
error pulling from an ostree repo, so the patch is still correct.
Currently for a "normal" refspec you can choose to use
ostree_repo_resolve_rev_ext() instead of ostree_repo_resolve_rev() if
you only want to look at local refs (in refs/heads/) not remote ones.
This commit provides the analogous functionality for
ostree_repo_resolve_collection_ref() by adding a flag
OSTREE_REPO_RESOLVE_REV_EXT_LOCAL_ONLY and implementing it. This
will be used by Flatpak.
Closes: #1825
Approved by: jlebon
Currently the flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_REMOTES for
ostree_repo_list_collection_refs() means that refs in refs/remotes/
should be excluded but refs in refs/mirrors/ should still be checked, in
addition to refs/heads/ which is always checked. However in some
situations you want to exclude both remote and mirrored refs and only
check local "owned" ones. So this
commit adds a new flag OSTREE_REPO_LIST_REFS_EXT_EXCLUDE_MIRRORS which
lets you exclude refs/mirrors/ from the listing.
This way we can avoid breaking API but still allow the listing of local
collection-refs.
The impetus for this change is that I'm changing Flatpak to make more
use of refs/mirrors, and we need a way to specify that a collection-ref
is local when using ostree_repo_resolve_collection_ref() in, for
example, the implementation of the repo command. The subsequent commit
will make the changes needed there.
Closes: #1825
Approved by: jlebon
My last commit "lib/repo-refs: Resolve collection-refs in-memory and in
parent repos" changed ostree_repo_resolve_collection_ref() to check the
in-memory set of refs *after* failing to find the ref on disk but that's
not what we want. We want to use the in-memory set of refs first,
because those are the most up to date commits, and then fall back to the
on-disk repo and finally fall back to checking any parent repo. This
commit makes such a change to the order of operations, which is
consistent with how ostree_repo_resolve_rev() works.
Aside from this change being logical, it also fixes some unit test
failures on an unmerged branch of flatpak:
https://github.com/flatpak/flatpak/pull/2705
Also, tweak the comments here.
Closes: #1825
Approved by: jlebon
On at least one user's computer, g_getenv("http_proxy") returns the
empty string, so check for that and treat it as no proxy rather than
printing a warning.
See https://github.com/flatpak/flatpak/issues/2790Closes: #1835
Approved by: cgwalters
Currently the P2P code requires you to trust every remote you have
configured to the same extent, because a remote controlled by a
malicious actor can serve updates to refs (such as Flatpak apps)
installed from other remotes.[1] The way this attack would play out is
that the malicious remote would deploy the same collection ID as the
victim remote, and would then be able to serve updates for it.
One possible remedy would be to make it an error to configure remotes
such that two have the same collection ID but differing GPG keys. I
attempted to do that in Flatpak[2] but it proved difficult because it is
valid to configure two remotes with the same collection ID, and they may
then each want to update their keyrings which wouldn't happen
atomically.
Another potential solution I've considered is to add a `trusted-remotes`
option to ostree_repo_find_remotes_async() which would dictate which
keyring to use when pulling each ref. However the
ostree_repo_finder_resolve_async() API would still remain vulnerable,
and changing that would require rewriting a large chunk of libostree's
P2P support.
So this commit represents a third attempt at mitigating this security
hole, namely to have the client specify which remote to use for GPG
verification at pull time. This way the pull will fail if the commits
are signed with anything other than the keys we actually trust to serve
updates.
This is implemented as an option "ref-keyring-map" for
ostree_repo_pull_from_remotes_async() and
ostree_repo_pull_with_options() which dictates the remote to be used for
GPG verification of each collection-ref. I think specifying a keyring
remote for each ref is better than specifying a remote for each
OstreeRepoFinderResult, because there are some edge cases where a result
could serve updates to refs which were installed from more than one
remote.
The PR to make Flatpak use this new option is here[3].
[1] https://github.com/flatpak/flatpak/issues/1447
[2] https://github.com/flatpak/flatpak/pull/2601
[3] https://github.com/flatpak/flatpak/pull/2705Closes: #1810
Approved by: cgwalters
Currently it's not an error to provide too many arguments to an ostree
config command. Change it so we print usage information in that case,
and update the unit tests.
Closes: #1743
Approved by: cgwalters
It doesn't make much sense to use SECTIONNAME in some places and
GROUPNAME in others when they mean the same thing.
Closes: #1743
Approved by: cgwalters
It seems cleaner to make the GKeyFile a g_autoptr variable and just
return rather than using the "goto out;" idiom.
Closes: #1743
Approved by: cgwalters
Currently there's a way to set a key to the empty string but there's no
way to unset it completely (remove the key from the group). This might
be helpful for instance if you want to temporarily set
"core.lock-timeout-secs" to a specific value for the duration of one
operation and then return it to the default after that operation
completes.
This commit implements an "unset" operation for the config command, adds
a unit test, and updates the man page.
Closes: #1743
Approved by: cgwalters
Currently the behavior of ostree_repo_resolve_rev() is that it tries to
resolve a ref to a commit by checking the refs/ directories, but also by
checking for in-memory ref-checksum pairs which are part of an
in-progress transaction and also by checking the parent repo if one
exists. Currently ostree_repo_resolve_collection_ref() only checks the
refs/ directories, so this commit makes its behavior analagous since it
is the analagous API which supports collection-refs.
The impetus for this was that currently Flatpak uses
ostree_repo_resolve_rev() to load a commit after doing a P2P pull in
flatpak_dir_do_resolve_p2p_refs(), but that assumes the ref came from
the same remote that originally provided it, which might not be the case
if more than one remote has the same collection ID configured. And
changing Flatpak to use ostree_repo_resolve_collection_ref() doesn't
work without this patch.
Closes: #1821
Approved by: pwithnall
Since commit a06bd82cd we no longer use OVERLAYFS_SUPER_MAGIC or
statfs() so remove the includes for linux/magic.h and sys/vfs.h
Closes: #1815
Approved by: cgwalters
OSTree's P2P support used to be a compile time option but is now always
enabled. One of the tests was still checking for the old feature flag,
"experimental", which has been renamed to "p2p" and skipping for that
reason. Delete the check so the test always runs.
Closes: #1804
Approved by: pwithnall
The way _ostree_repo_import_object() is written, a hardlink copy is only
attempted if the source repo is trusted, so update the docs for
ostree_repo_import_object_from_with_trust() to reflect that.
Closes: #1777
Approved by: cgwalters
This renames a config key to make its semantics more obvious. Despite
what the commit message says, it only applies when a set of repo finders
is not specified (either on the command line or in a library API call).
This also renames the corresponding ostree_repo_get function. We can do
this since it hasn't been released yet.
Closes: #1763
Approved by: pwithnall
This commit disables searching on the local network for refs, unless
explicitly requested by the user either by changing the value of the
"core.repo-finders" config option, or by passing an OstreeRepoFinderAvahi to
ostree_repo_find_remotes_async() / ostree_repo_finder_resolve_async(),
or by specifying "lan" in the --finders option of the find-remotes
command.
The primary reason for this is that ostree_repo_find_remotes_async()
takes about 40% longer to complete with the LAN finder enabled, and that
API is used widely (e.g. in every flatpak operation). It's also probable
that some users don't want ostree doing potentially unexpected traffic
on the local network, even though everything pulled from a peer is GPG
verified.
Flathub will soon deploy collection IDs to everyone[1] so these code
paths will soon see a lot more use and that's why this change is being
made now.
Endless is the only potential user of the LAN updates feature, and we
can revert this patch on our fork of ostree. For it to be used outside
Endless OS we will need to upstream eos-updater-avahi and
eos-update-server into ostree.
[1] https://github.com/flathub/flathub/issues/676Closes: #1758
Approved by: cgwalters
Currently libostree essentially has two modes when it's pulling refs:
the "legacy" code paths pull only from the Internet, and the code paths
that are aware of collection IDs try to pull from the Internet, the
local network, and mounted filesystems (such as USB drives). The problem
is that while we eventually want to migrate everyone to using collection
IDs, we don't want to force checking LAN and USB sources if the user
just wants to pull from the Internet, since the LAN/USB code paths can
have privacy[1], security[2], and performance[3] implications.
So this commit implements a new repo config option called "repo-finders"
which can be configured to, for example, "config;lan;mount;" to check
all three sources or "config;mount;" to disable searching the LAN. The
set of values mirror those used for the --finders option of the
find-remotes command. This configuration affects pulls in three places:
1. the ostree_repo_find_remotes_async() API, regardless of whether or
not the user of the API provided a list of OstreeRepoFinders
2. the ostree_repo_finder_resolve_async() /
ostree_repo_finder_resolve_all_async() API
3. the find-remotes command
This feature is especially important right now since we soon want to
have Flathub publish a metadata key which will have Flatpak clients
update the remote config to add a collection ID.[4]
This effectively fixes https://github.com/flatpak/flatpak/issues/1863
but I'll patch Flatpak too, so it doesn't pass finders to libostree only
to then have them be removed.
[1] https://github.com/flatpak/flatpak/issues/1863#issuecomment-404128824
[2] https://github.com/ostreedev/ostree/issues/1527
[3] Based on how long the "ostree find-remotes" command takes to
complete, having the LAN finder enabled slows down that step of the
pull process by about 40%. See also
https://github.com/flatpak/flatpak/issues/1862
[4] https://github.com/flathub/flathub/issues/676Closes: #1758
Approved by: cgwalters
This commit removes the recommendation in the create-usb man page for
the user to update the summary in the source repo before using the
create-usb command. I'm not sure where I got the idea that create-usb
depends on a summary in the source repo. I went back to the first commit
that introduced the create-usb command and even using that a summary
isn't required, so it seems unlikely that this changed recently.
This is good news because the exclusive lock that's taken for summary
updates has been causing problems on Endless (due to other processes
having a lock for the duration of the 30 second acquire time out
period).
Closes: #1746
Approved by: cgwalters
This commit fixes the bash tab completion handling of the "--repo"
argument. Before this commit, the completion only works if "--repo"
comes after the main command. After this commit, you can use "--repo"
directly after "ostree" in the command line, as is natural.
Closes: #1745
Approved by: jlebon
Currently the locking code checks if the value -1 was set for the config
key "lock-timeout-secs" and if so, a thread trying to acquire a lock
will block indefinitely. Positive values specify how long to attempt to
acquire a lock in a non-blocking way (the attempt is made once every
second). But when the value is read from the config file,
g_ascii_strtoull() is used, which converts it to an unsigned integer.
This commit makes libostree use g_ascii_strtoll() instead, so that it's
possible to set that key to -1 as intended.
Closes: #1737
Approved by: jlebon
Currently on Endless OS, the OSTree ref for the operating system is
something like os/eos/amd64/eos3, so that's what gets passed to `ostree
create-usb` when copying the OS to a USB drive (for offline updates).
However, when eos-updater checks for updates it pulls the metadata for a
candidate commit and in so doing updates that eos3 ref to point to the
partial commit being examined as a potential update rather than the
deployed commit. This causes `ostree create-usb` to fail with an error
like "No such metadata object
7fb045cb2d1f1f3a81bfc157c6128ff443eb56350315b9536bdb56aee0659863.dirtree".
OSTree creates deployment refs that look like "ostree/1/1/0" to maintain
a pointer to the deployed commit, but create-usb can't use these because
it shows up in the summary as just a ref, not a collection-ref.
So this commit adds a --commit option to the create-usb command, so we
can use the appropriate ref but copy the deployed commit rather than a
(potentially partial) update commit.
Closes: #1735
Approved by: cgwalters
Change the create-usb command so that it always creates the destination
repository using the "archive" mode, rather than using archive mode when
xattrs aren't supported and bare-user otherwise. This has a few
advantages:
1. The archive mode works with FAT filesystems, which is what most
USB drives are, and which doesn't support xattrs.
2. At least in some quick testing I did, archive mode is about
twice as performant as bare-user mode, in terms of how long it takes for
the create-usb command to complete.
3. This ensures that a tool can safely change the permissions on
".ostree/repo" and subdirectories after create-usb completes, which is
important for Endless since otherwise you can't use `ostree create-usb`
as root and then `flatpak create-usb` as a non-root user on the same USB
drive (or in other words copy OS updates and apps to the same USB).
Closes: #1733
Approved by: cgwalters
This commit defines a metadata key that tells clients to update their
remote config to add a collection ID. This functionality is currently
implemented in Flatpak for the key "xa.collection-id", but there are two
good reasons for moving the key to OSTree:
1) Servers such as Flathub shouldn't set xa.collection-id in their
metadata now or in the medium term future, because many users are still
using old versions of Flatpak and OSTree[1] which would hit various
bugs[2][3][4] on the P2P code paths that are enabled by collection IDs.
Defining a new key means that only clients running recent
(as-yet-unreleased) versions of Flatpak and OSTree will pay attention to
it and deploy the collection ID, leaving the users on old versions
unaffected.
2) OSTree is as "invested" in collection IDs as Flatpak, so there's no
reason the key should be defined in Flatpak rather than here. According
to Philip Withnall, the reason the key was put in Flatpak originally was
that at the time there was uncertainty about tying OSTree to collection
IDs.
[1] https://ahayzen.com/direct/flathub.html#downloadsbyflatpakstacked
[2] https://github.com/ostreedev/ostree/commit/e4e6d85ea
[3] https://github.com/flatpak/flatpak/commit/5813639f
[4] https://github.com/flatpak/flatpak/commit/5b21a5b7Closes: #1726
Approved by: pwithnall
This fixes typos and grammar in the docs for OstreeRepo, and copies the
information about OSTREE_REPO_MODE_BARE_USER_ONLY from ostree-core.h
Closes: #1725
Approved by: jlebon
In the OstreeRepoFinderAvahi implementation,
ostree_avahi_service_build_repo_finder_result() is where the DNS-SD
records are processed and turned into OstreeRepoFinderResult objects.
Each result object is supposed to have a hash table mapping refs to
checksums, so this is accomplished by first adding a placeholder (a ref
mapping to a NULL checksum) for each ref matched by the bloom filter,
and later filling in the checksums using the remote's summary file,
which happens in get_checksums(). The problem is that there's no
guarantee all the checksums will be resolved (non-NULL), so the
ostree_repo_finder_result_new() call then hits an assertion failure in
is_valid_collection_ref_map() leading to a crash (in the case that one
or more refs had NULL checksums).
There are at least two situations where the ref checksum might not be
found in the peer remote's summary file:
1) The bloom filter match was a false positive. This is going to happen
sometimes by design.
2) The peer remote's summary is out of sync with its DNS-SD records.
This shouldn't normally happen but it's still good to be robust to the
possibility; in Endless OS nothing guarantees the atomicity of updating
the summary and DNS-SD records.
This commit changes libostree to be robust to the possibility of refs
missing from the peer remote's summary, by removing any that still have
a NULL checksum associated with them after the summary has been fetched
and processed.
The other OstreeRepoFinder implementations don't have this issue because
they use summary files directly and therefore always have access to the
checksum.
Closes: #1717
Approved by: pwithnall
Normally, a configured remote will only serve refs with one associated
collection ID, but temporary remotes such as USB drives or LAN peers can
serve refs from multiple collection IDs which may use different GPG
keyrings. So the OstreeRepoFinderMount and OstreeRepoFinderAvahi classes
create dynamic OstreeRemote objects for each (uri, keyring) pair. So if
for example the USB mounted at /mnt/usb serves content from the
configured remotes "eos-apps" and "eos-sdk", the OstreeRepoFinderResult
array returned by ostree_repo_find_remotes_async() will have one result
with a remote called something like
file_mnt_usb_eos-apps.trustedkeys.gpg and the list of refs on the USB
that came from eos-apps, and another result with a remote
file_mnt_usb_eos-sdk.trustedkeys.gpg and the list of refs from eos-sdk.
Unfortunately while OstreeRepoFinderMount and OstreeRepoFinderAvahi
correctly only include refs in a result if the ref uses the associated
keyring, the find_remotes_cb() function used to clean up the set of
results looks at the remote summary file and includes every ref that's
in the intersection with the requested refs, regardless of whether it
uses a different remote's keyring. This leads to an error when you try
to pull from a USB containing refs from different collection IDs: the
pull using the wrong collection ID will error out with "Refspec not
found" and the result with the correct keyring will then be ignored "as
it has no relevant refs or they have already been pulled." So the pull
ultimately fails.
This commit fixes the issue by filtering refs coming from a dynamic
remote, so that only ones with the collection ID associated with the
keyring remote are examined. This only needs to be done for dynamic
remotes because you should be able to pull any ref from a configured
remote using its keyring. It's also only done when looking at the
collection map in the summary file, because LAN/USB remotes won't have a
"main" collection ID set (OSTREE_SUMMARY_COLLECTION_ID).
Closes: #1695
Approved by: pwithnall
This commits adds and implements a boolean repo config option called
"change-update-summary" which updates the summary file every time a ref
changes (additions, updates, and deletions).
The main impetus for this feature is that the `ostree create-usb` and
`flatpak create-usb` commands depend on the repo summary being up to
date. On the command line you can work around this by asking the user to
run `ostree summary --update` but in the case of GNOME Software calling
out to `flatpak create-usb` this wouldn't work because it's running as a
user and the repo is owned by root. That strategy also means flatpak
can't update the repo metadata refs for fear of invalidating the
summary.
Another use case for this relates to LAN updates. Specifically, the
component of eos-updater that generates DNS-SD records advertising ostree
refs depends on the repo summary being up to date.
Since ostree_repo_regenerate_summary() now takes an exclusive lock, this
should be safe to enable. However it's not enabled by default because of
the performance cost, and because it's more useful on clients than
servers (which likely have another mechanism for updating the summary).
Fixes https://github.com/ostreedev/ostree/issues/1664Closes: #1681
Approved by: jlebon
This ensures that commits aren't deleted and refs aren't added, removed,
or updated while the summary is being generated. This is in preparation
for adding a repo config option that will automatically regenerate the
summary on every ref change.
Closes: #1681
Approved by: jlebon
Now that it's possible to have both min-free-space-size and
min-free-space-percent set in a repo config, update the docs to make
the behavior clear in that case.
Closes: #1687
Approved by: jlebon
Since min_free_space_size_mb is considered before min_free_space_percent
in min_free_space_calculate_reserved_blocks(), it has to be considered
first when generating the error message in order for it to be accurate.
Closes: #1691
Approved by: jlebon
Currently when I run `ostree prune` it hits a seg fault when the
hash_func is used (in this case g_str_hash) from the call stack
_ostree_repo_prune_tmp() -> g_hash_table_contains() ->
g_hash_table_lookup_node(). So the key, in this case dent->d_name, must
be corrupt in some way.
glnx_dirfd_iterator_next_dent() uses readdir() to get the dirent struct.
And according to the man page for readdir(3), "POSIX.1 explicitly notes
that this field should not be used as an lvalue" (in reference to
d_name). So this commit avoids modifying d_name in place and copies it
instead. This seems to avoid the seg fault.
Closes: #1627
Approved by: jlebon
This adds subcommands that were missing from the ostree-admin man page,
and makes cosmetic fixes there and in the --help output to ensure
alphabetical order and remove trailing whitespace.
Closes: #1621
Approved by: jlebon
We need to include libglnx.h in places where ostree-autocleanups.h is
included, so that we get backports of G_DEFINE_AUTOPTR_CLEANUP_FUNC and
friends.
Closes: #1615
Approved by: jlebon
This reverts commit f1d9196076.
Since libglnx.h does not get installed, it can't be included in
ostree-autocleanups.h, which is included by ostree.h.
Closes: #1615
Approved by: jlebon
Currently the API that allows P2P operations (e.g. pulling an ostree ref
from a LAN or USB source) is hidden behind the configure flag
--enable-experimental-api. This commit makes the API public and makes
that flag essentially a no-op (leaving it in place in case we want to
use it again in the future). The P2P API has been tested over the last
several months and proven to work.
This means that since we're no longer using the "experimental" feature
flag, P2P builds of Flatpak will fail when using versions of OSTree from
this commit onwards, until Flatpak is patched in the near future. If you
want to build Flatpak < 0.11.8 with P2P enabled and link against OSTree
2018.6, you'll have to patch Flatpak. However, since Flatpak won't yet
have a hard dependency on OSTree 2018.6, it needs a new way to determine
if the P2P API in OSTree is available, so this commit adds a "p2p"
feature flag. This way the feature set is more semantically correct than
if we had continued to use the "experimental" feature flag.
In addition to making the P2P API public, this commit makes the P2P unit
tests run by default, removes the f27-experimental CI instance that's no
longer needed, changes a few man pages to reflect the changes, and
updates the bash completion script to accept the new commands and
options.
Closes: #1596
Approved by: cgwalters
This commit includes libglnx.h in ostree-autocleanups.h, so we get the
g_autoptr backports wherever they're needed. Also, remove the "#include
libglnx.h" lines elsewhere that are no longer needed.
Closes: #1596
Approved by: cgwalters
When building the OstreeBloom code against old versions of glib, we have
to have the libglnx headers included so that it defines
G_DEFINE_AUTOPTR_CLEANUP_FUNC and friends for us.
This is similarly true for test-repo-finder-mount.c which indirectly
includes ostree-autocleanups.h.
Closes: #1605
Approved by: cgwalters
This commit rearranges a few things in ostree-repo-pull.c so that OSTree
will successfully compile with experimental API enabled and without
libsoup, libcurl, or avahi:
./autogen.sh --enable-experimental-api --without-soup --without-curl
--without-avahi
This is accomplished with two sets of changes:
1. Move ostree_repo_resolve_keyring_for_collection() so it can be used
even without libsoup or libcurl.
2. Add stub functions for ostree_repo_find_remotes_async() and
ostree_repo_pull_from_remotes_async(), and their _finish() counterparts,
so they return an error when libsoup or libcurl isn't available.
Closes: #1605
Approved by: cgwalters
Currently if ostree is completing a file or directory for you it adds a
space to the end, but this is an inconvenience when it's an intermediate
directory in a tree. It's better to let the user add the space after the
final directory, so this commit changes the bash completion to avoid
adding a space when completing files or directories.
Closes: #1598
Approved by: jlebon
Currently the create-usb command only generates a summary file in the
destination repo if one doesn't already exist, which means if one does
exist it becomes out of date after the new refs are pulled. This commit
makes ostree regenerate the summary regardless of whether it exists, so
that consumers such as ostree_repo_find_remotes_async() (and at a higher
level, GNOME Software) get an accurate picture of the refs available on
the mount. This commit also updates one of the unit tests to check that
the summary is accurate after a second pull into the same repo.
Since any user of the create-usb command is using collection IDs they
are new enough to be using the unsigned summary support. While it would
technically be possible to use summary signatures on a repo and use the
create-usb command on it (a scenario broken by this commit), the
create-usb command is designed for P2P distribution of refs, which
requires use of unsigned summary support. So this is a legitimate
narrowing of the tool.
Fixes https://github.com/ostreedev/ostree/issues/1465Closes: #1543
Approved by: cgwalters
Currently OstreeRepoFinderResult, a data structure used by pull code
that supports P2P operations, has a hash table mapping refs to checksums
but doesn't include timestamp information. This means that clients have
no way of knowing just from the OstreeRepoFinderResult information if a
commit being offered by a peer remote is an update or downgrade until
they start pulling it. The client could check the summary or the commit
metadata for the timestamps, but this requires adding the temporary
remotes to the repo config, and ostree is already checking timestamps
before returning the results, so I think it makes more sense for them to
be returned rather than leaving it to the client. This limitation is
especially important for offline computers, because for online computers
the latest commit available from any remote is the latest commit,
period.
This commit adds a "ref_to_timestamp" hash table to
OstreeRepoFinderResult that is symmetric to "ref_to_checksum" in that it
shares the same keys. This is an API break, but it's part of the
experimental API, and none of the current users of that (flatpak,
eos-updater, and gnome-software) are affected. See the documentation for
more details on "ref_to_timestamp". One thing to note is the data
structure currently gets initialized in find_remotes_cb(), so only users
of ostree_repo_find_remotes_async() will get them, not users of, say,
ostree_repo_finder_resolve_all_async(). This is because the individual
OstreeRepoFinder implementations don't currently access the timestamps
(but I think this could be changed in the future if there's a need).
This commit will allow P2P support to be added to
flatpak_installation_list_installed_refs_for_update, which will allow
GNOME Software to update apps from USB drives while offline (it's
already possible online).
Closes: #1518
Approved by: cgwalters
In ostree_repo_remote_fetch_summary_with_options(), if no summary is
found on the server and summary verification is enabled, the error
message implies that it's the summary signature that's missing, which is
misleading. This commit adds a more specific error message for the case
of a missing summary, which has the side effect of explicitly checking
for the case that signatures != NULL && summary == NULL after
repo_remote_fetch_summary(), even though that should never happen.
One effect of this is that if you run "flatpak remote-add" with an
incorrect URL you get a more helpful error message, and similarly for
other flatpak operations and other users of ostree.
Closes: #1522
Approved by: cgwalters
In libostree, the phrase "commit metadata" has two meanings-- one is the
first dictionary in a commit GVariant that stores metadata such as ref
bindings, and the other is the commit metadata in the summary file,
which stores the commit size, checksum, and timestamp. In
find_remotes_process_refs(), the entire commit GVariant was being
referred to as commit metadata, so this commit changes the variable
name and a comment to make things more consistent.
Closes: #1528
Approved by: cgwalters
Ostree uses the OtChecksum data structure as a wrapper around GChecksum
(depending on what libraries are available at compile time). According
to the docs for g_checksum_get_digest(), a GChecksum value can no longer
be updated after that function is called. Ostree enforces this by
setting "initialized" to FALSE after getting the digest, but this leads
to ot_checksum_clear() avoiding freeing any memory, leading to leaks. So
this commit adds a "closed" value that gets set when getting a digest
and checked when updating the value, so the initialized value can be
used only for memory management.
Closes: #1521
Approved by: jlebon
ostree_repo_pull_from_remotes_async() passes along some options to
ostree_repo_pull_with_options(), so document them.
Closes: #1519
Approved by: cgwalters
The `ostree show` command is currently failing due to incorrect syntax,
but we want to check that it fails because the metadata isn't there.
Closes: #1520
Approved by: cgwalters
The _ostree_repo_get_remote() and _ostree_repo_get_remote_inherited()
methods transfer ownership of the returned OstreeRemote to the caller,
so this commit fixes a few call sites that weren't properly freeing it.
Closes: #1478
Approved by: cgwalters
This updates the gtk-doc comment for OstreeRepoFinderMount to match the
correct flatpak repo path, which was fixed in commit 6db6268df.
Closes: #1473
Approved by: cgwalters
The "ref_original_commits" hash table uses string values, not variants,
so fix the free function passed to g_hash_table_new_full (). Since
g_variant_unref isn't NULL safe, this prevents an assertion failure when
a NULL value is inserted.
Dan Nicholson suggested this patch; I'm just submitting it because he's
busy.
Fixes https://github.com/ostreedev/ostree/issues/1433Closes: #1474
Approved by: cgwalters
OstreeRepoFinderMount checks mounts for a few well-known directories
such as "ostree/repo" and ".ostree/repo" to try to find remotes. One of
the hard-coded directories is "var/lib/flatpak" but that's the flatpak
directory, not the ostree repo used by flatpak, which is at
"var/lib/flatpak/repo". So this commit changes the path so the repo can
be found.
For recent versions of Endless, flatpak uses /ostree/repo as its
repository, so this commit won't make a difference there. But it may
help on other operating systems.
Closes: #1471
Approved by: cgwalters
For P2P pulls ostree adds temporary remotes and removes them in
find_remotes_cb(). However, if an OstreeRepoFinderResult gets freed
during the course of that function, the OstreeRemote in the result is
freed but a pointer to it remains in the remotes_to_remove array. This
means that when _ostree_repo_remove_remote() gets called on it at the
end of the function it will fail. In my case the resulting error was
"OSTree-CRITICAL **: _ostree_repo_remove_remote: assertion 'remote->name
!= NULL' failed" but I think it could also seg fault.
This commit adds a reference to the remote so it can be properly removed
when we're finished with it.
Closes: #1450
Approved by: giuseppe
Currently users of the find_remotes_async()/pull_from_remotes_async()
functions have no way to specify a commit hash to use instead of the
latest one available. This commit implements an "override-commit-ids"
option analogous to the one used by ostree_repo_pull_with_options().
It's accomplished by returning OstreeRepoFinderResult objects pointing
to the given commit checksum(s) regardless of which ones were available
from the remotes, but in the future this implementation could be
improved to take into account the commits advertised by the remotes.
One effect of this is that flatpak will have the ability to downgrade
apps that use collection IDs
(https://github.com/flatpak/flatpak/issues/1309).
Closes: #1425
Approved by: pwithnall
Update the man page for the summary command to add the undocumented
options, make the syntax clear, and add examples.
Closes: #1416
Approved by: pwithnall
All the current uses of the find-remotes command in the tests use it to
find configured remotes or mounted (USB) remotes, so using
--finders=config and --finders=mount in the tests respectively shouldn't
affect the correctness of the tests. It does however allow the tests to
be run in an environment that doesn't have an Avahi daemon.
Closes: #1407
Approved by: cgwalters
It can be helpful to be able to choose which OstreeRepoFinder instances
to use when using the find-remotes command. For example, if the tests
need to run in an environment that can't have an Avahi daemon, this
allows you to disable the Avahi (LAN) finder. This commit adds the
--finders option for this purpose.
Closes: #1407
Approved by: cgwalters
This commit fixes an infinite loop that happens if you try to list the
remotes of a repo that has a parent repo set. It also adds a unit test
to ensure the right behavior, which is that both the child remotes and
parent remotes are listed.
Closes: #1366
Approved by: cgwalters
In case a filename contains invalid UTF-8 characters, libostree will
pass it to g_variant_builder_add() in create_tree_variant_from_hashes()
anyway, which leads to a critical warning from glib and an invalid
commit. This commit makes ostree print a useful error and exit instead.
Closes: #1271
Approved by: cgwalters
Update the comments and remove an unneeded variable to make it clear
that the find_remotes_async() / pull_from_remotes_async() functions use
the unsigned summary support.
This is a follow-up of commit 8c148eb7e "lib/repo-finder: Emit
gpg-verify-summary=false in dynamic remote config".
Closes: #1195
Approved by: pwithnall
We attempt to make deployments mutable in the test suite (as opposed to
immutable which is the default) to make it easier to chmod and clean up
the tmp files after each test. This is normally accomplished by setting
OSTREE_SYSROOT_DEBUG=mutable-deployments in libtest.sh, but that only
affects the environment variables for that bash instance, not the
process running gjs. So in test-sysroot.js OSTREE_SYSROOT_DEBUG wasn't
set when sysroot.deploy_tree() was called, which means the deployment
was made immutable which eventually causes the test to fail. This only
occurs when the test is run by the root user because for non-root users
_ostree_linuxfs_fd_alter_immutable_flag() would silently fail and the
deployment would be mutable.
This commit fixes this issue by setting the environment variable in
tests/test-sysroot.js.
Closes: #1122
Approved by: cgwalters
When working with collections it can be useful to see remote refs rather
than just local and mirrored ones. This commit changes the "ostree refs
-c" output to include remote refs, and includes remote refs with
collection IDs in summary file generation as well. The former behavior
is consistent with how "ostree refs" works, and the latter behavior is
useful in facilitating P2P updates even when mirrors haven't been
configured.
To accomplish this, OstreeRepoListRefsExtFlags was extended with an
EXCLUDE_REMOTES flag. This was done rather than an INCLUDE_REMOTES flag
so that existing calls to ostree_repo_list_refs_ext continue to have the
same behavior. This flag was added to ostree_repo_list_collection_refs
(which is an experimental API break).
Also, add unit tests for the "refs -c" and summary file behavior, and
update relevant tests.
Closes: #1069
Approved by: cgwalters
This commit sets prgname correctly so that the "ostree subcommand
--help" output prints the subcommand rather than just "ostree".
This was removed in commit f0519e541f because it tripped the thread
sanitizer, but it's being added back conditionally so most users who
don't compile with -fsanitize=adress see proper help output.
Closes: #1054
Approved by: cgwalters
Phaedrus Leeds