This patch makes it so that we mark the .commit file from a static delta as partial before writing the commit to the staging directory. This exactly mirrors what we do in meta_fetch_on_complete() when writing the commit on that codepath, which should lend some credibility to the correctness of this patch. I have checked that this fixes an issue Flatpak users have been encountering (https://github.com/flatpak/flatpak/issues/3479) which results in error messages like "error: Failed to install org.freedesktop.Sdk.Extension.texlive: Failed to read commit c7958d966cfa8b80a42877d1d6124831d7807f93c89461a2a586956aa28d438a: No such metadata object 8bdaa943b957f3cf14d19301c59c7eec076e57389e0fbb3ef5d30082e47a178f.dirtree" Here's the sequence of events that lead to the error: 1. An install operation is started that fetches static deltas. 2. The fetch is interrupted for some reason such as network connectivity dropping. 3. The .commit and .commitmeta files for the commit being pulled are left in the staging dir, e.g. "~/.local/share/flatpak/repo/tmp/staging-dfe862b2-13fc-49a2-ac92-5a59cc0d8e18-RURckd" 4. There is no `.commitpartial` file for the commit in "~/.local/share/flatpak/repo/state/" 5. The next time the user attempts the install, libostree reuses the existing staging dir, pulls the commit and commitmeta objects into the repo from the staging dir on the assumption that it's a complete commit. 6. Flatpak then tries to deploy the commit but fails in ostree_repo_read_commit() in flatpak_dir_deploy(), leading to the error message "Failed to read commit ..." 7. This happens again any subsequent time the user attempts the install, until the incomplete commit is removed with "flatpak repair --user". I will try to also add a workaround in Flatpak so this is fixed even when Flatpak links against affected versions of libostree. |
||
|---|---|---|
| .copr | ||
| .github | ||
| apidoc | ||
| bash | ||
| bsdiff@b817e9491c | ||
| build-aux | ||
| buildutil | ||
| ci | ||
| coccinelle | ||
| docs | ||
| libglnx@803adaf488 | ||
| man | ||
| manual-tests | ||
| src | ||
| tests | ||
| .cci.jenkinsfile | ||
| .dir-locals.el | ||
| .editorconfig | ||
| .gitmodules | ||
| .lgtm.yml | ||
| .packit.yaml | ||
| .vimrc | ||
| CONTRIBUTING.md | ||
| COPYING | ||
| Cargo.toml | ||
| GNUmakefile | ||
| Makefile-bash.am | ||
| Makefile-boot.am | ||
| Makefile-decls.am | ||
| Makefile-libostree-defines.am | ||
| Makefile-libostree.am | ||
| Makefile-man.am | ||
| Makefile-ostree.am | ||
| Makefile-otutil.am | ||
| Makefile-switchroot.am | ||
| Makefile-tests.am | ||
| Makefile.am | ||
| README.md | ||
| TODO | ||
| autogen.sh | ||
| cfg.mk | ||
| configure.ac | ||
| git.mk | ||
| maint.mk | ||
| mkdocs.yml | ||
| ostree.doap | ||
README.md
libostree
This project is now known as "libostree", though it is still appropriate to use the previous name: "OSTree" (or "ostree"). The focus is on projects which use libostree's shared library, rather than users directly invoking the command line tools (except for build systems). However, in most of the rest of the documentation, we will use the term "OSTree", since it's slightly shorter, and changing all documentation at once is impractical. We expect to transition to the new name over time.
As implied above, libostree is both a shared library and suite of command line tools that combines a "git-like" model for committing and downloading bootable filesystem trees, along with a layer for deploying them and managing the bootloader configuration.
The core OSTree model is like git in that it checksums individual files and has a content-addressed-object store. It's unlike git in that it "checks out" the files via hardlinks, and they thus need to be immutable to prevent corruption. Therefore, another way to think of OSTree is that it's just a more polished version of Linux VServer hardlinks.
Features:
- Transactional upgrades and rollback for the system
- Replicating content incrementally over HTTP via GPG signatures and "pinned TLS" support
- Support for parallel installing more than just 2 bootable roots
- Binary history on the server side (and client)
- Introspectable shared library API for build and deployment systems
- Flexible support for multiple branches and repositories, supporting projects like Flatpak which use libostree for applications, rather than hosts.
Documentation
For more information, see the project documentation or the project documentation website.
Operating systems and distributions using OSTree
Apertis uses libostree for their host system as well as Flatpak. See update documentation and apertis-update-manager
Endless OS uses libostree for their host system as well as Flatpak. See their eos-updater and deb-ostree-builder projects.
For Debian/apt, see also https://github.com/stb-tester/apt2ostree and the LWN article Merkle trees and build systems.
Fedora derivatives use rpm-ostree (noted below); there are 4 variants using OSTree:
Red Hat Enterprise Linux CoreOS is a derivative of Fedora CoreOS, used in OpenShift 4. The machine-config-operator manages upgrades. RHEL CoreOS is also the successor to RHEL Atomic Host, which uses rpm-ostree as well.
GNOME Continuous is where OSTree was born - as a high performance continuous delivery/testing system for GNOME.
GNOME OS is a testing OS that uses libostree for their host system as well as Flatpak.
Liri OS has the option to install their distribution using ostree.
Distribution build tools
meta-updater is a layer available for OpenEmbedded systems.
QtOTA is Qt's over-the-air update framework which uses libostree.
The BuildStream build and integration tool supports importing and exporting from libostree repos.
Fedora coreos-assembler is the build tool used to generate Fedora CoreOS derivatives.
Projects linking to libostree
rpm-ostree is used by the Fedora-derived operating systems listed above. It is a full hybrid image/package system. By default it uses libostree to atomically replicate a base OS (all dependency resolution is done on the server), but it supports "package layering", where additional RPMs can be layered on top of the base. This brings a "best of both worlds"" model for image and package systems.
eos-updater is a daemon that implements updates on EndlessOS.
Flatpak uses libostree for desktop application containers. Unlike most of the other systems here, Flatpak does not use the "libostree host system" aspects (e.g. bootloader management), just the "git-like hardlink dedup". For example, Flatpak supports a per-user OSTree repository.
Language bindings
libostree is accessible via GObject Introspection; any language which has implemented the GI binding model should work. For example, Both pygobject and gjs are known to work and further are actually used in libostree's test suite today.
Some bindings take the approach of using GI as a lower level and write higher level manual bindings on top; this is more common for statically compiled languages. Here's a list of such bindings:
Building
Releases are available as GPG signed git tags, and most recent versions support extended validation using git-evtag.
However, in order to build from a git clone, you must update the submodules. If you're packaging OSTree and want a tarball, I recommend using a "recursive git archive" script. There are several available online; this code in OSTree is an example.
Once you have a git clone or recursive archive, building is the same as almost every autotools project:
git submodule update --init
env NOCONFIGURE=1 ./autogen.sh
./configure --prefix=...
make
make install DESTDIR=/path/to/dest
Contact and discussion forums
OSTree has a mailing list and
there is also an #ostree channel on Libera.Chat. However, asynchronous+logged
communication is preferred for nontrivial questions.
Contributing
See Contributing.
Licensing
The licensing for the code of libostree can be canonically found in the individual files; and the overall status in the COPYING file in the source. Currently, that's LGPLv2+. This also covers the man pages and API docs.
The license for the manual documentation in the doc/ directory is:
SPDX-License-Identifier: (CC-BY-SA-3.0 OR GFDL-1.3-or-later)
This is intended to allow use by Wikipedia and other projects.
In general, files should have a SPDX-License-Identifier and that is canonical.