packaging
/
ostree
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ostree/src/libostree
History
Luca BRUNO c6b72f527b
lib/core/checksum: add flag to use canonical permissions 
This adds a new `OSTREE_CHECKSUM_FLAGS_CANONICAL_PERMISSIONS`
checksumming flag, which is needed in bare-user-only mode
to ignore local IDs.
 		2021-08-19 14:08:36 +00:00
..
README-gpg pull: Verify commits with gpg signatures from detached metadata 2013-09-29 14:49:47 -04:00
README.md core: Associate branches with remotes, move trigger runs into checkout 2012-04-03 23:46:34 -04:00
bupsplit.c lib: Move the bupsplit selftest into our test framework 2017-02-02 16:51:36 +00:00
bupsplit.h lib: Move the bupsplit selftest into our test framework 2017-02-02 16:51:36 +00:00
libostree-devel.sym Release 2021.3 2021-07-12 08:38:38 +00:00
libostree-released.sym Release 2021.3 2021-07-12 08:38:38 +00:00
ostree-1.pc.in libostree: Expose $OSTREE_FEATURES in the pkg-config file 2017-05-08 18:48:07 +00:00
ostree-async-progress.c libostree: Add ostree_async_progress_copy_state() 2019-11-20 12:24:26 -08:00
ostree-async-progress.h libostree: Add ostree_async_progress_copy_state() 2019-11-20 12:24:26 -08:00
ostree-autocleanups.h lib/sign: allow to build with glib version less than 2.44 2020-03-25 15:23:54 +03:00
ostree-bloom-private.h Fix building P2P code against old glib versions 2018-05-30 14:19:25 +00:00
ostree-bloom.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-bootconfig-parser.c lib/bootconfig: Add support for multiple initrd keys 2020-09-30 13:29:32 -04:00
ostree-bootconfig-parser.h lib/bootconfig: Add support for multiple initrd keys 2020-09-30 13:29:32 -04:00
ostree-bootloader-grub2.c Add configure option for unsuffixed GRUB2 commands 2021-03-18 14:31:28 +01:00
ostree-bootloader-grub2.h tree-wide: Remove Emacs modelines 2017-09-21 21:38:34 +00:00
ostree-bootloader-syslinux.c sysroot: Support /boot on root or as seperate filesystem for syslinux and u-boot 2020-07-15 20:37:49 +01:00
ostree-bootloader-syslinux.h tree-wide: Remove Emacs modelines 2017-09-21 21:38:34 +00:00
ostree-bootloader-uboot.c sysroot: Support /boot on root or as seperate filesystem for syslinux and u-boot 2020-07-15 20:37:49 +01:00
ostree-bootloader-uboot.h tree-wide: Remove Emacs modelines 2017-09-21 21:38:34 +00:00
ostree-bootloader-zipl.c bootloader: Add a zipl bootloader backend 2019-10-16 14:22:02 +00:00
ostree-bootloader-zipl.h bootloader: Add a zipl bootloader backend 2019-10-16 14:22:02 +00:00
ostree-bootloader.c bootloader: Add a zipl bootloader backend 2019-10-16 14:22:02 +00:00
ostree-bootloader.h bootloader: Add a zipl bootloader backend 2019-10-16 14:22:02 +00:00
ostree-chain-input-stream.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-chain-input-stream.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-checksum-input-stream.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-checksum-input-stream.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-cmdprivate.c lib: Move internal binding verification API to repo.c 2020-04-17 01:16:51 +00:00
ostree-cmdprivate.h Add concept of "staged" deployment 2018-04-12 14:55:12 +00:00
ostree-content-writer.c repo: Add ostree_repo_write_regfile 2021-04-09 21:54:44 +00:00
ostree-content-writer.h repo: Add ostree_repo_write_regfile 2021-04-09 21:54:44 +00:00
ostree-core-private.h deltas: Add _ostree_get_relative_static_delta_index_path() 2020-10-23 12:30:08 +02:00
ostree-core.c lib/core/checksum: add flag to use canonical permissions 2021-08-19 14:08:36 +00:00
ostree-core.h lib/core/checksum: add flag to use canonical permissions 2021-08-19 14:08:36 +00:00
ostree-date-utils-private.h tests: Split RFC 2616 date parsing code out and add tests 2020-10-22 21:03:34 +01:00
ostree-date-utils.c tests: Split RFC 2616 date parsing code out and add tests 2020-10-22 21:03:34 +01:00
ostree-deployment-private.h lib/deploy: Add support for overlay initrds 2020-09-30 13:29:32 -04:00
ostree-deployment.c deployment: Add a bunch of docs and fix annotations 2020-11-11 22:01:11 +00:00
ostree-deployment.h Add "transient" unlock 2020-08-07 18:57:56 +00:00
ostree-diff.c lib: add minimum version to various symbols based on libostree-released.sym 2019-05-19 14:50:08 +00:00
ostree-diff.h Drop use of `volatile` 2020-11-02 14:53:26 -05:00
ostree-dummy-enumtypes.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-dummy-enumtypes.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-enumtypes.c.template Drop use of `volatile` 2020-11-02 14:53:26 -05:00
ostree-enumtypes.h.template Make enum generation private by default, export stub symbol 2016-05-12 17:15:50 +00:00
ostree-fetcher-curl.c lib/fetch-curl: Unref timeout source 2020-10-27 12:01:01 -04:00
ostree-fetcher-soup.c Drop use of `volatile` 2020-11-02 14:53:26 -05:00
ostree-fetcher-uri.c pull: Cleanly error out on unknown schemes 2021-04-27 13:11:18 -04:00
ostree-fetcher-util.c libostree: Add support for ETag and Last-Modified headers 2020-10-22 21:03:34 +01:00
ostree-fetcher-util.h libostree: Add support for ETag and Last-Modified headers 2020-10-22 21:03:34 +01:00
ostree-fetcher.h pull: Cleanly error out on unknown schemes 2021-04-27 13:11:18 -04:00
ostree-gpg-verifier.c lib: Kill GPG agent when cleaning up tmp homedirs 2019-06-19 17:30:24 +00:00
ostree-gpg-verifier.h lib/repo: Search a list of paths in gpgkeypath for gpg keys 2018-11-21 17:03:10 +00:00
ostree-gpg-verify-result-dummy.c gpg: add dummy public interface 2019-08-01 02:06:47 +00:00
ostree-gpg-verify-result-private.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-gpg-verify-result.c lib/gpg: Add more specific OstreeGpgError codes 2020-01-24 13:02:17 -07:00
ostree-gpg-verify-result.h lib: Coerce flags enums to GIR bitfields 2020-04-30 15:13:31 -06:00
ostree-impl-system-generator.c Don't fail build when systemd unit path not defined 2021-06-18 12:35:41 -06:00
ostree-kernel-args.c lib/kernel-args: Store kernel args as key/value entries 2019-11-07 23:39:10 -05:00
ostree-kernel-args.h lib/kernel-args: Store kernel args as key/value entries 2019-11-07 23:39:10 -05:00
ostree-libarchive-input-stream.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-libarchive-input-stream.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-libarchive-private.h lib/commit: Added new ostree_repo_write_archive_to_mtree_from_fd() 2019-05-28 17:03:22 +00:00
ostree-linuxfsutil.c linuxfsutil: Pass int to ioctl, not long 2020-08-21 17:41:32 +00:00
ostree-linuxfsutil.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-lzma-common.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-lzma-common.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-lzma-compressor.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-lzma-compressor.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-lzma-decompressor.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-lzma-decompressor.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-metalink.c libostree: Add support for ETag and Last-Modified headers 2020-10-22 21:03:34 +01:00
ostree-metalink.h lib/repo-pull: Support retrying requests on transient network errors 2018-05-30 16:23:57 +00:00
ostree-mutable-tree.c lib: add some missing (out) annotations 2019-05-19 14:50:08 +00:00
ostree-mutable-tree.h OstreeMutableTree: add _remove method 2018-09-20 17:49:55 +00:00
ostree-ref.c lib: fix up some nullability annotations 2019-05-19 14:50:08 +00:00
ostree-ref.h Make P2P API public (no longer experimental) 2018-06-04 19:20:10 +00:00
ostree-remote-private.h Drop use of `volatile` 2020-11-02 14:53:26 -05:00
ostree-remote.c Make P2P API public (no longer experimental) 2018-06-04 19:20:10 +00:00
ostree-remote.h Make P2P API public (no longer experimental) 2018-06-04 19:20:10 +00:00
ostree-repo-checkout.c checkout: Don't hardlink zero sized files 2020-10-01 16:47:07 -04:00
ostree-repo-commit.c lib/commit: respect SOURCE_DATE_EPOCH for commit timestamp 2021-06-22 12:15:18 +00:00
ostree-repo-deprecated.h lib: Squash two gtk-doc warnings 2020-04-04 17:05:56 +00:00
ostree-repo-file-enumerator.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-repo-file-enumerator.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-repo-file.c lib: add some missing (out) annotations 2019-05-19 14:50:08 +00:00
ostree-repo-file.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-repo-finder-avahi-parser.c Avoid shadowing local variables 2020-09-13 10:08:02 -07:00
ostree-repo-finder-avahi-private.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-repo-finder-avahi.c lib: fix typo in function docs 2020-04-01 23:06:19 +02:00
ostree-repo-finder-avahi.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-repo-finder-config.c OstreeRepoFinderConfig: Fix guint/gsize confusion 2018-06-27 15:24:18 +00:00
ostree-repo-finder-config.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-repo-finder-mount.c Avoid shadowing local variables 2020-09-13 10:08:02 -07:00
ostree-repo-finder-mount.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-repo-finder-override.c Avoid shadowing local variables 2020-09-13 10:08:02 -07:00
ostree-repo-finder-override.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-repo-finder.c Fix building against old glib versions 2018-06-07 17:28:49 +00:00
ostree-repo-finder.h ostree-repo-finder: Clarify immutability after construction 2019-05-29 03:36:06 +00:00
ostree-repo-libarchive.c ostree commit --tree=tar: Import xattrs from tarballs 2020-11-17 16:54:32 +00:00
ostree-repo-os.c Add an API+CLI to inject metadata for bootable OSTree commits 2021-03-12 19:01:42 +00:00
ostree-repo-os.h Add an API+CLI to inject metadata for bootable OSTree commits 2021-03-12 19:01:42 +00:00
ostree-repo-private.h repo: Make locking per-OstreeRepo 2021-06-05 09:15:32 -06:00
ostree-repo-prune.c repo: Make locking APIs public 2021-06-05 09:00:21 -06:00
ostree-repo-pull-private.h pull: Error on depth pull with missing head commit 2021-01-12 14:19:01 -07:00
ostree-repo-pull-verify.c signapi: Change API to also return a success message 2020-06-17 00:33:47 +00:00
ostree-repo-pull.c pull: Cleanly error out on unknown schemes 2021-04-27 13:11:18 -04:00
ostree-repo-refs.c refs: Make ostree_repo_resolve_rev{,_ext}() use (nullable) 2021-02-19 01:11:43 +00:00
ostree-repo-static-delta-compilation-analysis.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-repo-static-delta-compilation.c deltas: Fix leak of matches 2021-02-02 21:26:43 +00:00
ostree-repo-static-delta-core.c repo: Make locking APIs public 2021-06-05 09:00:21 -06:00
ostree-repo-static-delta-private.h deltas: Update delta indexes when updating summary 2020-10-23 12:30:08 +02:00
ostree-repo-static-delta-processing.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-repo-traverse.c lib/repo: Add ostree_repo_traverse_commit_union_with_parents 2018-04-14 15:36:21 +00:00
ostree-repo-verity.c repo: Move fsverity bits to ostree-repo-verity.c 2021-01-11 14:36:40 +00:00
ostree-repo.c repo: Use g_new for OstreeRepoAutoLock 2021-06-07 13:14:05 -06:00
ostree-repo.h repo: Require lock type in ostree_repo_lock_pop 2021-06-05 09:07:39 -06:00
ostree-rollsum.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-rollsum.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-sepolicy-private.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-sepolicy.c lib: add minimum version to various symbols based on libostree-released.sym 2019-05-19 14:50:08 +00:00
ostree-sepolicy.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-sign-dummy.c lib/sign-dummy: Handle incorrect signatures correctly 2021-08-03 16:49:49 -04:00
ostree-sign-dummy.h signapi: Change API to also return a success message 2020-06-17 00:33:47 +00:00
ostree-sign-ed25519.c Merge pull request #2131 from cgwalters/sign-success 2020-06-24 18:24:41 -04:00
ostree-sign-ed25519.h signapi: Change API to also return a success message 2020-06-17 00:33:47 +00:00
ostree-sign.c signatures: Fix leak in _sign_detached_metadata_append() 2020-10-08 14:10:04 +02:00
ostree-sign.h lib: add some missing version tags 2020-08-26 22:32:47 +02:00
ostree-soup-form.c lib/repo: Add (transfer) annotations to various GHashTable arguments 2017-11-14 23:13:14 +00:00
ostree-soup-uri.c Avoid shadowing local variables 2020-09-13 10:08:02 -07:00
ostree-soup-uri.h libcurl backend 2017-02-09 16:37:45 +00:00
ostree-sysroot-cleanup.c repo: Make locking APIs public 2021-06-05 09:00:21 -06:00
ostree-sysroot-deploy.c Use generator to enable ostree-remount.service and ostree-finalize-staged.path 2021-06-16 09:40:28 -04:00
ostree-sysroot-private.h Use generator to enable ostree-remount.service and ostree-finalize-staged.path 2021-06-16 09:40:28 -04:00
ostree-sysroot-upgrader.c Drop use of `volatile` 2020-11-02 14:53:26 -05:00
ostree-sysroot-upgrader.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-sysroot.c lib/sysroot: Fix error message about creating `/var/lib` 2021-08-03 16:49:49 -04:00
ostree-sysroot.h sysroot: Add _require_booted_deployment() API 2021-03-17 19:55:56 +00:00
ostree-tls-cert-interaction.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-tls-cert-interaction.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-types.h repo: Add ostree_repo_write_regfile 2021-04-09 21:54:44 +00:00
ostree-varint.c Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-varint.h Add SPDX-License-Identifier to source files 2018-01-30 20:03:42 +00:00
ostree-version.h.in Add OSTREE_BUILT_FEATURES to ostree-version.h 2019-07-29 13:58:13 +00:00
ostree.h Add an API+CLI to inject metadata for bootable OSTree commits 2021-03-12 19:01:42 +00:00

README.md

Repository design

At the heart of OSTree is the repository. It's very similar to git, with the idea of content-addressed storage. However, OSTree is designed to store operating system binaries, not source code. There are several consequences to this. The key difference as compared to git is that the OSTree definition of "content" includes key Unix metadata such as owner uid/gid, as well as all extended attributes.

Essentially OSTree is designed so that if two files have the same OSTree checksum, it's safe to replace them with a hard link. This fundamental design means that an OSTree repository imposes negligible overhead. In contrast, a git repository stores copies of zlib-compressed data.

Key differences versus git

  • As mentioned above, extended attributes and owner uid/gid are versioned
  • Optimized for Unix hardlinks between repository and checkout
  • SHA256 instead of SHA1
  • Support for empty directories

Binary files

While this is still in planning, I plan to heavily optimize OSTree for versioning ELF operating systems. In industry jargon, this would be "content-aware storage".

Trimming history

OSTree will also be optimized to trim intermediate history; in theory one can regenerate binaries from corresponding (git) source code, so we don't need to keep all possible builds over time.

MILESTONE 1

  • Basic pack files (like git)

MILESTONE 2

  • Store checksums as ay
  • Drop version/metadata from tree/dirmeta objects
  • Add index size to superindex, pack size to index
    • So pull can calculate how much we need to download
  • Split pack files into metadata/data
  • pull: Extract all we can from each packfile one at a time, then delete it
  • Restructure repository so that links can be generated as a cache; i.e. objects/raw, pack files are now the canonical
  • For files, checksum combination of metadata variant + raw data
    • i.e. there is only OSTREE_OBJECT_TYPE_FILE (again)

MILESTONE 3

  • Drop archive/raw distinction - archive repositories always generate packfiles per commit
  • Include git packv4 ideas:
    • metadata packfiles have string dictionary (tree filenames and checksums)
    • data packfiles match up similar objects
  • Rolling checksums for partitioning large files? Kernel debuginfo
  • Improved pack clustering
    • file fingerprinting?
  • ELF-x86 aware deltas

Related work in storage

git: http://git-scm.com/ Venti: http://plan9.bell-labs.com/magic/man2html/6/venti Elephant FS: http://www.hpl.hp.com/personal/Alistair_Veitch/papers/elephant-hotos/index.html

Compression

xdelta: http://xdelta.org/ Bsdiff: http://www.daemonology.net/bsdiff/ xz: http://tukaani.org/xz/