From b7662aaf3345f17e234c2ba868b526a3c1baad4d Mon Sep 17 00:00:00 2001 From: Stefan Agner Date: Fri, 24 Apr 2020 13:02:05 +0200 Subject: [PATCH 1/3] docs: clarify archive repo type Today `archive-z2` is still used as the default string to indicate a `archive` type repository. Make clear that this is the way it is intended. Otherwise users might think they use an no longer supported OSTree repository since the mode string is still `archive-z2`. Signed-off-by: Stefan Agner --- docs/manual/formats.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/manual/formats.md b/docs/manual/formats.md index 1a669bb1..884b1b5e 100644 --- a/docs/manual/formats.md +++ b/docs/manual/formats.md @@ -22,8 +22,9 @@ where file/content objects are checksummed and managed individually. The `archive` format simply gzip-compresses each content object. Metadata objects are stored uncompressed. This means that it's easy -to serve via static HTTP. Note: this format used to be called `archive-z2` -for historical reasons. +to serve via static HTTP. Note: the repo config file still uses the +historical term `archive-z2` as mode. But this essentially indicates +the modern `archive` format. When you commit new content, you will see new `.filez` files appearing in `objects/`. From b43c0be3471feb19b1718da7ef3aa44a91b36393 Mon Sep 17 00:00:00 2001 From: Stefan Agner Date: Fri, 24 Apr 2020 13:05:15 +0200 Subject: [PATCH 2/3] docs: extend object type documentation Extend the object type documentation with file endings used for the individual type. Also clarify in which situation content type objects are used and why they do not match the SHA256 hash today. Signed-off-by: Stefan Agner --- docs/manual/repo.md | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/docs/manual/repo.md b/docs/manual/repo.md index 08fd5fe2..b8c3ea54 100644 --- a/docs/manual/repo.md +++ b/docs/manual/repo.md @@ -31,13 +31,16 @@ regenerate it from source code. A dirtree contains a sorted array of (filename, checksum) pairs for content objects, and a second sorted array of (filename, dirtree checksum, dirmeta checksum), which are -subdirectories. +subdirectories. These type of objects are stored as files +ending with `.dirtree` in the objects directory. ### Dirmeta objects In git, tree objects contain the metadata such as permissions for their children. But OSTree splits this into a separate object to avoid duplicating extended attribute listings. +These type of objects are stored as files ending with `.dirmeta` +in the objects directory. ### Content objects @@ -45,7 +48,13 @@ Unlike the first three object types which are metadata, designed to be `mmap()`ed, the content object has a separate internal header and payload sections. The header contains uid, gid, mode, and symbolic link target (for symlinks), as well as extended attributes. After the -header, for regular files, the content follows. +header, for regular files, the content follows. These parts toghether +form the SHA256 hash for content objects. The content type objects in +this format exist only in `archive` OSTree repositories. Today the +content part is gzip'ed and the objects are stored as files ending +with `.filez` in the objects directory. Because the SHA256 hash is +formed over the uncompressed content, these files do not match the +hash they are named as. The OSTree data format intentionally does not contain timestamps. The reasoning is that data files may be downloaded at different times, and by different build From ce5dfadbd7fa7ba00c4964bb86aa1e3579d0a911 Mon Sep 17 00:00:00 2001 From: Stefan Agner Date: Fri, 24 Apr 2020 13:12:47 +0200 Subject: [PATCH 3/3] docs: extend repository types Clarify where metadata are stored exactly in the `bare-user` case. Make the first sentence of `bare-user` and `bare-user-only` paragraph symetric to make it easier to jump to the right paragraph for readers in a hury. Stree out that `bare-user-only` may loose metadata. Signed-off-by: Stefan Agner --- docs/manual/repo.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/docs/manual/repo.md b/docs/manual/repo.md index b8c3ea54..07cc1a93 100644 --- a/docs/manual/repo.md +++ b/docs/manual/repo.md @@ -82,20 +82,21 @@ designed to be the source of a "hardlink farm", where each operating system checkout is merely links into it. If you want to store files owned by e.g. root in this mode, you must run OSTree as root. -The `bare-user` is a later addition that is like `bare` in that files -are unpacked, but it can (and should generally) be created as +The `bare-user` mode is a later addition that is like `bare` in that +files are unpacked, but it can (and should generally) be created as non-root. In this mode, extended metadata such as owner uid, gid, and -extended attributes are stored but not actually applied. +extended attributes are stored in extended attributes under the name +`user.ostreemeta` but not actually applied. The `bare-user` mode is useful for build systems that run as non-root but want to generate root-owned content, as well as non-root container systems. -There is a variant to the `bare-user` mode called `bare-user-only`. Unlike +The `bare-user-only` mode is a variant to the `bare-user` mode. Unlike `bare-user`, neither ownership nor extended attributes are stored. These repos are meant to to be checked out in user mode (with the `-U` flag), where this -information is not applied anyway. The main advantage of `bare-user-only` is -that repos can be stored on filesystems which do not support extended -attributes, such as tmpfs. +information is not applied anyway. Hence this mode may loose metadata. +The main advantage of `bare-user-only` is that repos can be stored on +filesystems which do not support extended attributes, such as tmpfs. In contrast, the `archive` mode is designed for serving via plain HTTP. Like tar files, it can be read/written by non-root users.