Commit Graph

48 Commits

Author SHA1 Message Date
Colin Walters 2346d5f4d5 libarchive: Handle `archive_entry_symlink()` returning NULL
The `archive_entry_symlink()` API can definitely return `NULL`,
reading through the libarchive sources.

I hit this in the wild when using old ostree-ext to try to unpack
a chunked archive.

I didn't try to characterize this more, and sorry no unit test right
now.
2022-04-04 10:25:35 -04:00
Colin Walters 48104f76ee lib/tar: Add some error prefixing
We're trying to debug a problem with a tar stream with hardlinks,
and I think this will be helpful.
2022-02-15 15:33:04 -05:00
Joseph Marrero 581a58067b Update FSF license notices to use URL instead of address 2021-12-07 08:34:25 -05:00
William Manley a88d2f5f7b ostree commit --tree=tar: Import xattrs from tarballs
If you specify an `xattr_callback` the xattrs will still be taken from
there for now.
2020-11-17 16:54:32 +00:00
Phaedrus Leeds c4f26bfdc8 Avoid shadowing local variables
This should help with code readability.

Fixes https://github.com/ostreedev/ostree/issues/2194
2020-09-13 10:08:02 -07:00
Dan Nicholson 1bbe674d91 libarchive: Support commit sizes metadata
Call the helper to set the generate_sizes boolean so that object size
data is stored while writing the mtree.
2020-01-20 20:42:27 -07:00
Tristan Van Berkom da4d0245af lib/commit: Added new ostree_repo_write_archive_to_mtree_from_fd()
Similar to ostree_repo_write_archive_to_mtree(), but takes
a file descriptor to read the archive from instead of mandating
a file path.

Usefull for importing archives into an OSTree repo over a socket
or from standard input in command line tools.

Closes: #1862
Approved by: jlebon
2019-05-28 17:03:22 +00:00
Marcus Folkesson 6bf4b3e1d8 Add SPDX-License-Identifier to source files
SPDX License List is a list of (common) open source
licenses that can be referred to by a “short identifier”.
It has several advantages compared to the common "license header texts"
usually found in source files.

Some of the advantages:
* It is precise; there is no ambiguity due to variations in license header
  text
* It is language neutral
* It is easy to machine process
* It is concise
* It is simple and can be used without much cost in interpreted
  environments like java Script, etc.
* An SPDX license identifier is immutable.
* It provides simple guidance for developers who want to make sure the
  license for their code is respected

See http://spdx.org for further reading.

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Closes: #1439
Approved by: cgwalters
2018-01-30 20:03:42 +00:00
Colin Walters a7f04a14b5 libarchive: Fix assertion error if used with CANONICAL_PERMISSIONS
I was trying to do a change for rpm-ostree to use
`OSTREE_REPO_COMMIT_MODIFIER_FLAGS_CANONICAL_PERMISSIONS`
for container builds with `bare-user-only,` but hit an assertion here
ultimtely because we weren't setting `standard::type`.

Rather than hand-rolling `GFileInfo` creation, use the stat buffer conversion
code which is more robust and used in multiple places already.

Closes: #1227
Approved by: jlebon
2017-09-28 19:59:24 +00:00
Colin Walters 6e4146a354 tree-wide: Remove Emacs modelines
We added a `.dir-locals.el` in commit: 9a77017d87
There's no need to have it per-file, with that people might think
to add other editors, which is the wrong direction.

Closes: #1206
Approved by: jlebon
2017-09-21 21:38:34 +00:00
Colin Walters 138c4d7aae libarchive: Add support for translating paths during commit
For rpm-ostree, I want to move RPM files in `/boot` to `/usr/lib/ostree-boot`.
This is currently impossible without forking the libarchive code.  Supporting
this is pretty straightforward; we already had pathname translation in
the libarchive code, we just need to expose it as an option.

On the command line side, I chose to wrap this as a regexp. That should be good
enough for a lot of use cases; sophisticated users should as always be making
use of the API. Note that this required some new `#ifdef LIBARCHIVE` bits to use
the new API. Following previous patterns here, we use the new API only if a
relevant option is enabled, ensuring unit test coverage of both paths.

For the test cases, I ended up changing the accounting to avoid having to
multiply the test count.

Closes: #1105
Approved by: jlebon
2017-08-30 14:30:30 +00:00
Colin Walters eb6f7c6db8 lib/commit: Honor commit filter for libarchive --tar-autocreate-parents
This makes `ostree commit --tree=tar` honor `--owner-uid` and `--owner-gid`
for the root directory.

Prep for further commit filtering work, although mostly for the unit test cases;
this ensures we can use `ostree checkout` after autocreating a root directory.

Closes: #1104
Approved by: jlebon
2017-08-23 14:48:12 +00:00
Colin Walters 4e068f3924 tree-wide: Fix the build with old glib (Ubuntu Trusty etc.)
This regressed with <https://github.com/ostreedev/ostree/pull/1040>
but currently the Travis builds aren't gating.

Closes: #1051
Approved by: jlebon
2017-08-03 16:23:41 +00:00
Colin Walters b929b620ae tree-wide: Use g_autoptr(Ostree*)
Part of cleaning up our usage of libglnx; we want to use what's in GLib where we
can.

Had to change a few .c files to `#include ostree.h` early on to pick up
autoptrs for the core types.

Closes: #1040
Approved by: jlebon
2017-08-03 13:48:12 +00:00
Colin Walters 40b41d2cc9 tree-wide: Use GLib autocleanups for libarchive
Define typedefs for read/write archives, and use the GLib
autocleanups for them.   Prep for updating libglnx to drop its
custom autocleanup macros.

Closes: #1042
Approved by: jlebon
2017-08-02 16:55:38 +00:00
Colin Walters d57410a7e6 lib: Add a helper to convert struct stat → GFileInfo
It's more natural for a few calling places. Prep for patches to go the other
way, which in turn are prep for adding a commit filter v2 that takes `struct
stat`.

`ot_gfile_type_for_mode()` was only used in this function, so inline it here.

Closes: #974
Approved by: jlebon
2017-06-29 18:17:28 +00:00
Jonathan Lebon 373dc4b66c codebase: start using GLNX_HASH_TABLE_FOREACH macros
Use the new macros introduced recently in libglnx to make iterating over
hash tables cleaner. This is just a start, it does not migrate the whole
tree.

Update submodule: libglnx

Closes: #971
Approved by: cgwalters
2017-06-28 16:37:15 +00:00
Colin Walters 88a1fc92a9 tree-wide: Add+run spatch to use glnx_throw()
I had to run a sed job to add whitespace after, but otherwise this was easy.

Closes: #890
Approved by: jlebon
2017-05-26 19:27:11 +00:00
Colin Walters 7bb0ff46a4 Define and use cleanup helpers for libarchive
This should fix some of the ASAN leaks around libarchive usage,
and is generally better.

Closes: #609
Approved by: jlebon
2016-12-05 15:20:56 +00:00
Colin Walters 49b8a72622 [ASAN] lib: Squash various leaks in library and commandline
The pull one is the most likely to affect users.  Otherwise mostly
just cleaning up `-fsanitize=address`.

Closes: #587
Approved by: jlebon
2016-11-21 16:34:06 +00:00
Colin Walters b77edf24a3 tree-wide: Remove unused variables detected by CLang
CLang finds these, whereas GCC treats having
`__attribute__((cleanup))` as a use.

This obsoletes https://github.com/ostreedev/ostree/pull/411

Closes: #548
Approved by: jlebon
2016-10-27 17:02:01 +00:00
Giuseppe Scrivano 0bac88e132 libostree: skip introspection for two functions
These two functions are not safe for gobject introspection, so annotate
them to be skipped:

1) ostree_repo_import_archive_to_mtree
2) ostree_repo_export_tree_to_archive

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>

Closes: #417
Approved by: cgwalters
2016-07-30 11:24:52 +00:00
Colin Walters 90b9a06277 lib: Use g_file_enumerator_iterate() if available, with fallback
Import `gs_file_enumerator_iterate()` for the next six months or
so...after RHEL 7.3 is released I'm strongly considering hard
requiring 2.46 or so.

Likely at some point we should figure out how to share more "glib
backport" code with NetworkManager at least.

Closes: #341
Approved by: jlebon
2016-06-21 18:24:17 +00:00
Mathnerd314 0e9a875393 repo: use OSTREE_TIMESTAMP (=1) for checked-out files
1 is a better choice than 0 because some programs use 0
as a special value; for example, GNU Tar warns of an
"implausibly old timestamp" with 0.

Closes: #330
Approved by: cgwalters
2016-06-09 18:04:55 +00:00
Krzesimir Nowak 862e6ecdcc libostree: Variant-related leak plugs and fixes
This tries to avoid leaking GVariantBuilders and GVariants in some
situations. The leaks were usually happening when some error occurred
or because of unclear variant ownership situation.

The former is mostly about making sure that g_variant_builder_clear is
called on builders that didn't finish their variant building process.

The latter is surely more work - sometimes the result of
g_variant_builder_end() should not be passed directly to a function,
but rather stored in a g_autoptr(GVariant), sunk and then passed to a
function. IMO, with an advent of g_autoptr, GVariants should be always
sunk instead of relying on some receiver function sinking it. This
would make an easy-to-follow policy of always sinking your
variants. Functions could then assume that the passed variant is
already sunk. These leaks are still happenning in commands, but they
are less harmful, since that code will not be used by some daemon as a
library routine.

Closes: #291
Approved by: cgwalters
2016-05-12 11:17:09 +00:00
Jonathan Lebon b1d3dd151c ostree-repo-libarchive.c: major refactor
- Make hardlink handling more generic. The previous strategy worked for
  tar archives, but not for cpio. It now works for both.
- Add support for SEL labeling (through the OstreeRepoCommitModifier)
- Add support for xattr_callback (through the OstreeRepoCommitModifier)
- Add support for filter (through the OstreeRepoCommitModifier)
- Add a use_ostree_convention option

Closes: #275
Approved by: cgwalters
2016-05-06 14:44:55 +00:00
Alexander Larsson 5595664e47 ostree export: Add --prefix option
This lets you set a prefix for the resulting archive patsh.
Especially useful in combination with --subpath, for instance
--subpath=subdir --prefix=subdir to extract just subdir.

Closes: #265
Approved by: cgwalters
2016-04-19 12:28:06 +00:00
Alexander Larsson 5079f70ec0 ostree export: Add --subpath support
This lets you export a subset of a commit.

Closes: #265
Approved by: cgwalters
2016-04-19 12:28:06 +00:00
Stef Walter 2454957e8d Fix building without libarchive
Although libarchive is an optional build option, the build
fails without it.

https://bugzilla.gnome.org/show_bug.cgi?id=764056
2016-03-23 13:55:34 -04:00
Colin Walters 61b4f26862 lib: Two more compiler warning fixes 2016-02-22 21:11:10 -05:00
Colin Walters 88ae6f1b9e libarchive: Fix a 32 bit format warning
This is actually an error by default with our default CFLAGS.
2016-02-22 20:52:33 -05:00
Colin Walters 30c5fb1a51 libarchive: Make autocreate_parents imply autocreating root dir
Some Docker layers are just metadata in the `layer.json`.  If one is
mapping Docker layers to OSTree commits, one needs to create a dummy
root directory, because OSTree doesn't support metadata-only commits.

Let's just push that logic down here because it's easier than special
casing it in higher levels.
2016-02-22 17:02:31 -05:00
Colin Walters 3a555114bc repo: Add ostree_repo_import_archive_to_mtree
This is a more flexible version of the previous
ostree_repo_write_archive_to_mtree() which took a file reference.
This has an extensible options structure, and in particular
now supports `ignore_unsupported_content`.

I plan to use this for importing Docker images which contain device
nodes.  (There's no reason for container images to have those, so
we'll just ignore them).

Also here, just like the export variant, the caller is responsible for
setting up libarchive.
2016-02-19 08:54:04 -05:00
Colin Walters b08b1abccd Rename libarchive write API to "export", matching command line
I was going to add new API for importing, and it was really confusing
that what I think of now as import and export both had "write" in the
name.  It's just clearer to talk about the direction.

At the same time, include `Export` in the options structure.

This isn't an ABI break as the API isn't in a release.
2016-02-19 08:54:04 -05:00
Colin Walters 355f8438ef Add an `export` builtin, and API to write to libarchive
At the moment I'm looking at using rpm-ostree to manage RPM inputs
which can then be converted into Docker images.  It's most convenient
if we can stream directly out of libostree rather than doing a
checkout + tar combination.

There are also backup/debugging etc. reasons to implement `export` as
well.
2016-02-14 09:53:01 -05:00
Matthew Barnes bb231fdf74 Use g_autoptr(GPtrArray) instead of gs_unref_ptrarray 2015-05-06 22:07:10 -04:00
Matthew Barnes 6a5f7b1288 Use glnx_unref_object instead of gs_unref_object
For non-GIO object types, at least until autocleanup support for GObject
based types becomes more widespread.
2015-05-06 22:07:04 -04:00
Matthew Barnes 4ee1acd981 Use g_autoptr() for GIO object types
GLib 2.44 supplies all the necessary autocleanup macros for GIO types,
and libglnx backports the relevant macros for ostree.
2015-05-06 21:51:19 -04:00
Matthew Barnes 7a62d64968 Use g_autofree instead of gs_free 2015-05-06 21:50:17 -04:00
Colin Walters 125889fd7e Enforce 'rdev' (device file major/minor) is 0
Historically OSTree supported device files, but it wasn't useful, and
added attack surface.  Support was removed in

https://git.gnome.org/browse/ostree/commit/?id=62a896350bd54bff5a9413d2ee0fad7ff4364f9a

Perform a further cleanup by enforcing internally that the device
major/minor must be 0.

Conflicts:
	src/libostree/ostree-core.c
2014-12-19 10:20:05 -05:00
Colin Walters c4efbf6718 core: Deduplicate code converting struct stat -> GFileInfo
We were doing the same thing in a number of places, make a helper
function.
2014-12-19 10:18:59 -05:00
Colin Walters 9f734494ff libostree: Make OstreeLibarchiveInputStream explicitly private
We weren't installing the headers, but at the moment all symbols
starting with ostree_ were being exported.  Fix that by prefixing
non-static symbols with '_'.

https://bugzilla.gnome.org/show_bug.cgi?id=731369
2014-06-09 14:04:53 -04:00
Jasper St. Pierre 6b4ef97c18 repo-libarchive: Apply commit modifiers to libarchive archives as well
And document the libarchive methods as well, so we can pass a NULL
commit modifier.

https://bugzilla.gnome.org/show_bug.cgi?id=707727
2013-09-09 22:51:43 -04:00
Jasper St. Pierre 5082e1d8e9 repo: Rename "stage" to "write" in the API
An earlier version of this API acted like git in that some objects
would be staged in a temporary directory which would be then committed
in one go by moving files around. The API doesn't match most users
expectations though, as while the stage is nice as a high-level API
it isn't really suited for low-level APIs.

While the stage was removed, the APIs were never renamed. Rename
them now so that they match expectations.

https://bugzilla.gnome.org/show_bug.cgi?id=707644
2013-09-06 20:31:12 -04:00
Colin Walters 23e2593b6d repo: Use non-deprecated API with sufficiently new libarchive
Based on a patch Tobias Hunger <tobias.hunger@gmail.com>
2013-08-30 14:34:32 -04:00
Colin Walters a5d43bb959 Install a shared library
This required a fair bit of surgery because previously ostree.h
included otutil.h, but that's supposed to be a private library.
2013-07-26 19:25:07 -04:00
Colin Walters 5aea9db3f2 Add API to retrieve statistics from transactions, use it in commit
It's just interesting to see =) This is also kind of prototyping out
some more "structured" output.
2013-07-25 19:23:25 -04:00
Colin Walters 3b6a94d272 libostree: Split off libarchive code 2013-07-09 19:11:37 -04:00