deploy: Add a 5s max timeout on global filesystem sync()

https://bugzilla.redhat.com/show_bug.cgi?id=2003532

Basically there's a systemd bug where it's losing the `_netdev`
aspect of Ceph filesystem mounts.  This means the network is taken
down before Ceph is unmounted.  In turn, our invocation of `sync()`
blocks on Ceph, which won't succeed.

And this in turn manifests as a failure to transition to the new
deployment.

I initially did this patch to just rip out the global `sync()`.  I
am pretty sure we don't need it anymore.  We've been doing individual
`syncfs()` on `/sysroot` and `/boot` for a while now, and those
are the only filesystems we should be touching.  But *proving* that
is a whole other thing of course.

To be conservative, let's instead just add a timeout of 5s on
our invocation of `sync()`.  It doesn't return any information on
success/error anyways.

To allow testing without the `sync()` invocation, we also support
a new `OSTREE_SYSROOT_OPT_SKIP_SYNC=1` environment variable.  For
staged deployments, this needs to be injected via e.g. systemd unit
overrides into `ostree-finalize-staged.service`.

Implementing this is a bit hairy - we need to spawn a thread.  I
debated blocking in arecursive mainloop, but I think `g_cond_wait_until()`
is also fine here.
This commit is contained in:
Colin Walters committed 2022-01-18 09:19:20 -05:00
1 parent 0095f7c472
commit cb73129483
3 files changed
+78 -1

No files matched your search

+6
View File
@@ -38,6 +38,11 @@ typedef enum {
OSTREE_SYSROOT_DEBUG_TEST_NO_DTB = 1 << 3, /* https://github.com/ostreedev/ostree/issues/2154 */
} OstreeSysrootDebugFlags;
typedef enum {
/* Skip invoking `sync()` */
OSTREE_SYSROOT_GLOBAL_OPT_SKIP_SYNC = 1 << 0,
} OstreeSysrootGlobalOptFlags;
typedef enum {
OSTREE_SYSROOT_LOAD_STATE_NONE, /* ostree_sysroot_new() was called */
OSTREE_SYSROOT_LOAD_STATE_INIT, /* We've loaded basic sysroot state and have an fd */
@@ -75,6 +80,7 @@ struct OstreeSysroot {
/* Only access through ostree_sysroot_[_get]repo() */
OstreeRepo *repo;
OstreeSysrootGlobalOptFlags opt_flags;
OstreeSysrootDebugFlags debug_flags;
};