deploy: Add a 5s max timeout on global filesystem sync()

https://bugzilla.redhat.com/show_bug.cgi?id=2003532

Basically there's a systemd bug where it's losing the `_netdev`
aspect of Ceph filesystem mounts.  This means the network is taken
down before Ceph is unmounted.  In turn, our invocation of `sync()`
blocks on Ceph, which won't succeed.

And this in turn manifests as a failure to transition to the new
deployment.

I initially did this patch to just rip out the global `sync()`.  I
am pretty sure we don't need it anymore.  We've been doing individual
`syncfs()` on `/sysroot` and `/boot` for a while now, and those
are the only filesystems we should be touching.  But *proving* that
is a whole other thing of course.

To be conservative, let's instead just add a timeout of 5s on
our invocation of `sync()`.  It doesn't return any information on
success/error anyways.

To allow testing without the `sync()` invocation, we also support
a new `OSTREE_SYSROOT_OPT_SKIP_SYNC=1` environment variable.  For
staged deployments, this needs to be injected via e.g. systemd unit
overrides into `ostree-finalize-staged.service`.

Implementing this is a bit hairy - we need to spawn a thread.  I
debated blocking in arecursive mainloop, but I think `g_cond_wait_until()`
is also fine here.
This commit is contained in:
Colin Walters committed 2022-01-18 09:19:20 -05:00
1 parent 0095f7c472
commit cb73129483
3 files changed
+78 -1

No files matched your search

+5
View File
@@ -184,6 +184,9 @@ ostree_sysroot_class_init (OstreeSysrootClass *klass)
static void
ostree_sysroot_init (OstreeSysroot *self)
{
const GDebugKey globalopt_keys[] = {
{ "skip-sync", OSTREE_SYSROOT_GLOBAL_OPT_SKIP_SYNC },
};
const GDebugKey keys[] = {
{ "mutable-deployments", OSTREE_SYSROOT_DEBUG_MUTABLE_DEPLOYMENTS },
{ "test-fifreeze", OSTREE_SYSROOT_DEBUG_TEST_FIFREEZE },
@@ -191,6 +194,8 @@ ostree_sysroot_init (OstreeSysroot *self)
{ "no-dtb", OSTREE_SYSROOT_DEBUG_TEST_NO_DTB },
};
self->opt_flags = g_parse_debug_string (g_getenv ("OSTREE_SYSROOT_OPTS"),
globalopt_keys, G_N_ELEMENTS (globalopt_keys));
self->debug_flags = g_parse_debug_string (g_getenv ("OSTREE_SYSROOT_DEBUG"),
keys, G_N_ELEMENTS (keys));