bubblewrap should fall back to MS_MOVE if pivot_root() fails

[DaanDeMeyer]

pivot_root() does not work in the initramfs. As a fallback, MS_MOVE can be used together with chroot() to mimick pivot_root() somewhat. bubblewrap should use this fallback if pivot_root() is not possible (e.g. when running bubblewrap from the initramfs) so that bubblewrap can be used from the initramfs.

Currently running bubblewrap from the initramfs fails with bwrap: pivot_root: Invalid argument

[smcv]

so that bubblewrap can be used from the initramfs

What's your use-case for using bubblewrap in an initramfs?

During early boot, at which point you are root and already have every privilege that could possibly exist, a tool that is not constrained by being designed for use by unprivileged users might be more successful.

[DaanDeMeyer]

What's your use-case for using bubblewrap in an initramfs?

I often boot straight into an initramfs as it is faster and easier than building and booting into a full disk image.

During early boot, at which point you are root and already have every privilege that could possibly exist, a tool that is not constrained by being designed for use by unprivileged users might be more successful.

Sure, but the tool I maintain (https://github.com/systemd/mkosi) does have those constraints and is specifically intended to be used by unprivileged users. It just so happens that I also want to be able to run it from an initramfs. Supporting an alternative to bubblewrap in mkosi just to be able to run mkosi from an initramfs seems like a huge amount of work compared to making sure bubblewrap can operate in an initramfs.

[rusty-snake]

Related: #595

[DaanDeMeyer]

Workaround:

unshare --mount
mount --rbind / /abc --mkdir
cd /abc
mount --move . /
chroot .
# bwrap will not get EINVAL from pivot_root() anymore

[aconz2]

sorry for the tangential comment, but that workaround is very helpful, thanks for sharing @DaanDeMeyer . I came across this issue in googling a pivot_root error from crun running from initramfs and that fixes it. Are there any caveats to using that approach? I don't fully understand/appreciate what it is doing. My notes on using it to run a container in a VM from initramfs are here. Now that I say that, kata must deal with the same thing since that is also from initramfs, I think somewhere in here but only took a glance right now

[DaanDeMeyer]

sorry for the tangential comment, but that workaround is very helpful, thanks for sharing @DaanDeMeyer . I came across this issue in googling a pivot_root error from crun running from initramfs and that fixes it. Are there any caveats to using that approach? I don't fully understand/appreciate what it is doing. My notes on using it to run a container in a VM from initramfs are here. Now that I say that, kata must deal with the same thing since that is also from initramfs, I think somewhere in here but only took a glance right now

I haven't encountered any caveats yet but I don't use this to run a container runtime so I wouldn't know.

[aconz2]

Could you explain why this works? My vague understanding was that pivot_root cannot be called on a rootfs and even after using the above trick, I still get rootfs on / type rootfs from mount, but pivot_root does work. EDIT: (to try not to spam this thread any further). I wrote up an explanation for why this works in case anyone finds this thread in the future. It is indeed as simple as setting up a parent mount, but that didn't make sense to me a few days ago.

[DaanDeMeyer]

pivot_root needs a parent mount that it can switch, this snippet sets up a parent mount.