Add general ignition-ostree module with rootfs replacement #184
Conversation
|
This is work for coreos/fedora-coreos-tracker#94 |
|
This is blocked by the rootfs mount options issue, right? |
Working on that! |
There was a problem hiding this comment.
I like the direction. My only question is whether reading it to an in-ram location (tar -cf ...) might be faster since you avoid the overhead of the delete process. For some cloud-based network attach storage, there's going to be a cost and avoiding that could make it marginally faster.
overlay.d/05core/usr/lib/dracut/modules.d/31ignition-ostree/ignition-ostree-dracut-rootfs.sh
Outdated
Show resolved
Hide resolved
|
Will rebase on #187 |
Yeah I started with that, but hit tar+SELinux issues. My plan is to put this in rpm-ostree, see coreos/rpm-ostree#1911 |
cgwalters
force-pushed
the
ignition-ostree-rootfs
branch
from
90f3db1
to
9540b70
Compare
overlay.d/05core/usr/lib/dracut/modules.d/31ignition-ostree/ignition-ostree-dracut-rootfs.sh
Outdated
Show resolved
Hide resolved
| fi | ||
| echo "Restoring rootfs from RAM..." | ||
| for x in boot ostree; do | ||
| mv -Tn ${tmproot}/${x} ${rootmnt}/${x} |
There was a problem hiding this comment.
What's the impact of this and hardlinks? Should we be doing ostree operations instead of just mv/cp?
There was a problem hiding this comment.
Yeah, I plan to move this portion into rpm-ostree.
overlay.d/05core/usr/lib/dracut/modules.d/31ignition-ostree/ignition-ostree-dracut-rootfs.sh
Outdated
Show resolved
Hide resolved
cgwalters
force-pushed
the
ignition-ostree-rootfs
branch
2 times, most recently
from
fbc96e6
to
a5dc2eb
Compare
This adds basic infrastructure units for "re-provisioning" the root filesystem. See: coreos/fedora-coreos-tracker#94 A unit first detects if the Ignition configuration has a filesystem with the label `root` - if so, we save the rootfs into RAM, let `ignition-disks.service` run, then restore it from RAM. Earlier attempts copied the files into `tmpfs`; this instead uses the `brd` kernel module which is a RAM-backed block device so we can just `dd`. This a faster and more reliable way to save the rootfs. However, `brd` doesn't support discards, so we require at minimum $rootfs_size RAM (e.g. 3G) until reprovisioning is complete. In the future I might investigate trying a `tmpfs` again as that better integrates with the page cache and doesn't have the discard issue. But it requires more work to save the rootfs. Future work here will likely more the `restore` phase into `rpm-ostree`.
- add check for `wipeFilesystem == true` in jq query - use `modprobe --first-time` instead of `lsmod` to avoid race condition - use `rsync` instead of `mv`; they're mostly equivalent in this context, but rsync is more tuned for this sort of massive operation and does things slightly more correctly (specifically, it correctly labels the root inode of the rootfs itself)
jlebon
force-pushed
the
ignition-ostree-rootfs
branch
from
a5dc2eb
to
f1cd51b
Compare
|
As mentioned in coreos/fedora-coreos-tracker#94 (comment), I've now sent the patch upstream to be able to read labels from the initrd. In order to be confident it worked, I rebased this on top of the latest "storage": {
"filesystems": [
{
"device": "/dev/disk/by-partlabel/root",
"format": "ext4",
"wipeFilesystem": true,
"label": "root"
}
]
}Anyway, took the liberty to force-push the rebase to this PR. I also added a follow-up commit with some tweaks. We can squash that in if you approve. |
| echo "Moved rootfs to RAM, pending redeployment: ${memdev}" | ||
| ;; | ||
| restore) | ||
| mount "$rootdisk" $rootmnt |
There was a problem hiding this comment.
Aside: I am still leaning towards "rootfs reprovisioning must add a root= kernel argument" (UUID is a sane default, but it doesn't have to be that). That way we avoid all races with changing the meaning of /dev/disk/by-label/root.
There was a problem hiding this comment.
Yeah, I think I agree with this. I feel like we should spend some time to write up some of the complex root device options we want to support (e.g. at least starting with multipath, RAID, and LUKS) and see how it fits into one cohesive picture.
|
So circling back, once we unblock the SELinux patch (I have an RHBZ pending for that), this is the next crucial piece in the puzzle for coreos/fedora-coreos-tracker#94. On its own, it will allow e.g. reconfiguring It's been pretty solid for me while hacking on coreos/fedora-coreos-tracker#94. Though we should get some fresh eyes on it too. |
|
This is now part of #503. |
* nav: Add link to Afterburn docs * nav: Improve projects docs naming and link names
Previously the
40coreos-varmodule was "special /varhandling for Ignition + OSTree".
This new module takes over that, renaming it to
ignition-ostree,and extends it with support for replacing the root filesystem.