Prairiedog and the kernel may disagree about parsing boot config keys without values

[markusboehme]

Image I'm using: Build of metal-dev variant fresh off develop

What I expected to happen:

When using the example user data from the boot settings section of the README.md, ...

[settings.boot.kernel-parameters]
"console" = [
  "tty0",
  "ttyS1,115200n8",
]
"crashkernel" = [
  "2G-:256M",
]
"slub_debug" = [
  "options,slabs",
]
"usbcore.quirks" = [
  "0781:5580:bk",
  "0a5c:5834:gij",
]

[settings.boot.init-parameters]
"log_level" = ["debug"]
"splash" = []

...I expected those settings to always reflect what the kernel and systemd would see on boot.

What actually happened:

Prairiedog may generate the following boot config...:

bash-5.1# cat /var/lib/bottlerocket/bootconfig.data 
kernel.slub_debug = "options,slabs"
kernel.crashkernel = "2G-:256M"
kernel.usbcore.quirks = "0781:5580:bk", "0a5c:5834:gij"
kernel.console = "tty0", "ttyS1,115200n8"
init.splash = 
init.log_level = "debug"

...which on next boot is read by the kernel as:

bash-5.1# cat /proc/bootconfig 
kernel.slub_debug = "options,slabs"
kernel.crashkernel = "2G-:256M"
kernel.usbcore.quirks = "0781:5580:bk", "0a5c:5834:gij"
kernel.console = "tty0", "ttyS1,115200n8"
init.splash = 'init.log_level = "debug"'

Note that the log_level key got pulled in as the value for the actually value-less splash key.

How to reproduce the problem:

Put the boot settings fragment from above into your user data. There seems to be a random component to it, and it may take a few (clean) boots to hit.

[markusboehme]

As for the semi-random occurrence of the issue: The value-less key must not be the last line in the configured boot config. The order in which prairiedog writes out these keys is determined by Rust's HashMap.iter() method, which is documented to produce an "arbitrary" order which in my tests mostly, but not always, happened to be one that put splash last.

Clearly there is disagreement about how to represent value-less keys in the boot config, and in case of doubt the kernel's implementation wins. Prairiedog could put an empty pair of quotes after the value-less key. I'll check whether there's other ways to represent such keys, though (IIRC init { splash } would be one).

Until a fix has been merged, one may work around the issue by providing "splash" = [""] in the user data, i.e. an empty string instead of an empty array.

[mchaker]

I've seen this issue and wasn't sure how to describe it in a bug report 😅 thanks for filing this!