models, sundog, prairiedog: add kernel boot config support

[etungsten]

Issue number:
N/A

Description of changes:

    models: new boot settings to specify kernel boot config

    This adds `settings.boot` to help configure kernel boot config settings.

    prairiedog: add boot config functionality

     - Adds new subcommand `generate-boot-config` to generate an initrd
    image file with boot config contents populated from Bottlerocket boot
    settings.
     - Adds new subcommand `generate-boot-settings` to generate bottlerocket
    settings from kernel's bootconfig file at `/proc/bootconfig`

    Moves prairiedog from `sources/prairiedog` to `sources/api/prairiedog`
    now that it needs to query from the Bottlerocket API and need settings
    model information.

    sundog: update settings retrieval to use prefix

    This updates the settings retrieval to use prefixes for matching and
    querying. We cannot use full keys for matching because we now have
    settings generator for keys that map to top level structs that
    encompasses multiple settings.

    models: add metadata for 'settings.boot'

    Add settings generator for `settings.boot` and restart commands for when
    boot settings change.

    migrations: add migrations for new `settings.boot` setting

    README: add description and example for 'settings.boot'

    metal: remove kernel console settings from variant metadata

    We've added bootconfig support so console settings should be specified
    through there. The console settings specified to the kernel commandline
    will always take effect, so we're removing them from the variant
    metadata.

Testing done:
Launched instance with the following user-data:

[settings.boot]
kernel.test1 = ["bark", "barkbark"]
kernel.test2 = ["woof", "woofwoof"]
init.test1 = ["good", "morning"]
init.test2 = ["hello", "goodbye"]

Prairiedog successfully generates the initrd image file containing boot config:

[ec2-user@ip-192-168-10-10 ~]$ hexdump -C bootconfig.data
00000000  69 6e 69 74 2e 74 65 73  74 31 20 3d 20 22 67 6f  |init.test1 = "go|
00000010  6f 64 22 2c 20 22 6d 6f  72 6e 69 6e 67 22 0a 69  |od", "morning".i|
00000020  6e 69 74 2e 74 65 73 74  32 20 3d 20 22 68 65 6c  |nit.test2 = "hel|
00000030  6c 6f 22 2c 20 22 67 6f  6f 64 62 79 65 22 0a 6b  |lo", "goodbye".k|
00000040  65 72 6e 65 6c 2e 74 65  73 74 31 20 3d 20 22 62  |ernel.test1 = "b|
00000050  61 72 6b 22 2c 20 22 62  61 72 6b 62 61 72 6b 22  |ark", "barkbark"|
00000060  0a 6b 65 72 6e 65 6c 2e  74 65 73 74 32 20 3d 20  |.kernel.test2 = |
00000070  22 77 6f 6f 66 22 2c 20  22 77 6f 6f 66 77 6f 6f  |"woof", "woofwoo|
00000080  66 22 00 00 84 00 00 00  fb 29 00 00 23 42 4f 4f  |f".......)..#BOO|
00000090  54 43 4f 4e 46 49 47 0a                           |TCONFIG.|
00000098

Verified that it's a valid initrd image file with valid boot config content with the linux kernel's bootconfig tool:

$ linux/tools/bootconfig/bootconfig -l bootconfig.data
init.test1 = "good", "morning"
init.test2 = "hello", "goodbye"
kernel.test1 = "bark", "barkbark"
kernel.test2 = "woof", "woofwoof"

• Integrated test on real bare-metal hosts with a provisioned boot config file

Built a metal-dev image and dd'd it to a baremetal server's /dev/sda.
I manually provisioned a bootconfig.data file that contained the following bootconfig:

$ ./bootconfig -l bootconfig.data
kernel.console = "tty0", "ttyS1,115200n8"
init.systemd.log_level = "debug"

and copied it into the BOTTLEROCKET-PRIVATE partition.
Then I booted the baremetal host from its harddrive.

Connected to the server's serial port over IP and saw that the host comes up successfully.
Checked settings and it's generated correctly from /proc/bootconfig:

[ec2-user@admin]$ apiclient get settings.boot
{
  "settings": {
    "boot": {
      "init": {
        "systemd.log_level": [
          "debug"
        ]
      },
      "kernel": {
        "console": [
          "tty0",
          "ttyS1,115200n8"
        ]
      }
    }
  }
}

If I check systemd, the log-level is correctly set to "debug"

bash-5.0# systemd-analyze get-log-level
debug

If I change the settings, /var/lib/bottlerocket/bootconfig.data gets updated as expected:

[ec2-user@admin]$ apiclient set --json '{"boot":{"kernel":{"console":["tty0","ttyS1,115200n8"]},"init":{"systemd.log_level":["info"]}}}'
[ec2-user@admin]$ cat /.bottlerocket/rootfs/var/lib/bottlerocket/bootconfig.data
init.systemd.log_level = "info"
kernel.console = "tty0", "ttyS1,115200n8"L#BOOTCONFIG

If I reboot, I can see that the systemd log-level gets updated to info as expected.

bash-5.0# cat /proc/bootconfig                                                                                                     
init.systemd.log_level = "info"                                                                                                    
kernel.console = "tty0", "ttyS1,115200n8"
                                                                              
bash-5.0# systemd-analyze get-log-level                                                                                            
info

• Integrated test on real bare-metal hosts with a provisioned user-data.toml file

Built a metal-dev image and dd'd it to a baremetal server's /dev/sda.
I manually provisioned a user-data.toml file that looks like this:

bash-5.0# cat /var/lib/bottlerocket/user-data.toml 
[settings.boot.kernel]
"console" = ["tty0", "ttyS1,115200n8"]
[settings.boot.init]
"systemd.log_level" = ["debug"]

and copied it into the BOTTLEROCKET-PRIVATE partition.
Then I booted the baremetal host from its harddrive and everything comes up fine.

The generated bootconfig.data file is as expected:

[ec2-user@admin]$ hexdump -C /.bottlerocket/rootfs/var/lib/bottlerocket/bootconfig.data
00000000  69 6e 69 74 2e 73 79 73  74 65 6d 64 2e 6c 6f 67  |init.systemd.log|
00000010  5f 6c 65 76 65 6c 20 3d  20 22 64 65 62 75 67 22  |_level = "debug"|
00000020  0a 6b 65 72 6e 65 6c 2e  63 6f 6e 73 6f 6c 65 20  |.kernel.console |
00000030  3d 20 22 74 74 79 30 22  2c 20 22 74 74 79 53 31  |= "tty0", "ttyS1|
00000040  2c 31 31 35 32 30 30 6e  38 22 00 00 4c 00 00 00  |,115200n8"..L...|
00000050  08 18 00 00 23 42 4f 4f  54 43 4f 4e 46 49 47 0a  |....#BOOTCONFIG.|
00000060

If I reboot the host, then the bootconfig above gets loaded by the kernel as expected.

bash-5.0# cat /proc/cmdline                                                                                                        
console="tty0" console="ttyS1,115200n8" .....

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[webern]

Nice

[etungsten]

Push above and below addresses @webern 's comments.

[etungsten]

Push above and below limits boot settings to bare metal variants only.

[etungsten]

I need to figure out conditional compilation for certain parts of prairiedog that deals with boot settings that only exist for metal variants.

[etungsten]

Push above add conditional compilation for boot config related functionalities in prairedog.

[etungsten]

Push above and below and tries to address comments.

Unit tests all pass.

[etungsten]

Push above simplifies padding calculation a bit and addresses some leftover comments.

[etungsten]

Push above rebases onto develop to pull the grub changes.

[etungsten]

Push above removes some extraneous comments and fixes a log message that was causing sundog to choke.

[etungsten]

Push above addresses @bcressey comments.

• Removes the bootconfig serde implementation, the bootconfig crate is now gone.
• Now parses and serializes bootconfig files in prairiedog directly to/from models::BootSettings without serde.

Tested all the changes and they still work as expected:

[erikson@admin]$ apiclient get settings.boot
{
  "settings": {
    "boot": {
      "init": {
        "systemd.log_level": [
          "debug"
        ]
      },
      "kernel": {
        "console": [
          "tty0",
          "ttyS1,115200n8"
        ]
      }
    }
  }
}

[erikson@admin]$ apiclient set --json '{"boot":{"kernel":{"console":["tty0","ttyS1,115200n8"]},"init":{"systemd.log_level":["info"]}}}'
[erikson@admin]$ sudo sheltie
bash-5.1# cat /var/lib/bottlerocket/bootconfig.data 
kernel.console = "tty0", "ttyS1,115200n8"
init.systemd.log_level = "info"
L#BOOTCONFIG

[etungsten]

Push above does some minor refactoring and clean-up

[etungsten]

Push above fixes initrd generation that I missed in my initial testing.

[bcressey]

Looks good to me except for some fussing about single- and double quotes.

[etungsten]

Pushes above addresses @bcressey 's comments.

• prairiedog will now error when parsing boot config files that have kernel and init as parent keys to a list of non-null values. This is disallowed since BootSettings model does not support it.
• BootConfigValue now rejects value strings that mix and contain both single quotes and double quotes. Quotes cannot be escaped in boot config values.

[bcressey]

Nice!

👢 🔧

[zmrow]

🐙

Nice work!