Migrate to schnauzer v2 for Bottlerocket Configuration Templates #3476

cbgbt · 2023-09-19T18:59:24Z

Issue number:
#3133

Description of changes:
This merges features which have been reviewed on the ootb-settings-extensions feature branch into develop for the next release cycle:

The only changes since those PRs has been a rebase onto develop, which included these modifications:

diff --git a/packages/nvidia-container-toolkit/nvidia-oci-hooks-json b/packages/nvidia-container-toolkit/nvidia-oci-hooks-json
index 647e3345..fad6e2c6 100644
--- a/packages/nvidia-container-toolkit/nvidia-oci-hooks-json
+++ b/packages/nvidia-container-toolkit/nvidia-oci-hooks-json
@@ -1,5 +1,3 @@
-[required-extensions]
-oci-hooks = "v1"
 +++
 {
   "hooks": {
diff --git a/packages/os/oci-default-hooks-json b/packages/os/oci-default-hooks-json
index f8d08de2..ab08bab5 100644
--- a/packages/os/oci-default-hooks-json
+++ b/packages/os/oci-default-hooks-json
@@ -1,5 +1,3 @@
-[required-extensions]
-oci-hooks = "v1"
 +++
 {
   "hooks": {

This also enables CI builds for that feature branch for continued feature development.

Testing done:
See the related PRs for details, but in summary:

Large-scale fuzz testing of template rendering on randomly generated settings objects
Smoke testing instance launches
Extensive unit tests in new code branches.

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

bcressey · 2023-09-19T23:21:52Z

The rebase is a bit deceptive in this case because we've added some new templates:

the copies in packages/kubernetes-1.28
the deprecated setting template in e3bc71d

Not sure if that's an exhaustive list. #3460 is potentially a mid-air collision depending on merge order.

bcressey · 2023-09-19T23:23:27Z

.github/workflows/build.yml

@@ -1,7 +1,7 @@
 name: Build
 on:
  pull_request:
-    branches: [develop]
+    branches: [develop, ootb-settings-extensions]


Can we back out the workflow changes since the branch is going away?

packages/docker-engine/daemon-json

cbgbt · 2023-09-20T18:13:23Z

@bcressey Nice catch on the rebase misses, I had been looking for changes to files that I had touched.

I did my testing using the new oci_defaults, so that should be fine.
I'll add commits to handle k8s 1.28 and the deprecated settings template.

cbgbt · 2023-09-20T21:28:34Z

I ran the following check for template additions, which were missed by my previous scan for changes during the rebase:

$ git diff -U0 2f7124a52685a342098e5b37c9f4744332f47bd3..origin/develop -Scross_templatedir

(2f7124a was the commit that my branch was previously based on)

The only changes that turned up were the ones mentioned by @bcressey:

k8s-1.28 template copies
deprecated settings

+%{_cross_templatedir}/kubelet-env
+%{_cross_templatedir}/kubelet-config
+%{_cross_templatedir}/kubelet-kubeconfig
+%{_cross_templatedir}/kubelet-bootstrap-kubeconfig
+%{_cross_templatedir}/kubelet-exec-start-conf
+%{_cross_templatedir}/kubernetes-ca-crt
+%{_cross_templatedir}/kubelet-server-crt
+%{_cross_templatedir}/kubelet-server-key
+%{_cross_templatedir}/credential-provider-config-yaml
+%{_cross_templatedir}/log4j-hotpatch-enabled

I'll chat with @foersleo about the midair collision with #3460.

cbgbt · 2023-09-20T22:04:35Z

Ugh, I ruined the diff by rebasing again. But ^ rebases on develop once more, then implements the mentioned changes.

I ran the following on the k8s templates as a check:

$ for f in kubelet-env kubelet-config kubelet-kubeconfig kubelet-bootstrap-kubeconfig kubelet-exec-start-conf kubernetes-ca-crt kubelet-server-crt kubelet-server-key credential-provider-config-yaml; do
> echo "checking $f"
> diff packages/kubernetes-1.28/$f packages/kubernetes-1.27/$f
> done
checking kubelet-env
checking kubelet-config
134a135,136
> featureGates:
>   RotateKubeletServerCertificate: true
checking kubelet-kubeconfig
checking kubelet-bootstrap-kubeconfig
checking kubelet-exec-start-conf
checking kubernetes-ca-crt
checking kubelet-server-crt
checking kubelet-server-key
checking credential-provider-config-yaml

EDIT: I tested these changes by launching a aws-k8s-1.28 and confirming that templates all render successfully, then modifying settings for log4j-hotpatch-enabled and checking the associated config file.

This change also uses a POSIX-shell-like lexer to tokenize setting-generators in sundog, rather than the previous approach of splitting on whitespace.

Bottlerocket invokes thar-be-settings whenever it needs to re-render configuration file templates defined in the Bottlerocket API. This change modifies thar-be-settings to use the schnauzer v2 renderer, and also modifies all existing configuration templates in Bottlerocket to use the new template format.

In order for helpers to be composable, they must implement the `call_inner` method for the `HelperDef` trait. By failing to implement this for `SettingExtensionTemplateHelper`, helpers used e.g. in an `if` block would not return correct results.

cbgbt · 2023-09-21T21:50:54Z

^ Rebases atop the changes from #3460

Tested by launching an aws-k8s-1.28 node and enabling autoload of a kernel module.

cbgbt requested review from bcressey, stmcginnis and webern September 19, 2023 18:59

bcressey reviewed Sep 19, 2023

View reviewed changes

cbgbt force-pushed the ootb-settings-extensions branch from 06b6f35 to 7e11db2 Compare September 20, 2023 21:56

bcressey approved these changes Sep 21, 2023

View reviewed changes

cbgbt added 10 commits September 21, 2023 20:50

schnauzer: move legacy api behind v1 namespace

2fd9e78

schnauzer: add schnauzer-v2 settings generator

7cbbc97

schnauzer: update README

0ba9c3a

Migrate setting-generators to schnauzer v2

b3b0e1b

This change also uses a POSIX-shell-like lexer to tokenize setting-generators in sundog, rather than the previous approach of splitting on whitespace.

Add schnauzer-v2 settings generator migrations

d5f623e

schnauzer: allow optional extensions

5a03c83

schnauzer: simplify template implementation

496ffd4

schnauzer: move any_enabled helper to std

9f3e345

schnauzer: fix helper composition bug

aae7b08

In order for helpers to be composable, they must implement the `call_inner` method for the `HelperDef` trait. By failing to implement this for `SettingExtensionTemplateHelper`, helpers used e.g. in an `if` block would not return correct results.

cbgbt force-pushed the ootb-settings-extensions branch from 7e11db2 to aae7b08 Compare September 21, 2023 21:50

cbgbt requested a review from etungsten September 22, 2023 17:36

etungsten approved these changes Sep 22, 2023

View reviewed changes

cbgbt merged commit a0f0173 into develop Sep 22, 2023
48 checks passed

cbgbt deleted the ootb-settings-extensions branch September 22, 2023 18:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate to schnauzer v2 for Bottlerocket Configuration Templates #3476

Migrate to schnauzer v2 for Bottlerocket Configuration Templates #3476

cbgbt commented Sep 19, 2023

bcressey commented Sep 19, 2023

bcressey Sep 19, 2023

cbgbt commented Sep 20, 2023

cbgbt commented Sep 20, 2023 •

edited

Loading

cbgbt commented Sep 20, 2023 •

edited

Loading

cbgbt commented Sep 21, 2023 •

edited

Loading

Migrate to schnauzer v2 for Bottlerocket Configuration Templates #3476

Migrate to schnauzer v2 for Bottlerocket Configuration Templates #3476

Conversation

cbgbt commented Sep 19, 2023

bcressey commented Sep 19, 2023

bcressey Sep 19, 2023

Choose a reason for hiding this comment

cbgbt commented Sep 20, 2023

cbgbt commented Sep 20, 2023 • edited Loading

cbgbt commented Sep 20, 2023 • edited Loading

cbgbt commented Sep 21, 2023 • edited Loading

cbgbt commented Sep 20, 2023 •

edited

Loading

cbgbt commented Sep 20, 2023 •

edited

Loading

cbgbt commented Sep 21, 2023 •

edited

Loading