feat: Add ReadDir as a templating function

[nlueb]

This PR adds a new templating function called readDir. With readDir users can read the contents of a specified directory and get a list of all contained files. This is useful when reading a bunch of files from a directory. The following example shows a snippet of a values file for configuring a Logstash deployment. Using only readFile, a user must specify each file by hand and adjust this list as the number of files to be read grows.

logstash:
  configs:
    logstash.yml: |
      {{- tpl (readFile "config/logstash.yml.gotmpl") . | nindent 6 }}
    jvm.options: |
      {{- readFile "config/jvm.options" | nindent 6 }}
    pipelines.yml: |
      {{- readFile "config/pipelines.yml" | nindent 6 }}
  pipelines:
    beats-log4j.conf: |
      {{- readFile "config/pipelines/beats-log4j.conf" | nindent 6 }}
    nginx-access.conf: |
      {{- readFile "config/pipelines/nginx-access.conf" | nindent 6 }}
    nginx-error.conf: |
      {{- readFile "config/pipelines/nginx-error.conf" | nindent 6 }}
    syslog-logs.conf: |
      {{- readFile "config/pipelines/syslog-logs.conf" | nindent 6 }}
    tcp-logs.conf: |
      {{- readFile "config/pipelines/tcp-logs.conf" | nindent 6 }}
    udp-debug.conf: |
      {{- readFile "config/pipelines/udp-debug.conf" | nindent 6 }}
    udp-logs.conf: |
      {{- readFile "config/pipelines/udp-logs.conf" | nindent 6 }}
  certificates:
    ca.crt: |
      {{- readFile "config/certificates/ca.crt" | nindent 6 }}
    logstash.crt: |
      {{- readFile "config/certificates/logstash.crt" | nindent 6 }}
    logstash.key: |
      {{- readFile "config/certificates/logstash.key" | nindent 6 }}

With readDir the above snippet can be rewritten as follows:

logstash:
  configs:
  {{- range readDir "config" }}
    {{ base . }}: |
      {{- if hasSuffix "gotmpl" . }}
          {{- tpl (readFile .) $ | nindent 6 }}
      {{- else }}
          {{- readFile . | nindent 6 }}
      {{- end }}
  {{- end }}
  pipelines:
  {{- range readDir "config/pipelines" }}
    {{ base . }}: |
      {{- readFile . | nindent 6 }}
  {{- end }}
  certificates:
  {{- range readDir "config/certificates" }}
    {{ base . }}: |
      {{- readFile . | nindent 6 }}
  {{- end }}

Disclaimers

• This is my first contribution to this project
• I haven't written any go code in quite some time
• I didn't add any tests as i didn't understand how i would do so. I would be very grateful if someone could point me in the right direction.

[mumoshu]

LGTM. Thanks for your contribution!

[cadeff01]

Looks like this is appending the path incorrectly to be used with readFile as the example shows. Based on the code below I'd expect a configmap that has each filename and the contents of those files but instead I get a duplicated base path appended to the file path.

apiVersion: v1
kind: ConfigMap
metadata
  name: realmconfig
data:
  {{- range readDir "resources/realms" }}
  {{ base . }}: |
    {{- readFile . | nindent 4 }}
  {{- end }}

[template: realmconfig.yaml.gotmpl:8:8: executing "realmconfig.yaml.gotmpl" at <readFile .>: error calling readFile: open resources/resources/realms/realm-admin.json: no such file or directory]

readFile checks for an absolute path and if it doesn't find it appends the basePath which is what you are seeing here. Suggest modifying

filenames = append(filenames, filepath.Join(path, entry.Name()))

to work better with readFile

Hacky work around as this is really great functionality overall:

apiVersion: v1
kind: ConfigMap
metadata:
  name: realmconfig
data:
  {{- range readDir "resources/realms" }}
  {{ base . }}: |
  {{- trimPrefix "resources/" . | readFile | nindent 4 }}
  {{- end }}

[mumoshu]

@bhester01 Thanks for reporting! Ah, so this is definitely broken. It looks like we had to make this aware of the base path as we do for ReadFile.

@nlueb Would you mind contributing a fix for that? You'd make ReadDir a method of *Context and use c.basePath as the base path for the context.

[nlueb]

Hey both, thanks for bringing this up @bhester01 and thanks for the suggestion @mumoshu. I created #2058 to hopefully resolve this issue. From my local testing it seems to be fixed.