pkgs.formats: Add HOCON format generator

[h7x4]

Description of changes

Taking some of the lessons learnt from #246115, I've made a generator for the HOCON format. This generator aims to support (almost) all the features of the HOCON format, including things like substitutions from envvars, different kinds of include statements both for declarative and mutual includes, append statements, etc.

HOCON does not seem to have a definitive governing body, nor a complete formal specification, so I've chosen to base my implementation upon the best I could find: https://github.com/lightbend/config/blob/main/HOCON.md. This means that there's bound to be some compatibility issues with certain implementations (in fact, I've already stumbled upon one with the include types when I tried writing the validator using the most popular go implementation), but I think this is unavoidable to some degree.

Up until now, I've only found these three that are using the program, as well as a pending package. Most of them seem to have copied each others implementation, so backwards compatability can probably be combined:

• nixos/modules/services/networking/jitsi-videobridge.nix
• nixos/modules/services/networking/jicofo.nix
• nixos/modules/services/networking/jibri/default.nix
• nixos/modules/services/web-apps/tachidesk-server.nix (nixos/tachidesk-server: init at 0.7.0 #257733)

TODO:

• write generator
• write tests
• replace existing implementations treewide
• provide some sort of backwards compatibility or warning to smooth migration from the existing implementations
  (I think the only thing we'd need to handle would be __hocon_envvar and __hocon_unquoted_string)

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

Some tests that now use this generator either directly or indirectly:

nix build ".#pkgs.tests.pkgs-lib.hocon.comprehensive"
nix build ".#pkgs.tests.pkgs-lib.hocon.backwards-compatibility"
nix build ".#nixosTests.jibri" # This one is failing for seemingly unrelated reasons
nix build ".#nixosTests.jitsi-meet"
nix build ".#nixosTests.prometheus-exporters.jitsi"
nix build ".#nixosTests.suwayomi-server.with-auth"
nix build ".#nixosTests.suwayomi-server.without-auth"

---

Add a 👍 reaction to pull requests you find important.

[ckiee]

hi, I have to focus my energy on other things so I won't be participating in this PR. best of luck!

[roberth]

This looks well thought out, follows an established implementation strategy (see libconfig), has tests, and the Nix code looks good to me.
Although I don't have the opportunity to dive into the intricacies of HOCON, to "definitively" claim that all the details are good, I am confident that this code is maintainable.

[h7x4]

Okay, this should be completely ready now.

pings for maintainers of affected modules:

• nixos/modules/services/networking/{jitsi-videobridge.nix,jicofo.nix,jibri/default.nix}: Jitsi team - @cleeyv, @ryantm, @Lassulus, @yayayayaka
• nixos/modules/services/web-apps/suwayomi-server.nix: @RatCornu

[h7x4]

@ofborg test jitsi-meet prometheus-exporters.jitsi suwayomi-server.with-auth suwayomi-server.without-auth

[infinisil]

Overall this is looking great, I'm impressed by the code quality! Just got some minor suggestions, but I'm happy to merge

[infinisil]

Found the problem, pushed the fix :D

[h7x4]

Found the problem, pushed the fix :D

Cool! I've reordered and rebased the fixup to make more sense.

[infinisil]

Looking good, let's merge!

[infinisil]

Forgot to mention that this should be documented! See #288163