✨ Clean up and simplify config management

[JonahSussman]

Config is now sourced from the environment as well as the config.toml via pydantic-settings. This makes Kai align more with 12-factor config principles.

pydantic-settings allows for easy config overrides (higher overrides lower):

• Config file that is declared on the command line / via init arguments. If using podman compose up, the config file in build/config.toml is supplied as a command line argument.
• Environment variables
• Local config file, kai/config_local.toml. This is for quick changes when running on bare metal.
• Global config file, kai/config_default.toml. Sane defaults.
• Default field values

Task list:

• Integrate pydantic-settings into the main project
• (Optional) Find a better solution to the config overriding thing
• Update documentation

[jwmatthews]

Looking good overall, but I did have a few questions.

• Lets verify end user can see/edit the config.toml used, concern mostly with workflow of how an end user sees the configured model and may change this model.
• Let's consider keeping ROADMAP.md at the main project root.
• I see the logs directory is no longer part of compose.yaml, unsure if that is intended.

[jwmatthews]

Will an end user running podman compose up have access to the config.toml on their local disk so they can see the settings and adjust?

The usecase we want to satisfy is to allow the end user to:

• View the config.toml from local disk
• Change settings on local disk to modify configured models

I'm not sure from a quick review of this PR if there is a mechanism for the end user to view/edit the config.toml that is used. My current impression is we are copying it into the container at build time. I see how that helps for running with the default configurations and I see the intent to override via environment variables, but unsure about how a user can modify the config.toml.

[JonahSussman]

The user can easily adjust their config.toml. The way it works in this PR is:

1. When the container is built, the kai/config.toml that is present in the repo is copied over to the container
2. When the container is run, it first checks if the file build/config.toml exists. If it does, it copies it over to the container.

Thus, the user can specify their config via build/config.toml. There is an example_config.toml also present in that directory. We could even add example_stable_config.toml or something so the user has a config that can match their version. We can change the location that entrypoint.sh looks for if it makes more sense for the user to specify their custom config elsewhere.

[JonahSussman]

I updated how this works to make more sense. See my comment below.

[JonahSussman]

I updated how config priority is done. It's now done in the following way (higher overrides lower):

• Config file that is declared on the command line / via init arguments. If using podman compose up, the config file in build/config.toml is supplied as a command line argument.
• Environment variables
• Local config file, kai/config_local.toml. This is for quick changes when running on bare metal.
• Global config file, kai/config_default.toml. Sane defaults.
• Default field values

Upon further evaluation, we could merge config_local.toml and config_global.toml. WDYT?

[jwmatthews]

I updated how config priority is done. It's now done in the following way (higher overrides lower):

• Config file that is declared on the command line / via init arguments. If using podman compose up, the config file in build/config.toml is supplied as a command line argument.
• Environment variables

Is it possible to swap this and allow Environment variables to have top precedence?

When I think ofpodman compose up my ideal preference would be:

1. Environment variables
2. Settings in compose.yaml
3. Configuration file (I'm assuming for this PR, build/config.toml would be the configuration file used.

Reason being is a user who is using podman compose up likely knows little about our project.
I think they will be expecting that:

• Environment variables, like API keys, will be magically available to Kai
• Settings explicitly called out in compose.yaml will take precedence

I assume in time they will understand we also have a configuration file they can view/edit at build/config.toml but I don't think it will be top of their mind on initial evaluation of Kai.

• Local config file, kai/config_local.toml. This is for quick changes when running on bare metal.
• Global config file, kai/config_default.toml. Sane defaults.
• Default field values

Upon further evaluation, we could merge config_local.toml and config_global.toml. WDYT?

I didn't see a config_global.toml. but I will assume you are talking about merging config_local.toml and config_default.toml?

When I see config_local.toml I think of a local config change I'm running for my own testing and this would be handled outside of the git tracking. For example, today I will tweak our kai/config.toml for various models while testing, then when I'm committing my PR I'll revert my changes to kai/config.toml. I could see something like a config_local.toml as an override that takes higher precedence and allows us to specify a few settings for convenience without needing to edit/revert the config between various PRs.

I'm not advocating we want/need that, just sharing what went through my head when I saw this.

I don't see the config_local.toml in .gitignore not sure if you were intending it for the same thing that was in my mind or something else.

[JonahSussman]

Going to hold off on merging due to #332 preventing me from testing. It was working earlier but after deleting the logs directory, which was having permission errors on bare metal, I can no longer run podman compose up. Very very frustrating.

If anyone else can verify everything works then I'm good to merge.

[jwmatthews]

Confirmed working on MacOS

[jwmatthews]

Closes #264

[JonahSussman]

Confirmed working on Ubuntu 24.04.1