sovereign.toml for SDK config

[vlopes11]

Is your feature request related to a problem? Please describe.
We need a way to parametrize the module compilation at compile time so the macros can be generated accordingly.

One of the problems this will solve is the gas config. We may set gas configuration on a sovereign.toml file, read it as literal string when generating the macros, and extract the relevant content.

Describe the solution you'd like
At compile time, read the same directory of the source file being worked by the macro, and search for a sovereign.toml file. It should have scoped sections so it can be used for different purposes. Example:

[gas]
base_cost = 15
variable_cost = 25

[foo]
bar = baz

The implementation should recurse to the parent dir and search for sovereign.toml, repeating the process until the root dir. If no sovereign.toml is found, then panic the compilation.

On the example above, let's take as example the Account ModuleInfo compilation. Let's assume the source path is located at /home/vlopes11/sovereign-sdk/module-system/module-implementations/sov-accounts/src/lib.rs, then, it would, in order, search for a toml file in the following paths:

/home/vlopes11/sovereign-sdk/module-system/module-implementations/sov-accounts/src/sovereign.toml
/home/vlopes11/sovereign-sdk/module-system/module-implementations/sov-accounts/sovereign.toml
/home/vlopes11/sovereign-sdk/module-system/module-implementations/sovereign.toml
/home/vlopes11/sovereign-sdk/module-system/sovereign.toml
/home/vlopes11/sovereign-sdk/sovereign.toml
/home/vlopes11/sovereign.toml
/home/sovereign.toml
/sovereign.toml

And finally fail, if none found.

There is, however, a problem. Currently, it is a nightly only feature to get the source path from a macro.

https://github.com/dtolnay/proc-macro2/blob/45ef770a5189ceca3adae9d4d6117a782da30a2a/src/lib.rs#L452-L459

This procmacro2_semver_exempt flag exists to shield backwards compatibility with older toolchains. It is not a feature, intentionally, and the user will have to set the RUSTFLAGS manually if he wants such functionality to be available. For more information, check the Unstable features documentation from proc-macro2.

With that said, it would be very inconvenient to require all downstream users to add the flags to their compilation as such flags will not replicate downstream and must be explicitly declared every time.

RUSTFLAGS='--cfg procmacro2_semver_exempt' cargo build

From an initial investigation, there is no other way to get the source path of the file that consumed the derive macro.

It is important to allow the user to freely set multiple sovereign.toml somehow, as a single crate might contain multiple modules with distinct configurations. One option would be to scope the sections and prioritize them in case of a match

[gas.Foo]
value = 15

[gas.Bar]
value = 30

[gas]
value = 45

Another problem to consider is how the user will update such file. There is no mechanism to notify the compiler to recompile the macro expansion when sovereign.toml is changed. It means the user will have to run a cargo clean every time he updates his manifest file in order for it to be integrated to the code.

Describe alternatives you've considered

CARGO_MANIFEST_DIR

This would work fine as we could just expect the sovereign.toml to be on the same dir of the Cargo.toml, except it isn't trivial to pass the manifest path to the macro implementation. Example:

let foo = env!("CARGO_MANIFEST_DIR");
#[derive(ModuleInfo)]
struct Accounts {
  #[gas]
  gas: Bar,
  ...
}

On the example above, foo will evaluate to /home/vlopes11/sovereign-sdk/module-system/module-implementations/sov-accounts. If we could pass it to the macro compilation, we could just append sovereign.toml. But, this is a compile-time macro, and if we try to

fn impl_module_info(...) -> ... {
  let foo = env!("CARGO_MANIFEST_DIR");
  ...
}

In that case, foo will evaluate as constant to /home/vlopes11/sovereign-sdk/module-system/sov-modules-macros. It is so because CARGO_MANIFEST_DIR is updated by cargo every time a compile scope is changed - as it should be.

If we try things like

#[derive(ModuleInfo)]
struct Accounts {
  #[gas = env!("CARGO_MANIFEST_DIR")]
  gas: Bar,
  ...
}

This ends up not being evaluated as a MetaNameValue, as this is not a string literal before the macro expansion - and it would also be inconvenient for the user to have to do it every time.

A simple solution would be to just start recursing from env!("CARGO_MANIFEST_DIR"): /home/vlopes11/sovereign-sdk/module-system/sov-modules-macros as this would exist under target for consumers of the library, and eventually we would recurse until the root of the workspace that may contain the sovereign.toml

Fork proc-macro2 and always require SourceFile

We currently use rust-toolchain.toml nightly-2023-06-25. This is enough to satisfy the requirements for proc_macro2::SourceFile so, for the users of the SDK, the functionality will always be available.

proc-macro2 took a defensive stance for backwards compatibility that is not applicable to us. We could fork the library and include SourceFile by default.

While this would be an easy solution without major complications, forking libraries is often negative as it increases the maintenance cost.