Explore the inclusion of &self as a parameter for Circuit::configure()

[CPerezz]

See this thread in halo2/zkevm-circuits: privacy-scaling-explorations/zkevm-circuits#948 (comment)

There is discussed the fact that it's really painful to pass a lot of const-generics or the need to edit the Config type to tuples or similar stuff when you're implementing the Circuit trait simply because it doesn't have &self as parameter.

Would be nice to explore a bit how that would feel and if we encounter any unexpected issues with it.

[CPerezz]

Candidate to be discussed in the Halo2-call/Office hours

[ed255]

I would like to recover this discussion! I think we can benefit greatly from having runtime configurable circuits; not only for the zkevm project (which will allow removing many generic const) but to have a better split between frontend and backend in halo2 which allows building circuits with new APIs, DSLs and languages.

Adding self to configure looks would like this:

pub trait Circuit<F: Field> {
    /// [...]

    fn configure(&self, meta: &mut ConstraintSystem<F>) -> Self::Config;
}

This is the approach I took with Plaf to support halo2 backend: https://github.com/Dhole/polyexen/blob/master/src/plaf/backends/halo2.rs

Pros:

• Simple trait change
• High flexibility, as we can reuse the self type to add anything there to be used as configuration parameters
  Cons:
• Breaking change: every implementation of Circuit needs a small change on the configure method to take self
• The self is the circuit type, which usually holds the witness. But now it holds the witness and configuration parameters. In this scenario a developer could write a configuration that depends on the witness. Nevertheless this problem already exists in the halo2 API because the synthesize does receive a self and it assigns fixed column values and sets copy constraints which are part of the circuit setup.

Alternatives:

• Maybe we could have a new trait method that has a default implementation so that we don't introduce a breaking change? For example:

pub trait Circuit<F: Field> {
    /// [...]

    fn configure_with_self(&self, meta: &mut ConstraintSystem<F>) -> Self::Config {
        Self::configure(meta)
    }

    fn configure(meta: &mut ConstraintSystem<F>) -> Self::Config;
}

• We could add a new associated type that holds the config parameters (to separate it from the witness values). This is also a breaking change

pub trait Circuit<F: Field> {
    /// [...]
    type Params: Default;

    fn params(&self) -> Self::Params {
        Self::Params::default()
    }

    fn configure_with_params(params: Self::Params, meta: &mut ConstraintSystem<F>) -> Self::Config {
        Self::configure(meta)
    }

    fn configure(meta: &mut ConstraintSystem<F>) -> Self::Config;

Please share your thoughts on which option you think is best! It would be nice to agree on one option and introduce it in our fork :D

@han0110 @CPerezz @kilic

[CPerezz]

I love the second option!!! 👏
It's really expressive and although it breaks API, i think we neede to break it in any case. And if we're already on this scenario, it's better to bet for a more curated API as in the second example.

I also like a lot the default impl for configure_with_params which will help to have an easy API transition.

Could you elaborate a bit more on the insights about why this will help to split halo2 into front and backend? And also on why does it provide so that DSLs are easier/possible to be implemented?

[han0110]

To use halo2 as a backend, I think the approach with associated type looks good and it should be relatively easy to migrate.

Just note we might also change other APIs like:

• VerifyingKey::read
• VerifyingKey::from_bytes
• ProvingKey::read
• ProvingKey::from_bytes
• create_domain
• CircuitGates::collect

Because they don't take circuit: &C as an input, they just use the C::configure directly, so we might need to add an extra argument param: C::Param to make sure the configure is expected.

[ed255]

Could you elaborate a bit more on the insights about why this will help to split halo2 into front and backend? And also on why does it provide so that DSLs are easier/possible to be implemented?

The interface that halo2 uses for the backend is the Circuit trait: anything that implements this trait can be used with the halo2 proving system. But the configure method of this trait doesn't take self or any user provided arguments, and this method is the one that defines the columns, the polynomial identities and the lookup expressions. Since this method is called like Circuit::configure(meta) it means that the configuration needs to be defined at compilation time, statically with hardcoded "rust code". You can't encode the circuit into your own format and then convert it into a halo2 Circuit, because the configure doesn't have access to any user-defined runtime variables. Right now everything needs to be defined inside the configure function, you can't have auxiliary data outside of it. And for a DSL, before defining the Circuit we would process a higher level circuit definition, store it in variables and then we would want to pass it to the configure method of a generic Circuit.

Here's an example for Plaf, which uses the configure(&self, ...) approach.
There's a single type that implements Circuit and is generic to define any circuit you want which is encoded in the plaf format:
https://github.com/Dhole/polyexen/blob/ba36325e9dd18125c4c7e4e7e34ba8bb75dfa110/src/plaf/backends/halo2.rs#L171-L181

Without access to self or another user provided parameter, you need a new rust type for each circuit; each instance of that type will be the same circuit with different witness. With access to self or another user provided parameter, you can have a single type and then each instance of that type is a different circuit.

[CPerezz]

Thanks for the insight @ed255 !

Makes complete sense. In favour for the change! Let's hear @han0110 and @kilic