Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Sync halo2 lib 0.4.0 #68

Closed
wants to merge 31 commits into from
Closed

Conversation

zhenfeizhang
Copy link

No description provided.

han0110 and others added 28 commits October 8, 2023 05:28
privacy-scaling-explorations#149)

* feat: expose mod ule `permutation` and re-export `permutation::keygen::Assembly`

* feat: derive `lone` for `permutation::keygen::Assembly`

* feat: bump MSRV for `inferno`
…ations#157)

* change: Migrate workspace to pasta_curves-0.5

This ports the majority of the workspace to the `pasta_curves-0.5.0`
leaving some tricky edge-cases that we need to handle carefully.

Resolves: privacy-scaling-explorations#132

* fix: Complete latest trait bounds to compile halo2proofs

* change: Migrate examples & benches to pasta 0.5

* change: Migrate halo2_gadgets to pasta-0.5

* change: Update gadgets outdated code with latest upstream

* fix: Sha3 gadget circuit

* fix: doc tests

* chore: Update merged main

* fix: Apply review suggestions
…plorations#168)

* Extend Circuit trait to take parameters in config

The Circuit trait is extended with the following:
```
pub trait Circuit<F: Field> {
    /// [...]
    type Params: Default;

    fn params(&self) -> Self::Params {
        Self::Params::default()
    }

    fn configure_with_params(meta: &mut ConstraintSystem<F>, params: &Self::Params) -> Self::Config {
        Self::configure(meta)
    }

    fn configure(meta: &mut ConstraintSystem<F>) -> Self::Config;
}
```

This allows runtime parametrization of the circuit configuration.  The extension to the Circuit trait has been designed to minimize the breaking change: existing circuits only need to define the associated `type Params`.

Unfortunately "Associated type defaults" are unstable in Rust, otherwise this would be a non-breaking change.  See rust-lang/rust#29661

* Implement circuit params under feature flag

* Don't overwrite configure method

* Fix doc test
…g-explorations#156)

* static ref to String type in Gates, Constraints, VirtualCell, Argument

* 'lookup'.to_string()

* return &str for gate name and constriant_name, also run fmt

* Update halo2_gadgets/Cargo.toml

Co-authored-by: Han <tinghan0110@gmail.com>

* upgrade rust-toochain

---------

Co-authored-by: Carlos Pérez <37264926+CPerezz@users.noreply.github.com>
Co-authored-by: Han <tinghan0110@gmail.com>
* return expression from cell

* add example

* selector

* recurse Expression to fill in index

* minimized changes from the original

* backword compatible meta.query_X & challange.expr()

* cargo fmt

* fixed lookup to pass all tests

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* Update comments

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* update

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>

* add primitives.rs back

* remove example2

* backward compatible meta.query_X & Column.cur(), next(), prev(), at(usize)

* impl Debug & make side effects only when query.index.is_none()

* change impl Debug for Expression instead & revert test in plonk_api

* upgrade rust-toolchain

* Update halo2_proofs/src/plonk/circuit.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* Update halo2_proofs/src/plonk/circuit.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* ran clippy

* Update halo2_proofs/src/plonk/circuit.rs

Co-authored-by: Han <tinghan0110@gmail.com>

---------

Co-authored-by: Brecht Devos <Brechtp.Devos@gmail.com>
Co-authored-by: Han <tinghan0110@gmail.com>
…#178)

Now the value returned when the number of selectors is a multiple of 8
is correct.

Resolves: privacy-scaling-explorations#175
* add a constructor

* add more comment

* fix as review

* remove clone

* remove

* no need to use new variable

* change comment

* fix clippy

* rename to from_parts

* remove n declaration
* feat: send / sync region

* Update layout.rs

* update

* lol

* debug

* Update keygen.rs

* Update keygen.rs

* Update keygen.rs

* Update keygen.rs

* thread-safe-region feature flag

* cleanup

* patch dev-graph

* patch non-determinism in mapping creation

* reduce mem usage for vk and pk

* mock proving examples

* swap for hashmap for insertion speed

* reduce update overhead

* replace BTree with Vec

* add benchmarks

* make the benchmarks massive

* patch clippy

* simplify lifetimes

* patch benches

* Update halo2_proofs/src/plonk/permutation/keygen.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* Update halo2_proofs/examples/vector-mul.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* rm benches

* order once

* patch lints

---------

Co-authored-by: Han <tinghan0110@gmail.com>
* fix parallelize workload imbalance

* remove the need of unsafe
* fix: explicitly define mds diff type

* rm paren
fix: remove nonsense comment

strictly check shuffle rows

address doc typos

move compression into product commitment

typo

add shuffle errors for `verify_at_rows_par`

dedup expression evaluation

cargo fmt

fix fields in sanity-checks feature
…rivacy-scaling-explorations#192)

* feat: public cells

* Update mds.rs

* Update mds.rs

* Update single_pass.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* bump toolchain to resolve errors

* fix clippy errors for CI run

* rustfmt post clippy

* plz let it be the last lint

* patch clippy lints in gadgets

* clippy lints for sha256 bench

* patch halo2proof benches

* Update assigned.rs

* Update halo2_gadgets/src/poseidon/primitives/mds.rs

Co-authored-by: Han <tinghan0110@gmail.com>

* Update halo2_gadgets/src/poseidon/primitives/mds.rs

Co-authored-by: Han <tinghan0110@gmail.com>

---------

Co-authored-by: Han <tinghan0110@gmail.com>
* refactor: add default impl for `SyncDeps` for backward compatability

* feat: pick changes from zcash#728 and changes of flag `test-dev-graph`

* feat: pick changes from zcash#622

* feat: pick changes about mod `circuit` and mod `dev`

* feat: pick rest changes of `halo2_proofs`

* fix: when `--no-default-features`

* ci: sync from upstream, and deduplicate jobs when
push to `main`, and remove always failing job `codecov`.

* fix: make `commit_zk` runnable when `--no-default-features`
…lorations#208)

* chore: Update rust-toolchain to 1.66 for testing

Note that tests will not compile due to the silent MSRV bump in
`blake2b_simd`.

Hence, we need to use `1.66` as toolchain.

Resolves: privacy-scaling-explorations#207

* change: UIpdate MSRVs in Cargo.toml
* fix: clippy

* fmt

* fix: Final clippy complains & adjustments

---------

Co-authored-by: CPerezz <c.perezbaro@gmail.com>
…s#209)

* Make it Eq to make it easier for tests

* Implement Sum and Product for Expression

* Make it readable
@Velaciela
Copy link

Velaciela commented Oct 31, 2023

Change Log

New feature

  • PSE#129: Call synthesize in MockProver multiple times to behave same as real prover
         related to scroll/halo2: #61, and commit 7a62a6bd: add phase-check feature
  • PSE#168: Extend Circuit trait to take parameters in config
         feature flag circuit-params and type Params = ();
  • PSE#180: feat: send sync region
         To enable parallel layouts for large circuits, the Region struct is modified to be Send and Sync by constraining its inner element
  • PSE#185: Implement native shuffle argument and api
         introduces ConstraintSystem::shuffle API, enabling efficient one-to-one mapping in dynamic lookup applications, saving permutation witness columns
  • PSE#209: Implement Sum & Prod trait impls for Expression

Improvement

  • PSE#150: feat(MockProver): replace errors by asserts
         In MockProver, replace all code that returns an error by an assert that panics instead of returning the error
  • PSE#156: Allow halo2 constraint names to have non static names
         name: &'static str >>> name: AsRef<str>
  • PSE#154: Improve halo2 query calls
         Remove the dependency on VirtualCells to query cells from columns
  • PSE#186: Fix parallelize workload imbalance
         performance: arithmetic.rs \ parallelize()
  • PSE#192: feat: public cells to allow for implementations of custom Layouter
         exposes the Layouter trait to allow for custom layouter implementation and makes cell.region_index and cell.row_offset public
  • PSE#199: Synchronize with zcash-halo2 [+2024, −1752]
         updates the impl for SyncDeps to maintain backward compatibility, modifies CI to test different feature sets, resolves issues
  • PSE#211: Improve root calculation for KZG setup()
         performance: ParamsKZG::setup

API change

  • PSE#129: Expose mod permutation and re-export permutation::keygen::Assembly
  • PSE#157: change: Migrate workspace to pasta_curves-0.5
         ports the the workspace to the pasta_curves-0.5.0, ff-0.13 and group-0.13 deps
         also to update a bit the halo2-gadgets
  • PSE#181: Add more getters to expose internal fields
  • PSE#164: Adding a constructor for ParamsKZG

Bug fix

  • PSE#178: fix: Fix serialization for VerifyingKey
         selector.len() / 8 + 1 >>> (selector.len() + 7) / 8

Commits that we didn't pick

lispc and others added 2 commits January 9, 2024 16:26
* Use thread pool for assign_regions (#57)

* feat: use rayon threadpool

* feat: add UT for many subregions

* refact: move common struct out to module level

* refact: reuse common configure code

* fix ci errors

---------

Co-authored-by: kunxian xia <xiakunxian130@gmail.com>

* Move `env_logger` dependency to dev-depdendencies (only for test). (#69)

* sync ff/group 0.13

* fix clippy

* fix clippy

* fmg

* [FEAT] Upgrading table16 for SHA256 (#73)

* upgrade sha256

* fix clippy

* Bus auto (#72)

* bus: expose global offset of regions

* bus-auto: add query_advice and query_fixed function in witness generation

* bus-auto: fix clippy

---------

Co-authored-by: Aurélien Nicolas <info@nau.re>

* fix-tob-scroll-21 (#59)

* fix-tob-scroll-21

* expose param field for re-randomization

* enable accessing for table16 (#75)

* chore: update poseidon link

* merge sha256 gadget changes

* Fix the CI errors (#78)

* cargo fmt

* fix clippy error

* Feat: switch to logup scheme for lookup argument  (#71)

* Multi-input mv-lookup. (#49)

* Add mv_lookup.rs

* mv_lookup::prover, mv_lookup::verifier

* Replace lookup with mv_lookup

* replace halo2 with mv lookup

Co-authored-by: ying tong <therealyingtong@users.noreply.github.com>

* cleanups

Co-authored-by: ying tong <therealyingtong@users.noreply.github.com>

* ConstraintSystem: setup lookup_tracker

Co-authored-by: Andrija <akinovak@gmail.com>

* mv_lookup::hybrid_prover

Co-authored-by: Andrija <akinovak@gmail.com>

* WIP

* mv_multi_lookup: enable lookup caching

Co-authored-by: therealyingtong <yingtong.lai@gmail.com>

* Rename hybrid_lookup -> lookup

* Chunk lookups using user-provided minimum degree

Co-authored-by: Andrija <akinovak@gmail.com>

* mv_lookup bench

Co-authored-by: Andrija <akinovak@gmail.com>

* Introduce counter feature for FFTs and MSMs

Co-authored-by: Andrija <akinovak@gmail.com>

* Fix off-by-one errors in chunk_lookup

Co-authored-by: Andrija <akinovak@gmail.com>

* bench wip

* time evaluate_h

* KZG

* more efficient batch inversion

* extended lookup example

* Finalize mv lookup

Author: therealyingtong <yingtong.lai@gmail.com>

* Remove main/

* Fix according to the comments

* replace scan with parallel grand sum computation

* Revert Cargo.lock

* mv lookup Argument name

* parallel batch invert

---------

Co-authored-by: Andrija <akinovak@gmail.com>
Co-authored-by: ying tong <therealyingtong@users.noreply.github.com>
Co-authored-by: therealyingtong <yingtong.lai@gmail.com>

* fmt

* fix unit test

* fix clippy errors

* add todo in mv_lookup's prover

* fmt and clippy

* fix clippy

* add detailed running time of steps in logup's prover

* fmt

* add more log hooks

* more running time logs

* use par invert

* use sorted-vector to store how many times a table element occurs in input

* par the process to get inputs_inv_sum

* use par

* fix par

* add feature to skip inv sums

* add new feature flag

* fix clippy error

---------

Co-authored-by: Sphere L <sph6r6.l1u@gmail.com>
Co-authored-by: Andrija <akinovak@gmail.com>
Co-authored-by: ying tong <therealyingtong@users.noreply.github.com>
Co-authored-by: therealyingtong <yingtong.lai@gmail.com>

* fix some simple building errs

* upgrade pathfinder_simd to newer version as it can't compile on mac m1 pro

* resolve merge conflict

* fmt

* clippy

* more clippy fix

* more lint fix

* fmt

* minor syntax fix

* fix ipa multiopen test failure

* fix clippy warning

* fmt

* fix par scan of log_inv diff

* remove uncessary clone

---------

Co-authored-by: alannotnerd <alan1995wang@outlook.com>
Co-authored-by: kunxian xia <xiakunxian130@gmail.com>
Co-authored-by: Steven <asongala@163.com>
Co-authored-by: Carlos Pérez <37264926+CPerezz@users.noreply.github.com>
Co-authored-by: zhenfei <zhenfei.zhang@hotmail.com>
Co-authored-by: Ho <noel.wei@gmail.com>
Co-authored-by: naure <naure@users.noreply.github.com>
Co-authored-by: Aurélien Nicolas <info@nau.re>
Co-authored-by: Sphere L <sph6r6.l1u@gmail.com>
Co-authored-by: Andrija <akinovak@gmail.com>
Co-authored-by: ying tong <therealyingtong@users.noreply.github.com>
Co-authored-by: therealyingtong <yingtong.lai@gmail.com>
@lispc lispc closed this Aug 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.