Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: (De)Serialization of points using GroupEncoding #88

Merged

Conversation

huitseeker
Copy link
Contributor

@huitseeker huitseeker commented Sep 12, 2023

Context

In #30 and in #39 this repo implements the SerdeObject trait that allows straight-line serialization and de-serialization of the internal point representation. This practice makes a lot of sense when (de)serialization is used to do local caching in zk-algorithms, and some sense when performing absorptions into e.g. a transcript.

However, as a format of exchange it isn't compatible with classical point (de)serialization which more traditionally uses the compressed point representation, which is helpfully defined here (for all curves in this repo) as the implementation for the zkcrypto/group::GroupEncoding trait for both projective and affine point representations (and is also what pasta uses).

Precisely because the SerdeObject trait allows an alternate route to get the raw representation of point objects (and, IIUC, downstream repos use these code paths), the default implementation of the Serialize/Deserialize traits could hence aim at compatibility and use the traditional compressed point representation for serde, which is what this PR suggests.

In a nutshell:

  • the SerdeObject code paths provide raw representation bytes,
  • the Serde code paths provide the classical compressed point representations.

Alternative

Because the implementation is limited to one single point of indirection (the serde annotation on $name, $name_affine becomes one macro call), we can also envision more optionality by e.g. opening a feature (e.g. "compressed_serde") allowing the user to pick and choose what serde would give them.

In detail

  • Updated curve point (de)serialization logic from the internal representation (generated by #[derive(Serialize, Deserialize)]) to the representation offered by the implementation of the GroupEncoding trait.

See also

#85

…ing` trait

- Updated curve point (de)serialization logic from the internal representation to the
  representation offered by the implementation of the `GroupEncoding` trait.
Copy link
Member

@CPerezz CPerezz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the direction this takes.
In any case, will ask the review of other team members to see their opinions.

I like that SerdeObject (which we could rename TBH) serves as raw/unchecked encoding and that then, regular serde is used for canonically checked en/decoding.
If we can also guarantee that with GroupEncoding we can both serialize and deserialize in compressed & uncompressed forms, that should be it.

Copy link
Contributor

@han0110 han0110 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@davidnevadoc davidnevadoc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the rationale of the solution proposed in the description. LGTM!

@CPerezz CPerezz added this pull request to the merge queue Sep 18, 2023
Merged via the queue into privacy-scaling-explorations:main with commit 2f3e388 Sep 18, 2023
7 checks passed
jonathanpwang added a commit to axiom-crypto/halo2curves that referenced this pull request Sep 23, 2023
* Add field conversion to/from `[u64;4]` (privacy-scaling-explorations#80)

* feat: add field conversion to/from `[u64;4]`

* Added conversion tests
* Added `montgomery_reduce_short` for no-asm
* For bn256, uses assembly conversion when asm feature is on

* fix: remove conflict for asm

* chore: bump rust-toolchain to 1.67.0

* Compute Legendre symbol for `hash_to_curve` (privacy-scaling-explorations#77)

* Add `Legendre` trait and macro

 - Add Legendre macro with norm and legendre symbol computation
 - Add macro for automatic implementation in prime fields

* Add legendre macro call for prime fields

* Remove unused imports

* Remove leftover

* Add `is_quadratic_non_residue` for hash_to_curve

* Add `legendre` function

* Compute modulus separately

* Substitute division for shift

* Update modulus computation

* Add quadratic residue check func

* Add quadratic residue tests

* Add hash_to_curve bench

* Implement Legendre trait for all curves

* Move misplaced comment

* Add all curves to hash bench

* fix: add suggestion for legendre_exp

* fix: imports after rebase

* Add simplified SWU method (privacy-scaling-explorations#81)

* Fix broken link

* Add simple SWU algorithm

* Add simplified SWU hash_to_curve for secp256r1

* add: sswu z reference

* update MAP_ID identifier

Co-authored-by: Han <tinghan0110@gmail.com>

---------

Co-authored-by: Han <tinghan0110@gmail.com>

* Bring back curve algorithms for `a = 0` (privacy-scaling-explorations#82)

* refactor: bring back curve algorithms for `a = 0`

* fix: clippy warning

* fix: Improve serialization for prime fields (privacy-scaling-explorations#85)

* fix: Improve serialization for prime fields

Summary: 256-bit field serialization is currently 4x u64, ie. the native format. This implements the standard of byte-serialization (corresponding to the PrimeField::{to,from}_repr), and an hex-encoded variant of
that for (de)serializers that are human-readable (concretely, json).

- Added a new macro `serialize_deserialize_32_byte_primefield!` for custom serialization and deserialization of 32-byte prime field in different struct (Fq, Fp, Fr) across the secp256r, bn256, and derive libraries.
- Implemented the new macro for serialization and deserialization in various structs, replacing the previous `serde::{Deserialize, Serialize}` direct use.
- Enhanced error checking in the custom serialization methods to ensure valid field elements.
- Updated the test function in the tests/field.rs file to include JSON serialization and deserialization tests for object integrity checking.

* fixup! fix: Improve serialization for prime fields

---------

Co-authored-by: Carlos Pérez <37264926+CPerezz@users.noreply.github.com>

* refactor: (De)Serialization of points using `GroupEncoding` (privacy-scaling-explorations#88)

* refactor: implement (De)Serialization of points using the `GroupEncoding` trait

- Updated curve point (de)serialization logic from the internal representation to the
  representation offered by the implementation of the `GroupEncoding` trait.

* fix: add explicit json serde tests

* Insert MSM and FFT code and their benchmarks. (privacy-scaling-explorations#86)

* Insert MSM and FFT code and their benchmarks.

Resolves taikoxyz/zkevm-circuits#150.

* feedback

* Add instructions

* feeback

* Implement feedback:  Actually supply the correct arguments to `best_multiexp`.

Split into `singlecore` and `multicore` benchmarks so Criterion's result
caching and comparison over multiple runs makes sense.

Rewrite point and scalar generation.

* Use slicing and parallelism to to decrease running time.

Laptop measurements:
k=22: 109 sec
k=16:   1 sec

* Refactor msm

* Refactor fft

* Update module comments

* Fix formatting

* Implement suggestion for fixing CI

---------

Co-authored-by: David Nevado <davidnevadoc@users.noreply.github.com>
Co-authored-by: Han <tinghan0110@gmail.com>
Co-authored-by: François Garillot <4142+huitseeker@users.noreply.github.com>
Co-authored-by: Carlos Pérez <37264926+CPerezz@users.noreply.github.com>
Co-authored-by: einar-taiko <126954546+einar-taiko@users.noreply.github.com>
@huitseeker huitseeker mentioned this pull request Oct 23, 2023
jonathanpwang added a commit to axiom-crypto/halo2curves that referenced this pull request Nov 13, 2023
* Add field conversion to/from `[u64;4]` (privacy-scaling-explorations#80)

* feat: add field conversion to/from `[u64;4]`

* Added conversion tests
* Added `montgomery_reduce_short` for no-asm
* For bn256, uses assembly conversion when asm feature is on

* fix: remove conflict for asm

* chore: bump rust-toolchain to 1.67.0

* Compute Legendre symbol for `hash_to_curve` (privacy-scaling-explorations#77)

* Add `Legendre` trait and macro

 - Add Legendre macro with norm and legendre symbol computation
 - Add macro for automatic implementation in prime fields

* Add legendre macro call for prime fields

* Remove unused imports

* Remove leftover

* Add `is_quadratic_non_residue` for hash_to_curve

* Add `legendre` function

* Compute modulus separately

* Substitute division for shift

* Update modulus computation

* Add quadratic residue check func

* Add quadratic residue tests

* Add hash_to_curve bench

* Implement Legendre trait for all curves

* Move misplaced comment

* Add all curves to hash bench

* fix: add suggestion for legendre_exp

* fix: imports after rebase

* Add simplified SWU method (privacy-scaling-explorations#81)

* Fix broken link

* Add simple SWU algorithm

* Add simplified SWU hash_to_curve for secp256r1

* add: sswu z reference

* update MAP_ID identifier

Co-authored-by: Han <tinghan0110@gmail.com>

---------

Co-authored-by: Han <tinghan0110@gmail.com>

* Bring back curve algorithms for `a = 0` (privacy-scaling-explorations#82)

* refactor: bring back curve algorithms for `a = 0`

* fix: clippy warning

* fix: Improve serialization for prime fields (privacy-scaling-explorations#85)

* fix: Improve serialization for prime fields

Summary: 256-bit field serialization is currently 4x u64, ie. the native format. This implements the standard of byte-serialization (corresponding to the PrimeField::{to,from}_repr), and an hex-encoded variant of
that for (de)serializers that are human-readable (concretely, json).

- Added a new macro `serialize_deserialize_32_byte_primefield!` for custom serialization and deserialization of 32-byte prime field in different struct (Fq, Fp, Fr) across the secp256r, bn256, and derive libraries.
- Implemented the new macro for serialization and deserialization in various structs, replacing the previous `serde::{Deserialize, Serialize}` direct use.
- Enhanced error checking in the custom serialization methods to ensure valid field elements.
- Updated the test function in the tests/field.rs file to include JSON serialization and deserialization tests for object integrity checking.

* fixup! fix: Improve serialization for prime fields

---------

Co-authored-by: Carlos Pérez <37264926+CPerezz@users.noreply.github.com>

* refactor: (De)Serialization of points using `GroupEncoding` (privacy-scaling-explorations#88)

* refactor: implement (De)Serialization of points using the `GroupEncoding` trait

- Updated curve point (de)serialization logic from the internal representation to the
  representation offered by the implementation of the `GroupEncoding` trait.

* fix: add explicit json serde tests

* Insert MSM and FFT code and their benchmarks. (privacy-scaling-explorations#86)

* Insert MSM and FFT code and their benchmarks.

Resolves taikoxyz/zkevm-circuits#150.

* feedback

* Add instructions

* feeback

* Implement feedback:  Actually supply the correct arguments to `best_multiexp`.

Split into `singlecore` and `multicore` benchmarks so Criterion's result
caching and comparison over multiple runs makes sense.

Rewrite point and scalar generation.

* Use slicing and parallelism to to decrease running time.

Laptop measurements:
k=22: 109 sec
k=16:   1 sec

* Refactor msm

* Refactor fft

* Update module comments

* Fix formatting

* Implement suggestion for fixing CI

* Re-export also mod `pairing` and remove flag `reexport` to alwasy re-export (privacy-scaling-explorations#93)

fix: re-export also mod `pairing` and remove flag `reexport` to alwasy re-export

* fix regression in privacy-scaling-explorations#93 reexport field benches aren't run (privacy-scaling-explorations#94)

fix regression in privacy-scaling-explorations#93, field benches aren't run

* Fast modular inverse - 9.4x acceleration (privacy-scaling-explorations#83)

* Bernstein yang modular multiplicative inverter (#2)

* rename similar to privacy-scaling-explorations#95

---------

Co-authored-by: Aleksei Vambol <77882392+AlekseiVambol@users.noreply.github.com>

* Fast isSquare / Legendre symbol / Jacobi symbol - 16.8x acceleration (privacy-scaling-explorations#95)

* Derivatives of the Pornin's method (taikoxyz#3)

* renaming file

* make cargo fmt happy

* clarifications from privacy-scaling-explorations#95 (comment) [skip ci]

* Formatting and slightly changing a comment

---------

Co-authored-by: Aleksei Vambol <77882392+AlekseiVambol@users.noreply.github.com>

* chore: delete bernsteinyang module (replaced by ff_inverse)

* Bump version to 0.4.1

---------

Co-authored-by: David Nevado <davidnevadoc@users.noreply.github.com>
Co-authored-by: Han <tinghan0110@gmail.com>
Co-authored-by: François Garillot <4142+huitseeker@users.noreply.github.com>
Co-authored-by: Carlos Pérez <37264926+CPerezz@users.noreply.github.com>
Co-authored-by: einar-taiko <126954546+einar-taiko@users.noreply.github.com>
Co-authored-by: Mamy Ratsimbazafy <mamy_github@numforge.co>
Co-authored-by: Aleksei Vambol <77882392+AlekseiVambol@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants