-
Notifications
You must be signed in to change notification settings - Fork 969
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
EIP4844: Update cryptography API and Fiat-Shamir logic #3038
Changes from 21 commits
429e597
22a4dcd
b5959a1
642f138
ff528a2
91476fe
090dc7e
30d19a3
0eb82cf
7631c18
fe7af4b
46b6b24
889deff
83ca385
cbc170b
89d4ae0
b9dfdaf
033567b
e81d54c
463948e
a33a423
31ad8a5
0174521
dfcf33c
d98c103
80d4d09
c8b8b53
5354a96
0e2e477
db619e2
cb46b11
af48987
1c9a8db
186a2eb
c130995
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,14 +18,20 @@ | |
- [`bit_reversal_permutation`](#bit_reversal_permutation) | ||
- [BLS12-381 helpers](#bls12-381-helpers) | ||
- [`bytes_to_bls_field`](#bytes_to_bls_field) | ||
- [`blob_to_field_elements`](#blob_to_field_elements) | ||
- [`hash_to_bls_field`](#hash_to_bls_field) | ||
- [`bls_modular_inverse`](#bls_modular_inverse) | ||
- [`div`](#div) | ||
- [`g1_lincomb`](#g1_lincomb) | ||
- [`vector_lincomb`](#vector_lincomb) | ||
- [`poly_lincomb`](#poly_lincomb) | ||
- [`compute_powers`](#compute_powers) | ||
- [KZG](#kzg) | ||
- [`blob_to_kzg_commitment`](#blob_to_kzg_commitment) | ||
- [`verify_kzg_proof`](#verify_kzg_proof) | ||
- [`compute_kzg_proof`](#compute_kzg_proof) | ||
- [`compute_aggregated_poly_and_commitment`](#compute_aggregated_poly_and_commitment) | ||
- [`compute_aggregate_kzg_proof`](#compute_aggregate_kzg_proof) | ||
- [`verify_aggregate_kzg_proof`](#verify_aggregate_kzg_proof) | ||
- [Polynomials](#polynomials) | ||
- [`evaluate_polynomial_in_evaluation_form`](#evaluate_polynomial_in_evaluation_form) | ||
|
||
|
@@ -46,13 +52,23 @@ This document specifies basic polynomial operations and KZG polynomial commitmen | |
| `BLSFieldElement` | `uint256` | `x < BLS_MODULUS` | | ||
| `KZGCommitment` | `Bytes48` | Same as BLS standard "is valid pubkey" check but also allows `0x00..00` for point-at-infinity | | ||
| `KZGProof` | `Bytes48` | Same as for `KZGCommitment` | | ||
| `Polynomial` | `Vector[BLSFieldElement, FIELD_ELEMENTS_PER_BLOB]` | a polynomial in evaluation form | | ||
asn-d6 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Constants | ||
|
||
*Note*: Domain separators are set to an empty string as they are not required according to current understanding (and are not "free"). | ||
Should we determine that they are required at a later point we can set | ||
|
||
| Name | Value | Notes | | ||
| - | - | - | | ||
| `BLS_MODULUS` | `52435875175126190479447740508185965837690552500527637822603658699938581184513` | Scalar field modulus of BLS12-381 | | ||
| `ROOTS_OF_UNITY` | `Vector[BLSFieldElement, FIELD_ELEMENTS_PER_BLOB]` | Roots of unity of order FIELD_ELEMENTS_PER_BLOB over the BLS12-381 field | | ||
| `DOMAIN_SEPARATOR_AGGREGATE_PROTOCOL` | `b""` | Fiat-Shamir domain separator random aggregation | | ||
| `DOMAIN_SEPARATOR_EVAL_PROTOCOL` | `b""` | Fiat-Shamir domain separator random evaluation | | ||
| `DOMAIN_SEPARATOR_FIELD_ELEMENT` | `b""` | Fiat-Shamir domain separator for field elements | | ||
| `DOMAIN_SEPARATOR_POINT` | `b""` | Fiat-Shamir domain separator for G1 points | | ||
| `DOMAIN_SEPARATOR_SQUEEZE` | `b""` | Fiat-Shamir domain separator before hashing | | ||
asn-d6 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
## Preset | ||
|
||
|
@@ -119,7 +135,52 @@ def bytes_to_bls_field(b: Bytes32) -> BLSFieldElement: | |
""" | ||
Convert bytes to a BLS field scalar. The output is not uniform over the BLS field. | ||
""" | ||
return int.from_bytes(b, "little") % BLS_MODULUS | ||
return int.from_bytes(b, ENDIANNESS) % BLS_MODULUS | ||
``` | ||
|
||
#### `blob_to_field_elements` | ||
|
||
```python | ||
def blob_to_field_elements(blob: Blob) -> Polynomial: | ||
dankrad marked this conversation as resolved.
Show resolved
Hide resolved
|
||
""" | ||
Convert blob to list of BLS field scalars. | ||
""" | ||
r = Polynomial() | ||
hwwhww marked this conversation as resolved.
Show resolved
Hide resolved
|
||
for i in range(FIELD_ELEMENTS_PER_BLOB): | ||
value = int.from_bytes(blob[i * BYTES_PER_FIELD_ELEMENT: (i + 1) * BYTES_PER_FIELD_ELEMENT], ENDIANNESS) | ||
assert value < BLS_MODULUS | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (Just a note) For reference, this check |
||
r[i] = value | ||
return r | ||
asn-d6 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` | ||
|
||
#### `hash_to_bls_field` | ||
|
||
```python | ||
def hash_to_bls_field(initializer: bytes, | ||
polys: Sequence[Polynomial], | ||
comms: Sequence[KZGCommitment]) -> Tuple[BLSFieldElement, bytes]: | ||
""" | ||
Compute 32-byte hash of serialised polynomials and commitments concatenated. | ||
This hash is then converted to a BLS field element. | ||
The output is not uniform over the BLS field. | ||
""" | ||
data = initializer | ||
|
||
# Append each polynomial which is composed by field elements | ||
for poly in polys: | ||
for field_element in poly: | ||
data += DOMAIN_SEPARATOR_FIELD_ELEMENT | ||
data += int.to_bytes(field_element, 32, ENDIANNESS) | ||
hwwhww marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
# Append serialised G1 points | ||
hwwhww marked this conversation as resolved.
Show resolved
Hide resolved
|
||
for commitment in comms: | ||
data += DOMAIN_SEPARATOR_POINT | ||
data += commitment | ||
|
||
data += DOMAIN_SEPARATOR_SQUEEZE | ||
h = hash(data) | ||
|
||
return bytes_to_bls_field(h), h | ||
``` | ||
|
||
#### `bls_modular_inverse` | ||
|
@@ -155,22 +216,37 @@ def g1_lincomb(points: Sequence[KZGCommitment], scalars: Sequence[BLSFieldElemen | |
return KZGCommitment(bls.G1_to_bytes48(result)) | ||
``` | ||
|
||
#### `vector_lincomb` | ||
#### `poly_lincomb` | ||
|
||
```python | ||
def vector_lincomb(vectors: Sequence[Sequence[BLSFieldElement]], | ||
scalars: Sequence[BLSFieldElement]) -> Sequence[BLSFieldElement]: | ||
def poly_lincomb(polys: Sequence[Polynomial], | ||
scalars: Sequence[BLSFieldElement]) -> Polynomial: | ||
""" | ||
Given a list of ``vectors``, interpret it as a 2D matrix and compute the linear combination | ||
of each column with `scalars`: return the resulting vector. | ||
Given a list of ``polynomials``, interpret it as a 2D matrix and compute the linear combination | ||
of each column with `scalars`: return the resulting polynomials. | ||
""" | ||
result = [0] * len(vectors[0]) | ||
for v, s in zip(vectors, scalars): | ||
result = [0] * len(polys[0]) | ||
for v, s in zip(polys, scalars): | ||
for i, x in enumerate(v): | ||
result[i] = (result[i] + int(s) * int(x)) % BLS_MODULUS | ||
return [BLSFieldElement(x) for x in result] | ||
``` | ||
|
||
#### `compute_powers` | ||
|
||
```python | ||
def compute_powers(x: BLSFieldElement, n: uint64) -> Sequence[BLSFieldElement]: | ||
""" | ||
Return ``x`` to power of [0, n-1]. | ||
""" | ||
current_power = 1 | ||
powers = [] | ||
for _ in range(n): | ||
powers.append(BLSFieldElement(current_power)) | ||
current_power = current_power * int(x) % BLS_MODULUS | ||
return powers | ||
``` | ||
|
||
### KZG | ||
|
||
KZG core functions. These are also defined in EIP-4844 execution specs. | ||
|
@@ -179,7 +255,7 @@ KZG core functions. These are also defined in EIP-4844 execution specs. | |
|
||
```python | ||
def blob_to_kzg_commitment(blob: Blob) -> KZGCommitment: | ||
return g1_lincomb(bit_reversal_permutation(KZG_SETUP_LAGRANGE), blob) | ||
return g1_lincomb(bit_reversal_permutation(KZG_SETUP_LAGRANGE), blob_to_field_elements(blob)) | ||
``` | ||
|
||
#### `verify_kzg_proof` | ||
|
@@ -204,7 +280,7 @@ def verify_kzg_proof(polynomial_kzg: KZGCommitment, | |
#### `compute_kzg_proof` | ||
|
||
```python | ||
def compute_kzg_proof(polynomial: Sequence[BLSFieldElement], z: BLSFieldElement) -> KZGProof: | ||
def compute_kzg_proof(polynomial: Polynomial, z: BLSFieldElement) -> KZGProof: | ||
""" | ||
Compute KZG proof at point `z` with `polynomial` being in evaluation form | ||
""" | ||
|
@@ -226,12 +302,64 @@ def compute_kzg_proof(polynomial: Sequence[BLSFieldElement], z: BLSFieldElement) | |
return KZGProof(g1_lincomb(bit_reversal_permutation(KZG_SETUP_LAGRANGE), quotient_polynomial)) | ||
``` | ||
|
||
#### `compute_aggregated_poly_and_commitment` | ||
|
||
```python | ||
def compute_aggregated_poly_and_commitment( | ||
blobs: Sequence[Blob], | ||
kzg_commitments: Sequence[KZGCommitment]) -> Tuple[Polynomial, KZGCommitment, bytes]: | ||
""" | ||
Return the aggregated polynomial and aggregated KZG commitment. | ||
""" | ||
# Generate random linear combination challenges | ||
r, h = hash_to_bls_field(DOMAIN_SEPARATOR_AGGREGATE_PROTOCOL, blobs, kzg_commitments) | ||
r_powers = compute_powers(r, len(kzg_commitments)) | ||
|
||
# Create aggregated polynomial in evaluation form | ||
aggregated_poly = Polynomial(poly_lincomb([blob_to_field_elements(blob) for blob in blobs], r_powers)) | ||
dankrad marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
# Compute commitment to aggregated polynomial | ||
aggregated_poly_commitment = KZGCommitment(g1_lincomb(kzg_commitments, r_powers)) | ||
|
||
return aggregated_poly, aggregated_poly_commitment, h | ||
``` | ||
|
||
#### `compute_aggregate_kzg_proof` | ||
|
||
```python | ||
def compute_aggregate_kzg_proof(blobs: Sequence[Blob]) -> KZGProof: | ||
commitments = [blob_to_kzg_commitment(blob) for blob in blobs] | ||
aggregated_poly, aggregated_poly_commitment, h = compute_aggregated_poly_and_commitment(blobs, commitments) | ||
x, _ = hash_to_bls_field(DOMAIN_SEPARATOR_EVAL_PROTOCOL + h, [aggregated_poly], [aggregated_poly_commitment]) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. small nit:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it should be the other way around:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Switching h to be appended first instead of second, should not change random oracle collisions I think, unless the hash function is bad. Instead of thinking of putting it at the beginning of the hash function, I'm thinking of putting it at the beginning of the sub protocol. Ie each new protocol if given a unique domain separator will effectively have a "different" hash function or RO There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I do not believe that what you suggest functions as a domain separator. If your hashed string is |
||
return compute_kzg_proof(aggregated_poly, x) | ||
``` | ||
|
||
#### `verify_aggregate_kzg_proof` | ||
|
||
```python | ||
def verify_aggregate_kzg_proof(blobs: Sequence[Blob], | ||
expected_kzg_commitments: Sequence[KZGCommitment], | ||
kzg_aggregated_proof: KZGCommitment) -> bool: | ||
aggregated_poly, aggregated_poly_commitment, h = compute_aggregated_poly_and_commitment( | ||
blobs, | ||
expected_kzg_commitments, | ||
) | ||
|
||
# Generate challenge `x` and evaluate the aggregated polynomial at `x` | ||
x, _ = hash_to_bls_field(DOMAIN_SEPARATOR_EVAL_PROTOCOL + h, [aggregated_poly], [aggregated_poly_commitment]) | ||
# Evaluate aggregated polynomial at `x` (evaluation function checks for div-by-zero) | ||
y = evaluate_polynomial_in_evaluation_form(aggregated_poly, x) | ||
|
||
# Verify aggregated proof | ||
return verify_kzg_proof(aggregated_poly_commitment, x, y, kzg_aggregated_proof) | ||
``` | ||
|
||
### Polynomials | ||
|
||
#### `evaluate_polynomial_in_evaluation_form` | ||
|
||
```python | ||
def evaluate_polynomial_in_evaluation_form(polynomial: Sequence[BLSFieldElement], | ||
def evaluate_polynomial_in_evaluation_form(polynomial: Polynomial, | ||
asn-d6 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
z: BLSFieldElement) -> BLSFieldElement: | ||
""" | ||
Evaluate a polynomial (in evaluation form) at an arbitrary point `z` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return type is wrong. how about:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in af48987
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, slightly annoying because
None
could also mean that the sidecar is not available (I know in python we would do this by raising an exception but not all languages would have that available).