Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update EIP-4844: Fee Market Update #5707

Merged
merged 27 commits into from
Nov 1, 2022
Merged
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
d232f24
remove MAX_BLOBS_PER_TX
adietrichs Sep 5, 2022
d440dc2
change fake_exponential to taylor expansion
adietrichs Sep 6, 2022
1bbcf00
add MIN_GASPRICE_PER_BLOB
adietrichs Sep 6, 2022
104d114
change update fraction and add motivation
adietrichs Sep 6, 2022
8dc23c0
align tx field naming with 1559
adietrichs Sep 22, 2022
9528d7f
move explanation to rationale
adietrichs Sep 22, 2022
b7d7ad1
introduce data gas
adietrichs Sep 22, 2022
e8e3c64
add max_fee_per_data_gas field and validity conditions
adietrichs Sep 22, 2022
5f0c9b3
set reasonable MIN_DATA_GASPRICE
adietrichs Sep 22, 2022
add8919
fix naming
adietrichs Sep 22, 2022
e1cb10a
Update EIPS/eip-4844.md
adietrichs Sep 22, 2022
b99749b
remove redundant per-block blob limit info
adietrichs Sep 22, 2022
f27ae40
Update EIPS/eip-4844.md
adietrichs Sep 26, 2022
1dce329
Apply suggestions from code review
adietrichs Sep 30, 2022
aec0160
remove calldata mention from blob gas
adietrichs Sep 30, 2022
df2886f
Update EIPS/eip-4844.md
adietrichs Oct 5, 2022
4ed2d02
change update fraction to more closely approximate EIP-1559
adietrichs Oct 5, 2022
946ba9a
charge 1 data gas per byte
adietrichs Oct 5, 2022
5e1753b
track excess data gas instead of excess blobs
adietrichs Oct 5, 2022
118b91d
move target from blobs to data gas
adietrichs Oct 5, 2022
e01fe36
move limit from blobs to data gas
adietrichs Oct 5, 2022
94d6b2c
adjust update fraction for excess data gas tracking
adietrichs Oct 6, 2022
27e719c
clarify update fraction rationale
adietrichs Oct 25, 2022
e86a179
set min data gasprice to 1
adietrichs Oct 31, 2022
bc93ac3
clarify fee burn
adietrichs Oct 31, 2022
c4ecf05
update mempool issues section
adietrichs Oct 31, 2022
9638cfe
fix linting
adietrichs Oct 31, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 76 additions & 59 deletions EIPS/eip-4844.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,17 +45,18 @@ Compared to full data sharding, this EIP has a reduced cap on the number of thes
| `BLOB_COMMITMENT_VERSION_KZG` | `Bytes1(0x01)` |
| `POINT_EVALUATION_PRECOMPILE_ADDRESS` | `Bytes20(0x14)` |
| `POINT_EVALUATION_PRECOMPILE_GAS` | `50000` |
| `MAX_BLOBS_PER_BLOCK` | `16` |
| `TARGET_BLOBS_PER_BLOCK` | `8` |
| `MAX_BLOBS_PER_TX` | `2` |
| `GASPRICE_UPDATE_FRACTION_PER_BLOB` | `64` |
| `MAX_DATA_GAS_PER_BLOCK` | `2**21` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SamWilsn and @lightclient can we quickly allowlist the https://github.com/ethereum/consensus-specswhich I think was a consensus?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Pandapip1 I see you approved the PR but the merge is failing because of the links to the CL spec. Should we force-merge this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have this power :-D ?

Copy link
Contributor

@MicahZoltu MicahZoltu Oct 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All constants, especially unintuitive ones, should include rationale for why the specific value has been chosen.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*rationale

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The EIP does do so implicitly with wording like "corresponding to a target of ~1 MB per block and a limit of ~2 MB". Do you feel strongly that an explicit explanation / calculation for how these parameters result in those sizes would add value?

| `TARGET_DATA_GAS_PER_BLOCK` | `2**20` |
| `MIN_DATA_GASPRICE` | `1` |
| `DATA_GASPRICE_UPDATE_FRACTION` | `8902606` |
| `MAX_VERSIONED_HASHES_LIST_SIZE` | `2**24` |
| `MAX_CALLDATA_SIZE` | `2**24` |
| `MAX_ACCESS_LIST_SIZE` | `2**24` |
| `MAX_ACCESS_LIST_STORAGE_KEYS` | `2**24` |
| `MAX_TX_WRAP_KZG_COMMITMENTS` | `2**24` |
| `LIMIT_BLOBS_PER_TX` | `2**24` |
| `GAS_PER_BLOB` | `120000` |
| `DATA_GAS_PER_BLOB` | `2**17` |
| `SIMPLE_GAS_PER_BLOB` | `120000` |
| `HASH_OPCODE_BYTE` | `Bytes1(0x49)` |
| `HASH_OPCODE_GAS` | `3` |

Expand Down Expand Up @@ -92,16 +93,18 @@ def kzg_to_versioned_hash(kzg: KZGCommitment) -> VersionedHash:
return BLOB_COMMITMENT_VERSION_KZG + hash(kzg)[1:]
```

Approximates `2 ** (numerator / denominator)`, with the simplest possible approximation that is continuous and has a continuous derivative:
Approximates `factor * e ** (numerator / denominator)` using Taylor expansion:

```python
def fake_exponential(numerator: int, denominator: int) -> int:
cofactor = 2 ** (numerator // denominator)
fractional = numerator % denominator
return cofactor + (
fractional * cofactor * 2 +
(fractional ** 2 * cofactor) // denominator
) // (denominator * 3)
def fake_exponential(factor: int, numerator: int, denominator: int) -> int:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a rationale for switching to the Taylor expansion?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This came out of a discussion with @vbuterin. If I recall correctly, reasons were:

  • simpler conceptually
  • more precise approximation
  • only slightly more computationally expensive (and only done once per block, so insignificant)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is "conceptually simpler" for a very small set of people (math lovers). I think for the vast majority of the rest of the world (which likely includes most of the people building Ethereum and building stuff on Ethereum) the original was "conceptually simpler".

I don't think the precision here is necessary, given that we are just guessing at capacity targets anyway.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conceptually simpler not from a "reading the code", but an "understanding where the code came from" perspective. Which I would argue is the main objective.

i = 1
output = 0
numerator_accum = factor * denominator
while numerator_accum > 0:
output += numerator_accum
numerator_accum = (numerator_accum * numerator) // (denominator * i)
i += 1
return output // denominator
```

### New transaction type
Expand All @@ -118,13 +121,14 @@ class SignedBlobTransaction(Container):
class BlobTransaction(Container):
chain_id: uint256
nonce: uint64
priority_fee_per_gas: uint256
max_priority_fee_per_gas: uint256
max_fee_per_gas: uint256
gas: uint64
to: Union[None, Address] # Address = Bytes20
value: uint256
data: ByteList[MAX_CALLDATA_SIZE]
access_list: List[AccessTuple, MAX_ACCESS_LIST_SIZE]
max_fee_per_data_gas: uint256
blob_versioned_hashes: List[VersionedHash, MAX_VERSIONED_HASHES_LIST_SIZE]

class AccessTuple(Container):
Expand All @@ -137,7 +141,7 @@ class ECDSASignature(Container):
s: uint256
```

The `priority_fee_per_gas` and `max_fee_per_gas` fields follow [EIP-1559](./eip-1559.md) semantics,
The `max_priority_fee_per_gas` and `max_fee_per_gas` fields follow [EIP-1559](./eip-1559.md) semantics,
and `access_list` as in [`EIP-2930`](./eip-2930.md).

[`EIP-2718`](./eip-2718.md) is extended with a "wrapper data", the typed transaction can be encoded in two forms, dependent on the context:
Expand All @@ -158,8 +162,7 @@ the `TransactionNetworkPayload` version of the transaction also includes `blobs`
The execution layer verifies the wrapper validity against the inner `TransactionPayload` after signature verification as:

- All hashes in `blob_versioned_hashes` must start with the byte `BLOB_COMMITMENT_VERSION_KZG`
- There may be at most `MAX_BLOBS_PER_TX` blob commitments in any single transaction.
- There may be at most `MAX_BLOBS_PER_BLOCK` total blob commitments in a valid block.
- There may be at most `MAX_DATA_GAS_PER_BLOCK // DATA_GAS_PER_BLOB` total blob commitments in a valid block.
- There is an equal amount of versioned hashes, kzg commitments and blobs.
- The KZG commitments hash to the versioned hashes, i.e. `kzg_to_versioned_hash(kzg[i]) == versioned_hash[i]`
- The KZG commitments match the blob contents. (Note: this can be optimized with additional data, using a proof for a
Expand All @@ -181,9 +184,9 @@ def get_origin(tx: SignedBlobTransaction) -> Address:

### Header extension

The current header encoding is extended with a new 256-bit unsigned integer field `excess_blobs`. This is the running
total of excess blobs included on chain since this EIP was activated. If the total number of blobs is below the
average, `excess_blobs` is capped at zero.
The current header encoding is extended with a new 256-bit unsigned integer field `excess_data_gas`. This is the running
adietrichs marked this conversation as resolved.
Show resolved Hide resolved
total of excess data gas consumed on chain since this EIP was activated. If the total amount of data gas is below the
target, `excess_data_gas` is capped at zero.

The resulting RLP encoding of the header is therefore:

Expand All @@ -205,18 +208,19 @@ rlp([
mix_digest,
nonce,
base_fee,
excess_blobs
excess_data_gas
])
```

The value of `excess_blobs` can be calculated using the parent header and number of blobs in the block.
The value of `excess_data_gas` can be calculated using the parent header and number of blobs in the block.

```python
def calc_excess_blobs(parent: Header, new_blobs: int) -> int:
if parent.excess_blobs + new_blobs < TARGET_BLOBS_PER_BLOCK:
def calc_excess_data_gas(parent: Header, new_blobs: int) -> int:
consumed_data_gas = new_blobs * DATA_GAS_PER_BLOB
if parent.excess_data_gas + consumed_data_gas < TARGET_DATA_GAS_PER_BLOCK:
return 0
else:
return parent.excess_blobs + new_blobs - TARGET_BLOBS_PER_BLOCK
return parent.excess_data_gas + consumed_data_gas - TARGET_DATA_GAS_PER_BLOCK
```

### Beacon chain validation
Expand Down Expand Up @@ -271,33 +275,48 @@ def point_evaluation_precompile(input: Bytes) -> Bytes:

### Gas price of blobs (Simplified version)

For early draft implementations, we simply change `get_blob_gas(parent)` to always return `GAS_PER_BLOB`.
___WARNING: This is only for testing___

### Gas price update rule (Full version)
For early draft implementations, we simply change `get_blob_gas(parent)` to always return `SIMPLE_GAS_PER_BLOB`.
adietrichs marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EIPs should be written toward their final target and not contain historical details (one can look at commit history and discussions for that). It is OK to not be fully committed to everything in the EIP yet, but the form of the EIP should target its final state (which means not saying things like "For early draft implementations do X instead of Y". Instead, just say "Do X" and then later that may change to "Do Y".

That being said, I'm not an EIP editor anymore so I have no actual authority over this matter.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that this PR does not introduce either of the [simplified/full] sections, but just modifies the full one, I think removing the simplified one should not be in scope for the PR. It does seem reasonable to me to remove it soon though.


We propose a simple independent EIP-1559-style targeting rule to compute the gas cost of the transaction.
We use the `excess_blobs` header field to store persistent data needed to compute the cost.
### Gas accounting (Full version)

We introduce data gas as a new type of gas. It is independent of normal gas and follows its own targeting rule, similar to EIP-1559.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really dislike introducing a new resource separate from gas and calling it "data gas". The term "data gas" implies a subtype of gas—this isn't.

As a radical suggestion, I'd suggest calling this resource teu.

I won't let this hold up the PR though.

We use the `excess_data_gas` header field to store persistent data needed to compute the data gas price. For now, only blobs are priced in data gas.

```python
def get_intrinsic_gas(tx: SignedBlobTransaction, parent: Header) -> int:
intrinsic_gas = 21000 # G_transaction
if tx.message.to == None: # i.e. if a contract is created
intrinsic_gas = 53000
# EIP-2028 data gas cost reduction for zero bytes
intrinsic_gas += 16 * len(tx.message.data) - 12 * len(tx.message.data.count(0))
# EIP-2930 Optional access lists
intrinsic_gas += 1900 * sum(len(entry.storage_keys) for entry in tx.message.access_list) + 2400 * len(tx.message.access_list)
# New additional gas cost per blob
intrinsic_gas += len(tx.message.blob_versioned_hashes) * get_blob_gas(parent)
return intrinsic_gas

def get_blob_gas(header: Header) -> int:
def calc_data_fee(tx: SignedBlobTransaction, parent: Header) -> int:
return get_total_data_gas(tx) * get_data_gasprice(header)

def get_total_data_gas(tx: SignedBlobTransaction) -> int:
return DATA_GAS_PER_BLOB * len(tx.message.blob_versioned_hashes)

def get_data_gasprice(header: Header) -> int:
return fake_exponential(
header.excess_blobs,
GASPRICE_UPDATE_FRACTION_PER_BLOB
MIN_DATA_GASPRICE,
header.excess_data_gas,
DATA_GASPRICE_UPDATE_FRACTION
)
```

The block validity conditions are modified to include data gas checks:

```python
def validate_block(block: Block) -> None:
...

for tx in block.transactions:
...

# the signer must be able to afford the transaction
assert signer(tx).balance >= tx.message.gas * tx.message.max_fee_per_gas + get_total_data_gas(tx) * tx.message.max_fee_per_data_gas

# ensure that the user was willing to at least pay the current data gasprice
assert tx.message.max_fee_per_data_gas >= get_data_gasprice(parent(block).header)
```

The actual `data_fee` as calculated via `calc_data_fee` is deducted from the sender balance before transaction execution and burned, and is not refunded in case of transaction failure.

### Networking

Transactions are presented as `TransactionType || TransactionNetworkPayload` on the execution layer network,
Expand Down Expand Up @@ -412,17 +431,19 @@ allowing the point verification precompile to work with the new format.
Rollups would not have to make any EVM-level changes to how they work;
sequencers would simply have to switch over to using a new transaction type at the appropriate time.

### Blob gasprice update rule
### Data gasprice update rule

The blob gasprice update rule is intended to approximate the formula `blob_gas = 2**(excess_blobs / GASPRICE_UPDATE_FRACTION_PER_BLOB)`,
where `excess_blobs` is the total "extra" number of blobs that the chain has accepted relative to the "targeted" number (`TARGET_BLOBS_PER_BLOCK` per block).
Like EIP-1559, it's a self-correcting formula: as the excess goes higher, the `blob_gas` increases exponentially, reducing usage and eventually forcing the excess back down.
The data gasprice update rule is intended to approximate the formula `data_gasprice = MIN_DATA_GASPRICE * e**(excess_data_gas / DATA_GASPRICE_UPDATE_FRACTION)`,
where `excess_data_gas` is the total "extra" amount of data gas that the chain has consumed relative to the "targeted" number (`TARGET_DATA_GAS_PER_BLOCK` per block).
Like EIP-1559, it's a self-correcting formula: as the excess goes higher, the `data_gasprice` increases exponentially, reducing usage and eventually forcing the excess back down.

The block-by-block behavior is roughly as follows.
If in block `N`, `blob_gas = G1`, and block `N` has `X` blobs, then in block `N+1`, `excess_blobs` increases by `X - TARGET_BLOBS_PER_BLOCK`,
and so the `blob_gas` of block `N+1` increases by a factor of `2**((X - TARGET_BLOBS_PER_BLOCK) / GASPRICE_UPDATE_FRACTION_PER_BLOB)`.
If block `N` consumes `X` data gas, then in block `N+1` `excess_data_gas` increases by `X - TARGET_DATA_GAS_PER_BLOCK`,
and so the `data_gasprice` of block `N+1` increases by a factor of `e**((X - TARGET_DATA_GAS_PER_BLOCK) / DATA_GASPRICE_UPDATE_FRACTION)`.
Hence, it has a similar effect to the existing EIP-1559, but is more "stable" in the sense that it responds in the same way to the same total usage regardless of how it's distributed.

The parameter `DATA_GASPRICE_UPDATE_FRACTION` controls the maximum rate of change of the blob gas price. It is chosen to target a maximum change rate of `e(TARGET_DATA_GAS_PER_BLOCK / DATA_GASPRICE_UPDATE_FRACTION) ≈ 1.125` per block.

## Backwards Compatibility

### Blob non-accessibility
Expand All @@ -433,17 +454,13 @@ instead, they go into the `BeaconBlockBody`. This means that there is now a part

### Mempool issues

Blob transactions are unique in that they have a variable intrinsic gas cost. Hence, a transaction that could be included in one block may be invalid for the next.
To prevent mempool attacks, we recommend a simple technique: only propagate transactions whose `gas` is at least twice the current minimum.

Additionally, blob transactions have a large data size at the mempool layer, which poses a mempool DoS risk,
Blob transactions have a large data size at the mempool layer, which poses a mempool DoS risk,
though not an unprecedented one as this also applies to transactions with large amounts of calldata.
The risk is that an attacker makes and publishes a series of large blob transactions with fees `f9 > f8 > ... > f1`,
where each fee is the 10% minimum increment higher than the previous, and finishes it off with a 21000-gas basic transaction with fee `f10`.
Hence, an attacker could impose millions of gas worth of load on the network and only pay 21000 gas worth of fees.

We recommend a simple solution: both for blob transactions and for transactions carrying a large amount of calldata,
increase the minimum increment for mempool replacement from 1.1x to 2x, decreasing the number of resubmissions an attacker can do at any given fee level by ~7x.
We recommend two solutions:

- include a 1.1x (or potentially higher) data gasprice bump requirement to the mempool replacement rules
- modify the Ethereum Wire Protocol to stop automatically broadcasting large transactions

## Test Cases

Expand Down