feat(fw,pytest): Implement state test fixture format

Co-authored-by: danceratopz <danceratopz@gmail.com>

docs/CHANGELOG.md

@@ -14,6 +14,7 @@ Test fixtures for use by clients are available for each release on the [Github r
 - 🔀 Rename test fixtures names to match the corresponding pytest node ID as generated using `fill` ([#342](https://github.com/ethereum/execution-spec-tests/pull/342)).
 - 💥 Replace "=" with "_" in pytest node ids and test fixture names ([#342](https://github.com/ethereum/execution-spec-tests/pull/342)).
 - 💥 Removed `--enable-hive` parameter, now all test types are generated by default ([#358](https://github.com/ethereum/execution-spec-tests/pull/358))
+- 💥 `StateTest`, spec format used to write tests, is now limited to a single transaction per test ([#361](https://github.com/ethereum/execution-spec-tests/pull/361))
 
 ### 🔧 EVM Tools
 
@@ -27,7 +28,8 @@ Test fixtures for use by clients are available for each release on the [Github r
    - `blockchain_tests`: Contains BlockchainTest formatted tests
    - `blockchain_tests_hive`: Contains BlockchainTest with Engine API call directives for use in hive
    - `state_tests`: Contains StateTest formatted tests
-2. In this release the pytest node ID is now used for fixture names (previously only the test parameters were used), this should not be breaking. However, "=" in both node IDs and therefore fixture names, have been replaced with "_", which may break tooling that depends on the "=" character.
+2. `StateTest`, spec format used to write tests, is now limited to a single transaction per test.
+3. In this release the pytest node ID is now used for fixture names (previously only the test parameters were used), this should not be breaking. However, "=" in both node IDs and therefore fixture names, have been replaced with "_", which may break tooling that depends on the "=" character.
 
    Pytest node ID example:
 

docs/tutorials/state_transition.md

@@ -1,6 +1,6 @@
 # State Transition Tests
 
-This tutorial teaches you to create a state transition execution specification test. These tests verify that a blockchain, starting from a defined pre-state, will reach a specified post-state after executing a set of specific transactions.
+This tutorial teaches you to create a state transition execution specification test. These tests verify that a starting pre-state will reach a specified post-state after executing a single transaction.
 
 ## Pre-requisites
 
@@ -13,7 +13,7 @@ Before proceeding with this tutorial, it is assumed that you have prior knowledg
 
 ## Example Tests
 
-The most effective method of learning how to write tests is to study a couple of straightforward examples. In this tutorial we will go over the [Yul](https://github.com/ethereum/execution-spec-tests/blob/main/tests/example/test_yul_example.py#L17) state test.
+The most effective method of learning how to write tests is to study a couple of straightforward examples. In this tutorial we will go over the [Yul](https://github.com/ethereum/execution-spec-tests/blob/main/tests/homestead/yul/test_yul_example.py#L19) state test.
 
 ### Yul Test
 
@@ -83,7 +83,7 @@ The function definition ends when there is a line that is no longer indented. As
     env = Environment()
 ```
 
-This line specifies that `env` is an [`Environment`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/types.py#L445) object, and that we just use the default parameters.
+This line specifies that `env` is an [`Environment`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/types.py#L878) object, and that we just use the default parameters.
 If necessary we can modify the environment to have different block gas limits, block numbers, etc.
 In most tests the defaults are good enough.
 
@@ -102,7 +102,7 @@ It is a [dictionary](https://docs.python.org/3/tutorial/datastructures.html#dict
         "0x1000000000000000000000000000000000000000": Account(
 ```
 
-The keys of the dictionary are addresses (as strings), and the values are [`Account`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/types.py#L264) objects.
+The keys of the dictionary are addresses (as strings), and the values are [`Account`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/types.py#L517) objects.
 You can read more about address fields [in the static test documentation](https://ethereum-tests.readthedocs.io/en/latest/test_filler/state_filler.html#address-fields).
 
 ```python
@@ -145,7 +145,7 @@ Generally for execution spec tests the `sstore` instruction acts as a high-level
     }
 ```
 
-[`TestAddress`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/constants.py#L8) is an address for which the test filler has the private key.
+[`TestAddress`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/constants.py#L7) is an address for which the test filler has the private key.
 This means that the test runner can issue a transaction as that contract.
 Of course, this address also needs a balance to be able to issue transactions.
 
@@ -163,7 +163,7 @@ Of course, this address also needs a balance to be able to issue transactions.
     )
 ```
 
-With the pre-state specified, we can add a description for the [`Transaction`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/types.py#L516).
+With the pre-state specified, we can add a description for the [`Transaction`](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/types.py#L1155).
 For more information, [see the static test documentation](https://ethereum-tests.readthedocs.io/en/latest/test_filler/state_filler.html#transaction)
 
 #### Post State
@@ -185,7 +185,7 @@ In this case, we look at the storage of the contract we called and add to it wha
 #### State Test
 
 ```python
-    state_test(env=env, pre=pre, post=post, txs=[tx])
+    state_test(env=env, pre=pre, post=post, tx=tx)
 ```
 
 This line calls the wrapper to the `StateTest` object that provides all the objects required (for example, the fork parameter) in order to fill the test, generate the test fixtures and write them to file (by default, `./fixtures/<blockchain,state>_tests/example/yul_example/test_yul.json`).

docs/tutorials/state_transition_bad_opcode.md

@@ -267,7 +267,7 @@ Over the entire for loop, it yields 255 different tests.
     yield StateTest(
         env=env, 
         pre=pre, 
-        txs=[tx],
+        tx=tx,
         post=(post_valid if opc_valid(opc) else post_invalid),
     )
 ```

src/ethereum_test_tools/common/types.py

@@ -1066,6 +1066,8 @@ def set_fork_requirements(self, fork: Fork, in_place: bool = False) -> "Environm
 
         if fork.header_zero_difficulty_required(number, timestamp):
             res.difficulty = 0
+        elif res.difficulty is None and res.parent_difficulty is None:
+            res.difficulty = 0x20000
 
         if (
             fork.header_excess_blob_gas_required(number, timestamp)

src/ethereum_test_tools/spec/base/base_test.py

@@ -6,7 +6,7 @@
 from itertools import count
 from os import path
 from pathlib import Path
-from typing import Any, Callable, Dict, Generator, Iterator, List, Mapping, Optional
+from typing import Any, Callable, Dict, Generator, Iterator, List, Mapping, Optional, TextIO
 
 from ethereum_test_forks import Fork
 from evm_transition_tool import FixtureFormats, TransitionTool
@@ -110,7 +110,7 @@ def format(cls) -> FixtureFormats:
 
     @classmethod
     @abstractmethod
-    def collect_into_file(cls, fixture_file_path: Path, fixtures: Dict[str, "BaseFixture"]):
+    def collect_into_file(cls, fd: TextIO, fixtures: Dict[str, "BaseFixture"]):
         """
         Returns the name of the subdirectory where this type of fixture should be dumped to.
         """

src/ethereum_test_tools/spec/blockchain/types.py

@@ -5,7 +5,7 @@
 from copy import copy, deepcopy
 from dataclasses import dataclass, fields, replace
 from pathlib import Path
-from typing import Any, Callable, ClassVar, Dict, List, Mapping, Optional, Tuple
+from typing import Any, Callable, ClassVar, Dict, List, Mapping, Optional, TextIO, Tuple
 
 from ethereum import rlp as eth_rlp
 from ethereum.base_types import Uint
@@ -865,13 +865,6 @@ class FixtureCommon(BaseFixture):
     Base Ethereum test fixture fields class.
     """
 
-    info: Dict[str, str] = field(
-        default_factory=dict,
-        json_encoder=JSONEncoder.Field(
-            name="_info",
-            to_json=True,
-        ),
-    )
     name: str = field(
         default="",
         json_encoder=JSONEncoder.Field(
@@ -905,16 +898,15 @@ def to_json(self) -> Dict[str, Any]:
         return self._json
 
     @classmethod
-    def collect_into_file(cls, fixture_file_path: Path, fixtures: Dict[str, "BaseFixture"]):
+    def collect_into_file(cls, fd: TextIO, fixtures: Dict[str, "BaseFixture"]):
         """
         For BlockchainTest format, we simply join the json fixtures into a single file.
         """
         json_fixtures: Dict[str, Dict[str, Any]] = {}
         for name, fixture in fixtures.items():
             assert isinstance(fixture, FixtureCommon), f"Invalid fixture type: {type(fixture)}"
             json_fixtures[name] = fixture.to_json()
-        with open(fixture_file_path, "w") as f:
-            json.dump(json_fixtures, f, indent=4)
+        json.dump(json_fixtures, fd, indent=4)
 
 
 @dataclass(kw_only=True)

src/ethereum_test_tools/spec/fixture_collector.py

@@ -164,7 +164,8 @@ def dump_fixtures(self) -> None:
             # Get the first fixture to dump to get its type
             fixture = next(iter(fixtures.values()))
             # Call class method to dump all the fixtures
-            fixture.collect_into_file(fixture_path, fixtures)
+            with open(fixture_path, "w") as fd:
+                fixture.collect_into_file(fd, fixtures)
 
     def verify_fixture_files(self, evm_fixture_verification: TransitionTool) -> None:
         """

src/ethereum_test_tools/spec/state/state_test.py

@@ -5,15 +5,18 @@
 from dataclasses import dataclass
 from typing import Callable, Generator, List, Mapping, Optional, Type
 
-import pytest
-
-from ethereum_test_forks import Fork
+from ethereum_test_forks import Cancun, Fork, is_fork
 from evm_transition_tool import FixtureFormats, TransitionTool
 
-from ...common import Environment, Number, Transaction
+from ...common import Account, Address, Alloc, Environment, Number, Transaction
 from ...common.constants import EngineAPIError
-from ..base.base_test import BaseFixture, BaseTest
+from ...common.json import to_json
+from ..base.base_test import BaseFixture, BaseTest, verify_post_alloc
 from ..blockchain.blockchain_test import Block, BlockchainTest
+from ..debugging import print_traces
+from .types import Fixture, FixtureForkPost
+
+BEACON_ROOTS_ADDRESS = Address(0x000F3DF6D732807EF1319FB7B8BB8522D0BEAC02)
 
 
 @dataclass(kw_only=True)
@@ -25,7 +28,7 @@ class StateTest(BaseTest):
     env: Environment
     pre: Mapping
     post: Mapping
-    txs: List[Transaction]
+    tx: Transaction
     engine_api_error_code: Optional[EngineAPIError] = None
     tag: str = ""
     chain_id: int = 1
@@ -78,7 +81,7 @@ def _generate_blockchain_blocks(self) -> List[Block]:
                 extra_data=self.env.extra_data,
                 withdrawals=self.env.withdrawals,
                 beacon_root=self.env.beacon_root,
-                txs=self.txs,
+                txs=[self.tx],
                 ommers=[],
             )
         ]
@@ -93,6 +96,69 @@ def generate_blockchain_test(self) -> BlockchainTest:
             post=self.post,
             blocks=self._generate_blockchain_blocks(),
             fixture_format=self.fixture_format,
+            t8n_dump_dir=self.t8n_dump_dir,
+        )
+
+    def make_state_test_fixture(
+        self,
+        t8n: TransitionTool,
+        fork: Fork,
+        eips: Optional[List[int]] = None,
+    ) -> Fixture:
+        """
+        Create a fixture from the state test definition.
+        """
+        env = self.env.set_fork_requirements(fork)
+        tx = self.tx.with_signature_and_sender()
+        pre_alloc = Alloc.merge(
+            Alloc(
+                fork.pre_allocation(block_number=env.number, timestamp=Number(env.timestamp)),
+            ),
+            Alloc(self.pre),
+        )
+
+        next_alloc, result = t8n.evaluate(
+            alloc=to_json(pre_alloc),
+            txs=to_json([tx]),
+            env=to_json(env),
+            fork_name=fork.fork(block_number=Number(env.number), timestamp=Number(env.timestamp)),
+            chain_id=self.chain_id,
+            reward=0,  # Reward on state tests is always zero
+            eips=eips,
+            debug_output_path=self.get_next_transition_tool_output_path(),
+        )
+
+        try:
+            verify_post_alloc(self.post, next_alloc)
+        except Exception as e:
+            print_traces(t8n.get_traces())
+            raise e
+
+        # Perform post state processing required for some forks
+        if is_fork(fork, Cancun):
+            # StateTest does not execute any beacon root contract logic, but we still need to
+            # set the beacon root to the correct value, because most tests assume this happens,
+            # so we copy the beacon root contract storage from the post state into the pre state
+            # and the transaction is executed in isolation properly.
+            if beacon_roots_account := next_alloc.get(str(BEACON_ROOTS_ADDRESS)):
+                if beacon_roots_storage := beacon_roots_account.get("storage"):
+                    pre_alloc = Alloc.merge(
+                        pre_alloc,
+                        Alloc({BEACON_ROOTS_ADDRESS: Account(storage=beacon_roots_storage)}),
+                    )
+
+        return Fixture(
+            env=env,
+            pre_state=pre_alloc,
+            post={
+                fork.fork(block_number=Number(env.number), timestamp=Number(env.timestamp)): [
+                    FixtureForkPost.collect(
+                        transition_tool_result=result,
+                        transaction=tx.with_signature_and_sender(),
+                    )
+                ]
+            },
+            transaction=tx,
         )
 
     def generate(
@@ -107,8 +173,7 @@ def generate(
         if self.fixture_format in BlockchainTest.fixture_formats():
             return self.generate_blockchain_test().generate(t8n, fork, eips)
         elif self.fixture_format == FixtureFormats.STATE_TEST:
-            # TODO: append fixture in state format
-            pytest.skip("StateTest fixture format not implemented.")
+            return self.make_state_test_fixture(t8n, fork, eips)
 
         raise Exception(f"Unknown fixture format: {self.fixture_format}")