refactor(fw): Refactor ethereum_test_tools into separate libraries

docs/CHANGELOG.md

@@ -50,6 +50,7 @@ Test fixtures for use by clients are available for each release on the [Github r
 - ✨ Enable loading of [ethereum/tests/BlockchainTests](https://github.com/ethereum/tests/tree/develop/BlockchainTests) ([#596](https://github.com/ethereum/execution-spec-tests/pull/596)).
 - 🔀 Refactor `gentest` to use `ethereum_test_tools.rpc.rpc` by adding to `get_transaction_by_hash`, `debug_trace_call` to `EthRPC` ([#568](https://github.com/ethereum/execution-spec-tests/pull/568)).
 - ✨ Write a properties file to the output directory and enable direct generation of a fixture tarball from `fill` via `--output=fixtures.tgz`([#627](https://github.com/ethereum/execution-spec-tests/pull/627)).
+- 🔀 `ethereum_test_tools` library has been split into multiple libraries ([#645](https://github.com/ethereum/execution-spec-tests/pull/645)).
 
 ### 🔧 EVM Tools
 

docs/consuming_tests/exceptions.md

@@ -15,8 +15,12 @@ the exceptions.
 
 ## `TransactionException`
 
-::: ethereum_test_tools.TransactionException
+::: ethereum_test_exceptions.TransactionException
 
 ## `BlockException`
 
-::: ethereum_test_tools.BlockException
+::: ethereum_test_exceptions.BlockException
+
+## `EOFException`
+
+::: ethereum_test_exceptions.EOFException

docs/library/ethereum_test_base_types.md

@@ -0,0 +1,3 @@
+# Ethereum Test Base Types package
+
+::: ethereum_test_base_types

docs/library/ethereum_test_exceptions.md

@@ -0,0 +1,3 @@
+# Ethereum Test Exceptions package
+
+::: ethereum_test_exceptions

docs/library/ethereum_test_fixtures.md

@@ -0,0 +1,3 @@
+# Ethereum Test Fixtures package
+
+::: ethereum_test_fixtures

docs/library/ethereum_test_specs.md

@@ -0,0 +1,3 @@
+# Ethereum Test Specs package
+
+::: ethereum_test_specs

docs/library/ethereum_test_types.md

@@ -0,0 +1,3 @@
+# Ethereum Test Types package
+
+::: ethereum_test_types

docs/library/ethereum_test_vm.md

@@ -0,0 +1,3 @@
+# Ethereum Test VM package
+
+::: ethereum_test_vm

docs/library/index.md

@@ -2,7 +2,13 @@
 
 Execution spec tests consists of several packages that implement helper classes and tools that enable and simplify test case implementation. This section contains their reference documentation:
 
-- [`evm_transition_tool`](./evm_transition_tool.md) - a wrapper for the transition (`t8n`) tool.
-- [`ethereum_test_tools`](./ethereum_test_tools.md) - provides primitives and helpers to test Ethereum execution clients.
+- [`ethereum_test_base_types`](./ethereum_test_base_types.md) - provides the basic types on top of which other testing libraries are built.
+- [`ethereum_test_exceptions`](./ethereum_test_exceptions.md) - provides definitions for exceptions used in all tests.
+- [`ethereum_test_fixtures`](./ethereum_test_fixtures.md) - provides definitions of all test fixture types that are produced in this repository and can be consumed by clients.
 - [`ethereum_test_forks`](./ethereum_test_forks.md) - provides definitions for supported forks used in tests.
+- [`ethereum_test_specs`](./ethereum_test_specs.md) - provides definitions for all spec types used to define test cases, and generate different kinds of test fixtures.
+- [`ethereum_test_tools`](./ethereum_test_tools.md) - provides primitives and helpers to test Ethereum execution clients.
+- [`ethereum_test_types`](./ethereum_test_types.md) - provides Ethereum types built on top of the base types which are used to define test cases and interact with other libraries.
+- [`ethereum_test_vm`](./ethereum_test_vm.md) - provides definitions for the Ethereum Virtual Machine (EVM) as used to define bytecode in test cases.
+- [`evm_transition_tool`](./evm_transition_tool.md) - a wrapper for the transition (`t8n`) tool.
 - [`pytest_plugins`](./pytest_plugins/index.md) - contains pytest customizations that provide additional functionality for generating test fixtures.

docs/navigation.md

@@ -34,7 +34,13 @@
     * [Enabling Precommit Checks](dev/precommit.md)
   * [Changelog](CHANGELOG.md)
   * [Library Reference](library/index.md)
-    * [EVM Transition Tool Package](library/evm_transition_tool.md)
-    * [Ethereum Test Tools Package](library/ethereum_test_tools.md)
+    * [Ethereum Test Base Types Package](library/ethereum_test_base_types.md)
+    * [Ethereum Test Exceptions Package](library/ethereum_test_exceptions.md)
+    * [Ethereum Test Fixtures Package](library/ethereum_test_fixtures.md)
     * [Ethereum Test Forks Package](library/ethereum_test_forks.md)
+    * [Ethereum Test Specs Package](library/ethereum_test_specs.md)
+    * [Ethereum Test Tools Package](library/ethereum_test_tools.md)
+    * [Ethereum Test Types Package](library/ethereum_test_types.md)
+    * [Ethereum Test VM Package](library/ethereum_test_vm.md)
+    * [EVM Transition Tool Package](library/evm_transition_tool.md)
     * [Pytest Plugins](library/pytest_plugins/index.md)

docs/writing_tests/exception_tests.md

@@ -4,7 +4,7 @@ Exception tests are a special type of test which verify that an invalid transact
 
 ## Creating an Exception Test
 
-To test for an exception, the test can use either of the following types from `ethereum_test_tools` library:
+To test for an exception, the test can use either of the following types from `ethereum_test_exceptions` library:
 
 1. [`TransactionException`](../consuming_tests/exceptions.md#transactionexception): To be added to the `error` field of the `Transaction` object, and to the `exception` field of the `Block` object that includes the transaction; this exception type is used when a transaction is invalid, and therefore when included in a block, the block is expected to be invalid too. This is different from valid transactions where an exception during EVM execution is expected (e.g. a revert, or out-of-gas), which can be included in valid blocks.
 

docs/writing_tests/writing_a_new_test.md

@@ -181,7 +181,7 @@ Within the `post` dictionary object, an account address can be:
 The `Account` object is used to specify the properties of an account to be
 verified in the post state.
 
-The python representation can be found in [src/ethereum_test_tools/common/types.py](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_tools/common/types.py).
+The python representation can be found in [src/ethereum_test_types/types.py](https://github.com/ethereum/execution-spec-tests/blob/main/src/ethereum_test_types/types.py).
 
 It can verify the following properties of an account:
 

setup.cfg

@@ -12,9 +12,15 @@ classifiers =
 
 [options]
 packages =
-    evm_transition_tool
+    ethereum_test_base_types
+    ethereum_test_exceptions
+    ethereum_test_fixtures
     ethereum_test_forks
+    ethereum_test_specs
     ethereum_test_tools
+    ethereum_test_types
+    ethereum_test_vm
+    evm_transition_tool
     pytest_plugins
 
 package_dir =
@@ -47,11 +53,23 @@ install_requires =
     solc-select>=1.0.4
 
 [options.package_data]
-ethereum_test_tools =
+ethereum_test_base_types =
+    py.typed
+ethereum_test_exceptions =
+    py.typed
+ethereum_test_fixtures =
     py.typed
 ethereum_test_forks =
     py.typed
     forks/contracts/*.bin
+ethereum_test_specs =
+    py.typed
+ethereum_test_tools =
+    py.typed
+ethereum_test_types =
+    py.typed
+ethereum_test_vm =
+    py.typed
 evm_transition_tool =
     py.typed
 pytest_plugins =

src/cli/check_fixtures.py

@@ -8,9 +8,9 @@
 import click
 from rich.progress import BarColumn, Progress, TaskProgressColumn, TextColumn, TimeElapsedColumn
 
-from ethereum_test_tools.common.json import to_json
-from ethereum_test_tools.spec.base.base_test import HashMismatchException
-from ethereum_test_tools.spec.file.types import Fixtures
+from ethereum_test_base_types import to_json
+from ethereum_test_fixtures.file import Fixtures
+from ethereum_test_specs.base import HashMismatchException
 
 
 def count_json_files_exclude_index(start_path: Path) -> int:

src/cli/evm_bytes_to_python.py

@@ -6,8 +6,8 @@
 
 import click
 
-from ethereum_test_tools import Macro
-from ethereum_test_tools import Opcodes as Op
+from ethereum_test_vm import Macro
+from ethereum_test_vm import Opcodes as Op
 
 
 def process_evm_bytes(evm_bytes_hex_string: Any) -> str:  # noqa: D103

src/cli/gen_index.py

@@ -18,10 +18,10 @@
     TimeElapsedColumn,
 )
 
-from ethereum_test_tools.common.base_types import HexNumber
-from ethereum_test_tools.spec.consume.types import IndexFile, TestCaseIndexFile
-from ethereum_test_tools.spec.file.types import Fixtures
-from evm_transition_tool import FixtureFormats
+from ethereum_test_base_types import HexNumber
+from ethereum_test_fixtures import FixtureFormats
+from ethereum_test_fixtures.consume import IndexFile, TestCaseIndexFile
+from ethereum_test_fixtures.file import Fixtures
 
 from .hasher import HashableItem
 

src/cli/gentest.py

@@ -59,8 +59,9 @@
 
 import click
 
-from ethereum_test_tools import Account, Address, Transaction, common
+from ethereum_test_base_types import Account, Address, ZeroPaddedHexNumber
 from ethereum_test_tools.rpc.rpc import BlockNumberType, EthRPC
+from ethereum_test_types import Transaction
 
 
 @click.command()
@@ -165,8 +166,8 @@ def _make_pre_state(
 
                 if account_obj.storage is not None:
                     for record, value in account_obj.storage.root.items():
-                        pad_record = common.ZeroPaddedHexNumber(record)
-                        pad_value = common.ZeroPaddedHexNumber(value)
+                        pad_record = ZeroPaddedHexNumber(record)
+                        pad_value = ZeroPaddedHexNumber(value)
                         state_str += f'{pad}    "{pad_record}" : "{pad_value}",\n'
 
                 state_str += pad + "}\n"

src/ethereum_test_base_types/__init__.py

@@ -0,0 +1,67 @@
+"""
+Common definitions and types.
+"""
+
+from .base_types import (
+    Address,
+    Bloom,
+    BLSPublicKey,
+    BLSSignature,
+    Bytes,
+    FixedSizeBytes,
+    Hash,
+    HashInt,
+    HeaderNonce,
+    HexNumber,
+    Number,
+    NumberBoundTypeVar,
+    ZeroPaddedHexNumber,
+)
+from .composite_types import Account, Alloc, Storage, StorageRootType
+from .constants import (
+    AddrAA,
+    AddrBB,
+    EmptyOmmersRoot,
+    EmptyTrieRoot,
+    TestAddress,
+    TestAddress2,
+    TestPrivateKey,
+    TestPrivateKey2,
+)
+from .conversions import to_bytes, to_hex
+from .json import to_json
+from .pydantic import CamelModel
+from .reference_spec import ReferenceSpec
+
+__all__ = (
+    "Account",
+    "AddrAA",
+    "AddrBB",
+    "Address",
+    "Alloc",
+    "Bloom",
+    "BLSPublicKey",
+    "BLSSignature",
+    "Bytes",
+    "CamelModel",
+    "EmptyOmmersRoot",
+    "EmptyTrieRoot",
+    "FixedSizeBytes",
+    "Hash",
+    "HashInt",
+    "HeaderNonce",
+    "HexNumber",
+    "Number",
+    "NumberBoundTypeVar",
+    "ReferenceSpec",
+    "Storage",
+    "StorageRootType",
+    "TestAddress",
+    "TestAddress2",
+    "TestPrivateKey",
+    "TestPrivateKey2",
+    "ZeroPaddedHexNumber",
+    "to_bytes",
+    "to_hex",
+    "to_json",
+)