This repository was archived by the owner on Jul 2, 2026. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 200
chore(docs): update README for archive #2326
Merged
Merged
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,220 +1,16 @@ | ||
| # ⚠️ NOTICE: All code from this repository has been migrated to ethereum/execution-specs | ||
| # Execution Spec Tests | ||
|
|
||
| As such, this repository is no longer accepting pull requests as of 2025-11-01. Please direct any changes to https://github.com/ethereum/execution-specs instead. | ||
| > [!IMPORTANT] | ||
| > This repository has been archived. All code from this repository has been migrated to [ethereum/execution-specs](https://github.com/ethereum/execution-specs). | ||
|
|
||
| ## What This Means for You | ||
|
|
||
| - **Client Teams:** No changes. Test fixture releases will continue to be published in this repository, [ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests). | ||
| - **Client Teams:** Test fixture releases are now published at [ethereum/execution-specs releases](https://github.com/ethereum/execution-specs/releases); they are no longer published in this repository. | ||
| - **Test Contributors:** | ||
| - Please direct PRs to [ethereum/execution-specs](https://github.com/ethereum/execution-specs). | ||
| - Read more about the changes effecting test development in our [blog announcement](https://steel.ethereum.foundation/blog/2025-11-04_weld_final/). | ||
| - **Documentation:** [eest.ethereum.org](https://eest.ethereum.org) is archived and now redirects to its new home at [steel.ethereum.foundation/docs/execution-specs](https://steel.ethereum.foundation/docs/execution-specs/). | ||
| - Read more about the changes affecting test development in our [blog announcement](https://steel.ethereum.foundation/blog/2025-11-04_weld_final/). | ||
| - **Documentation:** [eest.ethereum.org](https://eest.ethereum.org) redirects to its new home at [steel.ethereum.foundation/docs/execution-specs](https://steel.ethereum.foundation/docs/execution-specs/). | ||
|
|
||
| ### Questions or Feedback About the Migration? | ||
| ### Questions or Feedback | ||
|
|
||
| Please reach out to the STEEL team by writing to the `#el-testing` channel in the [Ethereum R&D Discord Server](https://discord.com/invite/qGpsxSA) or contacting any STEEL team member directly. | ||
|
|
||
| --- | ||
|
|
||
| ## Execution Spec Tests | ||
|
|
||
| [](https://github.com/ethereum/execution-spec-tests/releases/latest) | ||
|  | ||
| [](https://github.com/astral-sh/ruff) | ||
| [](https://github.com/astral-sh/uv) | ||
| [](https://github.com/ethereum/execution-spec-tests/blob/main/LICENSE) | ||
|
|
||
| ## About | ||
|
|
||
| The full execution-spec-tests documentation can be found [here](https://eest.ethereum.org/main/). | ||
|
|
||
| **Note:** All of [ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests) functionality (i.e., the spec tests, test frameworks and tooling) will move to [ethereum/execution-specs](https://github.com/ethereum/execution-specs) in Q4 2025 to streamline spec and test development. For more information please see our [blog post](https://steel.ethereum.foundation/blog/2025-09-11_weld-announcement/). For now, please continue contributing to both [ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests) and [ethereum/execution-specs](https://github.com/ethereum/execution-specs) as you previously have - we will reach out to you if and when need be! | ||
|
|
||
| [ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests) is both a collection of test cases and a framework implemented in Python to generate tests for Ethereum execution clients. | ||
|
|
||
| The framework collects and executes the test cases in order to generate _test fixtures_ (JSON) which can be consumed by any execution client to verify their implementation of [ethereum/execution-specs](https://github.com/ethereum/execution-specs). The fixtures, which define state transition and block tests, are generated by the framework using one of the `t8n` command-line tools that are provided by most execution clients, see below for an overview of the supported `t8n` tools. | ||
|
|
||
| ```mermaid | ||
| --- | ||
| title: Test Fixture Generation with execution-spec-tests | ||
| --- | ||
| flowchart LR | ||
| style C stroke:#333,stroke-width:2px | ||
| style D stroke:#333,stroke-width:2px | ||
| style G stroke:#F9A825,stroke-width:2px | ||
| style H stroke:#F9A825,stroke-width:2px | ||
|
|
||
| subgraph "ethereum/go-ethereum, ..." | ||
| C[<code>evm t8n</code><br/>external executable] | ||
| end | ||
|
|
||
| subgraph ethereum/solidity | ||
| D[<code>solc</code><br/>external executable] | ||
| end | ||
|
|
||
| subgraph ethereum/EIPs | ||
| E(<code>EIPS/EIP-*.md</code><br/>SHA digest via Github API) | ||
| end | ||
|
|
||
| subgraph "ethereum/execution-spec-tests" | ||
| A(<code>./tests/**/*.py</code><br/>Python Test Cases) | ||
| B([<code>$ fill ./tests/</code><br/>Python Framework]) | ||
| end | ||
|
|
||
| subgraph Test Fixture Consumers | ||
| subgraph ethereum/hive | ||
| G([<code>$ hive ...</code><br/>Go Test Framework]) | ||
| end | ||
| H([Client executables]) | ||
| end | ||
|
|
||
| C <-.-> B | ||
| D <-.-> B | ||
| A --> B | ||
| E <-.-> |retrieve latest spec version\ncheck tested spec version| B | ||
| B -->|output| F(<code>./fixtures/**/*.json</code>\nJSON Test Fixtures) | ||
| F -->|input| G | ||
| F -->|input| H | ||
| ``` | ||
|
|
||
| The generated test fixtures can be used: | ||
|
|
||
| 1. Directly by client teams' test frameworks, and, | ||
| 2. In the integration tests executed in the [ethereum/hive](https://github.com/ethereum/hive) framework. | ||
|
|
||
| ## Transition Tool Support | ||
|
|
||
| The following transition tools are supported by the framework: | ||
|
|
||
| | Client | "t8n" Tool | Tracing Support | | ||
| | -------| ---------- | --------------- | | ||
| | [ethereum/evmone](https://github.com/ethereum/evmone) | `evmone-t8n` | Yes | | ||
| | [ethereum/execution-specs](https://github.com/ethereum/execution-specs) | `ethereum-spec-evm` | Yes | | ||
| | [ethereum/go-ethereum](https://github.com/ethereum/go-ethereum) | [`evm t8n`](https://github.com/ethereum/go-ethereum/tree/master/cmd/evm) | Yes | | ||
| | [ethereumjs](https://github.com/ethereumjs/ethereumjs-monorepo) | [`ethereumjs-t8ntool.sh`](https://github.com/ethereumjs/ethereumjs-monorepo/tree/master/packages/vm/test/t8n) | No | | ||
| | [hyperledger/besu](https://github.com/hyperledger/besu/tree/main/ethereum/evmtool) | [`evm t8n-server`](https://github.com/hyperledger/besu/tree/main/ethereum/evmtool) | Yes | | ||
| | [status-im/nimbus-eth1](https://github.com/status-im/nimbus-eth1) | [`t8n`](https://github.com/status-im/nimbus-eth1/blob/master/tools/t8n/readme.md) | Yes | | ||
|
|
||
| ### Upcoming EIP Development | ||
|
|
||
| Generally, specific `t8n` implementations and branches must be used when developing tests for upcoming EIPs. | ||
|
|
||
| We use named reference tags to point to the specific version of the `t8n` implementation that needs to be used fill the tests. | ||
|
|
||
| All current tags, their t8n implementation and branch they point to, are listed in [.github/configs/evm.yaml](.github/configs/evm.yaml). | ||
|
|
||
| ## Getting Started | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| The tools provided by ethereum/execution-spec-tests use `uv` ([docs.astral.sh/uv](https://docs.astral.sh/uv/)) to manage their dependencies and virtual environment. `uv` downloads Python for your target platform if one of the required versions (Python 3.11 or 3.12) is not available natively. | ||
|
|
||
| `uv` can be installed via curl (recommended; can self-update): | ||
|
|
||
| ```console | ||
| curl -LsSf https://astral.sh/uv/install.sh | sh | ||
| ``` | ||
|
|
||
| or pip (requires Python, can't self-update): | ||
|
|
||
| ```console | ||
| pip install uv | ||
| ``` | ||
|
|
||
| ### Installation | ||
|
|
||
| Clone [execution-spec-tests](https://github.com/ethereum/execution-spec-tests) and install its dependencies: | ||
|
|
||
| ```console | ||
| git clone --depth 1 https://github.com/ethereum/execution-spec-tests | ||
| cd execution-spec-tests | ||
| uv python install 3.12 | ||
| uv python pin 3.12 | ||
| uv sync --all-extras | ||
| ``` | ||
|
|
||
| See [Installation Troubleshooting](https://eest.ethereum.org/main/getting_started/installation_troubleshooting/) in the online docs if you encounter issues. | ||
|
|
||
| ### Exploring and Filling Test Cases | ||
|
|
||
| By default, JSON test fixtures are generated from this repository's Python test cases using the [Ethereum Execution Layer Specification](https://github.com/ethereum/execution-specs) (EELS) reference implementation. The resulting JSON fixtures can be executed against execution clients to verify consensus. The process of generating fixtures is often referred to as "filling". | ||
|
|
||
| 1. Explore test cases via `--collect-only` and search for test cases that combine `PUSH0` and `DELEGATECALL` in the EVM functionality introduced in the Shanghai hard fork: | ||
|
|
||
| ```console | ||
| uv run fill --collect-only -k "push0 and delegatecall" tests/shanghai/ | ||
| ``` | ||
|
|
||
| The `fill` command is based on [`pytest`](https://docs.pytest.org/en/stable/). The above command uses the [optional pytest arguments](https://docs.pytest.org/en/stable/how-to/usage.html): | ||
|
|
||
| - `--collect-only` only collect test cases; don't execute them. | ||
| - `-k <expression>` filter test cases by their test case ID based on the given expression. | ||
| - `tests/shanghai` the directory containing the test cases (tells `fill` to only discover test cases in this directory; default: `tests/`). | ||
|
|
||
| Expected console output: | ||
|  | ||
|
|
||
| 2. Fill `state_test` fixtures for these test cases: | ||
|
|
||
| ```console | ||
| uv run fill -k "push0 and delegatecall" tests/shanghai/ -m state_test -v | ||
| ``` | ||
|
|
||
| where: | ||
|
|
||
| - `-m state_test` only fills test cases marked as a `state_test` (see all available markers via `uv run fill --markers`). | ||
| - `-v` enables verbose output. | ||
|
|
||
| Expected console output: | ||
|  | ||
|
|
||
| 3. Verify the generated fixtures: | ||
|
|
||
| a. Check the corresponding fixture file has been generated: | ||
|
|
||
| ```console | ||
| head fixtures/state_tests/shanghai/eip3855_push0/push0/push0_contract_during_call_contexts.json | ||
| ``` | ||
|
|
||
| b. Open the generated HTML test using the link provided at the bottom of the console output. This is written to the output directory at: | ||
|
|
||
| ```console | ||
| ./fixtures/.meta/report_fill.html | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| More information on how to obtain and consume the [released test fixtures](https://github.com/ethereum/execution-spec-tests/releases) can be found in the [documentation](https://eest.ethereum.org/main/running_tests/). | ||
|
|
||
| For further help with working with this codebase, see the [online documentation](https://eest.ethereum.org/main/): | ||
|
|
||
| 1. Learn [useful command-line flags](https://eest.ethereum.org/main/filling_tests/filling_tests_command_line/). | ||
| 2. [Execute tests for features under development](https://eest.ethereum.org/main/filling_tests/filling_tests_dev_fork/) via the `--from=FORK1` and `--until=FORK2` flags. | ||
| 3. _Optional:_ [Configure VS Code](https://eest.ethereum.org/main/getting_started/setup_vs_code/) to auto-format Python code and [execute tests within VS Code](https://eest.ethereum.org/main/filling_tests/filling_tests_vs_code/). | ||
| 4. Implement a new test case, see [Writing Tests](https://eest.ethereum.org/main/writing_tests/). | ||
|
|
||
| ## Coverage | ||
|
|
||
| The available test cases can be browsed in the [Test Case Reference doc](https://eest.ethereum.org/main/tests/). | ||
|
|
||
| ## Installation Troubleshooting | ||
|
|
||
| If you encounter issues during the installation process, please refer to the [Installation Troubleshooting](https://eest.ethereum.org/main/getting_started/installation_troubleshooting/) page. | ||
|
|
||
| ## Contributing | ||
|
|
||
| Contributions and feedback are welcome. Please see our [Contributing Guidelines](CONTRIBUTING.md) for detailed information on how to contribute, and the [online documentation](https://eest.ethereum.org/main/writing_tests/) for this repository's coding standards and help on implementing new tests. | ||
|
|
||
| We welcome earnest newcomers, no matter how small the contribution! However, we do not accept: | ||
|
|
||
| - Contributions that only fix spelling or grammatical errors in documentation, code or elsewhere | ||
| - Pull requests from obvious airdrop farmers | ||
| - Drive-by or vibe code contributions without proper engagement or context | ||
|
|
||
| Pull requests should have reasonable substance or resolve an existing repository open issue. | ||
|
|
||
| Care is required when adding PRs or issues for functionality that is live on Ethereum mainnet, please refer to the [Security Policy](SECURITY.md) for more information about reporting vulnerabilities and eligibility for the [bug bounty program](https://bounty.ethereum.org). | ||
|
|
||
| ## License | ||
|
|
||
| This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. | ||
| Please reach out to the [ethereum/execution-specs](https://github.com/ethereum/execution-specs) maintainers in the `#el-testing` channel in the [Ethereum R&D Discord Server](https://discord.com/invite/qGpsxSA). | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.