diff --git a/README.md b/README.md index 2466cad1b3f..d08cd04f7f1 100644 --- a/README.md +++ b/README.md @@ -1,220 +1,16 @@ -# ⚠️ NOTICE: All code from this repository has been migrated to ethereum/execution-specs +# Execution Spec Tests -As such, this repository is no longer accepting pull requests as of 2025-11-01. Please direct any changes to https://github.com/ethereum/execution-specs instead. +> [!IMPORTANT] +> This repository has been archived. All code from this repository has been migrated to [ethereum/execution-specs](https://github.com/ethereum/execution-specs). ## What This Means for You -- **Client Teams:** No changes. Test fixture releases will continue to be published in this repository, [ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests). +- **Client Teams:** Test fixture releases are now published at [ethereum/execution-specs releases](https://github.com/ethereum/execution-specs/releases); they are no longer published in this repository. - **Test Contributors:** - Please direct PRs to [ethereum/execution-specs](https://github.com/ethereum/execution-specs). - - Read more about the changes effecting test development in our [blog announcement](https://steel.ethereum.foundation/blog/2025-11-04_weld_final/). -- **Documentation:** [eest.ethereum.org](https://eest.ethereum.org) is archived and now redirects to its new home at [steel.ethereum.foundation/docs/execution-specs](https://steel.ethereum.foundation/docs/execution-specs/). + - Read more about the changes affecting test development in our [blog announcement](https://steel.ethereum.foundation/blog/2025-11-04_weld_final/). +- **Documentation:** [eest.ethereum.org](https://eest.ethereum.org) redirects to its new home at [steel.ethereum.foundation/docs/execution-specs](https://steel.ethereum.foundation/docs/execution-specs/). -### Questions or Feedback About the Migration? +### Questions or Feedback -Please reach out to the STEEL team by writing to the `#el-testing` channel in the [Ethereum R&D Discord Server](https://discord.com/invite/qGpsxSA) or contacting any STEEL team member directly. - ---- - -## Execution Spec Tests - -[![latest version](https://img.shields.io/github/v/release/ethereum/execution-spec-tests)](https://github.com/ethereum/execution-spec-tests/releases/latest) -![Python Versions](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-blue) -[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) -[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv) -[![License](https://img.shields.io/github/license/ethereum/execution-spec-tests)](https://github.com/ethereum/execution-spec-tests/blob/main/LICENSE) - -## About - -The full execution-spec-tests documentation can be found [here](https://eest.ethereum.org/main/). - -**Note:** All of [ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests) functionality (i.e., the spec tests, test frameworks and tooling) will move to [ethereum/execution-specs](https://github.com/ethereum/execution-specs) in Q4 2025 to streamline spec and test development. For more information please see our [blog post](https://steel.ethereum.foundation/blog/2025-09-11_weld-announcement/). For now, please continue contributing to both [ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests) and [ethereum/execution-specs](https://github.com/ethereum/execution-specs) as you previously have - we will reach out to you if and when need be! - -[ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests) is both a collection of test cases and a framework implemented in Python to generate tests for Ethereum execution clients. - -The framework collects and executes the test cases in order to generate _test fixtures_ (JSON) which can be consumed by any execution client to verify their implementation of [ethereum/execution-specs](https://github.com/ethereum/execution-specs). The fixtures, which define state transition and block tests, are generated by the framework using one of the `t8n` command-line tools that are provided by most execution clients, see below for an overview of the supported `t8n` tools. - -```mermaid ---- -title: Test Fixture Generation with execution-spec-tests ---- -flowchart LR - style C stroke:#333,stroke-width:2px - style D stroke:#333,stroke-width:2px - style G stroke:#F9A825,stroke-width:2px - style H stroke:#F9A825,stroke-width:2px - - subgraph "ethereum/go-ethereum, ..." - C[evm t8n
external executable] - end - - subgraph ethereum/solidity - D[solc
external executable] - end - - subgraph ethereum/EIPs - E(EIPS/EIP-*.md
SHA digest via Github API) - end - - subgraph "ethereum/execution-spec-tests" - A(./tests/**/*.py
Python Test Cases) - B([$ fill ./tests/
Python Framework]) - end - - subgraph Test Fixture Consumers - subgraph ethereum/hive - G([$ hive ...
Go Test Framework]) - end - H([Client executables]) - end - - C <-.-> B - D <-.-> B - A --> B - E <-.-> |retrieve latest spec version\ncheck tested spec version| B - B -->|output| F(./fixtures/**/*.json\nJSON Test Fixtures) - F -->|input| G - F -->|input| H -``` - -The generated test fixtures can be used: - -1. Directly by client teams' test frameworks, and, -2. In the integration tests executed in the [ethereum/hive](https://github.com/ethereum/hive) framework. - -## Transition Tool Support - -The following transition tools are supported by the framework: - -| Client | "t8n" Tool | Tracing Support | -| -------| ---------- | --------------- | -| [ethereum/evmone](https://github.com/ethereum/evmone) | `evmone-t8n` | Yes | -| [ethereum/execution-specs](https://github.com/ethereum/execution-specs) | `ethereum-spec-evm` | Yes | -| [ethereum/go-ethereum](https://github.com/ethereum/go-ethereum) | [`evm t8n`](https://github.com/ethereum/go-ethereum/tree/master/cmd/evm) | Yes | -| [ethereumjs](https://github.com/ethereumjs/ethereumjs-monorepo) | [`ethereumjs-t8ntool.sh`](https://github.com/ethereumjs/ethereumjs-monorepo/tree/master/packages/vm/test/t8n) | No | -| [hyperledger/besu](https://github.com/hyperledger/besu/tree/main/ethereum/evmtool) | [`evm t8n-server`](https://github.com/hyperledger/besu/tree/main/ethereum/evmtool) | Yes | -| [status-im/nimbus-eth1](https://github.com/status-im/nimbus-eth1) | [`t8n`](https://github.com/status-im/nimbus-eth1/blob/master/tools/t8n/readme.md) | Yes | - -### Upcoming EIP Development - -Generally, specific `t8n` implementations and branches must be used when developing tests for upcoming EIPs. - -We use named reference tags to point to the specific version of the `t8n` implementation that needs to be used fill the tests. - -All current tags, their t8n implementation and branch they point to, are listed in [.github/configs/evm.yaml](.github/configs/evm.yaml). - -## Getting Started - -### Prerequisites - -The tools provided by ethereum/execution-spec-tests use `uv` ([docs.astral.sh/uv](https://docs.astral.sh/uv/)) to manage their dependencies and virtual environment. `uv` downloads Python for your target platform if one of the required versions (Python 3.11 or 3.12) is not available natively. - -`uv` can be installed via curl (recommended; can self-update): - -```console -curl -LsSf https://astral.sh/uv/install.sh | sh -``` - -or pip (requires Python, can't self-update): - -```console -pip install uv -``` - -### Installation - -Clone [execution-spec-tests](https://github.com/ethereum/execution-spec-tests) and install its dependencies: - -```console -git clone --depth 1 https://github.com/ethereum/execution-spec-tests -cd execution-spec-tests -uv python install 3.12 -uv python pin 3.12 -uv sync --all-extras -``` - -See [Installation Troubleshooting](https://eest.ethereum.org/main/getting_started/installation_troubleshooting/) in the online docs if you encounter issues. - -### Exploring and Filling Test Cases - -By default, JSON test fixtures are generated from this repository's Python test cases using the [Ethereum Execution Layer Specification](https://github.com/ethereum/execution-specs) (EELS) reference implementation. The resulting JSON fixtures can be executed against execution clients to verify consensus. The process of generating fixtures is often referred to as "filling". - -1. Explore test cases via `--collect-only` and search for test cases that combine `PUSH0` and `DELEGATECALL` in the EVM functionality introduced in the Shanghai hard fork: - - ```console - uv run fill --collect-only -k "push0 and delegatecall" tests/shanghai/ - ``` - - The `fill` command is based on [`pytest`](https://docs.pytest.org/en/stable/). The above command uses the [optional pytest arguments](https://docs.pytest.org/en/stable/how-to/usage.html): - - - `--collect-only` only collect test cases; don't execute them. - - `-k ` filter test cases by their test case ID based on the given expression. - - `tests/shanghai` the directory containing the test cases (tells `fill` to only discover test cases in this directory; default: `tests/`). - - Expected console output: - ![Screenshot of pytest test collection console output](./docs/filling_tests/img/pytest_collect_only.png) - -2. Fill `state_test` fixtures for these test cases: - - ```console - uv run fill -k "push0 and delegatecall" tests/shanghai/ -m state_test -v - ``` - - where: - - - `-m state_test` only fills test cases marked as a `state_test` (see all available markers via `uv run fill --markers`). - - `-v` enables verbose output. - - Expected console output: - ![Screenshot of fill test collection console output](./docs/filling_tests/img/pytest_run_example.png) - -3. Verify the generated fixtures: - - a. Check the corresponding fixture file has been generated: - - ```console - head fixtures/state_tests/shanghai/eip3855_push0/push0/push0_contract_during_call_contexts.json - ``` - - b. Open the generated HTML test using the link provided at the bottom of the console output. This is written to the output directory at: - - ```console - ./fixtures/.meta/report_fill.html - ``` - -## Usage - -More information on how to obtain and consume the [released test fixtures](https://github.com/ethereum/execution-spec-tests/releases) can be found in the [documentation](https://eest.ethereum.org/main/running_tests/). - -For further help with working with this codebase, see the [online documentation](https://eest.ethereum.org/main/): - -1. Learn [useful command-line flags](https://eest.ethereum.org/main/filling_tests/filling_tests_command_line/). -2. [Execute tests for features under development](https://eest.ethereum.org/main/filling_tests/filling_tests_dev_fork/) via the `--from=FORK1` and `--until=FORK2` flags. -3. _Optional:_ [Configure VS Code](https://eest.ethereum.org/main/getting_started/setup_vs_code/) to auto-format Python code and [execute tests within VS Code](https://eest.ethereum.org/main/filling_tests/filling_tests_vs_code/). -4. Implement a new test case, see [Writing Tests](https://eest.ethereum.org/main/writing_tests/). - -## Coverage - -The available test cases can be browsed in the [Test Case Reference doc](https://eest.ethereum.org/main/tests/). - -## Installation Troubleshooting - -If you encounter issues during the installation process, please refer to the [Installation Troubleshooting](https://eest.ethereum.org/main/getting_started/installation_troubleshooting/) page. - -## Contributing - -Contributions and feedback are welcome. Please see our [Contributing Guidelines](CONTRIBUTING.md) for detailed information on how to contribute, and the [online documentation](https://eest.ethereum.org/main/writing_tests/) for this repository's coding standards and help on implementing new tests. - -We welcome earnest newcomers, no matter how small the contribution! However, we do not accept: - -- Contributions that only fix spelling or grammatical errors in documentation, code or elsewhere -- Pull requests from obvious airdrop farmers -- Drive-by or vibe code contributions without proper engagement or context - -Pull requests should have reasonable substance or resolve an existing repository open issue. - -Care is required when adding PRs or issues for functionality that is live on Ethereum mainnet, please refer to the [Security Policy](SECURITY.md) for more information about reporting vulnerabilities and eligibility for the [bug bounty program](https://bounty.ethereum.org). - -## License - -This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. +Please reach out to the [ethereum/execution-specs](https://github.com/ethereum/execution-specs) maintainers in the [Ethereum R&D Discord Server](https://discord.com/invite/qGpsxSA).