Firehose Ethereum Battlefield

This repository contains an Hardhat project and scripts to test the various Firehose Ethereum integrations we support. It contains Solidity smart contracts and test cases that exercises all corner cases of tracing an Ethereum node.

Install dependencies:

pnpm install

Ensure you have node version 20 or more recent as as the chain's binary compiled on your machine. Review Dependencies section to determine which tool you need to have locally for the commands below to work.

Run Test Suite

• Launch Firehose instance using geth --dev model (requires geth and fireeth to be available locally) in a terminal by running:

  ./scripts/run_firehose_geth_dev.sh 3.0

  This launches the Geth node and Firehose tracer version 3.0.

• Launch the test suite:

  pnpm test:fh3.0:geth-dev

Ensure that you run match command above with the Firehose Tracer Version you use when launching ./scripts/run_firehose_geth_dev.sh as well as for the chain your are testing against. See below for chain details and how you should launch the chain and the corresponding tests.

Chain Tests

Battlefield supports testing across various forks of Ethereum. Usually, you need to run the specific tests, here the list of currently supported/known chain and how to test them:

Chain	Firehose Launcher	Tests Launcher	Notes
Ethereum (Firehose 3.0)	./scripts/run_firehose_geth_dev.sh 3.0 prague	pnpm test:fh3.0:geth-dev	None
Reth Dev (fh 3.0)	./scripts/run_firehose_reth_dev.sh (Amsterdam fork by default, or ... prague)	pnpm test:fh3.0:reth-dev	Requires reth and fireeth
Arc Dev (fh 3.1, v5)	./scripts/run_firehose_arc_dev.sh	pnpm test:fh3.1:arc-dev	Requires arc-node-execution and fireeth. Own goldens tag
Sei	./scripts/run_firehose_sei.sh sequential	pnpm test:fh3.0:sei-dev	The sequential tag refers to transaction execution algorithm, test both
Sei	./scripts/run_firehose_sei.sh parallel	pnpm test:fh3.0:sei-dev	The parallel tag refers to transaction execution algorithm, test both
BNB	Docker miner: ./scripts/bnb/up.sh, then ./scripts/run_firehose_bnb.sh	pnpm test:fh3.0:bnb-dev	None
BNB Reth (fh 3.0, v5)	Docker miner: ./scripts/bnb/up.sh -c, then ./scripts/run_firehose_reth_bsc.sh	pnpm test:fh3.0:reth-bsc-dev	Requires reth-bsc. Own goldens tag, see BNB Reth
Polygon (fh 3.0)	./scripts/run_firehose_polygon.sh	pnpm test:fh3.0:polygon-dev, ./scripts/polygon-bridge	Heavy on dependencies (kurtosis, cast, polycli, bats...)
Geth Devnet (fh 3.0)	Terminal 1: ./scripts/ethereum_devnet/run_playground_devnet.sh Terminal 2: ./scripts/run_firehose_geth_devnet.sh	Terminal 3: pnpm test:fh3.0:geth-devnet	Requires geth, fireeth, and builder-playground
Reth Devnet (fh 3.0)	Terminal 1: ./scripts/ethereum_devnet/run_playground_devnet.sh Terminal 2: ./scripts/run_firehose_reth_devnet.sh	Terminal 3: pnpm test:fh3.0:reth-devnet	Requires reth, fireeth, and builder-playground
Besu (fh 3.0)	./scripts/run_firehose_besu_devnet.sh	pnpm test:fh3.0:besu-devnet	Requires besu binary, builder-playground
Optimism Geth Devnet (fh 3.0)	./scripts/optimism/run_optimism_devnet.sh then ./scripts/run_firehose_op_geth_devnet.sh.	pnpm test:fh3.0:op-geth-devnet	Requires builder-playground
Optimism Reth Devnet (fh 3.0)	./scripts/optimism/run_optimism_devnet.sh then ./scripts/run_firehose_op_reth_devnet.sh.	pnpm test:fh3.0:op-reth-devnet	Requires op-reth (firehose-instrumented Op Stack Reth on PATH as op-reth), and builder-playground
Nitro Dev (fh 2.3 because block version == 3)	./scripts/run_firehose_nitro_dev.sh	pnpm test:fh2.3:nitro-dev	Standalone mine-on-demand Arbitrum chain. Requires nitro and fireeth

Block Comparison

Every test run ends with a Compare blocks test that runs fireeth tools compare-blocks-rpc between the Firehose endpoint and the node's JSON-RPC, so there is nothing extra to run by hand. It takes the RPC url from the Hardhat network configuration and bounds the range at the chain's last irreversible block (the comparison streams final blocks only, an unfinalized stop block would hang). Genesis is excluded on purpose, Firehose synthesizes that block and genesis.test.ts asserts its content directly.

The test reports as pending when the chain has no final block to compare yet (a devnet running real consensus needs a few epochs before one exists) and on the public Amoy testnet. A missing fireeth binary fails the test rather than skipping it, it is a broken setup. On mine-on-demand chains the test first mines blocks until finality catches up with what the suite produced, and prints the blocks left out when it cannot get there within its budget.

Environment variable	Default	Purpose
SKIP_COMPARE_BLOCKS	unset	Set to 1 to skip the comparison entirely
COMPARE_BLOCKS_MAX_SPAN	2000	Maximum number of blocks compared in a single run
COMPARE_BLOCKS_TIMEOUT_MS	300000	Hard timeout for the fireeth comparison process
COMPARE_BLOCKS_FINALITY_TIMEOUT_MS	60000	How long to mine blocks waiting for finality to catch up
BATTLEFIELD_FIREHOSE_ENDPOINT	localhost:8089	Firehose gRPC endpoint of the stack under test

Specific Tests

You can run a specific test group or a specific test by using:

pnpm test:<id> --grep "<filter>"

The <filter> works by concatenating the series of describe and it in the tests handlers using a space. So for example, to run all tests found in calls.test.ts, you would use:

pnpm test:<id> --grep "Calls"

As the file as a single describe("Calls", ...) definition. And to run specific Delegate with value you would use:

pnpm test:<id> --grep "Calls Delegate with value"

Snapshot Tags

The test suite has different snapshot tags that can exercises different different tracer behavior. The snapshots tag to use when running tests is controlled by the environment variable SNAPSHOTS_TAG. Here the list of snapshot tags we currently support.

• SNAPSHOTS_TAG="fh3.0" expects a Firehose Tracer Version 3.0 ./scripts/run_firehose_geth_dev.sh 3.0

Note

You should use pnpm test:<id> to run the tests, it sets SNAPSHOTS_TAG environment variable for you as well as the correct network to use.

Network Snapshots Overrides

Some network like Arbitrum have bugs compared to the "canonical" Ethereum Firehose tracer which is Ethereum Mainnet. They trace almost the same thing as their respective model (fh3.0) for most transactions but have some differences for specific test cases. We leverage Hardhat networks to implement this functionality.

Instead of creating a brand new tag specific for this network, instead it's possible to override the snapshot to use on a test case basis:

await expect(contractCall(owner, Logs.logInSubFailedCallButTrxSucceed, [Contract.address])).to.trxTraceEqualSnapshot(
  "logs/log_sub_call_fails_top_level_succeed.expected.json",
  {
    $contract: Contract.addressHex,
  },
  {
    networkSnapshotOverrides: ["arbitrum-geth-dev"],
  },
)

Here, if the Hardhat network we run the tests against is named arbitrum-geth-dev, the snapshot that will be used for comparison will be logs/arbitrum-geth-dev/log_sub_call_fails_top_level_succeed.expected.json and in all other cases it will be logs/log_sub_call_fails_top_level_succeed.expected.json.

To add new overrides to a new network, modify ./hardhat.config.ts networks config element to add a new unique network name. Then simply modify the test case if the same way as presented above.

Global Field Exclusions

Some networks have fields that are not implemented or have different values compared to the canonical Ethereum Firehose tracer. Instead of creating separate snapshots for these cases, you can register globally excluded fields that will be stripped from both the actual and expected traces before comparison.

Register excluded fields in ./test/global.ts:

import { registerGlobalExcludedFields } from "./lib/field-exclusion"
import { besu_exclude_fields } from "./lib/constants"

// Register global excluded fields for specific networks
registerGlobalExcludedFields("besu-devnet", besu_exclude_fields)

Field paths support:

• Dot notation: calls.gasConsumed
• Array notation: calls[].gasConsumed (applies to all array elements)
• Indexed access: calls[0].gasConsumed (applies to specific element)

Keccak Preimage Filtering

The Firehose tracers built on evm-firehose-tracer-rs (reth, op-reth, reth-bsc, world-chain, arc) and Nitro keep only the keccakPreimages entries that explain a storage change key of the transaction. On those networks, both the actual and the expected traces go through the same filter before comparison (./test/lib/keccak-filter.ts), so the shared snapshots, which hold every preimage, still apply. Filtering an already filtered trace changes nothing, so nodes running a tracer from before the filter pass too. The list of networks is tracerFiltersKeccakPreimages() in the same file.

Development

A bunch of unit tests uses "snapshot" testing to avoid writing lengthy assertions. The test suite manages, uses and updates snapshots using a few environment variables. If you are adding new tests and the snapshot doesn't exist, the first test run will update the snapshot on disk. You should review the taken snapshot to ensure it fits the desired model state.

If there is differences, you should update a specific snapshot by using SNAPSHOTS_UPDATE=<id> variable, on differences, the test will report the exact <id> to use. You update a snapshot with a command like:

SNAPSHOTS_UPDATE="transfer_existing_address" pnpm test:<id>

If you modify a setting/config that affects all snapshots, you technically need to update all affected tags. You can use SNAPSHOTS_UPDATE=".*" to update all snapshots, normally you will need to do it for each tag

• SNAPSHOTS_UPDATE=".*" pnpm test:fh3.0:geth-dev

Snapshots

We support only TransactionTrace snapshot within the test project (we do have block-level test but they don't use snapshots). Snapshots are mostly implemented within ./test/lib/snapshots.ts and in ./test/lib/assertions.ts (search for trxTraceEqualSnapshot).

The transaction snapshot is dynamic through variables that are resolved at runtime based on the transaction as sent on the network. Indeed, since we are running transactions against a real network that is evolving instead of replay existing transactions, we need to account for "normal" varying data like transaction's hash, nonce, global balances, and many more.

The snapshots comparison process could be defined as:

• We load the <id>.expected.json file which contains human JSON format of the TransactionTrace protobuf. It's mainly ProtoJSON with bytes rendered as hex and BigInt Ethereum Protobuf Message inlined as hex directly. This file can contain $<variable> any where in the file and those will be replaced in the normalization step below.

• From the template above, we generate the (git ignored) file <id>.expected.resolved.json which has variables resolves with transaction's receipt and test provided variables like address of a contract the test interact with.

• The actual TransactionTrace Protobuf value is retrieved from the Firehose instance running against the network and written to <id>.actual.original.json. This file will contains the untouched transaction as fetched from Firehose.

• The actual TransactionTrace Protobuf is normalized and the normalized version is written to <id>.original.normalized.json:

  • Balance and nonce changes are deltaize, a process in which we change absolute values to become relative by computing the new - old delta, if positive, we update the change to become old: 0, new: <delta> otherwise if negative we do old: <delta>, new: 0.
  • Balance changes with REWARD_TRANSACTION_FEE are normalized to old: 0, new: 1, this is because most chains uses EIP-1159 which introduce dynamic fees making the fee changes without being controllable.

• [Optional] If SNAPSHOTS_UPDATE is set and the regex matches current snapshot <id>, then the <id>.original.normalized.json is taken and templatized:

  • First by explicitly changing some fixes "path" to variables ($hash, $index, $nonce, $cumulativeGasUsed, $logsBloom)
  • Second by taking the name => values variables map a specific test is providing and replacing all values within the JSON by the $<name> of the associated variable (this has the consequences that values and names must unique within a test case).

  The templatized value is then saved as the new expect value and written to <id>.expected.json. The test runner is updated to use the actual as the expect value which will yield no differences.

• The <id>.original.normalized.json and <id>.expected.resolved.json content are then compared deeply, if there is a difference the diff is presented.

Protobuf Generation

If you have a weird Protobuf decoding error like Error: Buffer is not a value in enum sf.ethereum.type.v2.TransactionTrace.Typ or in that vein, it probably means that the block you receives and the Protobuf representation of the Block have diverged.

You can re-generate the Ethereum Protobuf ES code by doing:

pnpm generate

Installation Instructions

Dependencies

Dependencies you will need to have locally to run the scripts contained in this project:

• Node.js (Installation)
• pnpm (Installation)
• jq (brew install jq, apt-get install jq, others)
• sd (brew install sd, apt-get install sd, others)
• fireeth (brew install tap/streamingfast/firehose-ethereum or others)

Get fireeth

To install fireeth, you can simply do brew install tap/streamingfast/firehose-ethereum or follow other installations section of the project's README.

Get arc-node-execution

Arc's execution client is reth-based and lives in the private streamingfast/arc-node-priv repository. There are no published binaries, so build it:

git clone git@github.com:streamingfast/arc-node-priv.git && cd arc-node-priv
git checkout release/0.x
cargo build --release --bin arc-node-execution

Put target/release/arc-node-execution on your PATH, or set ARC_NODE_BINARY=/path/to/arc-node-execution.

The reth-* crates come from the private pinax-network/reth fork, pinned over an HTTPS URL, and cargo fetches them with the git CLI. If the build stops on could not read Username for 'https://github.com', point git at your SSH key for those URLs:

git config --global url."git@github.com:".insteadOf "https://github.com/"

# Check
arc-node-execution --version
arc-node-execution node --help | grep -- --firehose

Get reth

The Firehose-instrumented Reth binary must be on your PATH as reth.

# Check
reth --version

Find the correct pre-built binary for your chain and architecture on the Firehose chains overview page:

https://firehose.streamingfast.io/firehose/overview/chains

Download the binary, rename it to reth, make it executable, and place it on your PATH. You can also override the binary path without renaming it:

export RETH_BINARY=/path/to/your/reth-binary

Get reth-bsc

The Firehose-instrumented BSC Reth binary (StreamingFast reth-bsc fork) must be on your PATH as reth-bsc.

# Check
reth-bsc --version

Find the correct pre-built binary for your architecture on the Firehose chains overview page:

https://firehose.streamingfast.io/firehose/overview/chains

Download the binary, rename it to reth-bsc, make it executable, and place it on your PATH. You can also override the binary path without renaming it:

export RETH_BSC_BINARY=/path/to/your/reth-bsc-binary

Get nitro

The Firehose-instrumented Arbitrum Nitro binary must be on your PATH as nitro.

# Check
nitro --version

Find the correct pre-built binary for your architecture on the Firehose chains overview page:

https://firehose.streamingfast.io/firehose/overview/chains

Download the binary, rename it to nitro, make it executable, and place it on your PATH. You can also override the binary path without renaming it:

export NITRO_BINARY../nitro/target/bin/nitro

run_firehose_nitro_dev.sh launches a standalone mine-on-demand Nitro chain (--dev) with the Firehose tracer (--node.firehose) wrapped by fireeth. Like geth-dev/reth-dev, it only mines when a transaction arrives — the test suite bootstraps Firehose readiness on its own, so launch the chain then run pnpm test:fh2.3:nitro-dev (wait only for JSON-RPC on http://127.0.0.1:8547, do not wait for port 8089).

Build Firehose geth

• Clone https://github.com/streamingfast/go-ethereum/tree/firehose-fh3.0

  git clone https://github.com/streamingfast/go-ethereum --branch firehose-fh3.0 --single-branch <folder>

• Install it locally:

  go install ./cmd/geth

  [!NOTE] If you have some problem running the built binary on OSX, it could be due to OSX code signing issue, fix signature of the built binary with codesign --sign - --force --preserve-metadata=entitlements,requirements,flags,runtime $(which geth)

Build tools for native addons (c-kzg)

The blob transaction tests (cancun.test.ts) use the c-kzg package which requires a C++ compiler and node-gyp. On Linux, if pnpm install fails or c-kzg fails to load at runtime:

# Option 1, via npm
npm install -g node-gyp
rm -rf node_modules && pnpm install

# Option 2, via apt
sudo apt-get install -y node-gyp
cd node_modules/.pnpm/c-kzg@4.1.0/node_modules/c-kzg && npx node-gyp rebuild

Geth Devnet and Reth Devnet

Both geth-devnet and reth-devnet run as a secondary execution layer client alongside an existing devnet spun up by builder-playground. The Firehose tracer is injected via fireeth.

Prerequisites

• builder-playground installed and available on PATH
• Firehose-instrumented geth binary (for geth-devnet) — see Build Firehose geth
• reth binary on PATH (for reth-devnet) — see Get reth
• fireeth binary — see Get fireeth

Setup for geth-devnet

1. Terminal 1 — Start the playground devnet (keep it running throughout):

   ./scripts/ethereum_devnet/run_playground_devnet.sh

2. Terminal 2 — Run Geth as a secondary EL client with the Firehose tracer:

   ./scripts/run_firehose_geth_devnet.sh

3. Terminal 3 — Run the tests:

   pnpm test:fh3.0:geth-devnet

Setup for reth-devnet

1. Terminal 1 — Start the playground devnet (keep it running throughout):

   ./scripts/ethereum_devnet/run_playground_devnet.sh

2. Terminal 2 — Run Reth as a secondary EL client with the Firehose tracer:

   ./scripts/run_firehose_reth_devnet.sh

3. Terminal 3 — Run the tests:

   pnpm test:fh3.0:reth-devnet

BNB Reth (reth-bsc)

reth-bsc does not produce blocks itself in this setup. It runs as a Firehose-instrumented follower of the same dockerized geth-BSC miner used by the bnb-dev chain, peering with it over devp2p. The miner produces the blocks, reth-bsc re-executes them with the Firehose tracer, and fireeth wraps reth-bsc as its reader node.

Prerequisites

• Docker (for the BSC miner) — see ./scripts/bnb
• reth-bsc binary on PATH — see Get reth-bsc
• fireeth binary — see Get fireeth

Setup

1. Terminal 1 — Start the BSC miner (keep it running throughout). Use -c to remove any existing container so the chain is fresh:

   ./scripts/bnb/up.sh -c

   Wait until the miner's JSON-RPC answers on http://localhost:8545.

2. Terminal 2 — Run reth-bsc as a Firehose follower:

   ./scripts/run_firehose_reth_bsc.sh

   The script pulls the genesis and enode from the running miner container, funds the test accounts, and starts reth-bsc on non-default ports (HTTP 9545, authrpc 9551, p2p 30404) so it does not conflict with the miner or any other local node.

3. Terminal 3 — Run the tests:

   pnpm test:fh3.0:reth-bsc-dev

Notes

• Run against a fresh chain. On a reused chain the fresh-chain-only tests fail (skip them with SKIP_FRESH_CHAIN_ONLY_TESTS=1) and prague/setcode_set_delegations shifts by one ordinal because its fixed authority EOA already exists.
• Separate goldens tag. The suite uses SNAPSHOTS_TAG=fh3.0/v5/reth-bsc-dev, not the geth-BSC fh3.0/bnb-dev tag. The reth Firehose tracer (protocol version 5) does not emit gas change events, so ordinals can never line up with the geth-BSC goldens. See ./test/snapshots/RETH_BSC_DEV.md for the full audited divergence list. To stop everything:

kill $(ps aux | grep -E "run_firehose_reth_bsc|fireeth|reth-bsc" | grep -v grep | awk '{print $2}') 2>/dev/null
./scripts/bnb/down.sh

Alternative: Use Besu

As an alternative to Geth, you can use Besu directly. This provides a Java-based Ethereum client option for battlefield testing.

Setup Besu for Battlefield

1. Prerequisites:

   • besu binary
   • builder-playground

2. Start the playground devnet:

./scripts/ethereum_devnet/run_playground_devnet.sh

3. Run Besu with Firehose:

./scripts/run_firehose_besu_devnet.sh

The Besu setup runs as a secondary client alongside the playground's Reth node, connecting via the Engine API.