Share one forkable hub between firehose and tier1

[sduchesneau]

When firehose and substreams-tier1 run in the same process, they now share one forkable hub instead of each connecting to the relayer and holding its own copy of the live blocks. Both hubs also keep the same number of final blocks: max(200, 2 × bundle size), from substreams' HubKeepFinalBlocks. Firehose used max(500, 2 × bundle size).

Depends on streamingfast/bstream#80 and streamingfast/substreams#983.

How

• The launcher records the launched apps in Runtime.Apps.
• cmd/apps/shared_hub.go builds the hub once with substreams' NewLiveHub, the hub tier1 has always used: partial blocks, burst from LIB. It starts the hub, updates both apps' head metrics from it, and passes it to both apps through their modules. Each app waits on its Ready and shuts down if it terminates.
• Firehose streams use stream.WithoutPartialBlocks(), so a hub fed with partial blocks for tier1 does not change what firehose sends: each block arrives once, complete, as before. Firehose's own hub never had partial blocks, so this does nothing when the hub is not shared.
• The hub is not shared when either app is missing, when there is no live source, or with --firehose-discard-partial-blocks, since tier1 needs the partial blocks.

Memory on Solana

Live blocks only, at today's 100-block bundles:

	firehose	tier1	total
before	1.3 GiB	0.5 GiB	1.8 GiB
after, separate processes	0.5 GiB	0.5 GiB
after, same process			0.5 GiB

The accounts stream goes from 4.1 GiB to 1.2 GiB in a process running both.

Behavior change

Firehose clients resuming between 200 and 500 blocks below the last irreversible block are now served from merged-blocks files instead of memory.

Tested

• Unit tests for when the hub is not shared.
• Local dummy-chain stack (devel/standard):
  • Both apps logged "using the forkable hub shared by the process".
  • Firehose streamed live blocks.
  • substreams run through tier1 streamed live blocks.
  • A firehose range crossing from merged-blocks files into the hub (150–330) came back with no gap. The only missing numbers were the dummy chain's own skips, every 13 blocks.
• Not tested on a chain with partial blocks.

[dfuse-bot]

🔍 Vulnerabilities of ghcr.io/streamingfast/firehose-core:b2e6701-amd64

📦 Image Reference ghcr.io/streamingfast/firehose-core:b2e6701-amd64

digest	sha256:8edcd7a8d09db10569cb43ce28f49a1b3c91230f1561dd52e9dc02d8c11a4427
vulnerabilities
platform	linux/amd64
size	164 MB
packages	509

📦 Base Image ubuntu:24.04

also known as	noble noble-20260917
digest	sha256:f610ab94648195aa356059f5b41d6085c9d4d903c072430cdd1af7bdb646106b
vulnerabilities