Skip to content

Batch Electrum witness resolution - #18

Draft
Jainakin wants to merge 1 commit into
rgb-protocol:masterfrom
Jainakin:hardik/esplora-witness-resolution
Draft

Batch Electrum witness resolution#18
Jainakin wants to merge 1 commit into
rgb-protocol:masterfrom
Jainakin:hardik/esplora-witness-resolution

Conversation

@Jainakin

@Jainakin Jainakin commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • resolve unique Electrum witness transactions and merkle proofs with native Electrum request batches
  • verify returned transaction identities and confirmation heights before constructing witness status
  • preserve the existing serial resolver for batch transport/protocol failures, missing-transaction recovery, malformed merkle-batch cardinality, and merkle-height drift

Motivation

High-history RGB consignments repeatedly resolve many historical witnesses. Serial remote lookups make transfer validation proportional to both contract history and network round-trip latency.

This is deliberately an Electrum-only batching change. It adds no persistent cache, operation cache, worker pool, Esplora change, or WASM change. The runtime batching logic is isolated to src/indexers/electrum_blocking.rs; the only non-runtime diff is the Cargo.toml feature dependency needed for serde_json parsing in the Electrum resolver.

Controlled 88-history results

A locally generated consignment with 88 witness bundles and 88 transitions was replayed through the exact resolver and RGB-lib validation/import/acceptance path.

Workload Serial Batched Improvement
Witness resolver, localhost median ~174 ms ~62 ms ~2.8x
Witness resolver, modeled 400 ms RTT 107.301 s 1.722 s ~62.3x
Full validate/import/accept, modeled 400 ms RTT 214.691 s 3.120 s ~68.8x

The RTT run used a deterministic local delay proxy; it is a controlled latency model, not a claim about production Signet infrastructure.

Correctness coverage

The deterministic local Electrum tests cover:

  • duplicate transaction-ID deduplication
  • out-of-order server responses
  • transaction identity mismatch rejection
  • confirmed and unconfirmed transactions
  • mixed found/missing transaction batches
  • batch-level protocol errors with preservation of serial missing-transaction semantics
  • merkle-height drift with deterministic serial fallback

Malformed merkle-batch cardinality also takes the serial fallback path; the Electrum client API itself guarantees one response slot per submitted request on a successful batch. Malformed transaction data and transaction-identity mismatches fail closed rather than being accepted through fallback.

Validation

  • cargo +nightly fmt --all -- --check
  • cargo test --workspace --all-features --no-fail-fast
    • 46 passed, 0 failed, 0 ignored
  • cargo clippy --workspace --all-features --all-targets -- -D warnings
  • cargo check --workspace --no-default-features
  • RUSTDOCFLAGS=\"-D warnings\" cargo doc --workspace --all-features --no-deps

The final Electrum-only revision's full GitHub matrix is green across Linux, macOS, Windows, stable, beta, nightly, MSRV, docs, WASM, lint, and tests.

Scope and rollout

Downstream integration is in UTEXO-Protocol/rgb-lib#79. That PR is temporarily pinned to this PR commit and must move to an official rgb-ops release before merge.

Production rollout must also exercise the exact authenticated Electrum deployment because native batch authorization semantics are provided by electrum-client; unsupported or rejected batches safely use the serial path but will not deliver the latency improvement.

This PR does not move RGB validation outside the LDK peer mutex and does not implement transactional channel-funding recovery. Those concerns remain separate from this resolver optimization.

@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.37275% with 68 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/indexers/electrum_blocking.rs 86.4% 68 Missing ⚠️

📢 Thoughts on this report? Let us know!

@Jainakin
Jainakin force-pushed the hardik/esplora-witness-resolution branch from 46c7aac to 6c98f4e Compare August 13, 2026 14:02
@zoedberg

Copy link
Copy Markdown
Member

Why did you change this PR from Optimize Esplora witness resolution to Batch Electrum witness resolution?

@Jainakin

Copy link
Copy Markdown
Contributor Author

Why did you change this PR from Optimize Esplora witness resolution to Batch Electrum witness resolution?

Good question. The original draft targeted Esplora by replacing the separate get_tx and get_tx_status requests with get_tx_info. That reduced each witness from two HTTP requests to one, but it still required one sequential network round trip per witness, so it did not address the main high-history latency problem.

After a protocol-team discussion, the agreed first step was to avoid introducing a cache and instead use the native batching supported by electrum-client, while keeping Esplora and WASM work separate. I therefore removed the Esplora implementation and repurposed this still-draft PR for Electrum batching.

The current PR deduplicates witness transaction IDs, batches verbose transaction retrieval and merkle-proof retrieval, verifies returned transaction IDs and block heights, and preserves the existing serial behavior as a fallback when batch operations are unsupported or fail. It no longer changes Esplora behavior.

I updated the title and description to match the final diff, but I should have added an explicit note explaining the scope change. The stale hardik/esplora-witness-resolution branch name also makes this more confusing. Sorry about that.
If you would prefer a clean review history, I can move the Electrum implementation to a separate PR and revert/close this one.

@St333p

St333p commented Aug 19, 2026

Copy link
Copy Markdown
Member

The original draft targeted Esplora by replacing the separate get_tx and get_tx_status requests with get_tx_info.

It still looks like a significant improvement, since it reduces the number of network calls by a half. A PR on it is still welcome, otherwise we will probably consider implementing it ourselves.

@Jainakin

Jainakin commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

The original draft targeted Esplora by replacing the separate get_tx and get_tx_status requests with get_tx_info.

It still looks like a significant improvement, since it reduces the number of network calls by a half. A PR on it is still welcome, otherwise we will probably consider implementing it ourselves.

Hey @St333p, new PR on this coming up, will notify as it becomes ready for review.

@Jainakin

Copy link
Copy Markdown
Contributor Author

Hey @St333p . The separate Esplora optimization is now ready for review in #20. I’ve kept this PR scoped exclusively to native Electrum batching.

#20 replaces the sequential get_tx() and get_tx_status() calls with one get_tx_info() request per witness resolution, validates both the advertised and reconstructed transaction IDs, and adds deterministic coverage for the success and failure paths.

In the controlled 88-witness replay, the complete validation/import/acceptance path improved from 234.59 seconds to 120.84 seconds, a 48.5% reduction before introducing parallel witness resolution. This is a controlled latency result, not a production Signet SLA.

All 25 checks, including Codecov patch coverage, are green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants