Batch Electrum witness resolution - #18
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
b7a0fc9 to
09350a5
Compare
b6a2f78 to
80a2608
Compare
46c7aac to
6c98f4e
Compare
|
Why did you change this PR from |
Good question. The original draft targeted Esplora by replacing the separate get_tx and get_tx_status requests with get_tx_info. That reduced each witness from two HTTP requests to one, but it still required one sequential network round trip per witness, so it did not address the main high-history latency problem. After a protocol-team discussion, the agreed first step was to avoid introducing a cache and instead use the native batching supported by electrum-client, while keeping Esplora and WASM work separate. I therefore removed the Esplora implementation and repurposed this still-draft PR for Electrum batching. The current PR deduplicates witness transaction IDs, batches verbose transaction retrieval and merkle-proof retrieval, verifies returned transaction IDs and block heights, and preserves the existing serial behavior as a fallback when batch operations are unsupported or fail. It no longer changes Esplora behavior. I updated the title and description to match the final diff, but I should have added an explicit note explaining the scope change. The stale hardik/esplora-witness-resolution branch name also makes this more confusing. Sorry about that. |
It still looks like a significant improvement, since it reduces the number of network calls by a half. A PR on it is still welcome, otherwise we will probably consider implementing it ourselves. |
Hey @St333p, new PR on this coming up, will notify as it becomes ready for review. |
|
Hey @St333p . The separate Esplora optimization is now ready for review in #20. I’ve kept this PR scoped exclusively to native Electrum batching. #20 replaces the sequential get_tx() and get_tx_status() calls with one get_tx_info() request per witness resolution, validates both the advertised and reconstructed transaction IDs, and adds deterministic coverage for the success and failure paths. In the controlled 88-witness replay, the complete validation/import/acceptance path improved from 234.59 seconds to 120.84 seconds, a 48.5% reduction before introducing parallel witness resolution. This is a controlled latency result, not a production Signet SLA. All 25 checks, including Codecov patch coverage, are green. |
Summary
Motivation
High-history RGB consignments repeatedly resolve many historical witnesses. Serial remote lookups make transfer validation proportional to both contract history and network round-trip latency.
This is deliberately an Electrum-only batching change. It adds no persistent cache, operation cache, worker pool, Esplora change, or WASM change. The runtime batching logic is isolated to
src/indexers/electrum_blocking.rs; the only non-runtime diff is theCargo.tomlfeature dependency needed forserde_jsonparsing in the Electrum resolver.Controlled 88-history results
A locally generated consignment with 88 witness bundles and 88 transitions was replayed through the exact resolver and RGB-lib validation/import/acceptance path.
The RTT run used a deterministic local delay proxy; it is a controlled latency model, not a claim about production Signet infrastructure.
Correctness coverage
The deterministic local Electrum tests cover:
Malformed merkle-batch cardinality also takes the serial fallback path; the Electrum client API itself guarantees one response slot per submitted request on a successful batch. Malformed transaction data and transaction-identity mismatches fail closed rather than being accepted through fallback.
Validation
cargo +nightly fmt --all -- --checkcargo test --workspace --all-features --no-fail-fastcargo clippy --workspace --all-features --all-targets -- -D warningscargo check --workspace --no-default-featuresRUSTDOCFLAGS=\"-D warnings\" cargo doc --workspace --all-features --no-depsThe final Electrum-only revision's full GitHub matrix is green across Linux, macOS, Windows, stable, beta, nightly, MSRV, docs, WASM, lint, and tests.
Scope and rollout
Downstream integration is in UTEXO-Protocol/rgb-lib#79. That PR is temporarily pinned to this PR commit and must move to an official rgb-ops release before merge.
Production rollout must also exercise the exact authenticated Electrum deployment because native batch authorization semantics are provided by
electrum-client; unsupported or rejected batches safely use the serial path but will not deliver the latency improvement.This PR does not move RGB validation outside the LDK peer mutex and does not implement transactional channel-funding recovery. Those concerns remain separate from this resolver optimization.