Skip to content
Open
7 changes: 7 additions & 0 deletions src/builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1754,6 +1754,13 @@ fn build_with_store_internal(
Arc::clone(&pending_payment_store),
));

// Fill the address pool up front so LDK's sync `SignerProvider` callbacks can hand out
// pre-persisted addresses without waiting on wallet persistence.
runtime.block_on(wallet.initialize_address_pool()).map_err(|e| {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to parallelize the reads on startup.

IMO it would also preferable to keep up the pre-existing patterns of doing the read_ methods here in parallel, and then hand that in to initialize via an AddressPool::new or so rather than doing all/obfuscating in an initialize_ method.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Done. The record read now runs in the tokio::join! with the other startup reads, its result is handed to Wallet::new, which validates it in AddressPool::new, and the builder only blocks on the initial pool top-up. One side effect: a failed read now fails the build with ReadFailed rather than WalletSetupFailed.

log_error!(logger, "Failed to initialize the wallet's address pool: {}", e);
BuildError::WalletSetupFailed
})?;

tx_broadcaster.set_wallet(Arc::downgrade(&wallet));

// Initialize the KeysManager
Expand Down
9 changes: 6 additions & 3 deletions src/chain/electrum.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ use lightning_transaction_sync::ElectrumSyncClient;

use super::WalletSyncStatus;
use crate::config::{
clamp_full_scan_stop_gap, Config, ElectrumSyncConfig, MAX_FULL_SCAN_STOP_GAP,
MIN_FULL_SCAN_STOP_GAP,
clamp_full_scan_stop_gap, Config, ElectrumSyncConfig, ADDRESS_POOL_SIZE,
MAX_FULL_SCAN_STOP_GAP, MIN_FULL_SCAN_STOP_GAP,
};
use crate::error::Error;
use crate::fee_estimator::{
Expand Down Expand Up @@ -598,7 +598,10 @@ impl ElectrumRuntimeClient {
bounded
);
}
bounded as usize
// Extend the gap by the address pool size: the pool keeps that many addresses standing
// revealed-but-unused, which a scan restoring the wallet from seed alone would otherwise
// count against the configured gap.
(bounded as usize).saturating_add(ADDRESS_POOL_SIZE as usize)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think with the FIFO design this (and below) is no longer needed.

If we'd want to account for the address pool size it should just be considered the minimum in clamp_full_scan_stop_gap rather than saturating_add to not unnecessarily slow down syncing in unexpected ways. But don't think we need this at all now.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Agreed, removed. Handouts consume the pool oldest-first, so pooled addresses sit past the handed-out ones, where they can't make a full scan stop short of funds.

One qualifier: a handout that fails while a concurrent one proceeds can briefly leave a pooled address below a handed-out one. The inversion is bounded by the pool size and consumed by the next handouts, so the default gap of 20 covers it — but a configured gap below the pool size could stop short during that window. Happy to make the pool size the minimum in clamp_full_scan_stop_gap, as you suggested, if you'd like that covered too.

Chasing this down also turned up a bug, fixed in a separate fixup: the refill flushed its reveals even when the pool-record write failed, permanently skipping those indices and widening what a restore from seed must scan across.

One narrow residual remains: reveals retained after a failed record write can still be flushed by a later unrelated persist, and after a restart they stay revealed-but-unused. Re-pooling them at load turns out unsound — a full scan (or an older version of the software) can also reveal indices past the record, and re-pooling those would reuse handed-out or even used addresses — so I left the residual in place: it's bounded by the pool size and needs a compound failure. Could revisit with a tx-graph check if you think it's worth closing.

}

async fn get_incremental_sync_wallet_update(
Expand Down
7 changes: 5 additions & 2 deletions src/chain/esplora.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ use lightning_transaction_sync::EsploraSyncClient;

use super::WalletSyncStatus;
use crate::config::{
clamp_full_scan_stop_gap, Config, EsploraSyncConfig, BDK_CLIENT_CONCURRENCY,
clamp_full_scan_stop_gap, Config, EsploraSyncConfig, ADDRESS_POOL_SIZE, BDK_CLIENT_CONCURRENCY,
MAX_FULL_SCAN_STOP_GAP, MIN_FULL_SCAN_STOP_GAP,
};
use crate::fee_estimator::{
Expand Down Expand Up @@ -256,7 +256,10 @@ impl EsploraChainSource {
bounded
);
}
bounded as usize
// Extend the gap by the address pool size: the pool keeps that many addresses standing
// revealed-but-unused, which a scan restoring the wallet from seed alone would otherwise
// count against the configured gap.
(bounded as usize).saturating_add(ADDRESS_POOL_SIZE as usize)
}

pub(super) async fn sync_lightning_wallet(
Expand Down
22 changes: 22 additions & 0 deletions src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,20 @@ pub const MIN_FULL_SCAN_STOP_GAP: u32 = 1;
/// Values above 1000 are clamped to 1000 when a full scan runs.
pub const MAX_FULL_SCAN_STOP_GAP: u32 = 1000;

/// The number of addresses the node keeps revealed and persisted ahead of use, from which it
/// serves fresh-address requests and channel destination and shutdown scripts.
///
/// Pooled addresses are revealed-but-unused wallet scripts, and a wallet restored from seed
/// alone has no record of them. On-chain wallet full scans therefore extend the configured stop
/// gap (e.g. [`EsploraSyncConfig::full_scan_stop_gap`]) by this amount, so that the pool's
/// unused tail can never exhaust the gap on its own.
///
/// After a restore from seed, the pool refills from the keychain's first indices before the
/// initial full scan runs, so it can serve addresses a previous installation of the wallet
/// already handed out — possibly even used ones. The cost is address reuse, not fund
/// visibility: the reveals keep the scripts watched and the scan discovers any prior use.
pub const ADDRESS_POOL_SIZE: u32 = 16;

// The number of concurrent requests made against the API provider.
pub(crate) const BDK_CLIENT_CONCURRENCY: usize = 4;

Expand Down Expand Up @@ -550,6 +564,10 @@ pub struct EsploraSyncConfig {
/// ([`MAX_FULL_SCAN_STOP_GAP`]), inclusive. Values outside this range will be clamped to the
/// nearest bound and a warning will be logged when the full scan runs.
///
/// The scan extends this value by [`ADDRESS_POOL_SIZE`] to account for the addresses the
/// node keeps revealed-but-unused ahead of use, which would otherwise count against the gap
/// when restoring a wallet from seed.
///
/// **Note:** Large values can cause many Esplora requests, hit server rate limits,
/// take a long time to complete, or cause syncs to fail with
/// [`SyncTimeoutsConfig::onchain_wallet_sync_timeout_secs`].
Expand Down Expand Up @@ -601,6 +619,10 @@ pub struct ElectrumSyncConfig {
/// ([`MAX_FULL_SCAN_STOP_GAP`]), inclusive. Values outside this range will be clamped to the
/// nearest bound and a warning will be logged when the full scan runs.
///
/// The scan extends this value by [`ADDRESS_POOL_SIZE`] to account for the addresses the
/// node keeps revealed-but-unused ahead of use, which would otherwise count against the gap
/// when restoring a wallet from seed.
///
/// **Note:** Large values can cause many Electrum requests, hit server rate limits,
/// take a long time to complete, or cause syncs to fail with
/// [`SyncTimeoutsConfig::onchain_wallet_sync_timeout_secs`].
Expand Down
5 changes: 5 additions & 0 deletions src/io/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,11 @@ pub(crate) const BDK_WALLET_INDEXER_PRIMARY_NAMESPACE: &str = "bdk_wallet";
pub(crate) const BDK_WALLET_INDEXER_SECONDARY_NAMESPACE: &str = "";
pub(crate) const BDK_WALLET_INDEXER_KEY: &str = "indexer";

/// The derivation indices of the wallet's address pool will be persisted under this key.
pub(crate) const BDK_WALLET_ADDRESS_POOL_PRIMARY_NAMESPACE: &str = "bdk_wallet";
pub(crate) const BDK_WALLET_ADDRESS_POOL_SECONDARY_NAMESPACE: &str = "";
pub(crate) const BDK_WALLET_ADDRESS_POOL_KEY: &str = "address_pool";

/// [`StaticInvoice`]s will be persisted under this key.
///
/// [`StaticInvoice`]: lightning::offers::static_invoice::StaticInvoice
Expand Down
Loading
Loading