diff --git a/CONTEXT.md b/CONTEXT.md index 45f702a7b..d118234d1 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -239,3 +239,41 @@ The network path a mixnet-only surface resolves to, either the Standing Client's tunnel or clearnet. zingolib derives it from the indicator, and the app never sees it. A price fetch that resolves to clearnet is refused, never sent. + +**Leg**: +One of the three links a mixnet send crosses: the client's websocket to its +entry gateway (leg one), the sphinx path from that gateway to the Exit Node +(leg two), and the Exit Node's TCP connection to the destination (leg +three). The local SOCKS5 listener is leg zero. + +**Exit Node**: +Our name for the node whose network requester opens leg three. Nym's +literature calls it an exit gateway, and its embedded service a network +requester (historically, a service provider). + +**Destination**: +The clearnet host a mixnet send is addressed to, reached only by the Exit +Node on leg three. +_Avoid_: correspondent (the term this one renames), recipient (the payment +addressee inside a transaction, never a network peer) + +**Sentinel**: +The fixed public resolver (`1.1.1.1:53`) that exit evidence is gathered +against, per zingolib ADR 0044: one DNS round trip through the tunnel, +where any reply proves the Exit Node carries traffic. Not a destination, +never eligible for a verdict. + +**Capture**: +The debug-only tracing layer in the proxy FFI that matches nym's own log +events to observe legs one and two, streamed to the host as diagnostics. + +**Probe**: +The debug-only FFI surface that gathers evidence through the running +proxy on a caller-supplied deadline: the Sentinel arm (one Sentinel round +trip) and the destination arm (one TLS handshake against a caller-supplied +destination, then hang up). + +**Verdict**: +What one probe arm classified its attempt into, combining the round trip's +outcome with the events captured in its window; silence that no capture +explains is reported as indeterminate, never dressed as a diagnosis. diff --git a/docs/adr/0005-the-proxy-component-is-named-for-the-mixnet.md b/docs/adr/0005-the-proxy-component-is-named-for-the-mixnet.md new file mode 100644 index 000000000..527522f6c --- /dev/null +++ b/docs/adr/0005-the-proxy-component-is-named-for-the-mixnet.md @@ -0,0 +1,115 @@ +# 5. The proxy component is named for the mixnet + +Date: 2026-08-19 + +## Status + +Accepted, ruled by the maintainer on 2026-08-19. The rename itself is +deferred to a follow-on pull request, because its scope reaches every +build system in the repository and does not belong inside PR #1276. + +## Context + +The crate is `zingo-nym-proxy-ffi`, and because it uses +`uniffi::setup_scaffolding!` rather than a UDL, that crate name becomes the +component namespace: Kotlin imports `uniffi.zingo_nym_proxy_ffi`, and the +generated scaffolding exports C-ABI symbols named +`ffi_zingo_nym_proxy_ffi_*`. + +Three separate things are wrong with that name, and each is visible from +inside the crate itself. + +The vendor name is the odd one out. Every type the crate exports is already +neutral about which mixnet it carries: `MixnetProxyHandle`, +`Socks5Endpoint`, `ProxyFfiError`, `ProxyDeathReason`, `ProxyDeathObserver`. +"Nym" survives only in the crate name and in an internal comment about +`NymProxy`. The wallet uses the same neutral vocabulary throughout — Mixnet +Mode, `MixnetProxy`, `MixnetSlot`, `MixnetTiming` — so the component name +is the single place the implementation choice leaks into the interface. + +The `_ffi` suffix names the mechanism rather than the thing, and it states +that mechanism a third time. Every consumer already reaches the component +as `uniffi.`, and the crate exists for no purpose other than being +bound. The wallet component settles the house convention by counterexample: +it is every bit as much an FFI component, with a UDL and generated Kotlin, +and it is named `zingo`, not `zingo_ffi`. + +The word "shim" describes the crate's posture toward its own dependencies +rather than what a caller receives. A consumer across the boundary holds a +handle to a running mixnet proxy and calls `start`, `socks5Endpoint`, +`exitNode`, and `stop`. That it happens to wrap `NymProxy` from +`zingo-netutils` is invisible to them and irrelevant to the name. + +## Decision + +The crate is `mixnet-proxy`. The component namespace is `mixnet_proxy`, +so Kotlin imports `uniffi.mixnet_proxy` and the scaffolding exports +`ffi_mixnet_proxy_*`. The word "shim" is retired from the vocabulary of +this repository, in prose, in file names, and in identifiers; where a name +is needed, the thing is the mixnet proxy. + +The wallet component keeps the name `zingo` for now. + +## Considered options + +**Keep `zingo-nym-proxy-ffi`.** Rejected. It names a vendor the interface +does not expose, and a mechanism the import path already states. + +**`zingo-mixnet-proxy`.** Rejected, though it carries a real argument. Both +components are bound into one application process, so their C-ABI symbols +share a global namespace with every other native library the application +links, and a `zingo_` prefix both reduces the chance of a collision with a +third-party UniFFI component and makes a symbol legible as ours in a crash +dump or a linker error. It also pairs visibly with `uniffi.zingo`. The +maintainer ruled against the prefix: within this repository the component +is unambiguous, and the shorter name says what the thing is without +restating where it lives. + +**`mixnet-proxy`.** Chosen. + +## Consequences + +The symbol prefix becomes generic. `ffi_mixnet_proxy_*` is a plausible name +for another party's component, so a future collision is possible where +`ffi_zingo_mixnet_proxy_*` would have been safe, and a symbol in a crash +dump no longer identifies itself as ours. This decision accepts that cost. + +The name will overstate the crate's reach until its couplings are cut. It +depends on `zingo-netutils` for `NymProxy` and bootstraps TLS through +`org.ZingoLabs.Zingo.NymTlsInit`, so a consumer outside Zingo cannot use it +today. A name that invites reuse should either acquire that reuse or lose +those couplings; whichever happens, it should happen deliberately. + +The rename is large and touches systems that cannot all be verified from +one place. Roughly 316 identifier sites across about 35 files carry the old +names, spanning Cargo manifests, `uniffi-bindgen`, two workbench binaries, +`android/app/build.gradle.kts`, `android/app/lint.xml`, +`ios/Zingo.xcodeproj/project.pbxproj`, `ios/NymTransportModule.swift`, +three `build_*.mjs` scripts, `rust/android/docker/Dockerfile`, the contract +tests, and ADR 0004. Seven paths change too, including the crate directory, +the checked-in generated bindings under `android/app/src/*/java/uniffi/`, +the workflow file `nym-proxy-ffi-check.yaml`, both `*-android-shim.rs` +workbench binaries, and this repository's ADR 0004, whose file name +contains the retired word. Cargo verifies only the Rust portion; Gradle, +Xcode, and CI each need their own build, which is the reason this is a +pull request of its own rather than a change carried inside another. + +ADR 0004 is renamed and its prose updated by the same change. Its decision +is untouched: the platform verifier must hold the application Context +before the proxy opens any connection. + +## Open + +Whether the wallet component becomes `zingo_wallet`. Beside +`uniffi.mixnet_proxy`, the bare `uniffi.zingo` claims the product name for +one component among several, and two peers would read better as +`uniffi.zingo_wallet` and `uniffi.mixnet_proxy`. That rename moves the +crate name `zingo`, hence the built library, the UDL namespace, the +checked-in bindings, every Kotlin and Swift import, and the loader +configuration, so it is a larger change than this one and wants its own +decision. + +Whether the React Native module classes `NymTransportModule` and +`NymTlsInit` are renamed with the component. They name the application's +bridge module rather than the component, so they are application +vocabulary and are left to a separate ruling. diff --git a/docs/adr/0007-the-mixnets-three-legs-are-observed-by-capture-and-probe.md b/docs/adr/0007-the-mixnets-three-legs-are-observed-by-capture-and-probe.md new file mode 100644 index 000000000..da40dc2ee --- /dev/null +++ b/docs/adr/0007-the-mixnets-three-legs-are-observed-by-capture-and-probe.md @@ -0,0 +1,152 @@ +# 7. The mixnet's three legs are observed by capture and probe + +Date: 2026-08-24 + +## Status + +Accepted. + +## Context + +Mixnet Mode routes a send through three links: the client's websocket to +its entry gateway, the sphinx path from that gateway to the exit node, +and the exit node's TCP connection to the destination. When a send +fails, remediation differs by leg. A dead gateway link wants a +reconnect, a refusing exit wants a redraw, and an unreachable +destination is not a mixnet fault at all. Today the app observes none +of them. The proxy FFI reports start failures as prose and the death of +its local SOCKS5 listener, which is leg zero. + +The SOCKS5 protocol cannot carry the discrimination. nym's local SOCKS5 +server answers every CONNECT with `Success` before anything enters the +mixnet (`nym-socks5-client-core` 1.21.5, `socks/client.rs`, +`acknowledge_socks5`), so `check_connectivity` in `zingo-netutils` +proves only the local listener. zingo-netutils' Sentinel (zingolib ADR +0044) already proves an Exit Node carries traffic, with a round trip +deliberately shaped like traffic the exit sees constantly, but that +evidence never crosses the FFI. When the exit node fails to reach the +destination, its network requester does send a typed +`ConnectionError { connection_id, network_requester_error }` back +through the mixnet, but the client core logs it at `error!` and drops +it (`socks/mixnet_responses.rs`). The gateway link is no better served. +`Socks5MixnetClient` in nym-sdk 1.21.4 exposes no connection state and +no event stream, and websocket failures surface only as `error!` events +inside `nym-gateway-client`. Every signal this decision needs exists in +the process, and none of it crosses an API. + +## Decision + +The proxy FFI defines the missing observations with a tracing capture +layer and a caller-driven probe, and `zingo-netutils` makes the +bootstrap race typed. + +A `tracing` layer inside `zingo-nym-proxy-ffi` matches events from two +target prefixes: `nym_socks5_client_core`, where the exit's +`ConnectionError` reason surfaces, and `nym_gateway_client`, where +websocket send and stream failures surface. Captured events and the +bootstrap narrative queue inside the crate, capped and drop-oldest, +and the host polls one `drain_diagnostics` export that surrenders the +queue in order with the count of any events lost to overflow. The host +implements no callback, so no host code runs on the mixnet client's +threads and no new trait object crosses the FFI. The existing +`ProxyDeathObserver` keeps its at-most-once production contract +unchanged. + +The probe has two arms. `probe_sentinel(deadline_millis)` opens one +tunnel through the running proxy and performs the Sentinel round trip +`zingo-netutils::sentinel` defines: a DNS lookup of a constant name, +sent to a public resolver whose silence indicts the tunnel rather than +itself. A Sentinel answer proves legs one and two: the Exit Node +carries traffic. A captured exit `ConnectionError` in the window +proves those legs and names the exit's refusal, and this is the +definition of "exit refused". Silence with captured gateway-client +errors reads as a dead gateway link. Clean silence at the deadline +reads as indeterminate beyond the gateway, and is reported as exactly +that. Because Exit Nodes filter destinations by policy, a refusal +verdict names the Sentinel, never the exit's health. + +`probe_destination(host, port, deadline_millis)` completes one TLS +handshake against the caller-supplied destination through the same +tunnel, verifying against the compiled-in Mozilla bundle (ADR 0006), +and hangs up without sending one byte of application protocol. A +completed handshake proves all three legs and the certificate chain. A +TLS-layer fault is its own verdict carrying rustls's reason, which is +the class of failure `nym_http_api_client` flattened into `client +error (Connect)` for weeks. A captured exit `ConnectionError` names +the exit's refusal of this destination. An attempt that ends without a +TLS fault classifies by the window, or stays indeterminate with the +local ending named. The crate holds the deadline and the caller's +host and port, and no wallet configuration. + +For the client-to-gateway leg, `zingo-netutils` gains a public +`BootstrapEvent` enum with five lifecycle variants: `DiscoveryStarted`, +`DiscoveryFinished` with the candidate count, `PullLaunched` and +`PullFailed` with the Exit Node address (carrying the pull's whole +typed `NetOpFailure`, ruled over prose so a diagnostics surface renders +structure instead of parsing it), and `Connected` with the winner. The +driver resolves arm indices to Exit Node addresses. The planner's +hedging mechanics stay private. Every race reports, the first +bootstrap and each reconnect alike, so the screen never goes blind at +the moment after a death. + +The whole surface compiles behind a cargo feature that only debug +wallet builds enable. The consumer is a hidden diagnostics screen in +the Zingo app whose bridge module registers from the debug source sets +(`androidDebug`, `#if DEBUG`). Release binaries never carry the log +coupling. + +## Consequences + +The test surface discriminates the three legs as sharply as an unforked +nym allows, and the exit's own refusal reason reaches the screen. The +price is coupling to nym's log targets and message shapes. The gate +confines that coupling to debug builds, so a nym upgrade that breaks +capture breaks a debug screen and its tests, never a release. The +diagnostics screen cannot help diagnose a release install in the field. +A new `zingo-netutils` surface means a zingolib change and a new tag +before the FFI work starts, and the backend and UI land as separate +changes. + +## Considered options + +**Read the exit's refusal from SOCKS5 reply codes.** Impossible. The +local server acknowledges success unconditionally before the mixnet is +involved. + +**Define the refusal behaviorally, as a stream closed without data.** +Rejected. It loses the refusal reason and cannot distinguish a +mid-mixnet drop from an exit-side refusal. + +**Fork nym's client core to forward `ConnectionError`.** Rejected. It +is the honest typed channel, but it adds a nym fork to maintain on top +of the existing zingolib fork, for a debug-build need. + +**A Sentinel-only probe, with leg three left to the send.** Rejected +after a first ruling for it. It cannot discern the TLS failure modes +of a live destination, and a certificate fault would keep presenting +as an unexplained stall exactly as it did through +`nym_http_api_client`. + +**A destination probe that sends an application request.** Rejected. +It puts protocol knowledge in the crate or pushes it across the FFI, +and the handshake alone already proves the three legs and the chain. + +**A streaming callback interface for the diagnostics events.** +Rejected after a first implementation. UniFFI hands the crate a boxed +trait object for every callback interface, host code runs on whatever +thread fires the event, and the only consumer is a debug screen that +polls its platform bridge anyway. The drain keeps every type static +and deletes the forwarder thread the callback needed. + +**Read gateway state from a nym-sdk API.** Impossible today. The +research found no connection state, no `is_connected`, and no event +stream on `Socks5MixnetClient`. + +**A separate harness APK, or instrumented tests alone.** Rejected as +the host. The diagnostics screen lives in the wallet app, debug builds +only, so a person watches a live device with the wallet's own +configuration. + +**Ship the capture layer in release builds.** Rejected. The log +coupling is too brittle to ship to users, and the field-diagnosis case +has not yet earned it. diff --git a/rust/nym-proxy-ffi/Cargo.lock b/rust/nym-proxy-ffi/Cargo.lock index 62321c1c1..b3e61356d 100644 --- a/rust/nym-proxy-ffi/Cargo.lock +++ b/rust/nym-proxy-ffi/Cargo.lock @@ -9105,12 +9105,12 @@ dependencies = [ [[package]] name = "zingo-net-diag" version = "0.1.0" -source = "git+https://github.com/zingolabs/zingolib?tag=zingolib_nym_rc0#f65251c5648ebdb96f00d3f2a9de92b11e139abb" +source = "git+https://github.com/zingolabs/zingolib?rev=8fc056a945e24fa5d36d274aed5d1cf5b725fafa#8fc056a945e24fa5d36d274aed5d1cf5b725fafa" [[package]] name = "zingo-netutils" version = "5.0.1" -source = "git+https://github.com/zingolabs/zingolib?tag=zingolib_nym_rc0#f65251c5648ebdb96f00d3f2a9de92b11e139abb" +source = "git+https://github.com/zingolabs/zingolib?rev=8fc056a945e24fa5d36d274aed5d1cf5b725fafa#8fc056a945e24fa5d36d274aed5d1cf5b725fafa" dependencies = [ "http 1.5.0", "hyper-util", @@ -9136,8 +9136,11 @@ version = "0.1.0" dependencies = [ "android_logger", "log", + "rustls-platform-verifier", "thiserror 2.0.20", "tokio", + "tokio-rustls 0.26.4", + "tokio-socks", "tracing", "tracing-subscriber", "uniffi", diff --git a/rust/nym-proxy-ffi/Cargo.toml b/rust/nym-proxy-ffi/Cargo.toml index 5b0ce33d7..46b07184f 100644 --- a/rust/nym-proxy-ffi/Cargo.toml +++ b/rust/nym-proxy-ffi/Cargo.toml @@ -12,7 +12,9 @@ crate-type = ["lib", "cdylib", "staticlib"] [dependencies] # NymProxy lives here, behind the nym feature; this shim always needs it. -zingo-netutils = { git = "https://github.com/zingolabs/zingolib", tag = "zingolib_nym_rc0", features = [ +# Pinned to the nym_bootstrap_events head (PR 2742) for BootstrapEvent; +# retarget to the next zingolib tag once that PR merges. +zingo-netutils = { git = "https://github.com/zingolabs/zingolib", rev = "8fc056a945e24fa5d36d274aed5d1cf5b725fafa", features = [ "nym", ] } # Proc-macro scaffolding (setup_scaffolding! + #[uniffi::export]); no build.rs @@ -23,15 +25,37 @@ uniffi = "0.28" tokio = { version = "1", features = ["io-util", "net", "rt-multi-thread", "time"] } thiserror = "2" tracing = "0.1.44" +# The diagnostics surface (ADR 0007): SOCKS5 tunnelling for the probes, TLS +# for the destination arm (verifying against the patched Mozilla-bundle +# verifier, ADR 0006), and the tracing capture layer. +tokio-socks = { version = "0.5", optional = true } +tokio-rustls = { version = "0.26", optional = true, default-features = false, features = [ + "ring", +] } +rustls-platform-verifier = { version = "0.7", optional = true } +tracing-subscriber = { version = "0.3.23", optional = true, default-features = false, features = [ + "registry", + "std", +] } [features] # Builds tests/live_mixnet.rs, which reaches the real Nym network. live-mixnet = [] +# The three-leg observation surface (ADR 0007): the tracing capture layer, +# the Sentinel and destination probes, and the observed bootstrap. Only +# debug wallet builds enable it; release binaries never carry it. +diagnostics = [ + "dep:rustls-platform-verifier", + "dep:tokio-rustls", + "dep:tokio-socks", + "dep:tracing-subscriber", + "zingo-netutils/socks5-transmit", +] [dev-dependencies] tokio = { version = "1", features = ["macros", "rt-multi-thread", "sync"] } # The tests read their temporal parameters from `time::test`. -zingo-netutils = { git = "https://github.com/zingolabs/zingolib", tag = "zingolib_nym_rc0", features = [ +zingo-netutils = { git = "https://github.com/zingolabs/zingolib", rev = "8fc056a945e24fa5d36d274aed5d1cf5b725fafa", features = [ "testutils", ] } diff --git a/rust/nym-proxy-ffi/src/debug_log.rs b/rust/nym-proxy-ffi/src/debug_log.rs index 7ea924105..2e07c357b 100644 --- a/rust/nym-proxy-ffi/src/debug_log.rs +++ b/rust/nym-proxy-ffi/src/debug_log.rs @@ -12,11 +12,29 @@ pub(crate) fn init() { .with_max_level(log::LevelFilter::Debug) .with_tag("MixnetProxy"), ); - let _ = fmt() - .with_ansi(false) - .with_max_level(tracing::Level::DEBUG) - .with_writer(Logcat) - .try_init(); + #[cfg(feature = "diagnostics")] + { + use tracing_subscriber::Layer as _; + use tracing_subscriber::layer::SubscriberExt as _; + use tracing_subscriber::util::SubscriberInitExt as _; + let _ = tracing_subscriber::registry() + .with( + fmt::layer() + .with_ansi(false) + .with_writer(Logcat) + .with_filter(tracing_subscriber::filter::LevelFilter::DEBUG), + ) + .with(crate::diagnostics::CaptureLayer) + .try_init(); + } + #[cfg(not(feature = "diagnostics"))] + { + let _ = fmt() + .with_ansi(false) + .with_max_level(tracing::Level::DEBUG) + .with_writer(Logcat) + .try_init(); + } }); } diff --git a/rust/nym-proxy-ffi/src/diagnostics.rs b/rust/nym-proxy-ffi/src/diagnostics.rs new file mode 100644 index 000000000..4b60ee0e9 --- /dev/null +++ b/rust/nym-proxy-ffi/src/diagnostics.rs @@ -0,0 +1,761 @@ +//! The three-leg observation surface (ADR 0007), compiled only under the +//! `diagnostics` feature and consumed only by debug wallet builds. +//! +//! A mixnet send crosses three legs: the client's websocket to its entry +//! gateway, the sphinx path to the Exit Node, and the exit's TCP connection +//! to the destination. nym exposes none of them across an API, so this +//! module captures nym's own `tracing` events from the two targets where +//! the legs surface (`nym_gateway_client`, `nym_socks5_client_core`) and +//! queues them, with the typed bootstrap narrative, for the host to poll +//! through [`drain_diagnostics`]; the host implements no callback, so no +//! host code ever runs on the mixnet client's threads. Two caller-driven +//! probes classify what a deadline-bounded round trip through the running +//! proxy proves: the +//! Sentinel arm rides `zingo_netutils::sentinel` (legs one and two), and +//! the destination arm completes a TLS handshake against a caller-supplied +//! destination, verifying against the compiled-in Mozilla bundle (ADR +//! 0006). The log coupling this module accepts is the reason it never +//! ships in a release binary. + +use std::collections::VecDeque; +use std::net::SocketAddr; +use std::sync::{Mutex, OnceLock}; +use std::time::{Duration, Instant}; + +use tokio_rustls::TlsConnector; +use tokio_rustls::rustls; +use zingo_netutils::BootstrapEvent; +use zingo_netutils::sentinel::{self, ExitEvidence}; + +use crate::MixnetProxyHandle; + +/// How many captured events the classification window retains. +const WINDOW_CAPACITY: usize = 256; + +/// How many undrained events the host-facing queue retains before dropping +/// the oldest. +const QUEUE_CAPACITY: usize = 1024; + +/// One step of the mixnet's observed life, streamed to the host as it happens. +#[derive(Clone, Debug, PartialEq, Eq, uniffi::Enum)] +pub enum MixnetDiagnosticsEvent { + /// The Exit Node discovery query left for the Nym directory. + DiscoveryStarted, + /// The directory answered the discovery query. + DiscoveryFinished { + /// The count of Exit Nodes the directory advertised. + candidate_count: u32, + }, + /// A pull of this Exit Node launched. + PullLaunched { + /// The Exit Node address the pull races. + exit_node: String, + }, + /// The pull of this Exit Node failed. + PullFailed { + /// The Exit Node address whose pull failed. + exit_node: String, + /// The failure rendered for a human. + error: String, + }, + /// The race kept this Exit Node and the local listener is up. + Connected { + /// The Exit Node address the proxy bound. + exit_node: String, + }, + /// A captured event from nym's gateway client, where leg one surfaces. + GatewayClientReport { + /// The event's level, rendered. + level: String, + /// The event's message, rendered. + message: String, + }, + /// A captured event from nym's SOCKS5 client core, where the exit's + /// refusal reason surfaces. + Socks5CoreReport { + /// The event's level, rendered. + level: String, + /// The event's message, rendered. + message: String, + }, +} + +/// One poll's worth of diagnostics: the queued events in order, and how +/// many older events the queue dropped since the previous drain. +#[derive(Clone, Debug, PartialEq, Eq, uniffi::Record)] +pub struct DiagnosticsDrain { + /// The events recorded since the last drain, oldest first. + pub events: Vec, + /// How many events overflowed the queue and were lost. + pub dropped_count: u32, +} + +/// What one Sentinel round trip through the running proxy proved. +#[derive(Clone, Debug, PartialEq, Eq, uniffi::Enum)] +pub enum SentinelVerdict { + /// The Sentinel answered, so legs one and two carry traffic. + ExitProven { + /// How long the round trip took, in milliseconds. + millis: u64, + }, + /// The exit refused the Sentinel, in its own captured words. + ExitRefused { + /// The captured refusal, rendered. + reason: String, + }, + /// Silence while the gateway client reported errors, so leg one is down. + GatewayLinkDead { + /// The captured gateway failure, rendered. + detail: String, + }, + /// Clean silence at the deadline, indeterminate beyond the gateway. + Indeterminate, +} + +/// What one TLS handshake against the destination proved. +#[derive(Clone, Debug, PartialEq, Eq, uniffi::Enum)] +pub enum DestinationVerdict { + /// The handshake completed, so all three legs and the certificate chain + /// hold. + DestinationProven { + /// How long the handshake took, in milliseconds. + millis: u64, + }, + /// The TLS layer failed with a named fault. + HandshakeRefused { + /// The TLS failure, rendered. + reason: String, + }, + /// The exit refused the destination, in its own captured words. + ExitRefused { + /// The captured refusal, rendered. + reason: String, + }, + /// Silence while the gateway client reported errors, so leg one is down. + GatewayLinkDead { + /// The captured gateway failure, rendered. + detail: String, + }, + /// The local SOCKS5 tunnel could not open, which is leg zero. + TunnelFailed { + /// The tunnel failure, rendered. + detail: String, + }, + /// The attempt failed without naming any leg. + Indeterminate { + /// What locally ended the attempt, rendered. + detail: String, + }, +} + +/// Which capture target a stored event came from. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum CaptureSource { + GatewayClient, + Socks5Core, +} + +/// One captured nym event, retained for probe classification. +#[derive(Clone, Debug)] +struct Captured { + at: Instant, + source: CaptureSource, + level: tracing::Level, + message: String, +} + +/// The undrained host-facing events and the count lost to overflow. +#[derive(Default)] +struct EventQueue { + events: VecDeque, + dropped: u32, +} + +/// The one crate-wide diagnostics state: the host-facing queue and the +/// classification window. +struct DiagnosticsHub { + queue: Mutex, + window: Mutex>, +} + +fn hub() -> &'static DiagnosticsHub { + static HUB: OnceLock = OnceLock::new(); + HUB.get_or_init(|| DiagnosticsHub { + queue: Mutex::new(EventQueue::default()), + window: Mutex::new(VecDeque::new()), + }) +} + +/// Queues `event` for the next drain, dropping the oldest on overflow. +pub(crate) fn publish(event: MixnetDiagnosticsEvent) { + let mut queue = hub() + .queue + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + if queue.events.len() == QUEUE_CAPACITY { + queue.events.pop_front(); + queue.dropped = queue.dropped.saturating_add(1); + } + queue.events.push_back(event); +} + +/// The diagnostics recorded since the last drain, surrendered in order. +#[uniffi::export] +pub fn drain_diagnostics() -> DiagnosticsDrain { + let mut queue = hub() + .queue + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + DiagnosticsDrain { + events: queue.events.drain(..).collect(), + dropped_count: std::mem::take(&mut queue.dropped), + } +} + +/// Records one captured nym event into the classification window and queues +/// it for the next drain. +fn record(source: CaptureSource, level: tracing::Level, message: String) { + { + let mut window = hub() + .window + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + if window.len() == WINDOW_CAPACITY { + window.pop_front(); + } + window.push_back(Captured { + at: Instant::now(), + source, + level, + message: message.clone(), + }); + } + let level = level.to_string(); + publish(match source { + CaptureSource::GatewayClient => { + MixnetDiagnosticsEvent::GatewayClientReport { level, message } + } + CaptureSource::Socks5Core => MixnetDiagnosticsEvent::Socks5CoreReport { level, message }, + }); +} + +/// The captured events at or after `since`, oldest first. +fn window_since(since: Instant) -> Vec { + hub() + .window + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()) + .iter() + .filter(|captured| captured.at >= since) + .cloned() + .collect() +} + +/// The first error-level message from `source` in the window, if any. +fn first_error(window: &[Captured], source: CaptureSource) -> Option { + window + .iter() + .find(|captured| captured.source == source && captured.level == tracing::Level::ERROR) + .map(|captured| captured.message.clone()) +} + +/// The `tracing` layer that captures nym's own events from the two targets +/// where the legs surface. +pub(crate) struct CaptureLayer; + +/// Renders an event's fields into one message line. +#[derive(Default)] +struct RenderedFields(String); + +impl tracing::field::Visit for RenderedFields { + fn record_debug(&mut self, field: &tracing::field::Field, value: &dyn std::fmt::Debug) { + use std::fmt::Write as _; + if !self.0.is_empty() { + self.0.push(' '); + } + if field.name() == "message" { + let _ = write!(self.0, "{value:?}"); + } else { + let _ = write!(self.0, "{}={value:?}", field.name()); + } + } +} + +impl tracing_subscriber::Layer for CaptureLayer { + fn on_event( + &self, + event: &tracing::Event<'_>, + _ctx: tracing_subscriber::layer::Context<'_, S>, + ) { + let target = event.metadata().target(); + let source = if target.starts_with("nym_socks5_client_core") { + CaptureSource::Socks5Core + } else if target.starts_with("nym_gateway_client") { + CaptureSource::GatewayClient + } else { + return; + }; + let mut fields = RenderedFields::default(); + event.record(&mut fields); + record(source, *event.metadata().level(), fields.0); + } +} + +/// Installs the capture layer as the process subscriber on hosts where no +/// platform log route claims it first. +#[cfg(not(target_os = "android"))] +pub(crate) fn install_capture() { + use tracing_subscriber::layer::SubscriberExt as _; + use tracing_subscriber::util::SubscriberInitExt as _; + let _ = tracing_subscriber::registry().with(CaptureLayer).try_init(); +} + +/// Carries the typed bootstrap narrative across the FFI unchanged. +pub(crate) fn bootstrap_event(event: BootstrapEvent) -> MixnetDiagnosticsEvent { + match event { + BootstrapEvent::DiscoveryStarted => MixnetDiagnosticsEvent::DiscoveryStarted, + BootstrapEvent::DiscoveryFinished { candidate_count } => { + MixnetDiagnosticsEvent::DiscoveryFinished { + candidate_count: u32::try_from(candidate_count).unwrap_or(u32::MAX), + } + } + BootstrapEvent::PullLaunched { exit_node } => { + MixnetDiagnosticsEvent::PullLaunched { exit_node } + } + BootstrapEvent::PullFailed { exit_node, error } => { + MixnetDiagnosticsEvent::PullFailed { exit_node, error } + } + BootstrapEvent::Connected { exit_node } => MixnetDiagnosticsEvent::Connected { exit_node }, + } +} + +/// Classifies what one Sentinel round trip proved, given the events captured +/// in its window. +fn classify_sentinel(evidence: ExitEvidence, window: &[Captured]) -> SentinelVerdict { + match evidence { + ExitEvidence::Answered { millis } => SentinelVerdict::ExitProven { millis }, + ExitEvidence::Silent => { + if let Some(reason) = first_error(window, CaptureSource::Socks5Core) { + SentinelVerdict::ExitRefused { reason } + } else if let Some(detail) = first_error(window, CaptureSource::GatewayClient) { + SentinelVerdict::GatewayLinkDead { detail } + } else { + SentinelVerdict::Indeterminate + } + } + } +} + +/// How one destination handshake attempt ended, before classification. +#[derive(Debug)] +enum HandshakeOutcome { + Established { + millis: u64, + }, + /// The local SOCKS5 tunnel could not open. + Tunnel { + detail: String, + }, + /// The TLS layer itself failed, in rustls's own words. + Tls { + reason: String, + }, + /// The attempt ended without a TLS-layer fault: a timeout, a closed + /// stream, or a transport error. + Ended { + detail: String, + }, +} + +/// Classifies what one destination handshake proved, given the events +/// captured in its window. +fn classify_destination(outcome: HandshakeOutcome, window: &[Captured]) -> DestinationVerdict { + match outcome { + HandshakeOutcome::Established { millis } => { + DestinationVerdict::DestinationProven { millis } + } + HandshakeOutcome::Tunnel { detail } => DestinationVerdict::TunnelFailed { detail }, + HandshakeOutcome::Tls { reason } => DestinationVerdict::HandshakeRefused { reason }, + HandshakeOutcome::Ended { detail } => { + if let Some(reason) = first_error(window, CaptureSource::Socks5Core) { + DestinationVerdict::ExitRefused { reason } + } else if let Some(gateway) = first_error(window, CaptureSource::GatewayClient) { + DestinationVerdict::GatewayLinkDead { detail: gateway } + } else { + DestinationVerdict::Indeterminate { detail } + } + } + } +} + +/// The TLS configuration every destination handshake shares: the ring +/// provider under the Mozilla-bundle verifier the workspace patch supplies. +fn tls_config() -> Result, rustls::Error> { + static CONFIG: OnceLock, rustls::Error>> = + OnceLock::new(); + CONFIG + .get_or_init(|| { + let provider = std::sync::Arc::new(rustls::crypto::ring::default_provider()); + let verifier = rustls_platform_verifier::Verifier::new(provider.clone())?; + let config = rustls::ClientConfig::builder_with_provider(provider) + .with_safe_default_protocol_versions()? + .dangerous() + .with_custom_certificate_verifier(std::sync::Arc::new(verifier)) + .with_no_client_auth(); + Ok(std::sync::Arc::new(config)) + }) + .clone() +} + +/// One deadline-free handshake attempt; the caller owns the deadline. +async fn destination_handshake(socks5: SocketAddr, host: &str, port: u16) -> HandshakeOutcome { + let started = Instant::now(); + let tunnel = match tokio_socks::tcp::Socks5Stream::connect(socks5, (host, port)).await { + Ok(tunnel) => tunnel, + Err(failure) => { + return HandshakeOutcome::Tunnel { + detail: failure.to_string(), + }; + } + }; + let config = match tls_config() { + Ok(config) => config, + Err(failure) => { + return HandshakeOutcome::Tls { + reason: failure.to_string(), + }; + } + }; + let server_name = match rustls::pki_types::ServerName::try_from(host.to_string()) { + Ok(server_name) => server_name, + Err(failure) => { + return HandshakeOutcome::Ended { + detail: failure.to_string(), + }; + } + }; + match TlsConnector::from(config) + .connect(server_name, tunnel.into_inner()) + .await + { + Ok(_established) => HandshakeOutcome::Established { + millis: u64::try_from(started.elapsed().as_millis()).unwrap_or(u64::MAX), + }, + Err(failure) => match failure + .get_ref() + .and_then(|inner| inner.downcast_ref::()) + { + Some(tls) => HandshakeOutcome::Tls { + reason: tls.to_string(), + }, + None => HandshakeOutcome::Ended { + detail: failure.to_string(), + }, + }, + } +} + +impl MixnetProxyHandle { + /// The listener address the probes tunnel through, if the handle still + /// renders one. + fn socks5_addr_for_probes(&self) -> Option { + format!("{}:{}", self.endpoint.host, self.endpoint.port) + .parse() + .ok() + } +} + +#[uniffi::export] +impl MixnetProxyHandle { + /// One Sentinel round trip through the running proxy, classified into + /// what it proved about legs one and two. + pub fn probe_sentinel(&self, deadline_millis: u64) -> SentinelVerdict { + let (Some(runtime), Some(socks5)) = (self.runtime.as_ref(), self.socks5_addr_for_probes()) + else { + return SentinelVerdict::Indeterminate; + }; + let started = Instant::now(); + let evidence = runtime.block_on(sentinel::probe_sentinel( + socks5, + Duration::from_millis(deadline_millis), + )); + classify_sentinel(evidence, &window_since(started)) + } + + /// One TLS handshake against `host:port` through the running proxy, + /// classified into what it proved about all three legs. + pub fn probe_destination( + &self, + host: String, + port: u16, + deadline_millis: u64, + ) -> DestinationVerdict { + let (Some(runtime), Some(socks5)) = (self.runtime.as_ref(), self.socks5_addr_for_probes()) + else { + return DestinationVerdict::Indeterminate { + detail: "the handle no longer runs a proxy".to_string(), + }; + }; + let deadline = Duration::from_millis(deadline_millis); + let started = Instant::now(); + let outcome = runtime + .block_on(tokio::time::timeout( + deadline, + destination_handshake(socks5, &host, port), + )) + .unwrap_or_else(|_| HandshakeOutcome::Ended { + detail: format!("no handshake within {}ms", deadline.as_millis()), + }); + classify_destination(outcome, &window_since(started)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn captured(source: CaptureSource, level: tracing::Level, message: &str) -> Captured { + Captured { + at: Instant::now(), + source, + level, + message: message.to_string(), + } + } + + /// HYPOTHESIS: a Sentinel answer proves the exit whatever the window + /// holds, and silence classifies by the window: an exit report beats a + /// gateway report, and a clean window stays indeterminate. Falsified if + /// any arm yields another verdict. + #[test] + fn sentinel_silence_classifies_by_the_captured_window() { + let exit = captured( + CaptureSource::Socks5Core, + tracing::Level::ERROR, + "network requester refused", + ); + let gateway = captured( + CaptureSource::GatewayClient, + tracing::Level::ERROR, + "websocket closed", + ); + assert_eq!( + classify_sentinel( + ExitEvidence::Answered { millis: 900 }, + std::slice::from_ref(&exit) + ), + SentinelVerdict::ExitProven { millis: 900 } + ); + assert_eq!( + classify_sentinel(ExitEvidence::Silent, &[gateway.clone(), exit.clone()]), + SentinelVerdict::ExitRefused { + reason: "network requester refused".to_string() + } + ); + assert_eq!( + classify_sentinel(ExitEvidence::Silent, &[gateway]), + SentinelVerdict::GatewayLinkDead { + detail: "websocket closed".to_string() + } + ); + assert_eq!( + classify_sentinel(ExitEvidence::Silent, &[]), + SentinelVerdict::Indeterminate + ); + } + + /// HYPOTHESIS: a non-error captured event never classifies a silence, + /// because nym narrates routine work below the error level. Falsified if + /// an info-level report yields a refusal or a dead gateway link. + #[test] + fn only_error_level_captures_classify_a_silence() { + let routine = [ + captured(CaptureSource::Socks5Core, tracing::Level::INFO, "connected"), + captured( + CaptureSource::GatewayClient, + tracing::Level::DEBUG, + "keepalive", + ), + ]; + assert_eq!( + classify_sentinel(ExitEvidence::Silent, &routine), + SentinelVerdict::Indeterminate + ); + } + + /// HYPOTHESIS: each destination outcome maps to its own verdict, and + /// only the untyped ending consults the window. Falsified if a TLS + /// fault or a tunnel fault is re-attributed by captured events. + #[test] + fn destination_outcomes_map_to_their_own_verdicts() { + let exit = captured( + CaptureSource::Socks5Core, + tracing::Level::ERROR, + "destination filtered", + ); + assert_eq!( + classify_destination(HandshakeOutcome::Established { millis: 1200 }, &[]), + DestinationVerdict::DestinationProven { millis: 1200 } + ); + assert_eq!( + classify_destination( + HandshakeOutcome::Tls { + reason: "invalid peer certificate: Expired".to_string() + }, + std::slice::from_ref(&exit) + ), + DestinationVerdict::HandshakeRefused { + reason: "invalid peer certificate: Expired".to_string() + } + ); + assert_eq!( + classify_destination( + HandshakeOutcome::Tunnel { + detail: "connection refused".to_string() + }, + std::slice::from_ref(&exit) + ), + DestinationVerdict::TunnelFailed { + detail: "connection refused".to_string() + } + ); + assert_eq!( + classify_destination( + HandshakeOutcome::Ended { + detail: "no handshake within 5000ms".to_string() + }, + &[exit] + ), + DestinationVerdict::ExitRefused { + reason: "destination filtered".to_string() + } + ); + assert_eq!( + classify_destination( + HandshakeOutcome::Ended { + detail: "connection reset".to_string() + }, + &[] + ), + DestinationVerdict::Indeterminate { + detail: "connection reset".to_string() + } + ); + } + + /// HYPOTHESIS: the capture layer records only the two nym targets, + /// carries the message text, and queues each capture for the next + /// drain. Falsified if a foreign target is captured, the text is lost, + /// or a drain misses a capture. + #[test] + fn the_capture_layer_records_the_two_nym_targets() { + use tracing_subscriber::layer::SubscriberExt as _; + let started = Instant::now(); + let subscriber = tracing_subscriber::registry().with(CaptureLayer); + tracing::subscriber::with_default(subscriber, || { + tracing::error!(target: "nym_gateway_client::socket", "gateway boom"); + tracing::error!(target: "nym_socks5_client_core::mixnet", "exit boom"); + tracing::error!(target: "some_other_crate", "unrelated"); + }); + let window: Vec = window_since(started) + .into_iter() + .filter(|captured| captured.message.contains("boom")) + .collect(); + assert_eq!(window.len(), 2, "exactly the two nym targets: {window:?}"); + assert_eq!(window[0].source, CaptureSource::GatewayClient); + assert!(window[0].message.contains("gateway boom")); + assert_eq!(window[1].source, CaptureSource::Socks5Core); + assert!(window[1].message.contains("exit boom")); + // The queue is crate-global and other tests drain it too, so only + // this test's own events are asserted, by their marker text. + let drained: Vec = drain_diagnostics() + .events + .into_iter() + .filter(|event| format!("{event:?}").contains("boom")) + .collect(); + assert_eq!( + drained, + vec![ + MixnetDiagnosticsEvent::GatewayClientReport { + level: "ERROR".to_string(), + message: "gateway boom".to_string(), + }, + MixnetDiagnosticsEvent::Socks5CoreReport { + level: "ERROR".to_string(), + message: "exit boom".to_string(), + }, + ] + ); + } + + /// HYPOTHESIS: the queue drops its oldest events on overflow and the + /// next drain reports how many were lost. Falsified if a drain after an + /// overflow keeps more than the capacity, loses order, or reports no + /// drops. + #[test] + fn an_overflowing_queue_drops_oldest_and_reports_the_loss() { + for ordinal in 0..QUEUE_CAPACITY + 3 { + publish(MixnetDiagnosticsEvent::PullLaunched { + exit_node: format!("overflow-marker-{ordinal}"), + }); + } + let drained = drain_diagnostics(); + assert!(drained.events.len() <= QUEUE_CAPACITY); + assert!( + drained.dropped_count >= 3, + "at least this test's own overflow is reported: {}", + drained.dropped_count + ); + let markers: Vec = drained + .events + .iter() + .filter_map(|event| match event { + MixnetDiagnosticsEvent::PullLaunched { exit_node } => exit_node + .strip_prefix("overflow-marker-") + .and_then(|ordinal| ordinal.parse().ok()), + _ => None, + }) + .collect(); + assert!( + markers.windows(2).all(|pair| pair[0] < pair[1]), + "the survivors keep their order: {markers:?}" + ); + assert_eq!( + markers.last(), + Some(&(QUEUE_CAPACITY + 2)), + "the newest event survives the overflow" + ); + } + + /// HYPOTHESIS: every bootstrap event crosses the FFI with its payload + /// intact. Falsified if a variant or a field is dropped or renamed. + #[test] + fn bootstrap_events_cross_the_ffi_intact() { + assert_eq!( + bootstrap_event(BootstrapEvent::DiscoveryFinished { + candidate_count: 828 + }), + MixnetDiagnosticsEvent::DiscoveryFinished { + candidate_count: 828 + } + ); + assert_eq!( + bootstrap_event(BootstrapEvent::PullFailed { + exit_node: "exit.example".to_string(), + error: "refused".to_string(), + }), + MixnetDiagnosticsEvent::PullFailed { + exit_node: "exit.example".to_string(), + error: "refused".to_string(), + } + ); + assert_eq!( + bootstrap_event(BootstrapEvent::Connected { + exit_node: "exit.example".to_string(), + }), + MixnetDiagnosticsEvent::Connected { + exit_node: "exit.example".to_string(), + } + ); + } +} diff --git a/rust/nym-proxy-ffi/src/lib.rs b/rust/nym-proxy-ffi/src/lib.rs index 2d91e2771..a57e7a00e 100644 --- a/rust/nym-proxy-ffi/src/lib.rs +++ b/rust/nym-proxy-ffi/src/lib.rs @@ -21,12 +21,15 @@ use std::{net::SocketAddr, sync::Mutex, time::Duration}; use tokio::runtime::Runtime; -use zingo_netutils::NymProxy; use zingo_netutils::time::{LISTENER_MONITOR_INTERVAL, LOOPBACK_DIAL_BOUND}; +use zingo_netutils::{BootstrapEvent, NymProxy}; #[cfg(target_os = "android")] mod debug_log; +#[cfg(feature = "diagnostics")] +mod diagnostics; + /// Consecutive check failures required before the proxy is declared dead. const LISTENER_MONITOR_STRIKES: u32 = 2; @@ -258,15 +261,12 @@ fn endpoint_from_listener_addr(addr: SocketAddr) -> Socks5Endpoint { } } -#[uniffi::export] impl MixnetProxyHandle { - /// Bring up a mixnet proxy and return, once its SOCKS5 listener is up, a - /// handle whose [`Self::socks5_endpoint`] the app hands to the wallet's - /// `attach_mixnet`, with a listener monitor that reports through - /// `observer`, at most once, if the proxy is lost. - #[uniffi::constructor] - pub fn start( + /// One start seam shared by the plain and the observed constructors, so + /// the two cannot drift. + fn start_with( observer: Option>, + on_bootstrap: impl FnMut(BootstrapEvent), ) -> Result, ProxyFfiError> { #[cfg(target_os = "android")] debug_log::init(); @@ -277,7 +277,7 @@ impl MixnetProxyHandle { reason: e.to_string(), })?; let proxy = runtime - .block_on(NymProxy::start()) + .block_on(NymProxy::start_observed(on_bootstrap)) .inspect_err(|e| tracing::error!(error = %e, "mixnet proxy start failed")) .map_err(|e| ProxyFfiError::Connect { reason: e.to_string(), @@ -301,6 +301,29 @@ impl MixnetProxyHandle { monitor, })) } +} + +#[uniffi::export] +impl MixnetProxyHandle { + /// Bring up a mixnet proxy and return, once its SOCKS5 listener is up, a + /// handle whose [`Self::socks5_endpoint`] the app hands to the wallet's + /// `attach_mixnet`, with a listener monitor that reports through + /// `observer`, at most once, if the proxy is lost. + #[uniffi::constructor] + pub fn start( + observer: Option>, + ) -> Result, ProxyFfiError> { + #[cfg(feature = "diagnostics")] + { + #[cfg(not(target_os = "android"))] + diagnostics::install_capture(); + Self::start_with(observer, |event| { + diagnostics::publish(diagnostics::bootstrap_event(event)) + }) + } + #[cfg(not(feature = "diagnostics"))] + Self::start_with(observer, |_| {}) + } /// The local SOCKS5 endpoint the app hands to `attach_mixnet`. pub fn socks5_endpoint(&self) -> Socks5Endpoint {