fix(openvpn): survive server-initiated soft reset / rekey - #3107
fix(openvpn): survive server-initiated soft reset / rekey#3107Lanlan13-14 wants to merge 25 commits into
Conversation
OpenVPN 2.6 starts a new SSL session and key epoch on P_CONTROL_SOFT_RESET_V1. The previous client reused the completed tls.Conn, always labeled data packets as key ID 0, and XOR-toggled control key IDs between 0 and 1, so the first server rekey tore the tunnel down. Each soft reset now installs a fresh tls.Conn, adopts the server key ID (0->1->...->7->1), encodes that ID on P_DATA_V1/V2, keeps the retiring data channel, carries auth-token across epochs, and preserves leftover key-method-2 / PUSH_REPLY bytes. Rekeys reuse the previous PUSH_REPLY instead of blocking on a second push that 2.6 often never sends. Verified on OpenVPN 2.6.14 with reneg-sec 10: official alpha-7ee0b05 dies on the first soft reset; this build keeps HTTP through three consecutive rekeys without restarting the process. Refs: MetaCubeX#3085
|
The current revision still has a deterministic blocker that can reproduce almost exactly the production timing reported in #3085. I do not think the PR is ready to merge yet. 1. Blocker: the 30-second rekey deadline is never cleared
mihomo/transport/openvpn/client.go Lines 459 to 494 in cede0b8
mihomo/transport/openvpn/client.go Lines 153 to 176 in cede0b8 The deadline is propagated through mihomo/transport/openvpn/control.go Lines 402 to 435 in cede0b8 The initial handshake clears its deadline, but the rekey path does not. After a successful rekey, The resulting sequence is: This is effectively the same The live test with At minimum, clear the deadline after a successful rekey, as is already done after the initial handshake: if _, err := c.doKeyExchange(renegCtx); err != nil {
return fmt.Errorf("rekey exchange: %w", err)
}
_ = c.tlsConn.SetDeadline(time.Time{})
return nilPlease also add a regression test that verifies the control-channel deadline is zero after rekey, or keeps the client idle for longer than 2. Rekey auth-token refresh is described but not implementedThe rekey branch in mihomo/transport/openvpn/client.go Lines 220 to 241 in cede0b8 It does not call OpenVPN explicitly sends a renewed token as an additional minimal https://github.com/OpenVPN/openvpn/blob/v2.6.20/src/openvpn/auth_token.c#L463-L482 https://github.com/OpenVPN/openvpn/blob/v2.6.20/src/openvpn/push.c#L754-L771 After The PR description should not claim refreshed-token support until the active TLS stream remains consumable after key exchange and the token-only push is parsed and installed. If auth-token support is intentionally out of scope, it would be cleaner to remove those unrelated changes and claims from this PR. 3. The shortened key-method-2 parser can truncate a standard fragmented record
mihomo/transport/openvpn/keymethod.go Lines 95 to 145 in cede0b8 TLS is a byte stream. OpenVPN 2.6.20's https://github.com/OpenVPN/openvpn/blob/v2.6.20/src/openvpn/ssl.c#L2263-L2320 The corresponding reader also always extracts username, password, and peer-info: https://github.com/OpenVPN/openvpn/blob/v2.6.20/src/openvpn/ssl.c#L2419-L2435 Consequently, a normal record fragmented immediately after the options string can be accepted prematurely. Its remaining bytes will then be treated as following TLS control data and can make PUSH parsing wait until timeout. If compatibility with a proven shortened record is required, only terminate early when the remaining bytes are positively identified as a following message such as 4. Soft-reset key ID validation accepts stale or invalid epochs
mihomo/transport/openvpn/control.go Lines 337 to 349 in cede0b8 That allows a delayed reset from a retiring epoch to move the client backwards. It also accepts key ID The server supplies the new key ID, but the client should still validate it: expected := NextKeyID(c.keyID)
return packet.KeyID == expected, packet.KeyID == expectedThe current test that expects key ID 5. UDP control packets are not retransmitted during rekeyThe client soft reset, TLS ClientHello, and subsequent TLS control records are inserted into mihomo/transport/openvpn/client.go Lines 531 to 557 in cede0b8 There is no retransmission loop during This may be separated from the minimal no-loss fix, but the PR should not claim reliable UDP rekey handling without a retransmission path and a packet-loss regression test. |
Review: fix(openvpn): survive server-initiated soft reset / rekeyThanks for the PR — I verified the changes against the OpenVPN 2.6 reference source ( Correct: key ID adoption with the 0→1→…→7→1 wrap (skipping 0), replying with a soft reset as the first packet of the new epoch, retiring/lame-duck data-channel routing by key ID, a fresh TLS session per epoch, and standard base64 decoding of Blocking1. Control-channel deadline is never cleared after a rekey — High2. Auth-token renewal during renegotiation is dropped — 3. No client→server retransmission during rekey (UDP only) — Medium4. peer-id is not carried across a rekey — 5. Data races — Minor6. Error wrapping pollution — 7. Dead code — several fields/accessors have no production callers: Happy to re-review once the deadline issue (and ideally the auth-token / UDP gaps) are addressed. |
Address review of the soft-reset/rekey path: - Clear the shared control-channel deadline after each renegotiation so waitForSoftReset is not torn down ~30s later (default reneg-sec is 3600). - Consume token-only PUSH_REPLY after rekey (leftover TLS, short read, or a P_CONTROL_V1 parked by the watcher) so the next key-method-2 auth uses the renewed auth-token instead of an expired one. - Inherit peer-id across rekey; mergePushReply cannot, because a zero PeerID is 0 not the PeerIDUnset sentinel. - Retransmit unacked control messages for the duration of a UDP rekey. - Store lastRekeyErr / tlsConn atomically; wrap ReadIPPacket with the rekey error only when the mux is closed. - Parse a shortened key-method-2 record only when the tail is a following TLS control message; do not accept a fragmented standard record early. - Drop unused lastDataKey / tlsEpoch / lastAuthToken accessors. Verified against OpenVPN 2.6.14: reneg-sec 45, auth-gen-token 0 180, two soft resets, 101/101 HTTP probes, token auth succeeded on both renegotiations; go test -race ./transport/openvpn/ clean.
|
The original 1. an ordinary control read can park a stale soft reset without validating the next key IDThe watcher path now correctly requires a soft reset to use mihomo/transport/openvpn/control.go Lines 303 to 315 in bee4126 This branch parks every same-session soft reset whose key ID merely differs from the current key ID. mihomo/transport/openvpn/control.go Lines 205 to 212 in bee4126 A focused in-memory test consistently reproduces the following sequence: This can move the client backwards to a retiring key epoch. It does not require an invalid key ID or a broken server; a delayed or retransmitted UDP packet is sufficient. The minimal fix is to apply the same next-key check before parking the packet: expected := NextKeyID(curKey)
if packet.Opcode == PControlSoftResetV1 &&
packet.KeyID == expected && sameSession {
// park the reset
}An invalid soft reset should not mutate ACK, pending-message, receive-sequence, or epoch state. 2. a token-only PUSH_REPLY split across two TLS reads is discarded
mihomo/transport/openvpn/client.go Lines 712 to 733 in bee4126 Additionally, mihomo/transport/openvpn/client.go Lines 308 to 315 in bee4126 I reproduced this with an in-memory TLS connection by writing the message in two parts: The helper returns The caller then discards that buffer and continues using the old token. TLS exposes a byte stream, so this fragmentation is valid and must not depend on packet or record boundaries. The minimal fix is to parse after every successful read, including the final allowed read, and always preserve buffered bytes when no complete message has been found. A cleaner interface would return 3. AUTH_FAILED during a rekey is treated as an absent tokenThe token helper explicitly converts mihomo/transport/openvpn/client.go Lines 721 to 723 in bee4126
mihomo/transport/openvpn/client.go Lines 736 to 747 in bee4126 Because mihomo/transport/openvpn/client.go Lines 232 to 255 in bee4126 A focused in-memory test with: consistently leaves the old token installed and reports no rekey error. This turns an explicit authentication failure into an apparently successful rekey followed by unexplained data loss or a later control-channel failure. The minimal fix is for the rekey push-consumption path to return an error when 4. retransmitting the client soft reset removes its original reset ACKThe first client soft reset carries the ACK for server soft-reset message 0. mihomo/transport/openvpn/control.go Lines 404 to 421 in bee4126 The original send has already cleared OpenVPN keeps recently transmitted ACKs in https://github.com/OpenVPN/openvpn/blob/v2.6.20/src/openvpn/reliable.c#L252-L304 The peer can sometimes recover by retransmitting its reset and causing another ACK, so this is less direct than the failures above. It nevertheless weakens the reliability mechanism added by this PR and can leave the server repeatedly retransmitting the reset. At minimum, retransmission should merge the stored packet's ACKs with current pending ACKs instead of replacing the stored ACK list. 5. the ClientHello loss test does not actually require a retransmission
mihomo/transport/openvpn/rekey_test.go Lines 402 to 473 in bee4126 It never records and ignores the first occurrence, nor does it require a second packet with the same message ID and payload. The test therefore passes even if the retransmission loop never retransmits ClientHello. The test should explicitly consume and discard the first ClientHello, then pass only after receiving the same reliable control message a second time. A second test should drop the first client soft reset and verify that its retransmission still acknowledges the server reset. |
…t ACK Round-2 review fixes on the rekey path: - Ordinary control read now validates the parked soft reset uses NextKeyID(current); a delayed reset from a retiring epoch is dropped without mutating ACK / pending / receive state. - readTokenPushReply parses after every read (including the last) and preserves partially-received bytes; a token-only PUSH_REPLY split across two TLS reads is no longer discarded. - AUTH_FAILED during a rekey is a hard error (errAuthFailed) and aborts doKeyExchange before a new data channel is installed, instead of being treated as "no token". - RetransmitPending merges the stored packet's ACKs with current pending ACKs, so a retransmitted client soft reset still acknowledges the server's reset. - Tests now require a real retransmission (second copy, same message ID) and add regression coverage for each case above. Live: OpenVPN 2.6.14, reneg-sec 45, auth-gen-token 0 180, two soft resets, 101/101 HTTP probes, token auth succeeded on both renegotiations, no TLS Error / link closed. go test -race ./transport/openvpn/ clean.
|
Two error-handling issues remain in the new token-consumption path. 1.
|
- readTokenPushReply processes bytes before err (io.Reader may return n>0 with io.EOF when app data is followed by close_notify), only treats a real timeout as "no token yet", and propagates EOF / unexpected EOF / TLS errors so a rekey never proceeds on a dead control stream or loses a bundled AUTH_FAILED. - watchControl propagates consumeRekeyPush errors: a parked AUTH_FAILED (deferred auth) aborts the tunnel instead of being replaced by the next renegotiation result. - ControlChannel now keeps an ACK MRU (lru_acks) mirroring OpenVPN reliable_ack_write: queued ACKs ride on this and subsequent control packets until replaced, and RetransmitPending emits one merged ack set. Live: OpenVPN 2.6.14, reneg-sec 45, auth-gen-token 0 180, two soft resets, two token auths, 101/101 probes, no TLS Error / link closed. go test -race ./transport/openvpn/ clean.
Thanks for the detailed review — all five points are real and I've fixed them. Round 3 of fixes is in 1. Ordinary control read parks a stale soft resetFixed in Test: 2. Token-only PUSH_REPLY split across two TLS reads is discardedFixed in
Tests: 3. AUTH_FAILED during a rekey treated as absent token
Tests: 4. Retransmitting the client soft reset loses its original reset ACK
Test: 5. ClientHello loss test didn't actually require a retransmission
Reliability layer alignment
The one deliberate difference: retransmission interval is 1s here vs the reference's Verification
|
|
The earlier rekey failure chain therefore appears to be fixed. However, one error-handling edge case remains, and the newly added ACK MRU introduces two protocol-compatibility issues. 1. a complete
|
takeAcksLocked treated the MRU as a deduplicated list: an ACK ID already present was skipped, so on a full MRU a re-acked ID (peer retransmitted that reliable packet) could be evicted. Reimplement as OpenVPN's copy_acks_to_mru: every pending ID is unconditionally moved to the front, shifting existing entries right and dropping the duplicate, so a re-requested ACK stays in the MRU. MRU [1..8], pending [8 9] now yields [8 9 1 2 3 4 5 6] exactly as the reference. TestMRUMoveToFrontMatchesReference pins the reference example. go test -race clean; live reneg-sec 45 / auth-gen-token 0 180: 2 soft resets, 2 token auths, 101/101, no errors.
MRU move-to-front. You're right — |
|
One substantive compatibility issue remains in the ACK MRU implementation. mihomo/transport/openvpn/control.go Lines 161 to 177 in 55744f6 mihomo/transport/openvpn/control.go Lines 196 to 239 in 55744f6
This does not preserve the reference client's per-packet limits:
mihomo supports configurations without A deterministic in-memory test queued and sent five ACKs through a plain The test reproduced this for ten consecutive runs. A SoftEther peer can discard that packet. When it retransmits the same reliable message, the MRU causes the client to send another packet containing more than four ACKs, so the control channel can remain stuck. The minimal fix is to separate MRU capacity from the number of ACKs serialized into a packet:
|
takeAcksLocked always serialized up to RELIABLE_ACK_SIZE (8) ACKs into every packet. OpenVPN limits ACKs per packet type: reliable control packets (including retransmissions) carry at most CONTROL_SEND_ACK_MAX (4); a dedicated ACK carries up to RELIABLE_ACK_SIZE (8), but only 4 when the channel is unprotected (TLS_WRAP_NONE, no tls-auth/tls-crypt) for SoftEther compatibility. Separate MRU capacity (8) from per-packet serialization: - Send / RetransmitPending: CONTROL_SEND_ACK_MAX (4) - SendAck: dedicatedAckMax() = 8, or 4 when c.crypt == nil (mihomo does not advertise TLS key-material export) TestAckSerializationCaps covers reliable-control / dedicated-ack / retransmit across protected and plain channels. go test -race clean; live reneg-sec 45 / auth-gen-token 0 180: 2 soft resets, 2 token auths, 101/101, no errors.
readTokenPushReply returned success as soon as the buffered bytes parsed into a complete reply, even when that read also returned io.EOF (tls.Conn returns (n, io.EOF) when app data is immediately followed by close_notify). The closed TLS stream was converted into a successful rekey. Now a complete reply is retained and only returned on a real timeout; EOF / unexpected EOF / TLS errors are propagated, with AUTH_FAILED keeping precedence for its diagnostic. TestReadTokenPushReplyCompleteReplyWithEOF pins the case. go test -race clean; live reneg-sec 45 / auth-gen-token 0 180: 101/101, 2 soft resets, 2 token auths, no errors.
All three are real — addressed in 1. Complete PUSH_REPLY suppresses EOF. 2. ACK MRU exceeds the four-ACK limit. Separated MRU capacity (8) from per-packet serialization: reliable control packets (incl. retransmits) carry ≤ 3. Existing ACKs not moved to the front. Verification: |
|
The original 1. capped ACK writes discard pending IDs that were not serializedmihomo/transport/openvpn/control.go Lines 197 to 226 in 28c2622
For example: OpenVPN's https://github.com/OpenVPN/openvpn/blob/v2.6.20/src/openvpn/reliable.c#L252-L304 The expected state after the first packet is therefore: Without that retention, ACK 5 is not sent on the next packet unless the peer retransmits message 5 and causes it to be queued again. This creates an unnecessary retransmission and does not match the reference reliability behavior. The new serialization-cap test queues five ACKs but only verifies that the emitted count is between one and the configured maximum: mihomo/transport/openvpn/rekey_test.go Lines 912 to 1052 in 28c2622 It does not verify that the fifth ACK remains pending and appears in a subsequent packet. A deterministic package-level regression test for that state failed ten consecutive runs on the current head. The minimal fix is to consume only the prefix that can be serialized: n := min(len(c.ackPending), max)
// Move only c.ackPending[:n] into the MRU.
c.ackPending = c.ackPending[n:]The test should assert the exact first packet and then verify that the next packet contains ACK 5. 2. complete token handling now depends on read-ahead boundariesmihomo/transport/openvpn/client.go Lines 746 to 790 in 28c2622 After reconsidering the The latest change now produces two different results for the same complete logical message: The fast path parses
The practical impact is lower because OpenVPN 2.6.20 frees an old OpenSSL key state with https://github.com/OpenVPN/openvpn/blob/v2.6.20/src/openvpn/ssl_openssl.c#L1963-L1975 Nevertheless, the current behavior is read-boundary-dependent and can abort an otherwise complete rekey exchange with a peer that closes the inner TLS epoch after sending the token. |
…h EOF - takeAcksLocked consumed the entire pending ACK list into the MRU and cleared it, dropping any ACKs beyond the per-packet cap. Now it moves only ackPending[:max] into the MRU and keeps ackPending[max:] pending for the next packet, matching reliable_ack_write. The fifth of five ACKs on a capped packet is no longer lost. - Revert the over-correction from 28c2622: a complete token PUSH_REPLY returned together with io.EOF is accepted (the bytes are valid and must be processed before the terminal error), so success does not depend on read-ahead boundaries. EOF propagates only when no complete message was obtained; AUTH_FAILED keeps precedence. Tests: TestAckCapRetainsUnsentPending (first packet [1 2 3 4], second carries retained ACK 5), TestReadTokenPushReplyCompleteReplyWithEOF (reply + io.EOF accepted). go test -race clean; live reneg-sec 45 / auth-gen-token 0 180: 101/101, 2 soft resets, 2 token auths, no errors.
I went through the OpenVPN 2.6.20 reference client ( What was copied from the referenceKey id. ACK MRU.
Per-packet ACK caps. From
Dedicated ACK trigger. Token / AUTH_FAILED. Complete reply + EOF. Immediate dedicated ACK. What was left as-is, and whyThese are not specified by those functions:
Verification
|
|
I re-reviewed the current head from the base commit, including the control-channel state machine, ACK handling, TLS/key-method framing, authentication-token flow, and data-channel epoch selection. The original 1. Late
|
- waitForSoftReset returns errParkedTLS when it parks a same-epoch P_CONTROL_V1 (token update / deferred AUTH_FAILED); watchControl consumes it right away instead of waiting for the next soft reset. A late AUTH_FAILED is now surfaced immediately rather than hidden until the next rekey (which may never come after an auth failure). - Retiring data epoch now expires after transitionWindow (3600s), mirroring OpenVPN's lame-duck must_die; packets labeled with an expired retiring key are rejected. - decryptDataPacket routes strictly by key ID: an unknown epoch is rejected instead of falling back to the current key (CBC HMAC excludes the outer opcode/key-ID header, so wrong-key "authentication" was possible). - RetransmitPending returns early when there are no pending packets, so an empty retransmit no longer consumes queued ACKs. Tests: TestWaitForSoftResetParksLateControlPayload (now expects errParkedTLS), TestRetransmitPendingEmptyDoesNotConsumeAcks, TestDecryptRejectsUnknownKeyID, TestDecryptRejectsExpiredRetiringKey, TestDecryptAcceptsRetiringKeyBeforeExpiry. go test -race clean; live reneg-sec 45 / auth-gen-token 0 180: 101/101, 2 soft resets, 2 token auths, no rekey-time errors.
Address the two pre-existing observations in review. - A new-epoch soft reset is always message 0 (the fresh reliable layer starts at 0). classifyWatchPacketLocked now rejects a next-epoch reset whose MessageID is not 0, so an invalid ID can no longer advance the receive sequence via MarkReceived. - takePushReply no longer commits a PUSH_REPLY as soon as "ifconfig" is present: it now requires the NUL terminator, so a reply fragmented across TLS reads is parsed only once complete and later route / DNS / cipher / token options are not lost. Matches the reference client, which parses the full PUSH message after it is fully received. Tests: TestClassifyWatchAcceptsNextEpochSoftReset (rejects msg id 5), TestTakePushReplyRequiresTerminator. go test -race clean; live reneg-sec 45 / auth-gen-token 0 180: 101/101, 2 soft resets, 2 token auths.
All six points in your review are addressed — the four merge-blockers in Merge-blockers (
|
|
I found three remaining regressions in the new rekey implementation and one incomplete validation fix. I do not think the PR is ready to merge yet. 1. A successful late token read leaves a 300 ms read deadline activeThe new mihomo/transport/openvpn/client.go Lines 491 to 507 in 4cbff69
mihomo/transport/openvpn/client.go Lines 793 to 823 in 4cbff69 This did not normally break the in-renegotiation call because mihomo/transport/openvpn/control.go Lines 550 to 561 in 4cbff69 The resulting sequence is: A deterministic in-memory reader test confirmed that the last installed deadline remains non-zero after a successful token read. The helper, or the new watcher branch, must clear the temporary deadline on every return path. A regression test should cover a successful delayed token followed by an idle established connection. 2.
|
…piry Address four regressions from review. - readTokenPushReply now clears the temporary 300 ms read deadline on every return (defer), so the errParkedTLS path cannot leave a stale absolute deadline that tears down the idle established connection. - ReadAll emits parkedTLS before subsequently contiguous recvPending: parked payloads were received out of order and already advanced recvMessage, so they must feed the TLS stream first after UDP reordering. - decryptDataPacket checks the retiring-epoch expiry before the dataByKey lookup, so a map hit cannot bypass the transition-window expiration. - waitForSoftReset defensively revalidates a parked soft reset (next-epoch + message 0) on consume, and the ordinary-read park path now also requires message 0. Tests: TestReadTokenPushReplyClearsDeadlineOnSuccess/OnError, TestReadAllEmitsParkedBeforeContiguous, TestWaitForSoftResetRejectsInvalidParkedReset. go test -race clean; live reneg-sec 45 / auth-gen-token 0 180: 101/101, 2 soft resets, 2 token auths, no errors.
All four regressions are addressed in 1. Successful late token read leaves a 300 ms read deadline active
2.
|
|
I reviewed the current PR head from the issue report and the OpenVPN 2.6.20 state machine, without carrying forward earlier review conclusions. The core correction is valid: an OpenVPN soft reset starts a fresh SSL/key state, and the PR now uses the peer's next key ID in both control and data headers. However, I still see two blocking issues in the delayed-authentication path. 1. Keep the retiring key active for outbound data during deferred authenticationThe rekey branch installs the new data channel immediately after receiving the server key-method-2 record: After OpenVPN assigns every new key state an With a management or plugin-based deferred authenticator, the server can send its KM2 record while the new key is still unauthenticated. Mihomo then starts sending new-key data after at most the short token probe, but the server has not generated that data key yet and drops those packets. This creates an asymmetric outage during each rekey and can break active connections when authentication takes more than a trivial amount of time. Please separate "new key available for eventual activation/decryption" from "key selected for outbound encryption." The old key must remain the outbound key for the OpenVPN deferred-auth grace period, or until there is reliable evidence that the peer has activated the new key. 2. Parse the TLS control stream as NUL-delimited messages before looking for a token reply
OpenVPN deferred authentication sends For this valid stream: the current code finds Please extract and consume complete NUL-delimited control messages in order. Handle |
…ntrol parse
Two correctness issues from review, both in the delayed-authentication path.
1. Outbound key was switched to the new epoch immediately after a rekey,
which is not OpenVPN's key selection. tls_select_encryption_key only
selects a key for outbound encryption once it is KS_AUTH_TRUE; during
deferred authentication the server sends its key-method-2 record before
generating its data key, so immediate switch-out sends packets the
server drops ("not authorized (deferred)"). Decryption can still use the
new key immediately (the peer may label packets with it), so outbound
key selection is now a separate slot: it keeps the lame-duck epoch until
a packet labeled with the new key ID decrypts successfully (the peer
only labels outbound packets with an authenticated key), with a
just-before-expiry backstop for asymmetric traffic.
2. takePushReply parsed only the bytes before the first NUL, so the
deferred-auth stream AUTH_PENDING\0INFO_PRE,...\0PUSH_REPLY,...\0 was
stuck parsing AUTH_PENDING and never reached the token. Control messages
are now extracted as complete NUL-delimited messages in order
(splitControlMessages), matching forward.c check_incoming_control_channel;
AUTH_FAILED wins, PUSH_REPLYs are merged, other messages consumed, only
the trailing incomplete message is retained.
Tests: TestOutboundKeyStaysLameDuckUntilPeerEvidence,
TestTakePushReplyAuthPendingBeforePush, TestReadTokenPushReplyDeferredAuth,
and TestWatchControlSurfacesParkedAUTHFailed now genuinely injects
AUTH_FAILED (it previously only exercised the no-TLS renegotiation error).
go test -race clean; live reneg-sec 45 / auth-gen-token 0 180: 101/101, no
errors. Deferred-auth management server: old and new binaries both ride the
lame-duck through rekey windows — with a valid auth-token OpenVPN skips
management deferral (skip_auth, ssl_verify.c:1653), so the outbound-key fix
is defensive alignment with the state machine, not a triggerable regression.
|
The NUL-delimited control-message problem from the previous review is fixed. However, I still do not think this revision is ready to merge. The new outbound-key selection has three correctness problems, and the shortened key-method-2 path still cannot enter the newly supported deferred-auth control flow. 1. A packet that fails decryption still promotes the new outbound key
Therefore, the code does not implement the condition stated by its comment ("decrypted successfully"). Any malformed packet carrying the current key ID sets the latch even when header validation, AEAD authentication, CBC HMAC verification, or replay validation fails. The next outbound write then switches away from the lame-duck key. This is especially concrete for CBC because the outer opcode/key-ID byte is not covered by its HMAC: changing an old packet's header to the new key ID makes decryption fail under the new key, but still promotes that key. During deferred authentication, that recreates the behavior this commit is intended to prevent: the server has not generated/authorized the new data key and drops the client's new-key packets. A pure in-memory regression test using a one-byte The promotion must happen only after cryptographic validation succeeds. It must also re-check 2. A valid old epoch is discarded for outbound selection if it has not sent a packet
The only initial-handshake case is The minimal in-memory sequence An existing old data channel should remain the outbound candidate regardless of its packet counter. 3. The asymmetric-traffic backstop uses the wrong OpenVPN window
That is not how OpenVPN selects an encryption key. OpenVPN assigns the new key an With defaults, the selection transition is around 60 seconds after the new epoch starts, not 60 seconds before the old key's 3600-second destruction time. The current policy makes a one-way tunnel use the retired key for almost the entire transition window, defeats the expected outbound key rotation, and can black-hole traffic much earlier when the peer uses a shorter Peer evidence can be an early positive signal, but the no-evidence deadline needs to model 4. A shortened KM2 record followed by
|
…rite Two issues found in a fresh review of the OpenVPN client against the 2.6.20 state machine. 1. newKeyEvidence lived on the Client, latched after decrypting a packet labeled with the newest key ID. The check `alt == c.data` ran under RLock, but the latch was applied after releasing it, so a rekey landing in between could attribute an older epoch's evidence to the newest key and promote the outbound key early across a back-to-back rekey. Move the evidence onto the DataChannel itself (MarkPeerActive / PeerActive, guarded by d.mu): each epoch carries its own activation state, so a subsequent rekey can never misattribute it. 2. encryptCBC wrote the HMAC via `_ = hmacAppend(..., out[len(header):len(header)])`, relying on mac.Sum(dst) appending into dst's backing array. Replace with an explicit hmacCopy that always writes the tag into dst, so the layout does not depend on append's capacity semantics. Verified: go test (1.23 + 1.25), remote go test -race clean; live reneg-sec 45 / auth-gen-token 0 180: 101/101, 2 soft resets, 2 token auths, no errors.
…rge, read-lock writes Address the three follow-ups from the OpenVPN client review against 2.6.20. 1. tls-auth / tls-crypt protected control packets carry [packet-id][timestamp] but mihomo never validated them against replay. Add an anti-replay window (replayState) mirroring OpenVPN packet_id_test backtrack mode: ids must advance within the current second, replays and timestamp backtracking are rejected, a new second resets the window. A valid next-epoch soft reset (classifyWatchPacketLocked / park path) seeds a fresh window since the server restarts its sequence. Active only when a ControlCryptor is set. 2. mergePushReply did not carry DNS / Redirect / BlockIPv6 across multi-segment PUSH_REPLY (push-continuation) streams, so a continuation segment lacking those fields would drop them. Merge them like the other fields. 3. writeDataPacket took the full dataLock write lock on every packet, serializing the data plane against reads. Use a read lock for the common path and upgrade to the write lock only when the outbound key needs promoting (re-checking under the write lock). Tests: TestCheckReplayAntiReplay (advance/replay/stale/timestamp-backtrack/ new-second/reset), TestMergePushReplyContinuation (DNS/PeerID/Cipher/prefix/ routes/redirect inheritance). go test 1.23 + 1.25 clean; remote -race clean; live reneg-sec 45 / auth-gen-token 0 180: 101/101, no errors.
The 1:1 comparison against OpenVPN 2.6.20 reliable.c surfaced that rec_reliable is a fixed RELIABLE_CAPACITY=12 buffer, and reliable_can_get / reliable_wont_break_sequentiality refuse packets that would overflow it. mihomo's recvPending was an unbounded map: a peer flooding control packets with arbitrarily high message IDs could grow it without limit (a DoS when no tls-auth/tls-crypt is present, since there is no pid/timestamp replay protection on the bare channel). Add recvWindowOK mirroring the OpenVPN window: an out-of-order packet is buffered only if its message ID is strictly ahead of recvMessage, within reliableCapacity of it, and the buffer is not already full. Replays are still ACKed regardless. Test: TestRecvPendingBounded. go test 1.23 + 1.25 clean; remote -race clean; live reneg-sec 45 / auth-gen-token 0 180: 101/101, no errors.
…2 boundary Address four review points on the outbound-key selection and the shortened key-method-2 record boundary. 1. Failed decryption no longer promotes (already fixed in 9e48a28): peer evidence is latched only after Decrypt succeeds, per-epoch. Regression test added (one-byte P_DATA_V2 with the new key id must not latch). 2. installDataChannel kept the old outbound key only when old.Started(), but the send counter says nothing about epoch validity: a quiet / receive-only tunnel never sends on the old key, so a server-initiated rekey selected the new key immediately. Keep the previous epoch as the outbound candidate whenever old != nil (only the first handshake starts on the new key), matching key_state_soft_reset + tls_select_encryption_key. 3. The no-evidence backstop used retiringExpiry - 60s (keeps the retired key for ~3540s). OpenVPN selects the new key within auth_deferred_expire = min(handshake_window, reneg/2) (~60s), so promote once outboundStart + authDeferredExpire elapses, with peer evidence as an earlier signal. 4. looksLikeFollowingTLSControl recognized only PUSH_REPLY / AUTH_FAILED / PUSH_REQUEST, so a shortened KM2 followed by AUTH_PENDING / INFO_PRE (the deferred-auth flow) was never recognized and the read loop stalled. Recognize every server-to-client command that can follow a shortened record (AUTH_PENDING, INFO_PRE, INFO, RESTART, HALT, EXIT, CR_RESPONSE). Tests: TestRekeyKeepsOldOutboundEvenIfNeverSent, TestFailedDecryptDoesNotPromote, TestOutboundPromotesAfterAuthDeferredExpire. go test 1.23 + 1.25 clean; remote -race clean; live reneg-sec 45 / auth-gen-token 0 180: 101/101, no errors.
|
The latest commit correctly fixes two previously reported points: an existing but unused old epoch is now retained as the outbound candidate, and the shortened-KM2 boundary recognizes the additional control commands. The core fix for #3085 also remains directionally correct: each soft reset creates a fresh TLS/key epoch, the peer's key ID is adopted, data headers use that key ID, and the original rekey error is preserved. However, I still do not think the PR is ready to merge. 1. the outbound-key backstop ignores
|
… don't ACK dropped packets
Address three review points on the deferred-auth path.
1. AUTH_PENDING,timeout N was consumed and discarded, so the fixed 60s
no-evidence backstop could promote the new outbound key while the peer
was still KS_AUTH_DEFERRED, and the server would drop those packets
("not authorized (deferred)"). Parse the advertised timeout into
PushReply.AuthPendingTimeout (parseAuthPendingTimeout) and record it as
a per-epoch deadline (applyAuthPendingTimeout / deferredUntil). The
writeDataPacket backstop never promotes before that deadline, falling
back to authDeferredExpire only when no timeout was advertised.
2. PUSH continuation was incomplete: takePushReply treated any complete
PUSH_REPLY as final, so an intermediate push-continuation 2 segment
returned early and remaining segments were lost; and mergePushReply
replaced repeatable fields (routes/dns/data-ciphers) instead of
appending them in wire order. takePushReply now tracks continuation
state (PushContinuation parsed by parsePushReplyInner) and returns
incomplete until the final segment; callers accumulate intermediate
segments across reads (readPushReply / readTokenPushReply /
consumeRekeyPush) via pushPending; mergePushReply appends repeatable
fields deduplicated. Extended to AUTH_PENDING too, so the timeout
survives across reads.
3. The recvPending bound ACKed packets it dropped: ackPending was appended
and sendAck set before the window decision, so an out-of-window packet
was dropped locally yet acknowledged, letting the sender delete it and
leave a permanent reliable hole. ACK now happens only for an accepted
in-window packet or an in-window replay, matching OpenVPN read_control_auth.
Tests: TestAuthPendingTimeoutRespected, TestTakePushReplyContinuation,
TestOutOfWindowPacketNotAcked. go test 1.23 + 1.25 clean; remote -race
clean; live reneg-sec 45 / auth-gen-token 0 180: 101/101, no errors.
|
I re-reviewed the PR from its base through the current head, including the latest commit ( I still do not think the PR is ready to merge. The latest implementation leaves the following protocol issues. 1.
|
Address the three latest review findings end to end. Standalone AUTH_PENDING now updates metadata without completing PUSH parsing; its timeout survives coalesced merges, is staged by key ID before data-channel install, transferred only to the matching epoch under dataLock, and extends control reads. PUSH continuation preserves prev-to-next wire order for routes/DNS/ciphers and survives across calls until a final segment. Already-buffered in-window duplicates are re-ACKed without reinsertion, while out-of-window packets remain unacknowledged. Add focused production-order and deterministic regression tests for all reported failures. Verified Go 1.23/1.25, remote -race, and live 101/101 with two rekeys.
|
The latest commit does fix the previously reported out-of-window ACK behavior, buffered duplicate re-ACK behavior, continuation wire order/cross-call accumulation, and the pre-buffered 1. mihomo does not advertise the
|
Advertise IV_PROTO_AUTH_PENDING_KW (IV_PROTO=22) now that keyword parsing is implemented, and give bare AUTH_PENDING the OpenVPN handshake-window fallback. When readTokenPushReply discovers AUTH_PENDING or push-continuation inside its own read loop, immediately promote the active wait policy instead of returning after the short token probe. AUTH_PENDING waits through its advertised timeout and continuation waits through the rekey window. Add deterministic message-timeout-final tests for both paths and update peer-info expectations. Verified Go 1.23/1.25, remote -race, and live 101/101 with two rekeys.
|
I re-reviewed the PR at the current head, including The latest commit fixes the two issues from my previous review at the parser/helper level: mihomo now advertises There is still one blocking deadline-propagation problem, plus a related timeout-anchor problem. 1. dynamically extending only the read deadline leaves reliable ACK writes on the expired 30-second deadline
That distinction matters because this is a reliable control channel. A deterministic in-memory The new regression test uses 2. the advertised timeout is restarted when the final push arrivesThe helper retains For example, an in-memory reader receiving OpenVPN stores an absolute push timeout for the current key state when it processes |
|
The latest commit correctly fixes both findings from my previous review: dynamically observed I found one remaining lifecycle bug and one timeout-update issue. 1. the parked-TLS success path leaves the operation deadline on the established connection
However, A deterministic in-memory reproduction invokes the same successful parked consume with: and observes: The parked branch needs the same clear-on-operation-exit ownership as 2. a later
|
|
The latest commit fixes the previously reported deferred-auth deadline lifecycle issues, but one correctness issue remains at head Preserve a continuation discovered by the final probe
When Consequently, if no I reproduced this deterministically with an in-memory reader result equivalent to: &PushReply{
HasPushReply: true,
PushContinuation: 2,
}
The minimal fix is to update the persistent continuation state whenever if more.HasPushReply {
c.pushContinuationPending = more.PushContinuation == 2
}This should be covered by a regression test where the intermediate continuation is discovered by the reader and the final segment never arrives. The existing dynamic-continuation test always supplies a final segment, so it does not exercise this exit path. |
Do not require a final
|
Install the new receive epoch before waiting out a standalone
|
Do not keep transmitting with a retiring key past its usable lifetime
This is observable with a valid deferred-auth configuration:
Those packets are dropped for the interval between old-key deletion and The two deadlines are separate in upstream OpenVPN. Please keep This behavior is introduced by this PR: the base version had no split |
Start the retiring-key window when the soft reset is acceptedmihomo/transport/openvpn/client.go Lines 848 to 851 in 5f2f57f
mihomo/transport/openvpn/client.go Lines 800 to 812 in 5f2f57f When no late token is available, that probe may wait for the full rather than: Upstream OpenVPN sets This matters now that the PR exposes short A deterministic in-memory reproduction used a no-data Please capture or install the retiring deadline immediately after This timing defect is introduced by the PR's split outbound/retiring-key lifecycle; it is not a pre-existing Alpha behavior. |
Fixes #3085.
OpenVPN 2.6 starts a new SSL session and a new key epoch on
P_CONTROL_SOFT_RESET_V1. It does not renegotiate TLS on the existing SSL object. Upstream OpenVPN allocates a new SSL object per key state and has TLS-layer renegotiation disabled.The previous mihomo client treated soft reset as TLS renegotiation on the already-completed
tls.Conn.HandshakeContextreturns immediately when the handshake is already done, so no ClientHello is sent. It then continued the key-method exchange on the old TLS state. Combined with a data-channel header that always used key ID0, and a control-channel key ID that XOR-toggled0 ↔ 1instead of following OpenVPN’s0 → 1 → 2 → 3 → 4 → 5 → 6 → 7 → 1, the first server-initiated rekey tore the tunnel down.On OpenVPN 2.6.14 with
reneg-sec 10the server log is:The mihomo process stays up. Only the OpenVPN transport closes (
use of closed network connection). Active connections drop until a full hard reset / new handshake. The original rekey error was discarded inwatchControl(), so production logs never showed whether TLS, key-method, PUSH, or token refresh failed.What each soft reset does now
P_CONTROL_SOFT_RESET_V1as a new key epoch. Adopt the key ID from that packet (0 → 1 → … → 7 → 1). Do not invent0 ↔ 1.MarkReceived). The server reset is new-epoch message0; without this, ServerHello (message1) sits inrecvPendingforever and the TLS handshake times out.tls.Connover the same control channel and run a full initial handshake. Do not callHandshakeContexton the old conn. Do not send TLSclose_notifyon the old epoch (that would go out under the new key ID and pollute it).P_DATA_V1/P_DATA_V2header.PUSH_REPLYorAUTH_FAILEDis not discarded.auth-token/auth-token-userinto the next epoch. Capture a refreshed token when the server sends one; keep the previous token when it does not.PUSH_REPLY(ifconfig / peer-id / cipher). OpenVPN 2.6 often does not send a second push. Waiting for one hung the second rekey for 30s.ControlConn.Readswallow it as an ordinary control payload.ReadIPPacket, instead of onlyuse of closed network connection.Test
Unit tests in
transport/openvpn:TestNextKeyIDFollowsOpenVPNSequence—0 → 1 → … → 7 → 1TestSoftResetAdvancesOpenVPNKeyID— second reset is key ID2, not0TestRekeyDataHeaderUsesActiveKeyID— rekeyed data packet key ID is1, not0TestParseServerKeyMethod2RecordShortenedPreservesTail— leftoverPUSH_REPLYkeptTestParsePushReplyAuthToken/TestCaptureAuthTokenUsedOnNextKeyMethodTestMarkReceivedUnblocksNextEpochControl— ServerHello after soft reset is deliveredTestStashedSoftResetIsNotSwallowedByControlRead— next-epoch reset is not eaten byReadLive check, same minimal config on both binaries:
reneg-sec 10,AES-128-GCM, username/password, bind127.0.0.1:1194mixed-port: 17890,MATCH→ OpenVPN outbound10.8.0.1:8080alpha-7ee0b05unexpected message, tunnel diesRefs: MetaCubeX/mihomo#3085