Skip to content

fix(openai): track HTTP/2 stream body failures - #6326

Open
HypoxanthineOvO wants to merge 1 commit into
Wei-Shaw:mainfrom
HypoxanthineOvO:fix/openai-h2-stream-body-circuit
Open

fix(openai): track HTTP/2 stream body failures#6326
HypoxanthineOvO wants to merge 1 commit into
Wei-Shaw:mainfrom
HypoxanthineOvO:fix/openai-h2-stream-body-circuit

Conversation

@HypoxanthineOvO

Copy link
Copy Markdown
Contributor

Summary

  • account for OpenAI HTTP/2 failures that occur while reading the response body, not only before response headers
  • prevent older in-flight successes from clearing newer HTTP/2 failure observations
  • keep an already-tripped proxy stream quarantine active until its TTL expires

Problem

The HTTP/2 compatibility fallback currently records success as soon as
http.Client.Do returns response headers. For SSE responses, the body can fail
later with errors such as http2: client connection lost. Each affected
request therefore clears the failure window before its body fails, and the body
failure is never reported to the fallback circuit.

The proxy stream circuit has a related concurrency race: any successful stream
deletes the circuit entry, including a request that started before concurrent
failures tripped the quarantine.

Fix

  • wrap response bodies with one-shot success/read-error callbacks
  • record success only after a clean body EOF
  • classify the observed HTTP/2 connection-loss signature
  • fence success resets using the request start time
  • do not let a late success clear an active quarantine
  • retain the existing quarantine fail-open selection behavior

Generic unexpected EOF is intentionally not classified as an HTTP/2
compatibility error at this layer because it can also be observed after a valid
terminal SSE event. The higher-level stream circuit keeps the protocol context
needed to distinguish those cases.

Tests

  • go test ./internal/repository -count=1
  • go test ./internal/service -count=1

Regression coverage includes repeated body read failures, a healthy request
resetting a pre-trip failure window, stale concurrent successes, generic
unexpected EOF, quarantine TTL behavior, terminal events, and cancellation.

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA).

To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant