Skip to content

chore(observability): add common internal HTTP metrics to AWS sinks#25508

Open
gwenaskell wants to merge 9 commits into
masterfrom
yoenn.burban/OPA-5365-add-aws-http-observability
Open

chore(observability): add common internal HTTP metrics to AWS sinks#25508
gwenaskell wants to merge 9 commits into
masterfrom
yoenn.burban/OPA-5365-add-aws-http-observability

Conversation

@gwenaskell

@gwenaskell gwenaskell commented May 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Wired common internal HTTP telemetry into the connector used by AWS sinks. This required some modifications to the internal events to provide compatibility with smithy requests and responses.

Vector configuration

How did you test this PR?

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details on the dd-rust-license-tool.

@gwenaskell gwenaskell changed the title Yoenn.burban/opa 5365 add aws http observability chore(observability): add common internal HTTP metrics to AWS sinks May 27, 2026
@gwenaskell gwenaskell force-pushed the yoenn.burban/OPA-5365-add-aws-http-observability branch from 3959a94 to 556590d Compare June 5, 2026 14:19
@gwenaskell

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 556590d4b5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/internal_events/http_client.rs
@gwenaskell gwenaskell force-pushed the yoenn.burban/OPA-5365-add-aws-http-observability branch from 556590d to aae5c92 Compare June 24, 2026 09:07
@gwenaskell gwenaskell marked this pull request as ready for review June 24, 2026 09:07
@gwenaskell gwenaskell requested a review from a team as a code owner June 24, 2026 09:07
@gwenaskell gwenaskell requested a review from bruceg June 24, 2026 11:53

@bruceg bruceg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pront

pront commented Jun 25, 2026

Copy link
Copy Markdown
Member

Note

The following is an automated Codex code review report.

Finding (Medium): AWS auth/region HTTP paths are still uninstrumented

The observability wrapper is only applied to the final service client connector (src/aws/mod.rs:212), but the raw connector is also used in:

  • src/aws/mod.rs:142 (region resolution)
  • src/aws/auth.rs:248, :314, :394 (IMDS, STS assume-role, profile/SSO credential providers)

This means HTTP failures from those paths (e.g., a failed IMDS call or STS assume-role timeout) won't emit the new common HTTP metrics/events.

Recommended fix: Either factor the wrapper into a shared helper and apply it to all connector construction sites (if the intent is full AWS observability), or add a comment + test explicitly documenting that only service calls are instrumented and the credential/region paths are out of scope.

Apply AwsHttpClient instrumentation to STS assume-role, profile/SSO
file, and IMDS credential providers. Region resolution remains
uninstrumented as no region is available at that point.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c186783669

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/internal_events/http_client.rs Outdated
Comment thread src/aws/auth.rs
Comment thread src/aws/auth.rs
@gwenaskell

Copy link
Copy Markdown
Contributor Author

@pront wired in telemetry for the AWS auth requests too.

@gwenaskell gwenaskell requested a review from pront June 29, 2026 13:46
@datadog-vectordotdev

datadog-vectordotdev Bot commented Jun 29, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: fe2ffcd | Docs | Give us feedback!

@pront

pront commented Jun 30, 2026

Copy link
Copy Markdown
Member

@pront wired in telemetry for the AWS auth requests too.

@gwenaskell Thanks! Did you test this end-to-end? There were some interesting finding about double counting bytes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AWS components lack custom HTTP metrics as of 0.21.0

3 participants