Skip to content

fix: return detailed vertex status message when unhealthy#3502

Open
AyushSrivastava1818 wants to merge 1 commit into
numaproj:mainfrom
AyushSrivastava1818:fix/vertex-status-message
Open

fix: return detailed vertex status message when unhealthy#3502
AyushSrivastava1818 wants to merge 1 commit into
numaproj:mainfrom
AyushSrivastava1818:fix/vertex-status-message

Conversation

@AyushSrivastava1818

Copy link
Copy Markdown

What this PR does / why we need it

When a vertex is unhealthy, CheckVertexStatus currently returns a generic message:

Vertex "<name>" is not healthy

This hides the more detailed diagnostic information already available in vertex.Status.Message.

This PR updates CheckVertexStatus to return vertex.Status.Message when it is available, while preserving the existing generic fallback message when no detailed status message exists.

Additionally, a unit test has been added to verify the new behavior without changing the existing fallback behavior.

Related issues

Fixes #3496

Testing

  • Added a unit test covering the detailed status message behavior.
  • Existing fallback behavior remains covered by the current tests.
  • Ran:
gofmt -w pkg/reconciler/util.go pkg/reconciler/util_test.go
go test ./pkg/reconciler/...

@adarsh0728

Copy link
Copy Markdown
Member

can you fix DCO check? (signed commit)

@adarsh0728 adarsh0728 requested a review from suryapratap-01 July 3, 2026 10:07
@AyushSrivastava1818 AyushSrivastava1818 force-pushed the fix/vertex-status-message branch from 8701bf1 to 7ff0df6 Compare July 3, 2026 10:27
Comment thread pkg/reconciler/util.go Outdated
}
if !vertex.Status.IsHealthy() {
if vertex.Status.Message != "" {
return false, "Unavailable", vertex.Status.Message

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return false, "Unavailable", vertex.Status.Message
return false, "Unavailable", `Vertex "` + vertex.Spec.Name + `" error: ` + vertex.Status.Message

@vaibhavtiwari33

Copy link
Copy Markdown
Contributor

Can you please also provide example of how this message appears in status from a running pipeline that is failing.

@AyushSrivastava1818 AyushSrivastava1818 force-pushed the fix/vertex-status-message branch from 7ff0df6 to e4f2215 Compare July 3, 2026 19:19
Signed-off-by: AyushSrivastava1818 <ayush.sri0705@gmail.com>
@AyushSrivastava1818 AyushSrivastava1818 force-pushed the fix/vertex-status-message branch from e4f2215 to 55f7712 Compare July 3, 2026 19:22
@AyushSrivastava1818

Copy link
Copy Markdown
Author

Thanks for the review!

I've updated the returned message to include the vertex name and pushed the changes.

Regarding the example, I don't currently have a failing pipeline available to capture an actual vertex.Status.Message. The intent of this change is to surface the existing vertex.Status.Message from the vertex status instead of returning only the generic Vertex "<name>" is not healthy message.

If needed, I can try to reproduce a failing pipeline and share an actual status output.

@vaibhavtiwari33

Copy link
Copy Markdown
Contributor

If needed, I can try to reproduce a failing pipeline and share an actual status output.

Yes pls

@codecov

codecov Bot commented Jul 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.07%. Comparing base (f834af7) to head (55f7712).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3502      +/-   ##
==========================================
- Coverage   83.08%   83.07%   -0.01%     
==========================================
  Files         310      310              
  Lines       81395    81397       +2     
==========================================
  Hits        67624    67624              
- Misses      13174    13176       +2     
  Partials      597      597              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@AyushSrivastava1818

Copy link
Copy Markdown
Author

Thanks! I'm setting up a local Numaflow environment to reproduce a failing pipeline and capture an actual vertex.Status.Message. I don't have a Kubernetes cluster configured on this machine yet, so it will take me a bit of time to set it up.

I'll update this PR with a real status output as soon as I have it. Thanks for your patience!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rollup unhealthy vertex reason to pipeline

3 participants