Skip to content

fix(app): keep superseded defaults as MEASURED with filtered measurements - #752

Merged
kstonekuan merged 4 commits into
Hebbian-Robotics:mainfrom
Adityakk9031:fix/default-superseded-measurement-mutation
Nov 6, 2026
Merged

kstonekuan merged 4 commits into
Hebbian-Robotics:mainfrom
Adityakk9031:fix/default-superseded-measurement-mutation

Conversation

@Adityakk9031

@Adityakk9031 Adityakk9031 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

_yield_defaults_superseded_by_the_pipeline mutated run.outcome to NotRun(SupersededByPipeline) after checks had already executed, but the computed measurements were still recorded by _check_run_rows. This left inconsistent state: a check marked SUPERSEDED while its measurements were still stored, risking duplicate-key collisions with pipeline steps.

Instead, keep the check as MEASURED, drop the superseded measurement keys to avoid collisions, and tag the dropped keys so the supersession remains visible in the catalog.

Validation: uv run ruff check, uv run ty check


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@github-actions

Copy link
Copy Markdown

👋 Hi @Adityakk9031 — thank you so much for your first contribution to HFlow!

A maintainer will review your pull request as soon as possible. In the meantime:

💡 Tip: one open pull request per contributor at a time. Before starting an issue, check its sidebar for an assignee or a linked pull request. Either one means somebody is already on it; everything else is fair game.

We are excited to have you here and appreciate your help making the project better! 🙌

@greptile-apps

greptile-apps Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the remaining test coverage request is non-blocking.

Findings

  1. P2 The fallback lacks test coverage ▶

Summary

Completed defaults stay MEASURED, retain non-overlapping measurements, and tag removed keys.

  • Automatic checks keep their non-overlapping measurements when pipeline steps measure the same keys.

Reviews (5) · Last reviewed commit: "fix(results): reject unknown projected m..." · Reviewed by Greptile

Comment thread src/hflow/app.py Outdated
Comment on lines +1431 to +1438
run.result.measurements = {
key: value
for key, value in run.result.measurements.items()
if key not in superseded_set
}
run.result.tags.extend(
f"superseded:{key}" for key in superseded_keys
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The fallback lacks test coverage

The existing supersession tests exercise the early skip, which still returns SUPERSEDED. They do not protect this new filtering or its tags, so they can pass even if duplicate keys survive or remaining evidence is lost.

Add a regression test that reaches this post-execution path. Cover partial and full overlap, including enrichment labels. Assert that the default keeps its remaining evidence, drops only shared measurements, and records the superseded:<key> tags in the catalog.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

…ents

_yield_defaults_superseded_by_the_pipeline mutated run.outcome to
NotRun(SupersededByPipeline) after checks had already executed, but the
computed measurements were still recorded by _check_run_rows. This left
inconsistent state: a check marked SUPERSEDED while its measurements
were still stored, risking duplicate-key collisions with pipeline steps.

Instead, keep the check as MEASURED, drop the superseded measurement
keys to avoid collisions, and tag the dropped keys so the supersession
remains visible in the catalog.
@Adityakk9031
Adityakk9031 force-pushed the fix/default-superseded-measurement-mutation branch from 1a158f2 to 3ea682f Compare October 29, 2026 06:01
@Adityakk9031

Copy link
Copy Markdown
Contributor Author

Recreating with rebased branch on latest main.

Comment thread src/hflow/app.py Resolved
@Adityakk9031

Copy link
Copy Markdown
Contributor Author

@kstonekuan have a look

@kstonekuan kstonekuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you keep project_results working when an enrichment supplies duration_s? With the episode_duration default enabled, selecting its duration_s now raises KeyError('duration_s'); main reports superseded. Please cover this post-execution path in a regression test.

… KeyError

project_results raised KeyError when a Measured default check had a
measurement key filtered out by _yield_defaults_superseded_by_the_pipeline.
Report the check as superseded instead, matching the pre-existing behavior
for fully-superseded defaults.

Add regression test covering the post-execution filtering path: a default
check runs in full, a later requires-bearing user step overlaps one of its
keys, the key is filtered out with a superseded:<key> tag, and project_results
returns superseded for the filtered key and measured for surviving keys.
Comment thread src/hflow/results.py Resolved
Comment thread tests/test_result_projection.py Resolved
@Adityakk9031

Copy link
Copy Markdown
Contributor Author

@kstonekuan done please have a look

@kstonekuan

kstonekuan commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Could you address the Greptile findings? project_results reports superseded for an unknown measurement, and the new tests match UnavailableCheck(status=...) even though the field is reason, so they pass without checking the outcome.

@kstonekuan kstonekuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! I added c093461 after the review went quiet to reject unknown projected keys, correct the tests, and update the docstring. Ruff, format, ty, and 2,628 tests passed (47 skipped).

@kstonekuan
kstonekuan merged commit d650654 into Hebbian-Robotics:main Nov 6, 2026
3 checks passed
@kstonekuan kstonekuan mentioned this pull request Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants