Repository navigation
fix(app): keep superseded defaults as MEASURED with filtered measurements - #752
Conversation
|
👋 Hi @Adityakk9031 — thank you so much for your first contribution to HFlow! A maintainer will review your pull request as soon as possible. In the meantime:
💡 Tip: one open pull request per contributor at a time. Before starting an issue, check its sidebar for an assignee or a linked pull request. Either one means somebody is already on it; everything else is fair game. We are excited to have you here and appreciate your help making the project better! 🙌 |
[Medium impact] The PR appears safe to merge; the remaining test coverage request is non-blocking. Findings
|
| run.result.measurements = { | ||
| key: value | ||
| for key, value in run.result.measurements.items() | ||
| if key not in superseded_set | ||
| } | ||
| run.result.tags.extend( | ||
| f"superseded:{key}" for key in superseded_keys | ||
| ) |
There was a problem hiding this comment.
The fallback lacks test coverage
The existing supersession tests exercise the early skip, which still returns SUPERSEDED. They do not protect this new filtering or its tags, so they can pass even if duplicate keys survive or remaining evidence is lost.
Add a regression test that reaches this post-execution path. Cover partial and full overlap, including enrichment labels. Assert that the default keeps its remaining evidence, drops only shared measurements, and records the superseded:<key> tags in the catalog.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
…ents _yield_defaults_superseded_by_the_pipeline mutated run.outcome to NotRun(SupersededByPipeline) after checks had already executed, but the computed measurements were still recorded by _check_run_rows. This left inconsistent state: a check marked SUPERSEDED while its measurements were still stored, risking duplicate-key collisions with pipeline steps. Instead, keep the check as MEASURED, drop the superseded measurement keys to avoid collisions, and tag the dropped keys so the supersession remains visible in the catalog.
1a158f2 to
3ea682f
Compare
|
Recreating with rebased branch on latest main. |
|
@kstonekuan have a look |
kstonekuan
left a comment
There was a problem hiding this comment.
Could you keep project_results working when an enrichment supplies duration_s? With the episode_duration default enabled, selecting its duration_s now raises KeyError('duration_s'); main reports superseded. Please cover this post-execution path in a regression test.
… KeyError project_results raised KeyError when a Measured default check had a measurement key filtered out by _yield_defaults_superseded_by_the_pipeline. Report the check as superseded instead, matching the pre-existing behavior for fully-superseded defaults. Add regression test covering the post-execution filtering path: a default check runs in full, a later requires-bearing user step overlaps one of its keys, the key is filtered out with a superseded:<key> tag, and project_results returns superseded for the filtered key and measured for surviving keys.
|
@kstonekuan done please have a look |
|
Could you address the Greptile findings? |
kstonekuan
left a comment
There was a problem hiding this comment.
Thanks for the fix! I added c093461 after the review went quiet to reject unknown projected keys, correct the tests, and update the docstring. Ruff, format, ty, and 2,628 tests passed (47 skipped).
_yield_defaults_superseded_by_the_pipeline mutated run.outcome to NotRun(SupersededByPipeline) after checks had already executed, but the computed measurements were still recorded by _check_run_rows. This left inconsistent state: a check marked SUPERSEDED while its measurements were still stored, risking duplicate-key collisions with pipeline steps.
Instead, keep the check as MEASURED, drop the superseded measurement keys to avoid collisions, and tag the dropped keys so the supersession remains visible in the catalog.
Validation: uv run ruff check, uv run ty check
Need help on this PR? Tag
@codesmith-botwith what you need.