Skip to content

fix(gate): compare NumPy scalar measurements instead of abstaining - #795

Open
Abhishek-B-R wants to merge 1 commit into
Hebbian-Robotics:mainfrom
Abhishek-B-R:fix/gate-numpy-measurements
Open

Abhishek-B-R wants to merge 1 commit into
Hebbian-Robotics:mainfrom
Abhishek-B-R:fix/gate-numpy-measurements

Conversation

@Abhishek-B-R

@Abhishek-B-R Abhishek-B-R commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A gate now compares NumPy scalar measurements. Before, a check that returned np.int64 or np.float32 (what np.count_nonzero, .sum() on an int array, or any float32 array reduction gives you) made its gate abstain, so a critical check never quarantined the episode however bad the number was.

Why

evaluate_gate keeps a matched value only if it is an int | float instance. np.int64 and np.float32 are not subclasses of either (only np.float64 is), so they fell into the "not a real number" branch. The catalog boundary already converts NumPy scalars with .item() (#126), and WeightedValue does the same (#647), but the gate runs on the raw CheckResult before the catalog sees it.

Repro:

gate = hflow.Gate(accept_when=(hflow.Threshold("value", hflow.Comparison.AT_MOST, 50.0),))

@app.check(version="1", critical=True, gate=gate)
async def counts_frames(ep):
    return hflow.CheckResult(measurements={"value": np.int64(99)})

On main the run reports measured, quarantine_tags == [] and tags gate-unevaluated:value. With this change it reports failed and quarantined:counts_frames, the same as for 99 or 99.0.

The fix converts np.generic values with .item() before the existing checks, looking NumPy up in sys.modules the way statistics.py does so steps.py stays NumPy free. np.bool_ becomes bool and still abstains, as True does today.

Validation

  • New test_numpy_scalar_measurements_compare_like_python_numbers in tests/test_gates.py: fails on main (GateAbstained(unevaluated_patterns=('value',)) == GateDecided(verdict=False)), passes with the fix.
  • uv run pytest -q tests/test_gates.py tests/test_checks.py tests/test_statistics.py: 167 passed.
  • uv run pytest -q -n 4: 2595 passed, 49 skipped, 3 failed. The 3 are tests/test_cli.py::test_the_module_form_*, which fail the same way on main in my macOS environment and do not touch this code.
  • uv run ruff check, uv run ruff format --check, uv run ty check: clean.

Checklist

  • I added or updated outcome-focused tests for changed business logic.
  • I updated documentation for changed behavior, flags, formats, or requirements. (No documented behavior changes.)
  • I ran uv run ruff check --fix, uv run ruff format, and uv run ty check.
  • I ran the relevant pytest suite.
  • I did not add recordings, generated media, credentials, private URLs, or runtime artifacts.
  • I preserved stored-data compatibility or documented an explicit version change.

Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

evaluate_gate only compared values that were int or float instances.
np.int64 and np.float32 are neither, so a check that returned a count
straight from NumPy made its gate abstain: the episode got a
gate-unevaluated tag and no verdict, and a critical check never
quarantined it. The catalog already stores those scalars as their Python
values, so the gate now reads them the same way before comparing.
@greptile-apps

greptile-apps Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge.

Summary

Gates now convert NumPy scalar measurements to Python values before comparing them.

  • Gates compare NumPy numbers and can quarantine failed critical checks.

Reviews (1) · Last reviewed commit: "fix(gate): compare NumPy scalar measurem..." · Reviewed by Greptile

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant