Skip to content

fix(catalog-ui): wait for the episodes file before binding an append - #775

Merged
kstonekuan merged 2 commits into
Hebbian-Robotics:mainfrom
Abhishek-B-R:fix/catalog-ui-wait-for-episodes
Nov 11, 2026
Merged

kstonekuan merged 2 commits into
Hebbian-Robotics:mainfrom
Abhishek-B-R:fix/catalog-ui-wait-for-episodes

Conversation

@Abhishek-B-R

@Abhishek-B-R Abhishek-B-R commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The open catalog UI no longer binds a half written first append. Before, hflow catalog ui on an empty catalog could print "Catalog Parquet detected. Views are now available in the open UI." while episodes was still empty, because the dependent tables of the append had landed and its episodes file had not. Now the first refresh waits for the episodes file, so it always shows the episode row, which is what hflow catalog ui --help already promises ("refreshes its views after the first completed append").

Why

Catalog.append_episode writes check_runs, measurements and observations first and the episodes file last, and _refresh_local_catalog_connection's docstring relies on that ("the first episodes Parquet is enough to rebind every table that append wrote"). The poll trigger, _empty_table_relations_have_landed_parquet, did not: it returned true for Parquet in any empty table. A poll that ran between the two writes bound the dependents, kept episodes as the empty shell, and spent the "first Parquet" message on it. The next poll repaired it, so the wrong state lasts one interval, but it is the window an operator watching the first ingest sees, and it is wider on a bucket catalog where each dependent is its own put plus a mirror sync.

The trigger now skips Parquet in tables from the append's write set (TABLE_COLUMN_DDL) until the episodes directory has a file. ingest_failures is outside that set, so a failure only catalog (#537) still binds without any episode.

Repro, as in the new test: start the UI on an empty catalog, then copy every table of a finished append except episodes into it. On main the UI refreshes within one poll with SELECT count(*) FROM episodes returning 0. With this change it waits, and refreshes once the episodes file is copied in.

Validation

  • New test_catalog_ui_waits_for_the_episodes_file_before_binding_an_append: on main it fails 5 of 5 runs (assert session.refresh_count == 0, got 1); with the fix it passes 20 of 20.
  • uv run pytest -q tests/test_catalog_ui.py tests/test_catalog_curation.py: 216 passed.
  • uv run pytest -q -n 4: 2627 passed, 49 skipped (macOS, Python 3.11 env from uv sync --locked).
  • uv run ruff check, uv run ruff format --check, uv run ty check: clean.

Checklist

  • I added or updated outcome-focused tests for changed business logic.
  • I updated documentation for changed behavior, flags, formats, or requirements. (No documented behavior changes.)
  • I ran uv run ruff check --fix, uv run ruff format, and uv run ty check.
  • I ran the relevant pytest suite.
  • I did not add recordings, generated media, credentials, private URLs, or runtime artifacts.
  • I preserved stored-data compatibility or documented an explicit version change.

Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

An append stores its dependent tables before its episodes file, which is
its commit marker. The poll loop rebound the open UI as soon as any empty
table gained Parquet, so a poll between those writes bound check_runs,
measurements and observations while episodes was still the empty shell,
and printed that the views were available. Dependent Parquet now counts
only once an episodes file exists; ingest_failures still binds on its own.
@greptile-apps

greptile-apps Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the remaining test timing gap is non-blocking.

Findings

  1. P2 Test can miss the poll ▶

Summary

The PR waits for an episodes Parquet file before dependent tables can trigger the first refresh. ingest_failures can still trigger a refresh without an episode.

  • Append tables wait for an episode file before triggering a UI refresh.

Reviews (2) · Last reviewed commit: "test(catalog-ui): wait for a poll over t..." · Reviewed by Greptile

Comment thread tests/test_catalog_ui.py Outdated
Comment on lines +279 to +280
time.sleep(0.2)
assert session.refresh_count == 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Test can miss the poll

The new test sleeps for 0.2 seconds without checking that the UI polled after the dependent files were copied. If the background thread is delayed, refresh_count == 0 passes even with the old bug. Wait for a completed poll against the dependents-only catalog before checking that no refresh happened, then copy the episodes file.

The fixed sleep could pass without the UI ever polling the
dependents-only catalog. Wait until a poll that started after the copy
has finished, then check that it did not refresh.

@kstonekuan kstonekuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Ruff, format, and ty pass; 2629 tests passed, 47 skipped. LGTM.

@kstonekuan
kstonekuan merged commit d2ee95e into Hebbian-Robotics:main Nov 11, 2026
3 checks passed
@kstonekuan kstonekuan mentioned this pull request Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants