Skip to content

Add additive SBD V6 metadata and timeline projection - #1313

Closed
luochen-amd wants to merge 4 commits into
feat/zgong/explore-mergefrom
feat/sbd-v6-metadata-outcome-timeline
Closed

Add additive SBD V6 metadata and timeline projection#1313
luochen-amd wants to merge 4 commits into
feat/zgong/explore-mergefrom
feat/sbd-v6-metadata-outcome-timeline

Conversation

@luochen-amd

@luochen-amd luochen-amd commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add an additive SBD V6 projection while preserving the existing V5 schema version and execution behavior
  • expose V6 metadata, outcome, timeline, and top-level close data
  • record fail-open install and model_gate telemetry without changing preflight, gate, or resume outcomes
  • implement the merged framework_agent timeline for configuration, source, Critic, macro-cycle, exit, and failure evidence
  • project Framework-internal candidate discovery success, empty, and failure outcomes in execution order
  • omit phase summaries and all token, pricing, and monetary cost fields

Review Fixes

  • persist hard preflight failures even when they occur outside an instrumented install step
  • acquire the session lock before writing resume-failure artifacts and fall back to an isolated failed-attempt session when the target session is busy
  • preserve specialist source phases and exclude non-Framework specialist evidence from framework_agent timeline creation
  • make legacy fixed-file timeline migration resumable after a partial migration

Compatibility

  • keeps V5 output and state-machine behavior intact
  • treats V6 collection as best-effort telemetry so collection failures do not affect optimization execution
  • retains legacy Explore evidence only as input compatibility for the merged Framework Agent stage

Validation

  • focused blocker reproductions: 4 passed
  • SBD, Framework Agent, specialist, enablement, and attribution regression selection: 266 passed
  • preflight regression selection: 237 passed
  • Ruff checks passed for all changed Python files
  • Ruff format checks passed for all changed Python files
  • git diff --check passed

Environment Note

  • Windows test runs inject a no-op fcntl compatibility module so POSIX-only KB imports can load; the production lock behavior remains covered by Linux CI and the focused busy-session test uses an explicit lock double

@luochen-amd
luochen-amd requested a review from a team as a code owner August 27, 2026 14:28
Comment thread src/hyperloom/inference_optimizer/breakdown/collectors/v6.py Fixed
Comment thread src/hyperloom/inference_optimizer/cli/preflight.py Fixed
Comment thread src/hyperloom/inference_optimizer/cli/preflight.py Fixed
Comment thread src/hyperloom/inference_optimizer/breakdown/collectors/v6.py Fixed
Comment thread src/hyperloom/inference_optimizer/cli/preflight.py Fixed
Comment thread src/hyperloom/inference_optimizer/cli/preflight.py Fixed
@github-actions

Copy link
Copy Markdown

CI E2E report — ❌ Timeout

item value
result ❌ Timeout
model Qwen/Qwen3-0.6B (dense)
resources 1× GPU, TP=1
PR branch feat/sbd-v6-metadata-outcome-timeline
commit 4dd5533d8e43e5fcdfbf9c1b76458eb8ced759c7
session_id 52dcbb00-77e1-4fe7-8705-7dc00196e1fe
queue → dispatch -68271s
run time 1163m 29s
total 25m 38s
reason Timed out — the run never reached a terminal state in time (task stuck, or the GPU stayed queued too long).
detail not terminal after 13200s

details

@luochen-amd
luochen-amd force-pushed the feat/sbd-v6-metadata-outcome-timeline branch from 4dd5533 to f09e319 Compare August 28, 2026 02:50
@luochen-amd
luochen-amd changed the base branch from main to feat/zgong/explore-merge August 28, 2026 02:50
@luochen-amd

Copy link
Copy Markdown
Contributor Author

Addressed the three blocking review findings:

  • Persist hard preflight/install failures by creating a safe failure session, writing the install event, and materializing session_breakdown.json before re-raising the original exception.
  • Preserve repeated install and model_gate executions in an ordered per-event timeline while keeping install.json and model_gate.json as latest-event compatibility projections. Legacy fixed-file sessions are migrated on the next write.
  • Exclude Kernel-phase explore rebench operations from the Framework Agent timeline and require matching cycle/time window evidence.

Regression coverage now includes hard preflight failure persistence, fresh-to-resume event retention, legacy event migration, and Kernel Explore exclusion.

Local validation:

  • SBD V6 focused tests: 26 passed
  • Related exporter/model-gate suites: 215 passed, 1 Windows-only path assertion deselected
  • Ruff check/format and git diff --check: passed



def test_preflight_hard_failure_creates_session_and_final_sbd(tmp_path, monkeypatch):
import hyperloom.inference_optimizer.cli as optimizer_cli


def test_preflight_hard_failure_creates_session_and_final_sbd(tmp_path, monkeypatch):
import hyperloom.inference_optimizer.cli as optimizer_cli


def test_unwrapped_preflight_failure_is_persisted_as_failed(tmp_path, monkeypatch):
import hyperloom.inference_optimizer.cli as optimizer_cli


def test_busy_resume_preflight_failure_uses_isolated_session(tmp_path, monkeypatch):
import hyperloom.inference_optimizer.cli as optimizer_cli
@luochen-amd

Copy link
Copy Markdown
Contributor Author

This combined PR has been split to keep the review boundaries explicit:

The Framework branch is intentionally kept separate while #1301 and #1331 are still open. A focused Framework PR can be opened after the dependency bases are aligned, avoiding duplicate core changes in review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants