Skip to content

Harden Windows diagnostics and session recovery - #4

Open
Tenali-Rama wants to merge 1 commit into
mainfrom
feature/windows-reliability
Open

Harden Windows diagnostics and session recovery#4
Tenali-Rama wants to merge 1 commit into
mainfrom
feature/windows-reliability

Conversation

@Tenali-Rama

Copy link
Copy Markdown
Collaborator

Summary

  • preserve versioned, validated project recovery snapshots and restore the newest valid candidate
  • export sanitized diagnostics with process evidence, Windows crash dumps, Event Viewer details, and media decode checks
  • classify missing session endings separately from native crashes and exclude the ABI-mismatched optional OpenFX plugin

Verification

  • 24 MVP/reliability tests passed
  • 30 reconstruction/segment tests passed (3 environment-dependent fixtures skipped)
  • full CMake build completed successfully
  • diagnostic export tested against the supplied failed Windows session archive

@parsewave-bot

parsewave-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown

TerminalBench Bot Commands

Run tasks:
/bot tb run [--dataset, --dataset-path, --dataset-config, --registry-url, --local-registry-path, --output-path, --run-id, --upload-results, --task-id, --n-tasks, --exclude-task-id, --no-rebuild, --cleanup, --use-subscription, --model, --agent, --agent-import-path, --agent-kwarg, --log-level, --livestream, --n-concurrent, --n-attempts, --global-timeout-multiplier, --global-agent-timeout-sec, --global-test-timeout-sec, --contributionsCommit]

Check:
/bot tb tasks check [--task-id, --tasks-dir, --unit-test-relative-path, --dockerfile-relative-path, --model, --agent, --fix, --output-path, --contributionsCommit]

Debug:
/bot tb tasks debug [--task-id, --run-id, --runs-dir, --tb-run-job-id, --tasks-dir, --agent, --model, --n-trials, --output-path, --contributionsCommit]

Full Check:
/bot full-check-v2 [--task-id <id>] [--analyze-failure] [...]
/bot full-check-v2 --opus-only - Run only tb_run_large with Claude Opus 4.7
/bot full-check-v2 --sonnet-only - Run only tb_run_large with Claude Sonnet 4.5
/bot full-check-v2 --tbench - Oracle + NOP + tb_run_large (Codex + openai/gpt-5.5, 5 parallel attempts, subscription; requires 0-3/5 resolved) + Codex harbor debug/analyze when ≤3/5 resolved
/bot full-check-v2 --tbench --opus - Same as --tbench, but tb_run_large uses Claude Opus 4.8 via claude-code
/bot full-check-v2 --fusion-reports - Oracle + NOP + harbor_run_large (Codex + openai/gpt-5.5, 1 attempt, subscription). No fallback stage. Harbor saves /output, /app/output, and traces per trial automatically (implicit --artifacts — needed for downstream re-verify).
/bot full-check --openclaw - Run Oracle + NOP + exactly 5 SecureHermes trials with GPT-5.6, judge with GPT-5.6-sol, then replace the PR's committed OpenClaw trace folder at the unchanged PR head.
/bot full-check-v2 --oracle-nop-only - Lightweight gate: only Oracle (5 attempts, 5 retries) + NOP, skip every model run / similarity / debug / quality check (~30-60s per task)
/bot full-check --tb-run-large-agent claude-code --tb-run-large-model claude-opus-4-7

Grok Trace Run:
/bot run-grok --5 - Shortcut for Oracle + NOP + 5 Grok Build trials with artifacts/traces.
/bot run-grok --5 --appends - Add 5 Grok traces after the existing S3 traces, then auto-rescore the full S3 trace set.
/bot run-grok --8 - Shortcut for Oracle + NOP + 8 Grok Build trials with artifacts/traces.

Trace Run:
/bot mm-trace-run - Oracle + NOP + 5 Codex GPT-5.5 xhigh trials (subscription) with artifacts/traces. Posts per-trial rewards and step counts, avg reward, avg steps, max/avg reward ratio, and artifact + trace viewer links.

Re-verify (re-score existing agent attempts against updated tests):
/bot re-verify [--task-id <id>] [--skip-oracle] [--skip-nop] - Re-run tasks/<task-id>/tests/test.sh against every trajectories*/<task-id>/<agent>/<N>/artifacts/output/ directory in the PR head ref (1 claude + 4 grok by convention). Also runs oracle (canonical solution/solve.sh, expected reward 1.0) and nop (empty /output, expected reward 0.0) sanity rows by default — pass --skip-oracle or --skip-nop to opt out. Produces fresh per-attempt rewards without re-running the agents — useful after editing tests/ during review.
/bot rejudge --openclaw - Rejudge the 5 OpenClaw Hermes traces already committed to the PR with GPT-5.6-sol. Solver trials are not rerun; refreshed judge evidence is committed only if the PR head is unchanged.
/bot rescore [--trace-s3-url s3://bucket/prefix] - Re-score existing S3 trace outputs against current tests after rubric/test-only changes; runs oracle + nop sanity rows by default and does not use PR-committed trajectories.
/bot fairness-review - Run the structured task fairness review and render a standardized PASS/WARN/FAIL comment.

Harbor format checker:
/bot harbor-format-check [--trace-s3-url s3://bucket/prefix[,s3://bucket/other-prefix]] [--policy mm-abc|compat] - Run the standardized pre-acceptance format checker on this PR, including LLM fuzzy checks and optional S3 trace checks.

Offline-search reviewer:
/bot offline-search-review [--agents 1-5] - Run the offline-search audit reviewer on this PR and post the auditrobot summary back here.
/bot offline-review [--agents 1-5] - Short alias for /bot offline-search-review.

Sapphire format checker:
/bot sapphire-format-check [--task-dir tasks/<id>] [--no-llm] [--check-traces] - Run the mm-sapphire-pipelines format checker on this PR without touching full-check or mm-trace-run. Trace checks are opt-in.

For detailed parameter descriptions, run tb --help or tb <command> --help locally.

Job Management:
/bot job list - List all running jobs
/bot job status <job_id> - Get status of a specific job
/bot job kill <job_id> - Kill a running job
/bot job restart <job_id> - Restart a failed job
/bot job info <job_id> - Show detailed information about a job
/bot job cleanup - Remove all failed-to-report jobs

Review:
/bot code-review - Trigger the generic AI code review service on this PR

Remove default flags: Use --no-{flag} to disable default flags (e.g., --no-use-subscription)

Aliases:
/bot /codex-attempts [--n, --task-id, --tasks-dir]/bot /tb run --agent codex
/bot /claude-attempts [--n, --task-id, --tasks-dir]/bot /tb run --agent claude-code
/bot /grok-attempts [--n, --task-id, --tasks-dir]/bot /tb run --agent terminus-2 --model xai/grok-4.3-internal --agent-kwarg reasoning_effort=high
/bot /tb-check [--task-id, --tasks-dir]/bot /tb tasks check
/bot /oracle [--task-id, --tasks-dir]/bot /tb run --agent oracle
/bot /nop [--task-id, --tasks-dir]/bot /tb run --agent nop
/bot /tb-debug [--task-id, --tasks-dir]/bot /tb tasks debug

Get help: /help or /bot help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant