Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/HANDOFF.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Current state of the project and active work. Read this at session start. Update before compaction or at natural breakpoints.

**Last updated:** 2026-06-02
**Last updated:** 2026-06-03

## Project State

Expand Down Expand Up @@ -60,6 +60,7 @@ Triage pattern: fix P1/P2 with code, create TODOs for P3/nitpicks, reply to ever

## Recent Context

- 2026-06-03 PR #42 (`feat: add brainstorm panel review pack`) merged through the merge queue after follow-up fixes. Added `refinery review-brainstorm-panels`, recorded Pi-backed L2 benchmark results and first-pass panel review docs, and kept `score-only` as the recommended default. Review feedback addressed: bare `--key-path` filenames no longer call `create_dir_all("")`; `--strategies` and artifact metadata now parse through `BrainstormIterationStrategy`; saved `propose-*.md` text is preserved verbatim in review packs; handoff timestamp refreshed. Final checks observed before merge: GitHub Actions Build/Check/Test passed, Buildkite build #36 passed, and CodeRabbit passed. Local verification included `cargo fmt --all`, `cargo test -p refinery_cli review_brainstorm_panels`, `cargo clippy -p refinery_cli --all-targets -- -D warnings`, `cargo build --workspace`, and `cargo test --workspace`.
- 2026-06-01 first-pass brainstorm L2 panel review completed (`todos/013`, `docs/brainstorms/2026-06-01-brainstorm-l2-panel-review.md`): reviewed the blind pack at `target/brainstorm-benchmark-2026-05-29-l2-pi-serial/logs/l2-panel-review-pack.md` and unblinded with `l2-panel-review-key.json`. Qualitative result: `score-only` looked strongest on useful diversity/non-overlap, `full-visibility` strongest on actionability/coverage, and `own-reviews` did not dominate globally but produced the strongest debugging/process panel. Recommendation remains: keep production default `score-only`; do not promote `full-visibility` despite higher automated quality scores until stronger human/calibrated judge evidence exists. For L3 prompt-reframing, use `score-only` as baseline and include `own-reviews` only if budget allows.
- 2026-05-30 brainstorm L2 iteration strategy benchmark completed (`todos/013`, `docs/brainstorms/2026-05-30-brainstorm-l2-iteration-strategy-benchmark.md`): ran 24 clean Pi-backed runs (6 prompts × `blind`, `score-only`, `own-reviews`, `full-visibility`) with `pi/openai-codex/gpt-5.4:off`, `pi/zai/glm-5.1:off`, `pi/kimi-coding/kimi-k2-thinking:off`, and `pi/minimax/MiniMax-M2.7:off`. Used `--max-concurrent 1` to avoid Pi local config lock contention and raised bounded stdout capture from 1MB to 64MB because Pi JSON event streams can exceed 1MB. Analyzer outputs live under `target/brainstorm-benchmark-2026-05-29-l2-pi-serial/logs/` (`run-dirs-clean.txt`, `l2-analysis-clean.json`, `l2-analysis-clean.txt`). Current `controversy_floor_7` aggregate: `full-visibility` highest quality (`mean=8.204`, `min=7.944`) but highest lexical overlap (`0.132`); `score-only` lowest lexical overlap (`0.097`) but lower quality (`mean=7.889`); `own-reviews` middle-ground (`mean=8.019`, disagreement `0.517`). Added `refinery review-brainstorm-panels` and generated blind review artifacts for `score-only`, `own-reviews`, and `full-visibility`: `l2-panel-review-pack.md` plus `l2-panel-review-key.json` in the same logs dir. Recommendation: keep production default `score-only` until whole-panel diversity/human or calibrated model-judge review checks semantic convergence and best-answer regret. Verified with `cargo fmt --all -- --check`, `cargo test -p refinery_cli review_brainstorm_panels`, `cargo clippy -p refinery_cli --all-targets -- -D warnings`, `cargo test -p tundish_providers`, and `cargo clippy -p tundish_providers --all-targets -- -D warnings`.
- 2026-05-31 PR #40 (`feat: add Pi provider and brainstorm benchmark variants`) passed final review/checks after follow-up commits. Addressed CodeRabbit/GHA feedback with nested `pi`/`opencode` model-spec validation, plan review-date refresh, and Clippy sort lint fixes; addressed Gemini feedback by comparing `ModelId` directly where compatible, accepting string evaluation scores, and preserving `USERPROFILE`; addressed Codex feedback by forwarding a whitelist of Pi credential/config env vars after `env_clear`. Final observed checks before merge: GitHub Actions Build/Check/Test passed, Buildkite build #31 passed, CodeRabbit approved. Local verification included `cargo fmt --all -- --check`, `cargo clippy --workspace -- -D warnings`, `cargo clippy --workspace --all-targets -- -D warnings`, `cargo build --workspace`, and `cargo test --workspace`.
Expand Down
Loading