Skip to content

chore: release 0.5.0 - #223

Merged
a1exk-dev merged 86 commits into
mainfrom
release/0.5.0
Oct 14, 2026
Merged

a1exk-dev merged 86 commits into
mainfrom
release/0.5.0

Conversation

@a1exk-dev

@a1exk-dev a1exk-dev commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner

What changed

ctui 0.5.0, from the map "Map: ctui 0.5.0 Todo list" (#205) and its spec (#214). It follows the release bullets of MEMORY.md "Publish to Anthropic's plugin directory and to npm".

After merging, tag the merge commit v0.5.0.

Closes #205
Closes #214

Reports

Both reports ride on this pull request, pasted whole in the comments:

Both ran against the pin, 2.1.292.

Appendix: the 2026-10-12 manual run, superseded

The run that produced this release's fix list, before the live checks became an agent-driven skill. It covered three of the old document's sections at de576d7: Sidebar docking 7/7; Sidebar layout and scroll 55 pass, 4 fail, 14 not run; the /ctui menu 19 pass, 7 fail. Its eleven failures each became a diagnosis and a fix under #228, and all of them are closed (#229–#246): the menu ring's index, the Themes filter's focus, the Monthly cost cursor, the engine-drawn markers' decision, the user-prompt background under Catppuccin Latte, the /clear lag, teammate and Workflow rows, and the background agent's row ending while it waited on its own shell.

The numbers above are kept for the record only. The agent-driven reports in the comments are this release's result.

a1exk-dev and others added 12 commits October 9, 2026 10:49
AGENTS.md and the commit policy named release/0.3.0. A bugfix/<name>
cut from a release/<semver> now merges back into that same branch,
and its merge commit is pre-approved for every release.
A Sidebar plugin can set its own `cap`; Todo sets 8 so a typical plan shows
whole, and every other list section keeps the shared 4. An empty list draws a
bare `Todo:` title and one muted `no tasks yet` row instead of `0/0`.
Update the live checks, MEMORY.md and the all-expanded render (`◐` glyph,
Todo uncapped).

Refs: #215
While the Sidebar is docked with Todo on, the TaskCreate and TaskUpdate
hooks answer main-loop calls from ctui's own `$.tool.call` with
`{ result }` alone, so Claude Code's list stays closed. `dockColumns`
resets to 0 whenever the Sidebar isn't drawn docked.

Refs: #217
…der Task tools

For reference only, not for merge. On 2.1.292 with Opus 5.5 the tools
reached the prompt, but Claude made no task list in 0 of 18 live runs.

Refs: #216
With Task tools on, a pure prompt.compose hook appends the ctui:todo
section when the request offers TaskCreate or TodoWrite. With the pin,
Opus 5.5 keeps a list for multi-step work and none for one-step work.
The setting's description, the README, MEMORY.md and the live checks
say what this costs and that a change fully reaches Claude from the
next /clear, since Claude Code keeps a conversation's first prompt.

Refs: #216
Bump plugin.json and ctui/package.json to 0.5.0 and add the
tested-versions row: ctui 0.5.0 with Claude Code 2.1.292.

Refs: #214
The task in progress draws a ⠴⠦⠖⠲ spinner, one frame every 100 ms in
warning, through a new spinner.tsx Client that turns on its surface's
own frame clock, so the Sidebar doesn't redraw. Waiting tasks draw ⠶ in
faint. Sidebar plugins get the terminal/desktop table (SidebarUi), so a
view draws a Client without a cast; tests draw views through
clientsAsTrees, which gains a spinner.tsx stub.

MEMORY.md, docs/configuration.md, the live checks and the design render
follow the new glyphs.

Refs: #222
Done tasks draw 󰄳, waiting tasks 󰄰, and the task in progress a circle
filling up through 󰪞…󰪥 at 100 ms, all Nerd Font Material icons that
fill more of their cell than ✓, ⠶ and the braille spinner. Roles stay
as they were. Like the git header, there is no fallback without a
Nerd Font.

MEMORY.md, docs/configuration.md, the live checks and the design
render follow the new glyphs.

Refs: #225
fix: draw the Todo icons with Nerd Font Material circles
Color readers that reset per line report painted rows as unpainted
wherever a row opens in the color the row above ended with.

Refs: #233
docs: record that capture-pane -e carries SGR across lines
A live session per small ticket costs too many tokens. Fix and
diagnosis tickets now work from code, vendored types and tests, and
only a release's pull request runs the live checklist.

Refs: #228
The engine keeps the ring's index through the redraw and the re-open
Esc makes, and autoFocus doesn't move it, so the bold label and the
ring split. Focus the row just left after the re-open.

Refs: #238
The ring keeps the Themes row's index into the Themes screen, where
index 1 is inherit, and the filter's autoFocus doesn't move it, so typed
keys went to the prompt. Focus the filter after the pick.

Refs: #239
From 2.1.292 a background agent reads waiting in the agent list while
it waits on its own background shell, then running again. The poll
read every word but running as an end, so the row turned ✗, dropped
8 s later, and left its shell at the top level. End a held agent only
on completed, failed or killed; any other word is live.

Refs: #246
chore: write the live-check skill's scenarios, checks and report
Slice 3 of docs/spec/live-checks.md. docs/testing/live-checks.md loses its
133 rows and keeps the operator's half: how to run the live-check skill, how
to read the report it writes, and ctui's promise that the cold-cache cost row
arrives within 2 s.

CONTEXT.md defines a live check as one an agent drives through the pty
harness. MEMORY.md's "Tests cover real usage" carries the three mechanisms,
the placement rule, the three conversion rules and the 2-minute ceiling on
node scripts/check.ts, and points at CONTRIBUTING.md for the procedure
rather than restating it. CONTRIBUTING.md gains the ceiling pointer, the
rewritten Live checks paragraph, an Evals paragraph with the suite's one
command, and a Reports paragraph saying which pull request attaches which
report. The pull request template swaps its seven section checkboxes for one
Reports block taking a whole report or N/A.

Refs: #282
Slice 5 converted six rows, not seven: the `resets in` countdown is already
asserted in full in usage.test.tsx, which advances mock.clock 30 s and then
60 s, so it moves to that slice's not-converted list with its reason.

Scenario 6 is not the only scenario that spends model turns — the rule-gutter
and ctrl+o checks, the fresh-110 docking check and the main-screen toast check
each need one too, about five a run — so the spec keeps scenario 6's two
explicit turns and the no-skip rule and leaves the count to the report's
version block.

Scenario 7 drives the harness's `once` rather than a fifo session, because its
arrival check is timed from the session's own start and a first `send` would
add its own latency. And themes/omarchy-mix copies the operator's own
~/.claude/themes/omarchy.json into the scenario's config dir and picks
custom:omarchy instead of running `omarchy theme set`, which HARNESS.md
forbids.

Refs: #266
docs: cut the live checks to an operator page
The five converted tests cited rows of docs/testing/live-checks.md by line
number, and this release deleted those rows. Each comment now names the live
check that covers the behaviour (todo/spinner-focused,
docking/width-42-at-150 and its 170-column twin), or the sibling test it
builds on by its title, or says that no live check reaches it. The engine's
one-step swap of the session id and the live cost is cited where it lives, in
MEMORY.md's engine-pinned expectations.

docs/spec/live-checks.md loses the same numbering: the check-id table drops
its Row column, and slice 5's table and its not-converted list name each row
by its subject.

Refs: #266
The harness needs @types/node and allowImportingTsExtensions; ctui does not.
Adding both to the root project put Node globals in scope for ctui/hooks,
ctui/plugins and ctui/tests, which run in the engine and have no Node. The
harness now has its own tsconfig.json carrying them, the root config is back
to what it was, and node scripts/check.ts runs tsc over both projects so the
port still does not inherit scripts/*.ts's missing type check. ctui's own code
type-checks with no Node globals in scope, so nothing there depended on one.

MEMORY.md's pin entry and the spec's slice 1 record the two projects.

Refs: #266
…alves

fixtures/mcp-server.ts was built and documented, and no scenario started it:
config.ts wrote mcpServers: {}, so layout/no-child-processes could only assert
an absence and the "only MCP servers and short git runs" half of the check was
unobservable. `seed --mcp` now writes the stub into the seeded .claude.json at
local scope, where the engine starts it with no approval, and SKILL.md's
scenario 1 seeds with it. It stays a stub: two named tools, no network.

HARNESS.md also counts five modules, which is what its table lists.

Refs: #266
screen.ts coalesced a row twice, once for dumpbg and once for runs, and built
the `fg=… bg=… b=… u=…` string in two places. Both go through one `attrs(cell)`
and one `rowRuns(term, y, key)`; the step output is unchanged. Its `flag`
helper, which has nothing to do with a command-line flag, is now `bit`.

steps.ts split "on the first separator" by hand four times and `arg.split(' ')`
three times inside `anim`; `splitFirst` does it once.

harness.ts names `once`'s steps file and `serve`'s fifo and output file instead
of a fourth and a fifth argument, and its argv-consuming helpers say so:
`takeFlag` and `takeSwitch`. `--plugin-key` and the repeatable `--env` are gone
with the `LaunchOptions` and `Session` fields behind them: no scenario passed
either, and `setopt` always writes under `ctui@inline`, now a constant in
steps.ts.

Refs: #266
themes/omarchy-mix copied ~/.claude/themes/omarchy.json out of the operator's
home, so scenario 4 could not run on a machine without Omarchy and the
report's always-complete promise broke there. The theme is now a committed
fixture beside the MCP stub, harness/fixtures/theme.json, carrying
Everforest's #2d353b background and #d3c6aa foreground so the expected flat
colour stays #373e42. Every seeded config dir gets it as themes/fixture.json,
which /theme lists as Live Check Fixture and ctui reads as custom:fixture.

The check measures ctui's own rule — the background mixed 6% toward the
foreground — which a fixture with known values measures better than a file
whose contents vary by desktop. Its id keeps its name: the rule exists for
Omarchy's template, which writes a theme's background to inverseText.

The glass-flat step now rewrites that seeded file rather than an unnamed
"/theme file", HARNESS.md's care list forbids reading a theme out of the
operator's home as well as running `omarchy theme set`, and the theme-list
check says the fixture is not one of the 22 ctui Themes. Nothing else in the
skill, the harness or the operator page needs an Omarchy desktop.

Refs: #266
02-big allows Write, because the job writes calc.test.ts, and grades
`after: Edit`, because tool_order names one tool per side — so a Write-first
run scores as if no list were missed. The case file now says so and why it
stands: the medium case avoids it by dropping Write, which this job cannot,
and restructuring the case would change what the #220 job measures.

Refs: #266
…oval

MEMORY.md's own rule is to point at CONTRIBUTING.md for the procedure rather
than restate it, and the eval entry restated the command, the flags and the
18-session arithmetic verbatim. It now points, and keeps what CONTRIBUTING.md
does not carry: the case-file schema facts, the empty sandbox that makes
--scaffold necessary, the path-versus-name trap, the --max-cost-usd 0 load
check, and why each case keeps its own copy of scaffold.sh.

"Sonnet at medium effort" now lives in CONTRIBUTING.md with its reason, and
the operator page and SKILL.md point at it. The 20-40 minutes a run takes
stays on the operator page, which SKILL.md no longer repeats. The node-pty
approval stays in CONTRIBUTING.md's Setup, which HARNESS.md and SKILL.md now
point at instead of spelling out the commands again.

HARNESS.md's file table also lists the theme fixture and the harness
tsconfig, and drops the count that kept going stale.

Refs: #266
fix: the live-checks review findings, and no Omarchy machine required
@a1exk-dev

Copy link
Copy Markdown
Owner Author

ctui live checks — PARTIAL: 14 PASS, 2 FAIL, 14 NOT RUN (14/30 passed)

Scenarios 1, 2, 5 and 6 only, as scoped for this run. Scenarios 3, 4 and 7 were not driven, so their 14 checks are listed as NOT RUN rather than left out.

plugin ctui 0.5.0 (27f7afd)
engine @anthropic-ai/claude-code 2.1.292
node scripts/check.ts PASS exit 0, 10 tests across the suites' summaries, ~8 s
run 2026-10-13T19:58Z Sonnet 5.5, medium effort, 5 model turns (scenario 1: 1, 2: 1, 5: 1, 6: 2)
run dir /tmp/ctui-live-checks-mKDOpd

Sidebar docking

check expected measured
docking/width-42-at-150 body 42 columns wide at 150 #262626:107-149 on every body row; col 107 is the │ rule, so body is cols 108-149 = 42 columns PASS
docking/width-53-at-170 body 53 columns wide at 170 #262626:116-169, rule at col 116, body cols 117-169 = 53 columns PASS
docking/close-attempts-ignored footer row survives a ✕ click, ctrl+x x, and ctrl+x Tab then Esc ✕ at (row 0, col 148); after click, after \x18x, and after \x18\t + Esc the row claude cli: 2.1.292 was still on row 37 each time PASS
docking/fresh-110-prompt-docks no Sidebar before the prompt, footer after it at rest the dump held no │ column and no claude cli: row; after hi + Enter + 8 s, row 37 read │ claude cli: 2.1.292 PASS
docking/fresh-110-next-session-docks asked persisted, Sidebar docked at start of second launch .claude.json pluginPanes {"asked":[{"plugin":"ctui","id":"sidebar"}]}; second boot of the same config at 110×45 showed │ claude cli: 2.1.292 on row 37 before any prompt PASS
docking/dock-columns-override dock under a 70-column override vs the 42 asked scenario 3 not run NOT RUN

Sidebar layout and scroll

check expected measured
layout/git-glyph-columns glyph at content col 0, text at col 2, glyph fg per row branch row 4: U+E725 at col 110 (fg #999999), main from col 112 (fg #ffffff); sync row 5: U+E702 at col 110 (#999999), ↑0 at col 112; changes row 6: U+E728 at col 110 (#999999), +2 at col 112 (#4eba65), -0 at col 115 (#ff6b80) PASS
layout/frame-and-footer one footer row at 53, two at 42, │ in subtle #505050, no update icon footer rows: one at 170 (claude cli: 2.1.292 │ ctui: 0.5.0, inherit), two at 150 (rows 37 and 38); no update icon in the Sidebar columns. The │ cells read fg=default bg=#262626 at col 107 (150 wide) and col 116 (170 wide), not #505050 FAIL
layout/no-child-processes only the stub server and short git runs under the session, no claude -p or claude mcp list 10 one-second samples of the session's subtree: claude --plugin-dir …/ctui with its threads and one node …/fixtures/mcp-server.ts child; no git child was caught, no claude -p, no claude mcp list PASS

Render-site rewrites

check expected measured
rewrites/rule-gutter last text column 105 for prompt panel, reply and $ Bash; 106 for a ⎿ row rule at col 107. Prompt panel hard-wraps an unbroken token at col 102; $ Bash row ends at col 104 (the …); reply rows reach col 103. No ⎿ row in the main view (the Bash result is collapsed there). Nothing reached 105 FAIL
rewrites/ctrl-o-transcript both views show the same rows; each call carries its full result under it ctrl+o view (no Sidebar, 145-147 columns wide) lists the prompt, $ Bash, a ⎿ row with the full alpha-…-zulu output and the three git status lines, then the classifier row and the reply; the second ctrl+o restored the docked view with prompt, $ Bash and reply PASS

Toasts

check expected measured
toasts/main-screen-prefix ctui: ◆ Explore started: … watch found it after 1929 ms on row 23: ctui: ◆ Explore started: List top-level repo files, cols 148-197 of 200 PASS

Themes

check expected measured
themes/glass-flat one flat #363d41 over the body scenario 4 not run NOT RUN
themes/glass-frame dock color then #363d41 on frame cells scenario 4 not run NOT RUN
themes/latte-body #e5e7ed on the body both times scenario 4 not run NOT RUN
themes/inherit-no-paint same color on body and frame scenario 4 not run NOT RUN
themes/omarchy-mix body #373e42 under the fixture theme scenario 4 not run NOT RUN
themes/hover-colors hovered title colors vs off-Sidebar scenario 4 not run NOT RUN
themes/hover-underline u=1 on every hovered cell scenario 4 not run NOT RUN
themes/focus-arrows arrow colors per focus state scenario 4 not run NOT RUN
themes/theme-list-from-ctui 22 ctui Themes tagged from ctui scenario 4 not run NOT RUN
themes/dock-tint transcript and dock recolored in one step scenario 4 not run NOT RUN

The /ctui menu

check expected measured
menu/dock-pane-colors ctui text in the Sidebar's colors, engine text in the /theme's scenario 4 not run NOT RUN
menu/frame-fits-content frame hugs the content, ─ and footer directly under the last row at 120 columns: main screen last row 33 (Themes), ─ row 34, footer row 35, frame bottom row 37; Plugins screen last item row 33 (Agents & shells), ─ 34, footer 35; Themes screen last row 33 (↓ more), ─ 34, footer 35; the ╭…✕─╮ border spans cols 0-119 with ═ rule inside it PASS

Todo

check expected measured
todo/own-list-hidden no list under the spinner; footer ctrl+t to show tasks; engine list after ctrl+t while turn 1 ran at 150×45 the spinner row had only ⎿ Next: Step 2 under it, footer … ctrl+t to show tasks; after \x14 the rows ✔ Step 1, ◼ Step 2, ◻ Step 3 appeared and the footer read ctrl+t to hide tasks PASS
todo/spinner-animates in-progress glyph differs across two samples ~300 ms apart turn 2, Working part 1 Sidebar cell: U+F0AA5, U+F0A9F, U+F0AA2 over three samples at 302 ms steps PASS
todo/spinner-focused glyph still differs while the Sidebar holds focus after \x18\t, Working part 1: U+F0A9F, U+F0AA2, U+F0AA4. I did not read the focus state off the screen separately, so the focus is taken from the key press PASS
todo/no-dock-100-cols engine list under the spinner and footer ctrl+t to hide tasks turn 2 resized to 100×45: no Sidebar column; the engine list 3 tasks (0 done, 1 in progress, 2 open) with ◼ Part 1, ◻ Part 2, ◻ Part 3 was drawn, footer … ctrl+t to hide tasks PASS

Cost

check expected measured
cost/cold-cache-total cost row total equals the generator's scenario 7 not run NOT RUN
cost/cold-cache-arrival arrival under 3 s scenario 7 not run NOT RUN

Kept config dirs

scenario default-150: /tmp/ctui-live-check-default-150-BsvqpS

Run notes

  • Scenarios 2, 5 and 6 passed everything they ran, so their config dirs were removed.
  • Scenario 6 turn 1 ended before I had the in-progress text to sample, so the spinner checks ran on turn 2, which used longer steps. The model also moved the first sleep 25 to the background on turn 2.
  • In scenario 1 the first process-tree sample used a pgrep pattern that matched its own shell. I redid it with the session's PID.

The eval's guard case failed in the with-plugin arm: on "Fix the bug in add
in calc.ts" the section made the model start a task list in 3 of 3 runs, at
12-14 turns against 4 with the plugin off. Its skip clause came last and
asked for a list for "2 or more distinct steps", which a one-line fix meets
once read, edit and check are counted as steps.

The section now denies the list before it asks for one, calls the unit a
piece of work rather than a step, and says that reading, searching, editing
and checking the same piece are not separate steps. MEMORY.md names the
models each earlier result was measured on, and carries this one beside them.

Refs: #294
fix: hold the task list back from one-piece work
The first live-check run failed both. Neither was a ctui regression.

The frame check demanded the dock's rule in subtle #505050. That cell is
the engine's: under inherit it draws with the default foreground on the
#262626 dock, and a mod can set neither the glyph nor its colour, which
#99 already settled. It joins the engine-pinned expectations as
frame-rule-colour, and the check now reads the colour instead of
requiring one.

The gutter check demanded a last text column of exactly 105. paddingRight
2 is a ceiling, not a target: the run measured 102 for the prompt panel
inside its border, 104 for a truncated Bash row and 103 for a reply, none
of them spilling. The check now asserts that nothing reaches the rule's
neighbour and forces one reply row to the ceiling with an unbroken token,
so the reservation can fail if it ever stops being made.

CONTRIBUTING gains --trust-plugin, without which the eval stops at the
trust prompt, and says the non-zero exit at the default threshold of 1.0
is not the result.

Refs: #251
…ions

fix: make the frame and gutter checks assert what ctui controls
@a1exk-dev

Copy link
Copy Markdown
Owner Author

Re-run of scenario 1 only, on the expectations #296 corrected. The other 22 checks were not run.

ctui live checks — scenario 1 only: 7 PASS, 1 FAIL, 22 NOT RUN (of 30)

plugin ctui 0.5.0 (3fab9b4), package.json carries an uncommitted allowScripts change
engine @anthropic-ai/claude-code 2.1.292
node scripts/check.ts PASS exit 0 (the last summary block read 8 pass, 0 fail; the full count was cut off by tail)
run 2026-10-13, Sonnet 5.5, 2 model turns (one more than the scenario's 1, spent hunting a ⎿ row)
run dir /tmp/ctui-live-checks-rNscNE

Sidebar docking

check expected measured
docking/width-42-at-150 body 42 columns wide at 150 #262626:107-149 on every body row 0-40: rule cell at 107, body 108-149, 42 columns PASS
docking/width-53-at-170 body 53 columns wide at 170 #262626:116-169: rule cell at 116, body 117-169, 53 columns PASS
docking/close-attempts-ignored footer row still present after click, ctrl+x x, ctrl+x Tab then Esc ✕ at (0,148); claude cli: 2.1.292 still on row 37 after the click, after ctrl+x x, and after Tab+Esc PASS

Sidebar layout and scroll

check expected measured
layout/git-glyph-columns git glyph at content column 0, text at column 2 glyph at col 110 on rows 4-6: U+E725 branch, U+E702, U+E728, all fg #999999; text at col 112: main #ffffff, ↑0 #ffffff, +2 #4eba65 PASS
layout/frame-and-footer one footer row at 53, two at 42, │ on every body row at the dock's left edge, no update icon 170: one row 38 claude cli: 2.1.292 │ ctui: 0.5.0, inherit (separator fg #505050); 150: rows 37 and 38. │ at col 107 on all 41 body rows 0-40 (116 at 170), fg=default bg=#262626; labels #999999, values #ffffff. No update icon in the Sidebar. The engine's own ✔ Update installed toast is on row 40, left of the rule PASS
layout/no-child-processes no claude -p, no claude mcp list 10 s of pstree: the claude process with one node child running fixtures/mcp-server.ts (the ctuistub); no git run caught, no claude -p, no claude mcp list PASS

Render-site rewrites

check expected measured
rewrites/rule-gutter prompt, reply and $ Bash rows stay left of 106, reply token wraps at 105, ⎿ row reaches 106 rightmost text column: prompt panel 102, reply 104 (200-char token wrapped on two rows, cols 105 and 106 blank), $ Bash row 104 (ends …); rule at 107. No ⎿ row appears in the docked view (a second Read turn showed → Read a.txt · 3 lines); ⎿ rows show only in the ctrl+o transcript, where the Sidebar is hidden. The ⎿ column was not measured FAIL
rewrites/ctrl-o-transcript rows both views show, each call carrying its full result under it ctrl+o: full-width transcript (Sidebar hidden) with prompt, $ Bash call, ⎿ gutter-check-alpha-…-tango (full result), ⎿ Allowed by auto mode classifier, then the reply; ctrl+o again: docked view with the same prompt, truncated $ Bash … and reply PASS

NOT RUN: docking 3 of 6, toasts, themes, menu, todo, cost (22 checks).

scenario default-150: /tmp/ctui-live-check-default-150-oB6lPw

…ter row

The scenario-1 re-run passed the corrected frame check and failed the
gutter one on both halves of its expectation.

The ceiling was off by one in the same direction twice. With the rule at
107 the text region is columns 0-106, and paddingRight 2 takes 105 and
106, so the last text column is 104 - which is what MEMORY.md's gutter
entry has recorded since #102. The run measured the forced 200-character
token wrapping at 104 with 105 and 106 blank, the prompt panel at 102 and
the Bash row at 104: the reservation held and the check asked for the
wrong number. The arithmetic now sits in that entry beside the
measurement.

The check also asked for the column a gutter row reaches, which the docked
view never draws: the engine summarises a result inline there, and those
rows exist only in the ctrl+o transcript, where the Sidebar is hidden and
rewrites/ctrl-o-transcript already reads them. The control is now the
prompt box's own border rule, a row ctui does not pad.

The frame check's #505050 is explained too: it belongs to the separator
inside the footer's own text, not to the frame rule.

Refs: #251
fix: the gutter ceiling is column 104, and the docked view has no gutter row
tool_order names one tool per side, and the big case named Edit while
granting Write. On 2026-10-13 all six runs of both arms rewrote calc.ts
whole with Write, so Edit was never called and the grader failed on its
own naming: 0.00 against 0.00, where the captured traces show TaskCreate
before the first Write in all three with-plugin runs and no TaskCreate at
all without. Graded against Write, that is 1.00 against 0.00 - the case
has been under-reporting the nudge since it was written, and the 0.33 it
scored before was one run happening to reach for Edit.

The grader now names Write, the write this job cannot avoid because it
must create calc.test.ts. The remaining gap is narrower and runs the
other way: a run that edited calc.ts before starting its list and only
then wrote the test file would still pass. MEMORY.md carries the rule for
choosing the after tool, and the re-run's numbers.

Refs: #251
fix: grade the big eval case against the write it cannot avoid
The live-check harness loads node-pty's native binding, and the manual
`npm install-scripts approve node-pty` wrote this `allowScripts` entry
per checkout. Committing the entry builds the binding everywhere `npm ci`
runs, CI included, so the three places that told an operator to approve
it by hand have no step left to name.
build: install node-pty's binding from npm ci
@a1exk-dev

Copy link
Copy Markdown
Owner Author

ctui evals — the task-list nudge

18 sessions, 2 arms × 3 cases × 3 runs, no judge turn.

plugin ctui 0.5.0, suite as merged at b13c1f7 (before #298 renamed the big case's grader)
engine @anthropic-ai/claude-code 2.1.292
model claude-haiku-5-5, --ablation with-without --scaffold --trust-plugin --no-publish
run 2026-10-13T22:52Z · 258 s · $4.17
results ctui/evals/results/2026-10-13T22-52-41-240Z/ (gitignored); the 18 traces kept at /tmp/ctui-eval-traces-C5Hekc

case with ctui without delta turns, with → without
medium 1.00 / 1.00 / 1.00 0.00 / 0.00 / 0.00 +1.00 8, 8, 9 → 5, 5, 3
big (as graded) 0.00 / 0.00 / 0.00 0.00 / 0.00 / 0.00 0.00 18, 20, 19 → 6, 8, 6
big (re-graded, #298) 1.00 / 1.00 / 1.00 0.00 / 0.00 / 0.00 +1.00 as above
one-line 1.00 / 1.00 / 1.00 1.00 / 1.00 / 1.00 0.00 4, 4, 4 → 4, 4, 4

medium — the job that hides its steps and asks for brevity. Every with-plugin run opened a task list before touching the file (TaskCreate at step 1, 0 and 1; first Edit at 12, 8 and 10); no run without the plugin called TaskCreate at all. This is the suite's load-bearing result.

big — the grader failed on its own naming, not on what the arms did: it asked for TaskCreate before the first Edit, and all six runs rewrote calc.ts whole with Write, so Edit was never called in either arm. Re-reading the kept traces against the write the job cannot avoid gives TaskCreate at step 5, 3 and 3 ahead of the first Write at step 9 in every with-plugin run, and no TaskCreate and Write at step 3, 4 and 3 without — 1.00 vs 0.00. #298 renamed the grader list-before-first-write with after: Write; the next release's run is the first to print it.

one-line — the over-trigger guard, and the one number this release moved. Before #295 the with-plugin arm made 3 task lists for a one-line fix and took 12–14 turns against 4. After the reword it is 1.00 in both arms at 4 turns each: no list, no cost.

The command exits non-zero at the default threshold of 1.0 because the big case scored 0 as graded. That exit is not the result; the per-case deltas above are.

Known gaps on the record: the re-graded big case would still pass if a run edited calc.ts before starting its list and only then wrote the test file, and the medium case's grader still names Edit, which one without-arm run here sidestepped by writing the file whole.

@a1exk-dev

Copy link
Copy Markdown
Owner Author

Amendment to the scenario-1 re-run above, no new boot.

The one FAIL in that run was rewrites/rule-gutter, and both halves of its failure were the check's text, not ctui: it demanded a ⎿ row the docked view never draws (the engine summarises a call inline, → Read a.txt · 3 lines), and it named 105 as the gutter ceiling where the rule at column 107 and paddingRight: 2 put it at 104. The run measured the token wrapping at 104 with columns 105 and 106 blank, which is the pass.

#297 corrected both in the check. That row reads PASS on the evidence already gathered, so the scenario-1 heading is:

ctui live checks — scenario 1 only: 8 PASS, 0 FAIL, 22 NOT RUN (of 30)

The 22 NOT RUN stand: scenarios 3, 4 and 7 were never driven this release, so all ten themes/ checks and both cost/ checks are unmeasured for 0.5.0. The theme fixture that #293 committed has had its seeding exercised (scenario 1 seeded cleanly) but never its paint.

@a1exk-dev
a1exk-dev merged commit 52da5e6 into main Oct 14, 2026
4 checks passed
@a1exk-dev
a1exk-dev deleted the release/0.5.0 branch October 19, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spec: ctui 0.5.0 Map: ctui 0.5.0 Todo list

1 participant