Repository navigation
feat(uipath-maestro-case,uipath-planner): SDD schema conformance — combined merge PR (MST-15966) - #3691
feat(uipath-maestro-case,uipath-planner): SDD schema conformance — combined merge PR (MST-15966)#3691roberts-cliff wants to merge 63 commits into
Conversation
…15966) Five fixture SDDs, each an existing fixture with one defect injected from the measured defect inventory: a table row that loses its leading pipe, curly quotes in one =js: cell, a renamed Case Variables heading, a trailing-(annotation) name collision, and an empty document. manifest.json names every defect, its line, its seed blob, and what one dev build did with it. That record is not the assertion: the future degraded_sdd_conformance eval (E4) asserts the gate names each line. Agent-authored, so it proves the gate catches the measured classes and nothing past them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…fest E4 asserts the gate names each defect's recorded line, so the manifest has to stay true to the fixtures. Runs in the required maestro-case checker job: every fixture listed, each recorded line holds the degraded text and the named defect, and each fixture is its seed plus one line while the seed is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…d corpus A row without its leading pipe is now read, not rejected (CLI stack A), so pipe-dropped-row expects read-as-seed; the other four expect the gate to reject naming their line or the document. Records stack A's results. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ure's expect Runs `uip maestro case sdd validate` on every fixture in the degraded corpus manifest: reject-naming-line exits non-zero naming the recorded line, reject-naming-document exits non-zero, read-as-seed exits zero and parses to exactly its seed's Model. Skips while the installed CLI has no `sdd validate`, like the corpus-clean check. Records what cli 367bfdb69's gate returned for each fixture. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rning at line 312 Cliff's call: a row without outer pipes stays readable and `sdd validate` reports it as the non-fatal SDD_ROW_WITHOUT_OUTER_PIPES. E4 now asserts that warning, code and line, alongside the read-as-seed model equality. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…the CLI enforces checklist_coverage.py seeds one violation per planner checklist item into sla_from_sdd and classifies what the installed CLI does: validate, validate-warns, convert (refusal or a new Unresolved item, compared by kind and location), or silent tagged by whether the parsed Model changes. Rows that convert also record what plain `case validate` raises on the converted plan, which is after the handoff and so is not gate coverage. On cli 367bfdb69, checklist_coverage.json: validate catches 0 of 26, convert reports 8, 18 are silent (10 of them change the Model). The checklist (skills S4) cannot be deleted until every row reads validate. Mutations regenerate from the seed each run; one that stops applying is reported, not skipped. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…date now catches items 7 and 20 cli#4678 (A7) turns a literal \n and an untyped recipient into parse errors (SDD_LITERAL_NEWLINE, SDD_RECIPIENT_UNTYPED). Tally: validate 2, convert 8, silent 16 of 26. The previous record (367bfdb69: 0/8/18) is in git history. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… — unchanged (2/8/16) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Claude finished @roberts-cliff's task in 5m 26s —— View job Coder-eval task lint (advisory)
Coder-eval task lint (advisory)4 task YAMLs changed; verdicts: 0 Critical, 0 High, 0 Medium, 0 Low, 4 OK. Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge. Evidence of passing run✅ Author confirms passing run: "CI on archive 36882965827 (cli Per-task lint
|
|
Claude finished @roberts-cliff's task in 2m 38s —— View job PR Review: SDD Schema Conformance (MST-15966)
Early finding: Personal path |
…ed CLI messages Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…-15966) S2 of the SDD-schema stack. Parses each case fixture SDD through the installed uip and requires Success, at least one stage, and no parse notes. Scoped to tests/tasks/uipath-maestro-case: a tests/tasks-wide glob would collect planner documents that are correctly refused as case SDDs. The degraded corpus is excluded wholesale. A refused document gets a recorded verdict, never an edit to make it pass. Runs in its own advisory workflow on @uipath/cli@latest, which has no `sdd` command yet: the test skips and says so, then checks by itself once one ships. Not required, since an npm outage or bad release would block every PR. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…finalization skips the layers guide before the first Read Checklist item 6 asked execute-connector-activity for **Resolved Resource:**, which the template's Connector Task Detail does not have; the gate built from it (cli#4766) fired SDD_TASK_BLOCK_INCOMPLETE 86 times across 48 S4 runs and drove 15-28 extra repair ops on the two dearest tasks. cli 54d04b951 drops the requirement; the checklist now matches the template. The finalization read budget sat ten lines below "read all three, in parallel", so 8 of 10 finalize runs read the layers guide anyway. SKILL.md is loaded before any Read, so the exception now lives there. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…nd sees The coverage record now seeds every checklist item (cli 54d04b951): validate 20, validate-warns 2, convert 7, excluded 1, silent 3 (17-19: data closure), needs-draft 4 (30-33: draft parity, which no CLI command can read). Items 28 and 35 are caught, by neighbouring rules (SDD_RULE_NOT_FOR_SLOT, SDD_WAIT_FOR_USER_UNPAIRED). The planner checks those seven inline on the written file — 17-19 always, 30-33 when finalizing a draft — each stated in full in the lane, no subagent, no checklist read. Everything else stays sdd validate's; the full walk remains only the no-sdd fallback. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… Value, as the template does The template's Timer Task Detail has no **Duration:**; the gate built from the checklist fired SDD_TASK_BLOCK_INCOMPLETE "lacks **Duration:**" 4 times in two S4 runs on cli 36b85f8e4. Same class as the Resolved Resource fix. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… 13 moves to the gate
SDD_SLA_REFERENCE_UNRESOLVED (cli#4766) catches both item 13 seeds: an
sla-status-change naming an undeclared SLA title, and the unclosed form from a
failed S4 run (`sla-status-change("Assess (`assess`)`). Tally: validate 22,
validate-warns 2, convert 6, excluded 1, silent 3 (17-19), needs-draft 4
(30-33). cli c2b3c72db (the #4743 rollup tip, adds #4774) changes no gate check
or convert output, so the verdicts hold there.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…th arms runnability_disclosed builds the all-placeholder sla_from_sdd case and grades that the completion report says the case cannot run and names where it stops. runnability_silent builds the runnable expense case (all resources deployed) and grades that the report says nothing about runnability. Each arm's precondition is deterministic (check_runnability_shape.py --expect blocked|runnable, falsified on a 29-placeholder user case and a resolved fixture); only the reply is judged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… line, not a numbered report field runnability_silent failed 2/2: on a fully runnable case both completion reports printed a Runnability section with zero counts, although field 6 said to omit it at zero. A numbered field reads as a slot to fill. The disclosure now sits after the field list as two lines written only when a count is non-zero, with an explicit 'write nothing' for the clean case. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ity items cli 6f3b4ba96 (the #4743 rollup; items 17-19 in cli#4766 at 7619dd18b) puts data closure behind sdd validate: SDD_VARIABLE_NEVER_PRODUCED, SDD_OUT_VARIABLE_UNSET, SDD_BUTTON_TARGET_UNKNOWN. Coverage over all 35 items: validate 25, validate-warns 2, convert 6, excluded 1, needs-draft 4, silent 0. The planner now checks only items 30-33, and only when finalizing a draft; checklist item 17 also names a Buttons assignment as a producer, as the gate does. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…; never stamp a marker into one resolve wrote #3671's "never change a field resolve wrote" did not move the refusal rate (3/6 runs before, 3/6 after, hitl smoke, cli f7c68a2e9): planning §3.4 still told the agent to record <UNRESOLVED: ...> in the entry's taskTypeId slot, and Step 9.1 / placeholder-tasks / Rule 9 keyed the placeholder on that marker. So the agent opened resolve's entry to stamp it, re-ran the lookup under the task name ("37 entries checked"), rewrote searchQuery.name and cacheFile, and convert --resolved refused. Resolve already writes selected:null and a rationale for an empty lookup; that is now the placeholder signal, and the marker belongs only to entries the agent writes itself. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…verbatim, not by reference runnability_disclosed passed 2 of 4: the misses listed every placeholder but never said a run would stop or where. The report paragraph now carries the two lines as a fill-in template instead of pointing at the Phase 6 section. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…le, receipt or not; normalize only what the CLI cannot read #3690 sent every SDD without the planner receipt to planner normalization. Most case-eval SDDs (44 of 49) are real designs written outside the planner, and the route fired inconsistently on them (init-validate normalized, case_exit_full and timer_connector_from_sdd built directly). Rule 2 now decides three ways: receipt present → build; no receipt but sdd parse reads it whole → build it directly, exactly the same; no receipt and the parse fails → normalize through the planner, keeping sdd.original.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…interrupts; parallel oversight is start-task or notify-only The docs taught a non-interrupting secondary SLA "oversight lane" (Interrupting: No). The case compiler sets cancelStagesAndTasks for any secondary-stage entry, whatever isInterrupting says, and cli#4804 now refuses the shape. Falsified 2026-10-05 rather than taken on report: an isInterrupting:false secondary case-entered entry packs to "cancelStagesAndTasks":true, and debug instance 6ecbe140 ran only the secondary lane while the primary stage, also on case-entered, never started. Rule 22, the secondary-lane anti-pattern, sla-response-shapes §1/§4/§5, the stages and stage-entry plugins, the planner layers guide (carve-out, response table, Interrupting cell) and the template's secondary rule now say Yes on every secondary entry, and route work-keeps-running requirements to start-task or notify-only. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… an at-risk SLA rule All 14 sla-status-change rows in the eval fixtures were 2-argument breach rows, so nothing exercised the template's 3-argument at-risk form. This variant of sla_from_sdd moves Intake's start-task to at-risk and grades the built rule: Intake's slaId plus the escalationId of the 'Intake at risk' at-risk escalation. Falsified on cli 6f3b4ba96, which read argument 3 as the SLA title: convert left the rule with no slaId at all. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… head (cli 90b5247e6) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…d-stage-exited carries an IF A secondary stage entered on selected-stage-exited(X) with no IF takes over every time X completes normally. Seen in a planner run (loan-origination, Withdrawn lane on any exit of Intake and Loan Setup) and in 10 corpus documents. cli#4766 now refuses it as SDD_SECONDARY_ENTERS_ON_COMPLETION (item 36). The checklist (the planner's no-sdd fallback walk), the template count and the coverage seed follow; coverage on cli fe4c3e3e2 reads it validate. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ceipt-carrying SDD; without the receipt it normalizes Rule 2's three-way decision sends an SDD with no planner receipt that fails sdd parse to planner normalization (c), but its refusal clause said any parse/format/resolve/convert refusal stops the build. The suite showed which one wins: all four no-receipt SDDs refused on a Case Variables Type `number` (io-binding-matrix, three hitl-quality tasks) stopped and asked the user, scoring 0.00 at $0.18-0.46. The stop now names case (a) only, and (c) names a single-cell refusal as its own. E1a's fixture carries the receipt, so its hand-back is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…lookup fields are untouchable registry-handoff-ambiguous passed 14/14 nightlies on main and failed both stack runs (0.14, 0.57). #3671's "never change a field resolve wrote" listed `selected`, so after the user picked a candidate the agent recorded a gateDecision but left selected null and bound nothing. convert --resolved compares stage, task and searchQuery, not selected: the rule now protects the lookup fields and names the one write a gate answer requires (selected to the chosen match, with gateDecision pick:<name>). An ambiguous entry goes to the gate before it can become a placeholder. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ompleted
The Lifecycle gates table allowed only required-tasks-completed and
wait-for-connector on a completing stage row, matching the CLI gate's item 9.
Both were wrong (cli-62, from @uipath/case-schema: the schema, compiler and
scheduler all accept selected-tasks-completed as a TasksCompleted completion
condition; cli fixed in 990b37cda). Planner normalization rewrote a real SDD's
selected-tasks-completed("Approve Contract") completion row to
required-tasks-completed, changing which tasks complete the stage, and
hitl quality_04 failed on it. Checklist item 9 reads this table, so the
fallback walk follows.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…gainst runtime truth case_breach_oversight graded isInterrupting false, which the runtime falsified (debug instance 72162d9b: the breach exited the active stage and cancelled its task). It now requires isInterrupting true on the still-secondary, non-required lane and judges whether the reply tells the user the work will pause. quality_01 accepts one trivial required task in Closed: an empty stage with a required-tasks-completed exit never completes (instance cc7223dc). io-binding accepts double or integer where the SDD wrote number, the form planner normalization produces. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…et layout The SDD declares no task set, so convert puts each timer, with its own current-stage-entered rule, in its own set. Debug instance 2df59883 shows two timers in separate sets start and finish together, so the shared-set assertion graded layout, not behaviour. The entry-rule assertion stays. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The oversight eval accepted only a lane, but the skill's parallel-oversight shape is a start-task. Debug instance 583c4102 shows a root-SLA breach during Intake starts such a task even in not-yet-entered Decision, cancels nothing, and the case completes, so it truly runs alongside. The checker now passes either a non-required breach task with no new stage, or a secondary lane with isInterrupting true; the judge grades the reply's claim against the shape built. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ng input The io_binding fixture writes the non-template type number on purpose: its eval grades the no-receipt, parse-fails route to planner normalization (run 37494190705: the planner rewrote exactly those three cells, build 1.00). None of the parse-clean test's three verdicts fit, and an exclusion would stop checking the file. A fourth verdict, routing-input, pins the exact refusal instead, so a new defect in the file or a CLI that starts accepting number both fail. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…st their seed read-as-seed required sdd validate to exit zero, but the seeds are hand-written pre-template SDDs that strict validate rejects on their own (32 issues on e2e_expense_runnable: no Planner Handoff, task envelopes or activation modes). The fixture now must give the seed's exit code and issues plus exactly its recorded warning, and the seed's Model. table-row-missing-leading-pipe passes on 1.205: the row and everything below it are read, with the warning at 312. Records the 1.205 measurement beside the old reader's. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…defect The norm() merge it was built for is fixed; what remained was a non-identifier variable name that convert wrote as an unreachable id, retyped to string, and both validators passed. cli #4804 now refuses it at line 70 with SDD_VARIABLE_NAME_NOT_IDENTIFIER, so reject-naming-line passes as recorded. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…names cells A no-receipt SDD one cell from parsing (a Case Variables Type of number) took the full re-render: about 150 s on Opus, 20 s of it re-typing the original and 75 s regenerating a document that needed one word changed. Nine of the ten normalizing evals were cell-level. The lane now copies the original with one cp, edits only the named cells when parse read the stages, and hands back once it parses whole (case Rule 2 (b)); the template render runs only for a refusal that is not cell-level. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…from sdd format The test hand-built a sidecar with no schemaVersion; the CLI's model schema moved to 2.0.0 and refuses an unversioned file as 1.0.0. The envelope now comes from a real sdd format run, so the test follows the CLI's schema. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On Opus the design lane's gate got 40% faster once sdd validate replaced the checklist walk, but the writing phase gave it back: both finalize tasks ran validate twice on half-written files and repaired errors the next append would have cleared, and 4 of 6 design tasks ran --help or --version to rediscover commands the lane already states. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Closing the MST-15966 SDD-quest stack; it isn't landing as is. The head is kept locally. |

MST-15966. The whole SDD-schema change in one PR, so it lands in one merge. Review it on the stack; merge this. This branch always equals the tip of the stack (#3717), so the diff here is the stack's combined diff. Don't merge the stack PRs one at a time; close them once this merges.
Lands with UiPath/cli#4743 (the CLI rollup). Merge the two close together. Two CLI PRs refuse SDDs that used to pass, and the repair loop that fixes such a document is in this PR. If the CLI lands alone, a planner-written SDD can fail its build with no way to repair it.
4e203e449)sdd validateand discloses repairs (E1b)sdd.mdroute to the planner. Activation gate: case 98.0%, planner 98.9% (85% floor)sdd formaton every handoffsdd formatkeeps every line a person wrotesdd resolvewrotecli_archive_run_id), with a manifest-exact auditsdd.original.md"placeholder"string (skills half of cli#4589); what the skill writes stays as it is until the CLI shipssdd validateis the planner's conformance gate; the 35-item walk survives only as a fallback when the CLI has nosddcommand. Walk vs no walk: $1.98 → $1.44 mean, 13.7 → 8.6 min, same quality on the measured items. Not ready: the coverage record measures 26 of the checklist's 35 items, and the walk is the only check on the other 9 (see #3717)Evidence
52ad1e565):Still owed
sdd resolvewrote (MST-15966) #3671) on one build.🤖 Generated with Claude Code