Skip to content

🍒 [v1.207] fix(uipath-maestro-bpmn): a request for a draft stops at the draft gate - #4999

Merged
rockymadden merged 1 commit into
release/v1.207from
claude-fix-bpmn-draft-mode-bcd04e-CP-to-release-v1-207
Nov 19, 2026
Merged

rockymadden merged 1 commit into
release/v1.207from
claude-fix-bpmn-draft-mode-bcd04e-CP-to-release-v1-207

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

Problem

A request that asks for a BPMN draft sometimes gets the whole process built instead of a skeleton and the draft gate. In Studio Web Autopilot (dogfood row 17), Haiku replied "JiraIssueSummary: built, not runnable yet". The reason it gave: "'create a BPMN draft' reads as building the process".

Root cause: the draft cue list never named a draft. The line "Judge the intent, not the word" read as permission to treat "draft" as "build". Haiku also built a fully specified "Draft a BPMN for PO approval …" request, reasoning "the spec is complete".

Fix (skill text only, uipath-maestro-bpmn)

  • SKILL.md § Draft first and references/draft-skeleton.md When: a request for a draft, an outline or a sketch is the gate even when it describes the whole process. Wording that leans toward a draft without saying it (a rough version, a first pass) gets the draft, unless the request also says outright to build or deploy the whole process ("Here's the rough idea … Build the complete process"); that exception never applies to an explicit draft, outline or sketch request. A request with no such cue is a build. A "draft project" that asks only for source files to import later, with no upload or run, stays a build, as do brownfield edits and discovery-only requests.
  • SKILL.md Workflow intro: one line at the point where the agent picks draft or build, so a complete spec does not override the draft request.
  • Capability-router row reworded to "A draft, or the shape before building it".
  • No Critical Rule added. No flavor override changed; studioweb inherits the text.

Test

New smoke task tests/tasks/uipath-maestro-bpmn/smoke/draft_request_is_gate.yaml: a natural "can you create a BPMN draft for our expense claims? …" request. It is graded by check_draft_skeleton.py (bare boxes, a real gateway, full DI, validates), plus no registry discovery, no upload, no mid-turn question, and an llm_judge check that the reply ends at the gate.

The prompt says "don't upload it". The coder-eval sandbox has a real uip login, and an early red run uploaded a skeleton to the tenant. That line stops sandbox side effects and does not change what is graded. Earlier wording ("don't upload or run anything yet") was dropped: "yet" is itself a show-me-first cue and hid the bug on main.

Local coder-eval on the final prompt and wording (deterministic criteria; the llm_judge needs CODEX_BASE_URL and cannot run locally):

Task Model main (b79a350) fix
draft_request_is_gate Haiku 5.5 skeleton check failed 2/3 (red) 3/3
Sonnet 5.5 — 3/3
Opus 5.5 — 3/3
draft_word_is_not_gate Haiku 5.5 — 2/2 (still builds)
draft_skeleton Haiku 5.5 — 2/2

On the first prompt (no email step), main Haiku built 5/5 while Sonnet and Opus gated. One Opus skeleton had only 2 boxes; the shared checker requires 3, so the prompt now has a natural third step.

Autopilot (Studio Web)

Each build was swapped into the local Autopilot stack, the watchers restarted, and every chat opened new. Proof that the agent ran the intended build: the iframe's getBundledPluginReleases() equals sha256 of the plugin's dist/index.json, and the served skill files hash-match the build.

Build in-iframe release (= sha256 dist/index.json) SKILL.md draft-skeleton.md
stock 1.207.0-dev.1868 (draft text matches main) a8b15ef850db — —
fix, head 9878425 52b869f1a76b e41293cf1c6a 315e0a206f2d
fix, final head 6b27d3f 7c7a9d7e2d7f 426aa6870f6c afac81acf556
Prompt Model stock fix
Row 17: "Create a maestro BPMN draft to look at jira ticket for a project and get an agent summary for the issue, after done it should post in a channel. The variables should be input arguments" Haiku 5.5 build path 3/3 gate 3/3
"can you make a BPMN draft for how we onboard new vendors? procurement collects the vendor's details, legal reviews their contract, if the vendor is outside the US finance also runs a tax check, and then the vendor gets set up in SAP by our robot" Haiku 5.5 built 2/3, gate 1/3 gate 3/3
Row 17, final head 6b27d3f Haiku 5.5 — gate 1/1 (no questions asked)
Row 17 Sonnet 5.5 gate 1/1 gate 1/1
Row 17 Opus 5.5 gate 1/1 gate 1/1
Control: "create a BPMN process for leave requests: the employee submits a request, their manager approves or rejects it, and HR gets an email either way" Haiku 5.5 — build path (binds the email connection, no gate)

Sonnet and Opus already stopped at the gate on stock; the bug shows on Haiku.

Autopilot's clarifying questions got a real user's answers by one rule, the same on stock and fix: leave a connection or robot step unbound or a placeholder when offered; free-text values ("Which Jira project key?", "Which Slack channel?") → "make it an input argument"; otherwise the recommended or first option. The fix runs asked only about the Studio Web template's uipath:maestroEngineVersion tag, which refresh/validate reject. That issue is unrelated to drafting and tracked separately. Two stock row-17 runs ended in an Autopilot connection error after their build questions.

Terminal prompt matrix (default flavor, branch skill only, uploads blocked)

11 prompts across roles: beginner, power user, sales, FDE, analyst, manager, IT admin, ops, and three build controls including a source-only "draft project". Fix v2 ran on all three models, main on Haiku.

  • Sonnet and Opus on fix: every draft prompt that produced a BPMN was a bare-box skeleton. Every build control built.
  • Haiku on fix v2: improved over main on IT admin. Still built the power-user PO request, which led to v3; the Autopilot results above confirm v3 on Haiku.
  • Prompts that never say BPMN ("outline an employee offboarding process for me") sometimes get a prose document instead of a process. That is skill triggering, out of scope; in a Studio Web BPMN project the skill is always active.

🤖 Generated with Claude Code


Automatically cherry-picked from #4899 to release/v1.207 by @rockymadden.

…te (#4899)

* fix(uipath-maestro-bpmn): a request for a draft stops at the draft gate

"Judge the intent, not the word" let models read "create a BPMN draft"
as a build. Name the draft itself as a gate cue and keep the source-only
"draft project" as the one build exception.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(uipath-maestro-bpmn): an unclear draft-or-build request gets the draft, however complete its spec

Haiku still built a fully specified "Draft a BPMN ..." request ("the spec is
complete"). Name outlines and sketches with drafts, send a request that could
mean either to the draft, and say so where the Workflow starts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(uipath-maestro-bpmn): give the lean-to-draft rule a test and its exceptions

The routing line now defers to Draft first for the build cases, the
draft-skeleton exception no longer claims to be the only one, and a draft
leaning request needs a draft-like cue. The smoke task gains a natural third
step so a compliant skeleton clears the 3-box check, and fails on a mid-turn
question.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(uipath-maestro-bpmn): the draft routing line exempts brownfield edits; every draft cue counts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(uipath-maestro-bpmn): Draft first names the brownfield exception too

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(uipath-maestro-bpmn): an explicit build instruction outweighs a draft-leaning cue

"Here's the rough idea ... Build the complete process" must build, not gate.
Explicit draft, outline and sketch requests stay the gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(uipath-maestro-bpmn): scope the build exception to draft-leaning wording

Explicit draft, outline and sketch requests always gate; only leaning wording
yields to an outright build or deploy instruction. The Workflow line now names
the explicit cue only and points to Draft first for the rest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@rockymadden
rockymadden merged commit 91caca5 into release/v1.207 Nov 19, 2026
3 checks passed
@rockymadden
rockymadden deleted the claude-fix-bpmn-draft-mode-bcd04e-CP-to-release-v1-207 branch November 19, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants