Repository navigation
🍒 [v1.207] fix(uipath-maestro-bpmn): a request for a draft stops at the draft gate - #4999
Merged
rockymadden merged 1 commit intoNov 19, 2026
Conversation
…te (#4899) * fix(uipath-maestro-bpmn): a request for a draft stops at the draft gate "Judge the intent, not the word" let models read "create a BPMN draft" as a build. Name the draft itself as a gate cue and keep the source-only "draft project" as the one build exception. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(uipath-maestro-bpmn): an unclear draft-or-build request gets the draft, however complete its spec Haiku still built a fully specified "Draft a BPMN ..." request ("the spec is complete"). Name outlines and sketches with drafts, send a request that could mean either to the draft, and say so where the Workflow starts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(uipath-maestro-bpmn): give the lean-to-draft rule a test and its exceptions The routing line now defers to Draft first for the build cases, the draft-skeleton exception no longer claims to be the only one, and a draft leaning request needs a draft-like cue. The smoke task gains a natural third step so a compliant skeleton clears the 3-box check, and fails on a mid-turn question. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(uipath-maestro-bpmn): the draft routing line exempts brownfield edits; every draft cue counts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(uipath-maestro-bpmn): Draft first names the brownfield exception too Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(uipath-maestro-bpmn): an explicit build instruction outweighs a draft-leaning cue "Here's the rough idea ... Build the complete process" must build, not gate. Explicit draft, outline and sketch requests stay the gate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(uipath-maestro-bpmn): scope the build exception to draft-leaning wording Explicit draft, outline and sketch requests always gate; only leaning wording yields to an outright build or deploy instruction. The Workflow line now names the explicit cue only and points to Draft first for the rest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
github-actions
Bot
requested review from
a team,
dmetzgar and
tmatup
as code owners
November 19, 2026 15:30
dmetzgar
approved these changes
Nov 19, 2026
rockymadden
deleted the
claude-fix-bpmn-draft-mode-bcd04e-CP-to-release-v1-207
branch
November 19, 2026 20:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A request that asks for a BPMN draft sometimes gets the whole process built instead of a skeleton and the draft gate. In Studio Web Autopilot (dogfood row 17), Haiku replied "JiraIssueSummary: built, not runnable yet". The reason it gave: "'create a BPMN draft' reads as building the process".
Root cause: the draft cue list never named a draft. The line "Judge the intent, not the word" read as permission to treat "draft" as "build". Haiku also built a fully specified "Draft a BPMN for PO approval …" request, reasoning "the spec is complete".
Fix (skill text only,
uipath-maestro-bpmn)SKILL.md§ Draft first andreferences/draft-skeleton.mdWhen: a request for a draft, an outline or a sketch is the gate even when it describes the whole process. Wording that leans toward a draft without saying it (a rough version, a first pass) gets the draft, unless the request also says outright to build or deploy the whole process ("Here's the rough idea … Build the complete process"); that exception never applies to an explicit draft, outline or sketch request. A request with no such cue is a build. A "draft project" that asks only for source files to import later, with no upload or run, stays a build, as do brownfield edits and discovery-only requests.SKILL.mdWorkflow intro: one line at the point where the agent picks draft or build, so a complete spec does not override the draft request.Test
New smoke task
tests/tasks/uipath-maestro-bpmn/smoke/draft_request_is_gate.yaml: a natural "can you create a BPMN draft for our expense claims? …" request. It is graded bycheck_draft_skeleton.py(bare boxes, a real gateway, full DI, validates), plus no registry discovery, no upload, no mid-turn question, and anllm_judgecheck that the reply ends at the gate.The prompt says "don't upload it". The coder-eval sandbox has a real
uiplogin, and an early red run uploaded a skeleton to the tenant. That line stops sandbox side effects and does not change what is graded. Earlier wording ("don't upload or run anything yet") was dropped: "yet" is itself a show-me-first cue and hid the bug on main.Local coder-eval on the final prompt and wording (deterministic criteria; the
llm_judgeneedsCODEX_BASE_URLand cannot run locally):draft_request_is_gatedraft_word_is_not_gatedraft_skeletonOn the first prompt (no email step), main Haiku built 5/5 while Sonnet and Opus gated. One Opus skeleton had only 2 boxes; the shared checker requires 3, so the prompt now has a natural third step.
Autopilot (Studio Web)
Each build was swapped into the local Autopilot stack, the watchers restarted, and every chat opened new. Proof that the agent ran the intended build: the iframe's
getBundledPluginReleases()equals sha256 of the plugin'sdist/index.json, and the served skill files hash-match the build.dist/index.json)SKILL.mddraft-skeleton.mda8b15ef850db52b869f1a76be41293cf1c6a315e0a206f2d7c7a9d7e2d7f426aa6870f6cafac81acf556Sonnet and Opus already stopped at the gate on stock; the bug shows on Haiku.
Autopilot's clarifying questions got a real user's answers by one rule, the same on stock and fix: leave a connection or robot step unbound or a placeholder when offered; free-text values ("Which Jira project key?", "Which Slack channel?") → "make it an input argument"; otherwise the recommended or first option. The fix runs asked only about the Studio Web template's
uipath:maestroEngineVersiontag, whichrefresh/validatereject. That issue is unrelated to drafting and tracked separately. Two stock row-17 runs ended in an Autopilot connection error after their build questions.Terminal prompt matrix (default flavor, branch skill only, uploads blocked)
11 prompts across roles: beginner, power user, sales, FDE, analyst, manager, IT admin, ops, and three build controls including a source-only "draft project". Fix v2 ran on all three models, main on Haiku.
🤖 Generated with Claude Code
Automatically cherry-picked from #4899 to
release/v1.207by @rockymadden.