Repository navigation
Conversation
…eb output (run 2026-11-19) Accept =js:vars.<id> and start-event-scoped inputs, prefer the edited file over a work-folder copy, read get-by-id from the node metadata, resolve the Studio Web layout in customer_escalation_triage, grade solution creation by outcome, zero the advisory paginated criteria, and preflight the JDBC connection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Claude finished @DevMomo's task in 1m 29s —— View job Coder-eval task lint (advisory)I linted the 7 task YAMLs this PR changes. Verdicts: 0 Critical, 2 High, 0 Medium, 4 Low, 1 OK. (Both High verdicts come from a Critical theme, downgraded one level under the rubric's theme rule.) Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge. I scored each whole task, not only the lines this PR touches, so some findings below predate the PR. Evidence of passing run
Per-task lint
|
|
Claude finished @DevMomo's task in 2m 12s —— View job SummaryThis PR fixes BPMN coder_eval graders that rejected correct Studio Web output in run 2026-11-19. It accepts Change-by-Change Review1.
|
| Area | Status | Notes |
|---|---|---|
| Frontmatter | OK | No skill files touched |
| E2E Tests | Issue | Prompt and grader disagree in customer_escalation_triage; advisory weight inconsistent; no passing-run claim |
| Skill Body | OK | Untouched |
| References & Assets | OK | Untouched |
| Repo Hygiene | OK | Scoped to tests/tasks/uipath-maestro-bpmn/; fixtures contain tenant GUIDs (Low) |
Issues for Manual Review
- Fix 1 risk (the PR body raises this too): if the canvas only treats a literal
=vars.start/end source as an argument bridge (ArgumentUtils.ts:7-23), then accepting=js:vars.<id>lets through output that looks right but doesn't bridge at runtime. The live canvas check is still pending; this should block merge until it lands. - Fix 4: whether the runtime treats
GET+ metadataGETBYIDas get-by-id is unverified, as the PR body says. - JDBC preflight folder: the preflight asserts the JDBC connection lives in folder
uipath-maestro-flow. Confirm that is where the BPMN suite's Databricks fixture connection lives, and not just the Flow suite's. - I couldn't run
pytest tests/tasks/uipath-maestro-bpmn/_sharedhere because the command needed approval in this runner. The "2692 passed" figure comes from the PR body and was not re-verified.
Conclusion
The grader changes are well targeted and well evidenced. Before merging:
- Settle the
customer_escalation_triagemismatch (Medium): the prompt pins a path the graders no longer require. - Post the live canvas check for
=js:vars.: it decides whether fix 1 is "accepted" or actually "correct".
The rest is Low polish: the stale comment in _operation_matches, the advisory weights on the four new outcome checks, and lowercasing the local variables in both checkers' main().
Fixes graders in
tests/tasks/uipath-maestro-bpmn/that rejected correct Studio Web output in run 2026-11-19_09-23-02 (72.2%). Test files only; no skill text touched. Fixtures under_shared/fixtures/run_2026_11_19/are the agents' real artifacts, byte for byte.Fixes
=js:vars.<id>— newbpmn_check.vars_ref()(accepts=vars.<id>and=js:vars.<id>, rejects scripts like=js:vars.a + 1); used incheck_simple_approval_bpmn.py,check_script_jint_guidance.py(bridges, end mapping, args, business result) andbpmn_live.input_echo_ids. Other=vars.greps: only a docstring example remains.find_bpmn_file: among several hinted matches the shallowest wins (Foo-work/Foo.bpmnsorted beforeFoo.bpmnbecause-<.).is_scratch_bpmnunchanged._operation_matchesreads the catalog verb from the node metadata (activityMetadata.activity.configuration.httpMethod); a bareGETwith no metadata stays a list read;GETBYIDin contextmethodstill matches.customer_escalation_triagepackage/behavior graders and the validate criterion resolve the project viaresolve_bpmn_file(new_shared/print_bpmn_path.pyfor the shell command).paginated_reference_lookupadvisory criteria set toweight: 0(+pass_threshold: 0), per fix(tests): route-specific command telemetry must not move the score #3516:pass_threshold: 0alone never fails a task but still moves the score.solution new|initcommand criterion replaced by the workspace-outcome check (.uipxorproject.uiproj, same as the Flow twin) inevaluate/evaluator_type_choice,local_crud,no_auto_upload,simulation/simulation_crud(same pattern in all four).jdbc_databricks_querygetspreflight_connections.py uipath-uipath-jdbc=uipath-maestro-flowinpre_run.Evidence
FAIL: completion end must bridge…→ OK; script-jintFAIL: riskScore is neither bridged…→ OK; testmanager--all-opsget a record … missing→ OK;find_bpmn_file('ExpenseClaim')returnedExpenseClaim-work/ExpenseClaim.bpmn→ExpenseClaim.bpmn.=js:vars.riskScoreValue * 2(jint),=js:vars.decisionValue + 'x'/ undeclared id / no prefix / missing mapping (approval), input scoped to another element or unscoped (jint), bareGETwithout metadata verb, list op vs get-by-id node.python3 -m pytest tests/tasks/uipath-maestro-bpmn/_shared -q: 2692 passed (new:test_run_2026_11_19_graders.py, 33 tests).@uipath/maestro-validation0.1233.0validateBpmnXmlon the three real artifacts: 0 errors. It does not flag an undeclared=js:vars.xor a wrongly scoped input either (known limit), so it supports "accepted", not "correct".origin/develop:sourceis serialized verbatim:src/services/serialization/bpmn-moddle.ts:606,bpmn-from-xml-headless.ts:885,966. Prefixes are=vars./=js:(src/services/uipath/variables/VariableConstants.ts:7-8);removeVariablePrefixstrips=js:thenvars.(src/services/validation/shared/VariableExpressionUtils.ts:62-68,114-123); reference regex is prefix-agnostic (:16-17); non-custom mapping sources are not validated (VariableValidationUtils.ts:323-331).src/reducers/canvasSlice.ts:224-247,2191,UnifiedBuildDataManager.tsx:271,322;VariableValidationUtils.ts:154-164accepts a ScriptTask read via upstream node outputs.riskScore←scriptResponse(the=js:vars.<id>end mapping, start-event-scoped internal inputs).'Auto-approve standard expense' writes 'decisionValue' without a variable update), unrelated to the mappings. End event Arguments showdecision←decisionValue; the XML view keepssource="=js:vars.decisionValue".decisionValuein the End event's variable picker produced<uipath:output name="decision" type="string" source="=js:vars.decisionValue" var="decision" custom="true" />.Not proven
GET+ metadataGETBYIDas canonical canvas output; it copies whatever method string the activity config carries (IntsvcActivityConfigurationUtils.ts:333-345), quick-add copieshttpMethodintomethod(QuickAddSelectionUtil.ts:49,63), and the only in-repo fixture has contextmethod=GETBYID(xml-serialization.test.ts:812). The grader now accepts both shapes; whether runtime treatsGET+GETBYIDas get-by-id is unverified.command_executedtelemetry (e.g.bpmn init, eval list inlocal_crud) left untouched.🤖 Generated with Claude Code