Repository navigation
feat(uipath-doc-strategy-select): add file handling strategy selection skill - #3506
SummerWangUiPath wants to merge 43 commits into
Conversation
|
Claude finished @SummerWangUiPath's task in 3m 36s —— View job Coder-eval task lint (advisory)2 task YAMLs changed; verdict counts: 0 Critical, 1 High, 0 Medium, 0 Low, 2 OK. Rubric: .claude/commands/lint-task.md. This check is advisory and never blocks merge. Evidence of passing run❌ High — PR body does not claim the changed coder-eval tasks have been run and passed. The body mentions the e2e check script (
Per-task lint
|
|
Claude finished @SummerWangUiPath's task in 5m 38s —— View job PR Review: feat(uipath-doc-mode-select)
SummaryAdds a new Change-by-Change Review1.
|
| Area | Status | Notes |
|---|---|---|
| Frontmatter | OK | Valid YAML, name matches folder, description under 1024 chars, starts with brand, has → redirects |
| E2E Tests | OK | 1 smoke + 1 e2e, correct tags (all 4 required dimensions), side-effect criteria, co-located validation script |
| Skill Body | OK | Full section order, 8 numbered Critical Rules, routing table, anti-patterns |
| References & Assets | OK | 6 reference files, all kebab-case with -guide.md/-template.md suffixes, all links resolve |
| Repo Hygiene | OK | CODEOWNERS, skill-status.json, skills.sh.json, README all updated. No secrets. Provenance scrub clean |
Issues for Manual Review
- Cross-skill link accuracy. The 35 cross-skill delegation links all resolve to existing files, but I cannot verify that the specific sections referenced (e.g., "§ Classifier Variant", "§ Plugin Index and § Node Selection Heuristics") exist within those target files. The PR description notes that
skills:check-linkspasses (6960 links, all resolve). - Domain accuracy of routing decisions. The 21-case library encodes routing heuristics based on UiPath product behavior. I cannot verify whether the described capabilities (e.g., Summarize serializing as
uipath.pattern.deep-rag, classifier models sharing theuipath.ixp.*pattern) are factually accurate against the current UiPath platform. - Sibling skill frontmatter. The PR description deliberately defers adding
→redirect clauses touipath-ixp,uipath-maestro-flow,uipath-agents, anduipath-planneruntil this skill's own recall is measured. This means those skills won't redirect confused queries touipath-doc-mode-selectyet — reasonable for an introducing PR, but should be tracked.
Conclusion
Clean PR. Structure, conventions, test coverage, registry updates, and provenance scrub all pass. The skill fills a real gap — mode routing before authoring — and does so without duplicating the content owned by sibling skills. The 21-case library is well-anonymized and teaches boundaries rather than recipes. No issues to block merge.
…skill Choosing between Extract, Classify & Split, Summarize, Analyze files, and persistent-index search is a decision nobody owned. uipath-maestro-flow documents each node once you know which node you want; uipath-agents documents each context-grounding mode once you know which mode you want; uipath-planner routes across products, not within document processing. The question "which mode does this workload need?" had no entry point, and the expensive mistakes are made before any file exists. This skill selects and hands off. It deliberately does not restate what the 26 plugins/<node>/planning.md files and context-grounding-patterns.md already own — mode-routing-guide.md is a handoff map, and every row points at the owning skill's build guide. The registry, not this skill, is ground truth for node types; the guide says so and gives the discovery commands, because canvas labels and wire types diverge (Summarize serializes as uipath.pattern.deep-rag). Three areas carry real content rather than redirects: - agent-harness-guide.md — standard vs advanced. The published guidance defaults to "standard" with no feature matrix, so the rule is to start there and select advanced only for a behavior verified in the target, recording what that behavior is. Lists what does not select advanced, since length, a persistent index, node count and industry are routinely mistaken for harness requirements. - search-mode-guide.md — semantic by default, agentic only for dependent retrieval, and the completeness boundary in both. Retrieval supports a positive finding and never proves absence. - classify-split-guide.md — corrects the drafted claim that classification is unavailable. Classifier models exist and share the uipath.ixp.* pattern; what is undocumented is the node's configuration and output shape. The guide says discover, do not invent. The 21-case library is anonymized for a public repository: customer names became workload shapes, ticket and memo locators were dropped, and measured figures became qualitative. Every surviving number is a routing threshold that changes the design, never a result. case-template.md carries those constraints forward with a grep-able scrub so the next case added does not reintroduce them. Sibling frontmatter is deliberately untouched. Adding redirect clauses to uipath-ixp, uipath-maestro-flow, uipath-agents or uipath-planner would fire the activation gate against floors of 100/95/90/60 with DROP_PP 10 — worth doing once this skill's own recall is measured, not on the introducing PR. Standalone. The seven agentic-search pointers across SKILL.md, search-mode-guide.md, mode-routing-guide.md and case-library-guide.md are unlinked backticked paths rather than relative links, so skills:check-links resolves without feat/agents-agentic-search and this PR no longer stacks on it. Restore them as links once that branch merges. Validation: skills:check-links (6953 links, all resolve), skills:validate, skills:pack (2 packages), check-skill-status, check-skills-sh, check-skill-verbs and validate-skill-descriptions (774 chars) all clean. All four case-library anchors resolve. The e2e check script was self-tested against five flows — no flow, correct Summarize, IxP instead, Summarize plus IxP, and node_modules noise. skills:test has one pre-existing failure on main (an npm publish --dry-run registry assertion), unrelated to these changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The case library is derived from meeting memos kept locally under assets/memos/. This repository is public and skills/uipath-doc-mode-select/references/case-template.md § Publishing constraints requires every derived case to pass a provenance scrub — customer names, memo locators, internal dates and measured figures never land. Ignoring the source directory removes the one-command path to committing ~33 MB of unscrubbed customer material. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e the case library Review feedback asked for three changes: add batch transform to the mode set, make every case readable by a person rather than only matchable by an agent, and keep classify & split. Agentic search stays — its locations are inventoried separately for a manual decision rather than removed here. Batch transform (BT) is the eighth category. The handoff row in mode-routing-guide.md already pointed at both build guides, so what was missing was the routing signal and the cases. The Fast Routing row states the decisive signal as per-row LLM work against a fixed output contract, and the reject condition as the three things a row cannot do: see another row, reach an external document, or cause a side effect. Workflow step 2 now draws the line that matters in practice — per-row work enters BT only when the row needs language reasoning; a formula, regex or date reformat is a Transform. The three BT cases are sourced from engagement material and carry distinct boundaries rather than three versions of one shape: - BT1 parses a format-free bilingual operational note into charge intent. It teaches the two constraints people hit first: reference data the row cannot see, and one input row that has to produce several output rows. - BT2 classifies a line item against a condensed reference matrix. The pivotal design question was not the model but whether the reference table fits alongside the row — which is precisely the BT/PA1 line. Its negative is an observed one: no labeled evaluation set existed, so the classifier could not be scored, and reference-data quality gated harder than model quality. - BT3 runs five independent judgements per line item feeding a case-level no-touch decision. It teaches that per-row accuracy multiplies rather than averages when every row must be right, and that a per-field confidence gate that looks strong in isolation did not produce a safe case-level result. Case structure is now Problem description / Best-fit solution / Decision reasoning / Negative example / Boundary, applied to all 24 cases and to case-template.md so future cases follow it. No content was dropped: the old Acceptance check folded into Problem description as "working behaviour looks like this", Placement became Best-fit solution, and Match plus Route became Decision reasoning. Problem description is new prose — enough context that a reviewer who has not seen the source can follow the workload before meeting the routing argument. Counts move to 24 cases across 8 categories in case-library-guide.md, its maintenance note, and case-template.md. The Boundary Lookup gains four rows for the BT confusions, including the one that sends deterministic column work to Transform or Script. Provenance: every BT case passed the case-template.md scrub plus a name sweep over the customers, internal applications, people, ticket identifiers and dates in the source material. Surviving numbers are routing thresholds — how many categories, how many rows in the condensed matrix, how many judgements per line item — never results. The measured figures in the sources were converted to qualitative statements. Validation: skills:check-links (all resolve, every enforced anchor lands), check-skill-status, check-skills-sh and validate-skill-descriptions all clean. Frontmatter description is 793 characters against the 1024 cap. All 24 cases carry all five sections; no old section label survives. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ow mode-selection eval
The skill shipped with two coder_eval tasks covering one mode between them, and
was absent from the activation suite entirely. Both gaps are load-bearing:
tests/scripts/activation_gate.py returns 0 for any skill missing from
BASELINES_PCT, so every frontmatter change to this skill has passed the
activation gate without measuring anything.
Activation. tests/tasks/activation/uipath-doc-mode-select.jsonl adds 30 positive
rows, registered in activation.yaml's dataset list with a 26th skill_triggered
criterion at the same recall.yes floor as its siblings. Prompts deliberately sit
on the seam with uipath-maestro-flow, uipath-ixp and uipath-agents — they ask
which mode a workload needs before anything is authored, which is the line that
separates this skill from the ones that own the build. No BASELINES_PCT entry
yet: that needs a baseline run, and the gate correctly no-ops until one exists.
Mode selection. routing/mode_selection.yaml scales the ad-hoc seven-case run
(one prompt per mode) to 27 rows — three per mode across all eight, plus three
whose ground truth is NONE. Ground truth is the case library itself, so a miss
points at either a non-discriminative Fast Routing signal or a case that does
not teach its decisive signal.
The NONE rows matter more than the count suggests. Every other row has a
document mode as its answer, so without them a skill that replies BT to anything
tabular scores well. They carry a date difference, an exact-key join and a
group-by aggregate — the three shapes Critical Rule 3 sends to Data, Transform
or Script. The prompt states plainly that NONE is an answer and not a refusal.
Prompts describe workloads the way a practitioner would, with no mode vocabulary
and no case IDs, and the allowed IDs are listed bare with no glosses — an agent
that never loads the skill cannot know what AH means, so guessing is 1-in-9
against a 0.70 floor.
CS is expected to be the weakest mode and the description says so: there is no
published Classify & Split node configuration in Flow, so an agent reasoning
about what it can actually build may legitimately route elsewhere. Read a CS
miss as a product-gap signal until that configuration lands.
One unverified assumption, flagged in the task description and guarded in code:
the grading command interpolates ${row.expected_mode} into a run_command string,
and this is the only place in tests/ that puts ${row.*} inside a command field.
Interpolation is proven in initial_prompt, expected_label, expected_skill and
description. If it does not extend to command, check_mode.py detects the literal
template and fails with an explicit message rather than grading against it —
verify this on the first run before trusting a green result.
Tagged integration, not smoke, so it stays out of the PR gate; 27 rows is an
opt-in benchmark. check_mode.py self-tested against match, mismatch, lowercase
with punctuation and trailing prose, bare prose, a missing file, an
uninterpolated argument, and both NONE directions.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2fda0ab to
59acd20
Compare
Links the skill's manifest entry to the internal page that maps each case in references/case-library-guide.md back to the FDE meeting memo it was derived from. That mapping cannot live in this repository — case-template.md § Publishing constraints forbids customer names, memo locators, internal dates and measured figures, and the mapping is made of exactly those. page_id carries the numeric identifier rather than the URL, matching the uipath-insights entry, which is the only other populated confluence block. last_synced is ISO 8601 for consistency with dates elsewhere in the repo. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s in their sources Both cases claim an *observed* negative — that a failure really happened — and a provenance review could not initially find either engagement. Re-reading the source material found both; the searches had been aimed at the wrong words. Neither case was wrong, but both stated their failure thinly enough that the detail an agent needs to recognise the same situation was missing. AH3 now carries what the source actually records: a single file-reading tool used for everything with no deterministic tools built, despite heavy spreadsheet work; an evidence size ceiling on that tool putting roughly a tenth of the controls out of reach with no mitigation designed; calculation logic sitting with the model so numeric accuracy could not be verified; and controls bucketed easy, medium and complex with only the easy bucket built. The rerun that returned a different pass/fail result for the same control is what broke confidence — not any individual wrong answer. That last distinction is the routing lesson, and it was previously implicit. PS1 now names the formats and the remedy. Legacy office formats — .doc, .xls, .ppt — were silently dropped at ingestion, and the symptom reached users as "missing source" complaints rather than as an ingestion error, so it was investigated as a retrieval problem first. The fix was a pre-processing conversion step before re-ingestion, not a change to search. An agent that knows the symptom presents as a retrieval fault will look in the right place. The size ceiling and the affected-control share are routing thresholds — they change whether a mitigation path is needed — and stay as bands. Everything identifying stays out: the scrub and a name sweep over the accounts, applications, people and identifiers in the sources both come back clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…sourced ones AF3 and SU1 were the only cases in the library with no engagement behind them. Both were labelled illustrative, so that was honest rather than wrong, but a constructed case earns its place only while no real one teaches the same boundary. Two do, and both are better evidenced than what they replace. AF3 becomes "Inspect a document for signs of alteration". A production fraud detection engagement: a visual inspection step reads claim documents for alteration or fabrication — fonts, alignment, letterhead, signatures, formatting — and a weighting step turns findings into a severity band that routes to investigators. This teaches the one boundary nothing else in the library covered: the evidence is visual, and every mode that reads through digitization destroys it before the model sees it. Font inconsistency does not survive into extracted text. The case says so, and says to verify the input path actually delivers page images before committing to the route. Its negative is now observed rather than constructed. Across a review of several hundred findings a meaningful minority were severity miscalibrations, with a few hallucinated findings. The separation between fraud-tested claims and controls was real and useful, but the severity number is still the model rating itself — the bands had to be set against reviewed outcomes. Treating it as calibrated would have shipped the miscalibrations into the routing decision. That is the library's standing rule about self-rated confidence, now with a production case behind it. SU1 becomes "Identify open gaps against a guideline set" and keeps the role the old case held: absence claims require complete review, not retrieval. An agentic system reviews patient records for open clinical care gaps against guidelines. An open gap asserts that nothing anywhere in the record shows the required action happened, which is the same shape as "no contrary clause exists" but with sharper consequences in both directions — a missed gap leaves a patient without care, a fabricated one erodes clinical trust. Because the role is unchanged, every inbound reference to SU1 still holds: CS3's absence boundary, PS2's no-contrary-provision boundary, and both Boundary Lookup rows. Its negative records what the engagement actually hit: proving accuracy required live clinician review of real records, with per-gap-type false-positive and false-negative rates as the acceptance measure — no model confidence substituted and no aggregate score answered it. The complete-review path also carried a tracked latency cost, resolved through infrastructure and model choice rather than by narrowing coverage, which would have quietly turned the absence claim into a retrieval result. One inbound link needed repair: EX1's boundary pointed at AF3 for "a low-volume incidental three-field read", which the replacement no longer is. It now points at AF2, which is the bounded-read-without-a-schema case. Provenance: both passed the case-template scrub and a name sweep over the accounts, systems, clinical identifiers and measured figures in the sources. Volumes became bands, review effort became a range, and every accuracy figure became qualitative. The surviving specifics — the asymmetry between missed fraud and over-flagging, per-gap-type error measurement — are routing thresholds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ced them Replacing AF3 and SU1 in the case library left their eval rows behind. Row af3 still described three fields read off a clean letter; row su1 still described a contract reviewed against standard terms. Neither case exists any more. The failure was silent, which is what makes it worth a commit message. Both rows kept expected_mode AF and SU, and both old workloads genuinely do route there, so the rows passed while testing cases the library no longer teaches — and the two replacement cases had no coverage at all. Measured against the Problem description now holding each ID, the stale rows shared 9% and 14% of their content words; a row that has drifted off its case looks exactly like a row that is merely worded differently, so nothing flagged it. Both are rewritten against the current cases, and deliberately in a different domain from the case prose: af3 describes doctored proof-of-delivery paperwork rather than benefit claims, su1 describes vessel maintenance files checked against a required-inspection list rather than clinical care gaps. The decisive signals are unchanged — visual tells with no field schema and an asymmetric error cost for af3; an absence claim over a defined record set with both error directions tracked per type for su1. Choosing a different domain is not incidental. The other 22 rows were written from the case prose and share 65-97% of their content words with it, so they substantially measure whether an agent can match a paraphrase of text it just read. These two now sit at 24% and 30%, which is closer to what an unfamiliar prompt looks like. Whether to bring the remaining rows down to that level is a separate decision: it makes the suite harder and would invalidate any baseline taken before the change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…kloads the library does not use Every row in the previous set was written from the Problem description of the case it tested. Measured against those cases, rows shared 65-97% of their content words and peaked at 0.47 Jaccard, with verbatim runs like "Short, consistent forms of a few pages each" and "through five separate judgements — diagnosis". That set largely measured whether an agent could match a paraphrase of text it had just read. An agent doing string comparison over the library scores well on it without reasoning about a workload at all, and the score is optimistic against real prompts, which do not share the library's vocabulary. The 27 rows are rebuilt from workloads the case library does not use. Peak overlap with any case is now 0.16, and no row exceeds 0.20. Eighteen come from separate engagements in the same source material the library was built from; nine are constructed in unrelated domains — instrument diagnostic dumps, grant panels, incident post-mortems, underwriting manuals, regional tariff terms, sanctions-to-control tracing, support-ticket triage, product-feed attributes, rebate accruals — for the modes where no distinct engagement existed. A provenance field records which is which, so a reader can tell field-grounded rows from constructed ones without a separate document. Distribution is unchanged: three per mode across all eight, plus three NONE rows whose answer is that the work is deterministic and leaves the taxonomy. Two honest limits are now stated in the task description rather than left for someone to discover from a red result. AH is the weakest ground truth in the set and structurally always will be: its decisive signal is a capability need verified to exist only in the advanced harness, which is a design conclusion and not something a business says about its own workload. Its rows carry the long-running, precedence and resumability signals the library's AH cases use, and a near miss onto EX or SU should be read as weaker evidence than a miss elsewhere. CS is expected to underperform for the documented product reason. Provenance: the rows carry no account, system or ticket identifier. An earlier revision of this change did — a per-row source field naming engagements and FOR ids — and the sweep that should have caught it excluded every line in the file, because the pattern it filtered on appeared in all of them. The field is gone and the sweep now runs unfiltered over the whole file. The row-to-source mapping lives in the local-only provenance map alongside the case mapping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…guage and consistency-aware Closes the two gaps between this suite and the eval plan it is meant to serve. Held-out coverage was already there — three cases per mode drawn from workloads the library does not use. What was missing was that the prompts did not read like a user wrote them, and that nothing measured consistency. Nine prompts stated the routing conclusion instead of the business fact. "Each ticket stands on its own, we are not calling anything out to another system" is BT's decisive signal handed to the model; no one describes their support tickets that way. Same for "the two are dependent: which vendor records are even candidates depends on which company code the first step landed on", and for three AF rows announcing that nothing downstream consumes a structured result. All nine now carry business facts only and leave the inference to be made. The line drawn is between a requirement a business really would state — "every line has to point back at a source" — and a conclusion it would not; requirements are kept. Each workload now appears under two phrasings, variant a and b joined by a pair key, taking the set from 27 rows to 54 and six per mode. The phrasings differ in voice, ordering and vocabulary rather than by synonym swap; mean lexical overlap between the two halves of a pair is 0.31. Overlap against the library is unchanged at a 0.16 peak, with no row above 0.20, and the file carries no account, system or ticket identifier. analyze_consistency.py turns a run into the three numbers that matter. Accuracy by mode against the modal answer; rows whose replicates disagreed; and pairs that answered differently when reworded. It singles out the flips where one phrasing was right and the other wrong, because that is the most actionable result the suite can produce — it says the routing copy works for one framing of a workload and not another, which is a different defect from a signal that is missing altogether. Accuracy on its own cannot separate a genuinely uncertain row from a stable-but-wrong one; only the per-replicate breakdown can. Self-tested against a synthetic run carrying one seeded instance of each of those three failures. Stability needs repeats to mean anything, so the description gives the run line — coder-eval run ... --repeats 5 — and the analyzer says plainly that stability is not measurable at one replicate rather than reporting a misleading 100%. The suite threshold moves from 0.70 to 0.65. The set is harder than the one that produced no baseline: the prompts no longer hand over the answer, and half the rows are phrasings never tuned against. Both changes should cost accuracy, and that is the intent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The analyzer took the row id from mode.txt's grandparent. A real coder-eval run
lays the artifact out as
<run>/<variant>/<task>/<row>/<NN>/artifacts/<task>/<row>/mode.txt
so the grandparent is the task id, not the row. Every row would have collapsed
into one bucket named after the task, and the script would have reported a
single row with 270 replicates rather than 54 rows with 5 each — wrong, and
wrong in a way that still prints a plausible-looking table.
It now walks up for the two-digit replicate directory and takes its parent.
That is the one landmark the layout guarantees, and it survives changes to how
the sandbox is preserved beneath it.
Found by running a three-row preflight before the full pass and checking where
mode.txt actually landed, rather than trusting the path shape I had assumed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ion set NONE asked the model to answer that no document-processing mode applies because the work is deterministic and leaves the taxonomy. It is a defensible thing for the skill to teach — Critical Rule 3 and the Boundary Lookup both say it — but it is a meta-answer rather than a mode, and it did not explain itself to readers of the results. Six rows removed, three workloads times two phrasings; 48 rows remain, six per mode across all eight. The prompt no longer offers NONE, so the instruction and the answer space now agree. check_mode.py still recognises the token, deliberately: an agent may still reach for it from the skill's own deterministic-work rule, and reporting "selected NONE, ground truth BT" says the model thought the workload left the taxonomy. Dropping it from the accepted set would collapse that into "not a valid mode", indistinguishable from garbage. Worth recording what this costs, since it is not nothing. The NONE rows were added because every other row has a document mode as its answer, so a skill that replies BT to anything tabular scores well without the guard. That guard is gone. The deterministic boundary is still covered inside the library — BT1 and BT2 both carry it in their Boundary sections, and the Boundary Lookup sends formula, regex and date-reformat columns to Transform or Script — it is simply no longer measured here. If a future run shows BT over-firing, this is the first thing to reinstate. Scored without the NONE rows, the last run reads 40/48 (83%) rather than 46/54 (85%); the six NONE rows were all correct, so removing them lowers the headline without changing any finding. AH is still 0/6 and af-clinical-qa still misses under both phrasings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… not modes
"Mode" was doing two jobs in this skill: naming the eight things an agent picks
between, and appearing in the repo's own test tag vocabulary. The first is the
skill's vocabulary and is now "file handling strategy"; the second is not ours
to rename.
Renamed across SKILL.md, all six references, the eval set, the activation
dataset and the README catalog row. Two guides were renamed with their subject
— mode-routing-guide.md to strategy-routing-guide.md, search-mode-guide.md to
search-strategy-guide.md — and every inbound link repointed.
In the eval the term is structural, not just prose, so the rename reaches the
data: expected_mode became expected_strategy across all 48 rows, the artifact
the agent writes is strategy.txt rather than mode.txt, check_mode.py became
check_strategy.py, and mode_selection.{yaml,jsonl} became strategy_selection.
Three things were deliberately NOT renamed:
- The skill folder, uipath-doc-mode-select. Renaming it pulls in
assets/skill-status.json, skills.sh.json, CODEOWNERS, the README status
table, the tests/tasks/<skill>/ directory and the activation dataset
filename, and CLAUDE.md requires both registries to move in the same PR.
It also changes the frontmatter name, which fires the activation gate. That
is a separate change with its own blast radius, not a rider on a vocabulary
fix.
- The short codes AH, AF, CS, SU, PS, PA, EX, BT. The grader keys on them, the
case IDs derive from them, and 272 replicates of eval data are labelled with
them. They are identifiers, not vocabulary.
- mode:build and friends in task tags, and retrievalMode in the PS
configuration. The first is the repo's closed tag taxonomy from
tests/README.md; the second is a product field name. Both were shielded
through the substitution and verified afterwards.
The sweep that confirms this needed fixing twice. A line-level grep exclusion
for the protected tokens hid every line that also contained them — including
the README row, which still said "Document-processing mode selection" while
the sweep reported clean. The check now strips protected tokens from each line
and searches what remains. That is the third time in this work that a filter
excluded the thing it was meant to inspect.
PA-LOCATIONS.md carries a staleness banner: it indexes agentic-search
touchpoints by line number, and both the rename and the guide renames shifted
every one of them. File and section names in it are still correct.
Validation: skills:check-links, check-skill-status, check-skills-sh,
validate-skill-descriptions, check-task-host-paths and check-task-driver all
clean. Frontmatter description is 795 characters against the 1024 cap.
check_strategy.py self-tested on match, mismatch and missing artifact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The skill calls the eight things it picks between file handling strategies; its own folder still said "mode". Frontmatter name and folder now agree, which is a hard requirement rather than a preference — .claude/rules/skill-structure.md requires the name field to match the parent folder exactly. Both registries move with it, as CLAUDE.md requires in the same PR: assets/skill-status.json and skills.sh.json are the only records of a skill's lifecycle status and its skills.sh grouping, and neither is discovered from disk, so a rename that misses one leaves drift nothing else in the build notices. CODEOWNERS covers both the skill and its test directory. The README status table regenerates and was verified current. The test surface moves too: tests/tasks/<skill>/ is matched by path, so smoke-skills.yml maps a change under skills/uipath-doc-strategy-select/ to tests/tasks/uipath-doc-strategy-select/. Three task YAMLs carry the name twice each — once as the tier tag and once as the skill_triggered criterion's skill_name and expected_skill — and all six were updated. The activation dataset is renamed alongside its 30 row ids, with its entry and criterion in activation.yaml repointed. One thing this could have broken silently: the two local-only files in the skill folder, CASE-MEMO-MAP.md and PA-LOCATIONS.md, are excluded by path in .git/info/exclude. Moving the directory would have left those rules pointing at a path that no longer exists, and both files — which name accounts, ticket ids and memo locators — would have become untracked and visible to the next git add -A in a public repository. The exclude rules were repointed and re-verified with git check-ignore. Checked before committing: no sibling SKILL.md references this skill, so no other skill's frontmatter changes and no other activation baseline is touched. The activation gate no-ops here regardless, since activation_gate.py returns 0 for any skill absent from BASELINES_PCT and this one is absent by design. Git detects all 15 moves as renames, so the diff reads as moves rather than deletions plus additions. PR #3506 is unaffected — a pull request tracks its head branch, not file paths. Validation: check-skill-status (28 skills, manifest valid; README table current), check-skills-sh (28 grouped across 4 sections), skills:check-links, validate-skill-descriptions, check-task-host-paths and check-task-driver all clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nal; unbrittle the smoke The smoke task scored 1.0 before the AF/SU boundary edit and 0.647 after. Two separate causes, one mine and one the test's. SU covers two deliverables and the rewrite collapsed it to one. The row had read "Cited synthesis, or complete review of a defined document set"; separating AF from SU on the deliverable replaced that with a signal leading on citations, and "complete review" disappeared. A workload whose claim IS the coverage — "which of these still reference the retired terminology, I need the complete list" — still routed to SU but no longer matched a case, so the agent wrote a bare SU where the smoke expects SU1. It also left Critical Rule 4 without a destination: retrieval never proves absence, and SU is where a completeness claim goes. The row now carries both deliverables and keeps the AF and AH boundaries. The smoke asserted on a word, not a behaviour. `includes: ["enumerat"]` failed an answer that applied the rule correctly in different words — "every document in the corpus must be processed", "coverage reconciliation must confirm the processed count matches". That is the rule, stated well. Replaced with a regex over the ways the boundary is actually expressed, verified to match both observed answers and to reject a lazy one that proposes a larger result limit instead. Verified: smoke 1.0, routing suite 7/7. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`npm pack --dry-run` shipped skills/uipath-doc-strategy-select/CASE-MEMO-MAP.md into the public @uipath/skills package. That file maps library cases to the engagements they came from and names real customer accounts and ticket ids. It was kept out of git only by .git/info/exclude, which is local to one machine and which npm never reads, so every `npm pack` on a machine that had the file packed it. PA-LOCATIONS.md shipped too. That one carries no customer data, but it is the reinstatement spec for a strategy this release deliberately withholds, and an agent reading an installed skill could find it and route to a strategy that is not there. Excluded at the canonical skill walk in compose-skill-flavor.mjs, which is the only entry point into a composition plan, so one predicate covers every built tree, every staged package and every tarball. The same test is applied to the tarball denylist beside the existing source-only-paths assertion, so a future bypass fails closed rather than silently packing. Not done with a `files` negation in package.json: checkedPayloadPath rejects any entry containing a glob character, so `!skills/*/CASE-MEMO-MAP.md` fails prepack outright. `files` is a literal copy list in this repo, not an npm pattern list. Verified: root tarball 1876 -> 1874 files with both names absent and the skill's 7 real files intact; both built trees carry SKILL.md and references/ only; skills:validate clean; all 51 compose-skill-flavor tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…OCATIONS.md Classify & Split is not available when this skill merges. The capability exists — IxP publishes classifier models and they appear in the Flow registry — but the node's configuration and output shape are undocumented, so an agent routed to CS cannot finish the build. CS comes out, and CS-LOCATIONS.md holds everything removed, verbatim, with where each piece goes back. This is not the PA withdrawal repeated. PA was something a workload escalated TO, so deleting it left nothing dangling. CS is something Extract goes THROUGH: three places encoded "separate the documents before extracting", and the obligation survives the strategy. Those are rewritten, not deleted — the skill still forbids extracting a packet as one document, it just no longer names a strategy that separates them: - workflow step 3 keeps its place and states the rule without a destination, adding the failure it prevents (merging unrelated headers, producing fields from the wrong form); - EX's reject column still fires on mixed documents, now describing boundary resolution as a prerequisite this release does not route; - EX1's boundary says document boundaries must be established, not "CS1 first". Deleting step 3 instead would have left a mortgage packet routing straight to EX with nothing warning against it. Two mechanical traps worth recording. The guide and its studioweb override are coupled in both directions: compose-skill-flavor.mjs fails at :524 if the canonical target is missing and at :538 if the markers are, so both files go in one commit. And BT2's boundary already carried a PA-era "no strategy here covers it yet", so patching its CS clause separately would have left two dangling clauses in one paragraph; it is rewritten whole. Also in this change: the smoke's case-ID assertion is widened from the literal SU1 to any matched SU case. It asserted one id where two are defensible, and was ~50% flaky on correct answers — observed SU1, SU2, SU1, bare SU across four runs. A bare "SU" still fails, which is the defect the criterion exists to catch. Verified: 18 cases across 6 categories with grep agreeing; workflow steps 1-6 with no gap; every Boundary Lookup row names a case that exists; description 702/1024; both flavor trees build; links, status manifest, skills.sh and host-path checks clean; routing suite 6/6; smoke 3/3 at 1.0 after the widening. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ow step 2 Missed in the CS withdrawal. Step 2 sends deterministic work outside the strategy taxonomy and still counted CS in it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ode-select # Conflicts: # CODEOWNERS # skills/uipath-genome/scripts/check-genome.py
joe-prosser
left a comment
There was a problem hiding this comment.
Requesting changes on three blockers: assets/memos/ packs into the public tarball; dependent retrieval routes to AH in the Fast Routing table but to nothing in the guides; and the Extract and Classify handoffs predate the v2 document nodes already documented on main. Two smaller items ride along: the packaging fix has no test, and CASE-MEMO-MAP.md is not in .gitignore.
Detail and line references are in the inline comments.
| | Strategy selected | Owner | Build guide | | ||
| |---|---|---| | ||
| <!--skill-flavor:handoff-ixp:start--> | ||
| | Extract, inside a Flow | `uipath-maestro-flow` | [/uipath:uipath-maestro-flow — ixp.md](../../uipath-maestro-flow/references/ixp.md) | |
There was a problem hiding this comment.
Picking up the Slack thread: Caleb said this skill targets the new extract v2 / classify / split nodes rather than the existing IxP node. The PR routes the other way. The only Extract-in-Flow handoff is ixp.md (published-project ixpExtract), and Classify & Split is withheld because "its node configuration and output shape are not documented" (CS-LOCATIONS.md L3-6).
Since 2026-09-30 (#3624), uipath-maestro-flow/references/document-pipeline.md documents documentClassify({ fileRef, pageRange?, splitPages?, modelConfig? }) as uipath.document.classify, and dynamicExtract({ fileRef, schema, model, … }) as uipath.ixp.extract-document-builder with an inline schema and no published project. The CS withdrawal commit is dated 2026-10-13, so the prerequisite in CS-LOCATIONS looks already met.
Three things follow. EX needs a Dynamic Extract row here and a rule for inline-schema versus published-project extraction. SKILL.md L70 and L101-105 ("this release routes no strategy for that separation") and case-library-guide.md L625 are contradicted by splitPages living in the same plugin, so either reinstate CS or reword them. And the Extract v2 skill itself has no skills PR yet: it is being scoped in #ixp-coding-agents-extractor-v2 (RE-13407), where the 12 Oct notes record "extraction routing in Flow: no strategy yet" with Caleb on the scenario-to-skill mapping, so this PR is the heuristic that group is waiting on.
Given that, please request a review from @samuelfekete, who is building the Extract v2 skill under RE-13407 (RE-13424 and RE-13425 are in progress; the CLI side is landing in UiPath/cli#5069, UiPath/cli#5158 and UiPath/cli#5204), and from the uipath-ixp owners (@cezara98t @misupantea-uipath @alexandrujircan). I own that epic, so happy to take the handoff-target question to a call with Sam and Caleb if that is quicker than threading it here.
There was a problem hiding this comment.
Taking these in your order.
1. Dynamic Extract row and the inline-vs-published rule — done, then reworked.
32874a5eb split the handoff, 3180f1177 added the rule. Then @alexandrujircan reviewed it and found the framing was wrong, so 0ad786469 redid both.
His point: in IXP, publishing a project and deploying one are distinct, and what my row called "published" is a deployment. He is right, and swapping the word would not have been enough — the axis was wrong. dynamicExtract requires model.modelName and model.folderKey, which is the same {DeploymentName, FolderKey} pair ixpExtract resolves, so "no published IxP project" read as "no IXP needed" when both nodes run against a model deployed to an Orchestrator folder.
The rows now split on where the field definitions come from:
| Extract, inside a Flow — a project's trained taxonomy fields | uipath-maestro-flow | ixp.md |
| Extract, inside a Flow — a JSON Schema authored inline in the node | uipath-maestro-flow | document-pipeline.md |
§ Which Extract node now states that neither choice avoids IXP, that both need uip ixp deployments create --folder-key, and that publishing pins a version inside the project rather than supplying the folder identity. uipath-ixp/references/cli-reference.md L198 and uipath-ixp/SKILL.md L71 already said this; my skill contradicted them.
One knock-on worth flagging: document-pipeline.md L5 opens with "two document steps that need no published IxP project", which is where I took the phrasing from. Same problem, and it is the page my row links to — but it is the Maestro Flow owners' file, so I have not touched it here. Raised with Alexandru as a candidate separate issue.
2. The CS rationale was wrong; the three contradicted sites are still open.
You were right on the dates, and worse than you put it — document-pipeline.md landed 2026-09-30, the withdrawal is 89208cd70 on 2026-10-13. The stated reason was false when written. The real reason is product availability at merge, not documentation, and that is now what it says.
SKILL.md L70, L101-105 and case-library-guide.md L625 are unchanged, deliberately. They turn on a question I cannot answer from the repo: is uipath.document.classify the same surface as the Classify & Split that will not be available, or a different one? It needs no published IxP project, which suggests different — in which case I reword and point separation at documentClassify/splitPages, CS stays withdrawn, the table stays at six rows. If you and Caleb read them as one capability, it is a reinstate instead.
I would rather take your offer of a call with Sam and Caleb than guess, since the answer also decides whether "Extract v2" belongs in the routing table as a term. I have deliberately kept that phrase out of the skill so far — once it is in a routing table it is what agents match on, and my mapping of it to dynamicExtract + documentClassify is inferred from your sentence, not from anything the repo states.
Also note the classify half is still unrouted: documentClassify is not in the handoff map and the skill does not name it or splitPages. That falls out of the same decision.
3. One housekeeping change that affects your line references.
CS-LOCATIONS.md and PA-LOCATIONS.md are out of the repository as of a47076f46 — they are held on Confluence now, where the people who would reinstate them can read them. So "CS-LOCATIONS.md L3-6" no longer resolves on this branch, and my earlier replies about [STALE] flags refer to a file that has since moved. The composer rule and its tests stay, keyed to the -LOCATIONS.md suffix rather than those two names.
4. Reviewers. Alexandru has reviewed and his finding is fixed above. Sam and the other two ixp owners are not requested yet — I would rather settle the handoff-target question on the call first, so they review a direction rather than a proposal. Say the word if you would prefer them on now.
There was a problem hiding this comment.
Follow-up on the second of your three points — the contradicted sites are reworded, in 8e2341ac1.
The problem was the claim, not the ordering. "This release routes no strategy for that separation" asserted that no capability exists, which is false with splitPages on documentClassify, and it pushed agents toward hand-rolled page ranges. All five places that touch separation now name Classify & Split as the step a packet goes through before EX, and the two that face a routing decision say plainly that it is not routed in this release, so the dependency stays visible instead of being silently skipped:
SKILL.mdEX reject — "Classify & Split has to separate them first, and this release does not route it yet"SKILL.mdworkflow step 3 — states the obligation, names the step, says it is unroutedcase-library-guide.mdBoundary Lookup, plus the EX1 and BT2 boundaries, which named the obligation without naming the step
Deliberately not added, and this is the part I would like your eye on: no CS row in the handoff map, no classify-split guide, and no mention of documentClassify or splitPages. Whether the Flow node is the same surface as the Classify & Split that is not yet available is exactly the question still open with you and Caleb, and putting it in the routing table would commit to an answer I do not have. The strategy table stays at six rows and CS remains unselectable.
Classify & Split is expected to ship shortly, so the plan is a small follow-up patch that adds the handoff row and the build-guide reference once it does — at which point the "not routed yet" clauses come out in the same change.
That leaves only the handoff-target question on this thread, which I still think is better settled on the call than written down by me.
There was a problem hiding this comment.
Follow-up on the Classify & Split half, in eb66265: rather than reinstating it as a seventh strategy, it's now a prerequisite step with a real handoff — a row to uipath-maestro-flow / document-pipeline.md (Document classify, uipath.document.classify, splitPages) and a Packets paragraph requiring each logical document to reach Extract on its own, reported as unresolved if the build can't show that. The "not routed in this release" wording is gone from SKILL.md and the Boundary Lookup row. The Extract half now mirrors the flow skill's published-model default (33ba75f).
…it the Extract handoff, guard the packaging exclusion Addresses Joe's review on #3506. Dependent retrieval contradicted itself: SKILL.md's Fast Routing rejects PS into AH for it, while both guides still said no strategy covered that shape. search-strategy-guide.md and agent-harness-guide.md now route it to AH, with the open-ended/fixed-length boundary stated so a two-hop keyed lookup stays deterministic (Critical Rule 3). case-library-guide.md's BT2 boundary carried the same dangling clause and now routes the same way. The Extract handoff predated document-pipeline.md, so it offered only ixp.md — a published project's trained fields — and had no route for a schema authored inline. Split into two rows; studioweb drops the new one, since its classic tree documents no such node. CS stays withdrawn, but the stated reason was wrong: uipath.document.classify has been documented on main since document-pipeline.md landed, including splitPages and the output port. The withdrawal is a product-availability decision, so CS-LOCATIONS.md now says that, and the four archived passages asserting "not yet documented" are flagged [STALE] so reinstatement rewrites them instead of pasting them back. Packaging: the memo sources and the provenance map move out of the repository to a local-only path, both are gitignored, and two tests guard it — one over a fixture (no working document reaches a plan or a built tree, default and flavor) and one over this repository (no memo under a packaged root, no working document in the default plan). Verified by mutation: blanking isWorkingDocument fails three tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ished-project Extract rule The split Extract rows carried the distinction only in their labels. Joe asked for the rule behind them: a published, trained IxP project is ixpExtract; no published project is dynamicExtract with a JSON-Schema authored in the node and a model identity copied from the tenant; and an inline schema is not a way to skip publishing a project when the type recurs at volume and needs calibrated review. Studio Web has no inline-schema node, so its override says so in one line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…o the table's dependent-retrieval answer 32874a5 settled two of the five sites Joe listed. The PS1 and PS2 boundaries and the Boundary Lookup row still said dependent retrieval was unrouted, so a reader of the table answered AH and a reader of a case said unroutable. All three now give the same answer as SKILL.md, with the carve-out stated once per site: a chain of known length is a deterministic sequence of scoped lookups, and an open-ended one -- where the agent works out each next lookup from the last -- is AH. BT3's boundary already routed a fixed dependency to a Flow sequence and is unchanged. The PS reject cell in the table carries the same qualifier now, so the table cannot be read as sending a two-hop keyed lookup to AH against Critical Rule 3. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rate about the AH sites PA-LOCATIONS section 0 quoted PS's reject cell verbatim and named two files as the other places routing dependent retrieval. 8bb3a11 changed the quote and raised the count to six. Both are corrected, so reinstating PA does not miss four sites that would keep pointing at AH. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…d-out Joe asked for CASE-MEMO-MAP.md in .gitignore alongside assets/memos/, since .git/info/exclude is per-clone and anyone else's `git add -A` would publish real account and ticket IDs. 32874a5 added it; this makes the block say what each entry is and why ignoring alone is not enough, given npm's `files` allowlist overrides .gitignore. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ments assets is a whole-root files entry, and stagePackages copies it wholesale through copyPayload with no filtering, so memo sources under it were invisible to the skill plan and to verifyTarball alike. .gitignore does not stop a pack, which is the same mechanism that shipped CASE-MEMO-MAP.md. The sources are out of the repository, and that remains the first line. This is the second, for a machine that still holds a copy: isPrivateWorkingPath matches any path with a memos segment as well as the working documents, copyPayload skips those entries, and verifyTarball denies them on sight rather than trusting copyPayload to have dropped them. Two tests: a real pack over a fixture that still has assets/memos/ ships none of it while keeping its ordinary assets, and the denylist rejects the three names whatever staged them. Verified by mutation -- dropping the memos segment check fails both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…parately Joe's two tests. The PR body claimed the exclusion was asserted on the tarball denylist while nothing under tests/ referenced isWorkingDocument, -LOCATIONS.md or CASE-MEMO-MAP.md; the 51 passing tests were the pre-existing suite. The plan fixture now carries an invented FOO-LOCATIONS.md alongside the two real names, so what is proven is the suffix rule rather than a list of the files that happen to exist today. The denylist test replaces a predicate-only check with the real thing: a tarball built to contain the offending file, staged with it so the tree comparison passes, and verifyTarball asserted to throw. A clean tarball of the same shape verifies first, so the throws cannot pass for the wrong reason. The two are independent, which is the point of having both. Disabling the tarball denylist fails only the verifyTarball test; removing the canonical walk exclusion fails only the plan tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nally, not in the repo CS-LOCATIONS.md and PA-LOCATIONS.md recorded, verbatim, everything removed when Classify & Split and agentic search were withheld, plus where each piece goes back. They are now kept on Confluence, where the people who would reinstate them can read them, so they come out of the repository. Two test comments pointed at those paths and would have rotted silently -- neither skills:check-links nor any other check scans task YAML or Python comments for repo paths. Both now say the specs are held internally. The composer rule stays: isWorkingDocument still matches the -LOCATIONS.md suffix, the tarball denylist still rejects it, and the tests still prove both against an invented FOO-LOCATIONS.md rather than the two names that existed. .gitignore gains the same unanchored pattern, so the shape cannot come back by accident. The eval contract is untouched -- CS and PA stay answerable but never expected, so a stale answer still reads as a selection rather than a parse failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… source, not on publishing
Alexandru Jircan, reviewing as a uipath-ixp owner: publishing a project and
deploying one are distinct IXP concepts, and what the row called "published"
is a deployment.
He is right, and swapping the word would not have been enough -- the axis was
wrong. dynamicExtract requires model.modelName and model.folderKey, which is
the {DeploymentName, FolderKey} pair, so "no published IxP project" read as
"no IXP needed" when both nodes resolve a deployed model from an Orchestrator
folder. What actually differs is where the field definitions come from: a
trained taxonomy, or a JSON Schema authored in the node.
Both rows are relabelled accordingly, and the rule now states that neither
choice avoids IXP, and that publishing pins a version inside the project while
deploying is what supplies the folder identity these nodes resolve. The repo
already said this in skills/uipath-ixp/references/cli-reference.md and
uipath-ixp/SKILL.md; this skill contradicted it.
Studio Web carries the same correction.
Not changed here: document-pipeline.md opens with "two document steps that
need no published IxP project", which is where the phrasing came from. That
file belongs to the Maestro Flow owners.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ion step, without routing it Three sites said this release routes no strategy for separating a mixed document. That claimed no capability exists, which is false -- splitPages is documented on documentClassify -- and it sent agents off to hand-roll page ranges instead. All five separation sites now name Classify & Split as the step a packet goes through before EX, and the two that face a routing decision say plainly that it is not routed in this release, so the dependency is visible rather than silently skipped. Deliberately NOT added: a CS row in the handoff map, a classify-split guide, and any reference to documentClassify or splitPages. Whether the Flow node is the same surface as Classify & Split is still open with the Extract v2 group, and naming it in the routing table would commit to that answer. The handoff lands as a small patch once Classify & Split ships. The strategy table stays at six rows; CS remains unselectable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@joe-prosser — ready for another look. Ten commits since your review; each inline thread has a reply with the detail, so this is just the map. Three blockers
Two smaller items
Also landed, from @alexandrujircan's review He caught that publishing and deploying an IXP project are distinct, and that what my row called "published" is a deployment. The axis was wrong, not just the word — Two changes that move your line references
Still open, and I do not think I should decide it Whether CI is green — 39 passing, nothing failing. |
samuelfekete
left a comment
There was a problem hiding this comment.
This looks good to me overall. The main thing to look out for is changes landing in the CLI and skills PRs: UiPath/cli#5158, UiPath/cli#5204 (the nodes create / nodes schema get|set commands, releasing in 1.207.x) and #4122 (the Extract document node and taxonomy guides in uipath-maestro-flow). Instances where this PR conflicts with those are pointed out inline.
| its fields need calibrated review. An inline schema carries no taxonomy and no trained field | ||
| evaluation, so that is a training decision deferred, not removed. | ||
|
|
||
| **Neither choice avoids IXP.** Both nodes address the model by `{FolderKey, DeploymentName}`, so both |
There was a problem hiding this comment.
The model requirement belongs to the builder SDK's dynamicExtract() factory, not to the node. The node itself requires only fileRef: the validator exempts uipath.ixp.extract-document-builder from the model rule and rejects a top-level model block on the instance, and at run time the extraction service resolves its processor from inputs.processorKey (written from the idpProcessor solution resource when the flow is saved in Studio Web) — it never reads a model identity. The CLI route (uip maestro flow nodes create / nodes schema set, UiPath/cli#5158 and UiPath/cli#5204) authors the node with no IXP project and no deployment. So the inline-taxonomy choice does not need an IXP model; only the published-model node resolves a deployed model by {FolderKey, DeploymentName}. The factory requiring model fields the node never reads is tracked as RE-14355. #4122's rewrite of document-pipeline.md (§ "Two ways to author it") covers this split.
There was a problem hiding this comment.
Fixed in 33ba75f. The guide now says the node needs only fileRef and no IXP model or deployment, and that only the published-model node resolves a deployed model by {FolderKey, DeploymentName}. Since the CLI now provisions the processor during authoring, the handoff carries forward what nodes create reported about it (linking extract-document-node.md § Making it actually extract) rather than relying on a Studio Web save.
| | Extract, inside a Flow — a project's trained taxonomy fields | `uipath-maestro-flow` | [/uipath:uipath-maestro-flow — ixp.md](../../uipath-maestro-flow/references/ixp.md) | | ||
| <!--skill-flavor:handoff-ixp:end--> | ||
| <!--skill-flavor:handoff-document-pipeline:start--> | ||
| | Extract, inside a Flow — a JSON Schema authored inline in the node | `uipath-maestro-flow` | [/uipath:uipath-maestro-flow — document-pipeline.md](../../uipath-maestro-flow/references/document-pipeline.md) | |
There was a problem hiding this comment.
The node's schemaDocument is not a bare JSON Schema — it is a taxonomy, an object with prompt and outputSchema, where every field group and field must carry a stable id (x-uipath-field-group-id / x-uipath-field-id). flow validate rejects a JSON-Schema object with properties alone. The CLI's nodes schema set mints the ids; a schema passed to the SDK factory is emitted verbatim and fails validation. After #4122, the guides that own this node are extract-document-node.md and extract-document-taxonomy.md, which this row's target (document-pipeline.md) points at.
There was a problem hiding this comment.
Fixed in 33ba75f. The Extract document facts now describe inputs.schemaDocument as a taxonomy — prompt plus outputSchema, every field group and field carrying x-uipath-field-group-id / x-uipath-field-id — and say a bare JSON-Schema object with properties alone is not valid. The handoff row now points straight at extract-document-node.md and extract-document-taxonomy.md.
| 1. **A trained taxonomy** — use `ixpExtract`, an `uipath.ixp.*` node. The fields are trained and named, | ||
| and the model's taxonomy can be looked up when the Flow must select particular extracted fields. | ||
| 2. **A schema you write** — use `dynamicExtract` (`uipath.ixp.extract-document-builder`). The schema is | ||
| a JSON-Schema object authored in the node, so a new document type ships without a training cycle. |
There was a problem hiding this comment.
This routes "a schema you write" to the Extract document node so "a new document type ships without a training cycle". #4122 encodes the opposite default, agreed with the uipath-ixp owners: document extraction defaults to a published IXP model (building and deploying the project via uipath-ixp when none exists), and the Extract document node is used only when the user asks for it by name or the flow already holds one. The flow skill's e2e task e2e_03_project_creation_handoff pins the no-model-named case to the uipath-ixp handoff. As written, an agent with both skills loaded gets contradictory routing for a new document type with no trained model.
There was a problem hiding this comment.
Agreed, and fixed in 33ba75f. The routing now mirrors the flow skill's default: a new flow uses a published IXP model (handing off to uipath-ixp to build and deploy one when none exists); Extract document only when the user names it; an existing flow keeps the extraction node it has. It also follows #4652 — IXP not_supported offers Extract document, Extract document unavailable falls back to a published model, neither → stop. The "ships without a training cycle" framing is gone, replaced by an explicit rule that setup cost is not a routing signal.
|
|
||
| 1. **A trained taxonomy** — use `ixpExtract`, an `uipath.ixp.*` node. The fields are trained and named, | ||
| and the model's taxonomy can be looked up when the Flow must select particular extracted fields. | ||
| 2. **A schema you write** — use `dynamicExtract` (`uipath.ixp.extract-document-builder`). The schema is |
There was a problem hiding this comment.
dynamicExtract is the SDK factory, which currently cannot author this node without a published model's identity (RE-14355); the CLI is the authoring route. The node's user-facing name since the 2026-09-22 rename (RE-13766) is "Extract document" — packages/flow-core/locales/en.json gives both display.label and canvasLabel as "Extract document".
There was a problem hiding this comment.
Fixed in 33ba75f. The node is called "Extract document" throughout, and the guide says to author it with uip maestro flow nodes create / nodes schema set, not the SDK dynamicExtract() factory, since that demands a model identity the node never reads and emits the schema without ids.
| ``` | ||
|
|
||
| Tenant-specific nodes (IxP models, connectors) appear only after `uip login` and | ||
| `uip maestro flow registry pull`. An empty search is not proof of absence until you are authenticated |
There was a problem hiding this comment.
The Extract document node never appears in a tenant's registry. Studio Web appends it to the canvas client-side behind a feature flag, so it is in no tenant manifest, and registry list/search merge the CLI's bundled catalogue — the row shows AvailableOnTenant: false on every tenant, enabled or not, even after uip login and registry pull. nodes create also succeeds against a tenant that cannot run the node, so the failure moves to runtime. "Confirm against the registry" holds for wire types and tenant-published IxP models, but cannot confirm this node's availability; RE-13410 tracks a real check.
There was a problem hiding this comment.
Fixed in 33ba75f. § Registry before labels now states the registry cannot confirm this node (AvailableOnTenant: false everywhere, registry get → Node not found, nodes create / validate succeed regardless) and that the registry check holds only for wire types and tenant-published IxP models. Since RE-13410 shipped as uip maestro flow extraction check --output json (#4644), the guide points there: Available can run it, FeatureDisabled / ModelUnavailable / ServiceUnavailable route to a published model, Unknown or an error settles nothing. Critical Rule 3 in SKILL.md carries the same exception.
…ode-select # Conflicts: # skills/uipath-genome/scripts/check-genome.py # tests/scripts/compose-skill-flavor.test.mjs # tests/tasks/activation/activation.yaml
… skill's default Address samuelfekete's review and follow what main has since shipped: - Default a new flow's extraction to a published IXP model; use the Extract document node only when named, when the flow already holds one, or when IXP is not enabled. Each falls back to the other. - Describe the node's taxonomy (prompt + outputSchema with stable ids), author it with the CLI rather than dynamicExtract(), and say it needs no IXP model or deployment. - The registry cannot confirm this node; point at `uip maestro flow extraction check` instead, and force one registry pull before treating an empty search as absence. - Use the node's current name, "Extract document". - Drop the studioweb overrides: the classic tree they pinned is gone, so both flavors now build from canonical. - Fix the planner link moved to references/shared/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
skill-lint: 0 new errors, 35 new warnings
What these rules check
Only what this PR adds is listed, compared with the commit it branched from; issues already on the base branch are not. Updated on every push. |
…kill's classify node joe-prosser's review asked to reinstate Classify & Split or reword the sites that called it unrouted, since uipath-maestro-flow documents `uipath.document.classify` with `splitPages`. Keep it a prerequisite step rather than a seventh strategy, but stop calling it unrouted: add a handoff row to document-pipeline.md and a Packets paragraph that requires each logical document to reach Extract on its own, and point SKILL.md and the Boundary Lookup row at it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What
A skill that picks the file handling strategy before any file exists, then hands off. Strategies:
Extract, Classify & Split, Summarize, Analyze files, Batch transform, persistent-index semantic search, persistent-index agentic search and the advanced agent harness.
Why
The choice was a decision nobody owned.
uipath-maestro-flowdocuments each node once you know whichnode you want;
uipath-agentsdocuments each context-grounding pattern once you know which one youwant;
uipath-plannerroutes across products, not within document processing. The expensive mistakesare made before any file exists.
This skill selects and hands off.
strategy-routing-guide.mdis a handoff map — every row points at theowning skill's build guide, and there are six owners downstream. The registry, not this skill, is ground
truth for node types.
Changes
The skill —
SKILL.md(6-row Fast Routing table, 8 Critical Rules, 6-step workflow), five referenceguides, an 18-case library. Each case runs Problem description / Best-fit solution / Decision reasoning /
Negative example / Boundary.
Registration — six hand-maintained surfaces, none discovered from disk:
assets/skill-status.json(preview),skills.sh.json(Authoring),CODEOWNERS,README.mdcatalog rowand generated status table, and
skills/uipath-genome/scripts/check-genome.py(OPERATE_SKILLS). The lastwas found by a failing test rather than a check anyone reads —
uipath-genomeclassifies skills by whetherthey author an artifact, and this one selects and hands off, so it sits with
uipath-planner. Thatclassification is load-bearing: a skill in that set is flagged if a genome names it under Build With
rather than Platform Dependencies.
A packaging fix, unrelated to this skill but found here.
npm packshippedskills/uipath-doc-strategy-select/CASE-MEMO-MAP.md— real customer accounts and ticket ids — into thepublic
@uipath/skillspackage. It was excluded only by.git/info/exclude, which is local to one machineand which npm never reads, so every pack on a machine that had the file packed it.
PA-LOCATIONS.mdshipped too, making a withheld strategy discoverable to an installed agent. Both are now dropped at the
canonical skill walk in
compose-skill-flavor.mjs— the only entry point into a composition plan, so onepredicate covers every built tree, staged package and tarball — with the same test asserted on the tarball
denylist so a future bypass fails closed. A
filesnegation inpackage.jsondoes not work here:checkedPayloadPathrejects any entry containing a glob, sofilesis a literal copy list, not an npmpattern list.
Provenance — every case traces to a real engagement. The mapping names accounts and ticket ids, so it
lives in Confluence, not here:
INTERNAL Mode-selection skill — case library.
assets/skill-status.jsonrecords the page id.Details — more in Confluence:
Shipping uipath-doc-strategy-select skill.
Eval results
Measured on 42 rows (21 workloads × 2 phrasings, six per strategy), held out from the case library.
Peak content-word overlap with any case is 0.090. One replicate per row.
CS scored 6/6 in that run and is omitted: the strategy is withheld, so the number describes something this
build no longer contains.
Activation 95/95. Smoke 1/1, verified 3/3 after the fix below. End-to-end not run — needs
uip login.The original full Activation and Routing sets are in Confluence:
Activation & Routing Eval set.
Deliberately NOT in this PR
Classify & Split (CS). The capability exists — IxP publishes classifier models and they appear in the
Flow registry — but the node's configuration and output shape are undocumented, so an agent routed to CS
cannot finish the build. CS comes out rather than shipping as a dead end, and
CS-LOCATIONS.mdholdseverything removed verbatim.
This was not the PA withdrawal repeated. PA was something a workload escalated to, so deleting it left
nothing dangling. CS is something Extract goes through: three places encoded "separate the documents
before extracting", and the obligation survives the strategy. Workflow step 3, Extract's reject column and
EX1's boundary were rewritten, not deleted — the skill still forbids extracting a packet as one
document, it just no longer names a strategy that separates them. Deleting step 3 would have left a
mortgage packet routing straight to Extract with nothing warning against it.
Two mechanical traps worth a reviewer's eye: the guide and its studioweb override are coupled both ways
(
compose-skill-flavor.mjsfails if the canonical target is missing and fails if the markers are, soboth move in one commit), and BT2's boundary already carried a PA-era "no strategy here covers it yet", so
patching its CS clause separately would have left two dangling clauses in one paragraph — it is rewritten
whole.
Agentic search (PA). Waiting on the page-window API via the
uipath-agentsreference material. Sametreatment, in
PA-LOCATIONS.md. Dependent retrieval currently routes to AH; PA takes that shape back whenit lands. PA does not displace AH, it narrows it.
A two-axis strategy + harness grade. Built, then reverted. The harness axis is ungradeable today: the
guide's own exclusion lists resolve every business-statable condition to
standard, so a row expectingadvancedwould mark the agent down for obeying the skill.Reviewer notes
The shipped eval sets are smaller than the ones measured above. Routing is 6 rows (one per shipped
strategy), activation positives 30 → 5. What the cut stops measuring is recorded in the task description:
paraphrase invariance is gone entirely (only variant
asurvives, so the analyzer reports 0 of 0 pairs),and
af-clinical-qais not in the cut because a permanently-red row in a small gate leaves the suite nevergreen. The 42-row set is not in git — it existed only between the AH rework and the trim.
Reconstructable on request.
The shared negatives were restored, not trimmed. An earlier revision cut
negative.jsonlfrom 65 to 5.That file is scored against by all 26 skills, so the trim reduced precision coverage for the other 25 — and
main had meanwhile grown it to 74, making it a hard merge conflict as well as a cross-skill regression. It
is back at main's 74. Cost savings on activation come from scoping a local run to one dataset, not from
editing a shared one.
Threshold arithmetic at the new sizes. Routing
mean: 0.78× 6 = 4.68, so 5 of 6 must pass. Activationrecall.yes: 0.70× 5 = 3.5, so 4 of 5. A single miss moves a small gate a long way; the numbers were setfor 42 and 30 rows and are worth a deliberate look.
The activation gate passes without measuring anything.
activation_gate.pyreturns 0 for any skillabsent from
BASELINES_PCT, and this one is absent by design — a baseline needs a run first. The 95/95above is the baseline to record.
${row.expected_strategy}interpolates into arun_commandstring — the only place intests/thatdoes.
check_strategy.pyfails loudly rather than silently mis-grading if that ever stops working.Stability is unmeasured. Everything above is one replicate. A five-replicate run is the outstanding
gate, and the AH rows have no stability history at all.
The red check is not from this skill. "Build and inspect every skill package" fails on two dead
anchors in
uipath-maestro-bpmn, which this branch never touched. A pristine checkout of this branchpasses the check (exit 0);
origin/mainfails it on its own. Because a pull-request check runs on themerge of branch and base, the PR inherits main's breakage.
uipath-maestro-bpmnis the only tree whoseanchors are enforced — the other 30 are explicitly "not enforced" — so that one skill blocks every open PR.
Validation
skills:check-links,skills:validate(both flavor trees),check-skill-status(README table current),check-skills-sh,check-task-host-pathsand the frontmatter check (702 chars against the 1024 cap) allclean. All 18 cases carry all five sections; the case library reports 18 across 6 categories and
grepagrees; workflow steps run 1–6 with no gap; every Boundary Lookup row names a case that exists.
check_strategy.pyself-tested on match, mismatch, lowercase, prose-below-the-answer, stale labels,uninterpolated
${row.*}and missing artifact. All 51compose-skill-flavortests pass, and the roottarball drops from 1876 to 1874 files with both working documents absent and the skill's 6 real files
intact.
Four bugs fixed along the way:
agent-harness-guide.mdnamedsettings.strategywhere the owning skill calls the fieldmode.analyze_consistency.pyreported 100% agreement for rows where every replicate failed to answer —None == Nonecounted, so a total failure read as perfect stability.SU1and the literalenumerat. Both failed answers that applied therule correctly in different words; both are now behaviour patterns. 3/3 after.
SKILL.mdstep 2 still counted seven strategies after the withdrawal.🤖 Generated with Claude Code