Skip to content

ci(skill-lint): bump the linter to skill-linter#19 and fail on new errors and warnings - #4759

Merged
CarlesUIPath merged 3 commits into
mainfrom
ci/skill-lint-pin-fail-on-new
Nov 18, 2026
Merged

CarlesUIPath merged 3 commits into
mainfrom
ci/skill-lint-pin-fail-on-new

Conversation

@CarlesUIPath

@CarlesUIPath CarlesUIPath commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Bumps the skill-lint PR check to the current linter and makes it fail on what a PR adds. Findings already on main still never fail it.

What

  • .github/workflows/skill-lint.yml
    • Pin: 427217ff (skill-linter#8) → c71f4de0 (skill-linter#19).
    • Fails on new findings: fail-on-findings: "false" is removed and --fail-on warning is added, so a new error or warning fails the check. Info and report-only (shadow) findings never fail it.
    • The header comment describes the new behavior.
  • .skill-lint.yaml, kept in step with the linter's examples/uipath-skills.skill-lint.yaml:
    • LINK-CROSS-SKILL.shared: the files platform, admin, tasks and maestro-flow offer to other skills, from the cross-skill review (skill-linter#2). Links into them become info. Case shares none since feat(uipath-maestro-case): promote the builder-SDK Case skill; Studio Web pins v1 #4529.
    • CONTENT-CRITICAL-LATE.chars_per_token: 2.56: the same token ratio AGENT-READ-TRUNCATED already uses.
    • SAFETY-DESTRUCTIVE-NO-CONFIRM: safe_paths (the skills' own working files) and safe_commands (uip maestro flow eval … remove).
    • Override: UIP-REQUIRED-SECTIONS is info for uipath-platform, which keeps its hard rules as top-level sections of their own.
    • Override: SEC-BROAD-BASH is info: every skill that declares allowed-tools grants bare Bash on purpose, and scoping it is the maintainers' security call (skill-linter#18).
    • EVAL-GATE-BASELINE.baselines_file: tests/scripts/activation_gate.py: new rule in skill-linter#19. Warns for a skill with activation prompts but no BASELINES_PCT entry, which the required Skill activation gate skips. Today: activity-migrator, bundles, connector-builder, insights, process-mining, rules; test(activation): gate baselines for 6 skills the activation gate skipped #4944 adds their baselines.
  • CONTRIBUTING.md and docs/REQUIRED-CHECKS.md: "phase 1" is replaced by the new behavior. The check is still not required; promoting it stays a separate step after a week or two of PRs with no false alarms.

What the linter bump brings

Full list in the linter's CHANGELOG.

How "new" is decided

The PR and its merge base are linted with the same linter and the same config (the base branch's), and only findings that are not on the merge base are reported. Findings are matched by rule, file and content key, not by line number, so:

  • Editing a file that already has a finding doesn't make it new. Tested on main (a89fd49) by changing files that already had findings:

    • appending to api-workflow/references/troubleshooting.md (over Claude Code's read cap) and to ixp/SKILL.md (over Codex's);
    • pushing planner's Critical Rules further down;
    • lengthening troubleshoot's description (listing already over the cap).

    None of these was reported.

  • New problems are reported and fail the check. In the same test, the troubleshoot description going over 1,024 characters (error) and losing its "UiPath" prefix (warning) were reported, with exit code 1.

Verification

  • With the new linter, this PR's A/B run reports 0 new findings and passes, as the check on this PR should show.

  • The new .skill-lint.yaml loads. On main:

    Errors Warnings Info Report-only
    Old config 7 125 456 181
    New config 7 92 496 177

    The warnings drop because 33 declared cross-skill links become info, and platform's Critical Rules warning becomes info.

  • The PR comment still starts with <!-- skill-lint pr-comment -->, and a clean run's title is still ### skill-lint: no new errors or warnings, which the comment job matches.

  • The new config is read from the base branch, so it applies from the PR after this one merges. This PR's own run uses the current config with the new linter.

Not changed

  • Check name: it stays No new skill-lint errors, so nothing that refers to it breaks.
  • Not required yet: see docs/REQUIRED-CHECKS.md § Open items.

🤖 Generated with Claude Code

…rors and warnings

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @CarlesUIPath's task in 1m 17s —— View job


Summary

This PR moves the skill-lint check to skill-linter#17 (0a795c75) and makes it fail on new errors and warnings (--fail-on warning, with fail-on-findings: "false" removed). It also brings .skill-lint.yaml in line with the linter's example config: cross-skill shared files, CONTENT-CRITICAL-LATE token ratio, destructive-command safe lists, and a platform override. The docs drop the "phase 1" wording.

Change-by-Change Review

1. .github/workflows/skill-lint.yml

OK

  • The job name No new skill-lint errors and the trigger are unchanged, so CLAUDE.md rule 10 still holds.
  • The new header comment (lines 7–10) matches the new behavior.
  • The comment job still works with the new failure mode. A job that fails on findings has a non-clean body, so it gets posted as is. A failed job whose body says clean (a config error or crash) still becomes "skill-lint failed".
  • The "PR's .skill-lint.yaml loads" step (line 112) and the comment step both run on !cancelled(). Both still run when the lint step fails on findings.

2. .skill-lint.yaml — LINK-CROSS-SKILL.shared

OK

  • All 19 declared paths resolve under skills/.
  • Matching exact files is the safer choice.
  • Low: a PR that adds a legitimate new cross-skill link now fails the check. It can't fix that by declaring the file in the same PR, because the config is read from the base branch. The comment at lines 19–25 says this. Contributors only have two options: land a config PR first, or silence the link with a reason. The CONTRIBUTING.md § skill-lint bullet doesn't mention this, and one sentence there would help.

3. .skill-lint.yaml — SAFETY-DESTRUCTIVE-NO-CONFIRM

Low

  • .flow-work/* and .flow-work/** match nothing. grep -rn "flow-work" skills/ only hits flow-workbench (uipath-agents/.../inline-in-flow.md:346, uipath-maestro-flow/references/hitl.md:180). If these entries are only there to match the linter's example config, keep them. Otherwise drop them.
  • safe_commands: ["uip maestro flow eval *remove *"] correctly covers evaluate.md:35,60,109.
  • The comment calls these edits "reversible (git-tracked)". That is only true if the user committed first. Maybe say "local file edits" instead.
  • **/dist and **/node_modules are fine as build output. They match the rm -rf in uipath-agents/references/coded/quickstart.md:210 style.

4. .skill-lint.yaml — CONTENT-CRITICAL-LATE.chars_per_token: 2.56

OK. It uses the same ratio as AGENT-READ-TRUNCATED, and the comment explains why chars ≈ bytes.

5. .skill-lint.yaml — platform UIP-REQUIRED-SECTIONS: info override

Low

  • This is consistent with uipath-platform/SKILL.md, which has no ## Critical Rules and has ## Use the CLI. Don't roll your own REST. at line 21.
  • The override drops every UIP-REQUIRED-SECTIONS finding for platform to info, not only the Critical Rules one. If platform later loses ## When to Use This Skill or Quick Start, that no longer warns.
  • If the linter supports per-section options, narrow the override to the Critical Rules section. Otherwise this is acceptable as is, since the comment documents the reason.

6. CONTRIBUTING.md:565, docs/REQUIRED-CHECKS.md:225

OK

  • Both are accurate.
  • The promotion steps are now just "add the context to § Current target set", which matches the change.
  • No other "phase 1" or fail-on-findings text remains in the repo outside skills/.

What's Missing

  • CONTRIBUTING.md doesn't say what to do when a new cross-skill link fails the check (land the shared declaration first, or silence it with a reason). This is optional (Low).
  • Nothing else identified. No skill folders are added or renamed, so CODEOWNERS and both registries are unaffected.

Area Ratings

Area Status Notes
Frontmatter OK No skill files changed
E2E Tests OK N/A
Skill Body OK N/A
References & Assets OK All shared paths exist
Repo Hygiene OK No secrets; the pinned SHA has a PR reference; the required-check name is unchanged

Issues for Manual Review

  • From here I can't verify that the internal linter's action defaults fail-on-findings to true. Removing the input only makes findings fail if it does. I also can't verify that --fail-on warning excludes shadow and report-only findings. The PR's A/B evidence (exit code 1 on the troubleshoot description test) suggests both hold.
  • The workflow edit needs CODEOWNERS review of /.github/, as the Trust note in the workflow says.

Conclusion

Approve. I found no blocking issues. The only notes are Low: dead .flow-work safe paths, a broad platform override, and an optional CONTRIBUTING sentence about cross-skill links.

CarlesUIPath and others added 2 commits November 18, 2026 16:39
…er#18)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CarlesUIPath CarlesUIPath changed the title ci(skill-lint): bump the linter to skill-linter#17 and fail on new errors and warnings ci(skill-lint): bump the linter to skill-linter#19 and fail on new errors and warnings Nov 18, 2026
@CarlesUIPath
CarlesUIPath merged commit b4763d2 into main Nov 18, 2026
52 checks passed
@CarlesUIPath
CarlesUIPath deleted the ci/skill-lint-pin-fail-on-new branch November 18, 2026 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants