Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@
{
"name": "aviator",
"source": "./aviator",
"description": "Verify and Runbooks workflow automation with Aviator MCP",
"version": "1.2.0"
"description": "Verify and Runbooks workflow automation via the aviator CLI",
"version": "1.3.0"
}
]
}
20 changes: 7 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ run `av pr --all` to create PRs for the entire stack.

### aviator

Connects Claude Code to Aviator via MCP for spec submission to [Runbooks](https://aviator.co/runbooks) and Verify, to automate development workflows.
Submits specs from Claude Code to Aviator [Runbooks](https://aviator.co/runbooks) and Verify — through the `aviator` CLI, to automate development workflows.

**Verify vs Runbooks.** Aviator has two ways to hand a spec off from your Claude session:

Expand All @@ -81,10 +81,9 @@ Connects Claude Code to Aviator via MCP for spec submission to [Runbooks](https:

**What this plugin does:**

- Connects to the Aviator MCP server for spec submission and runbook operations
- Submits Verify specs and creates runbooks from your current Claude session context
- Handles OAuth authentication automatically
- Provides access to Aviator's workflow automation tools
- Guides Claude through writing the intent, spec, and acceptance criteria from your current session context
- Submits Verify specs and creates runbooks via the `aviator` CLI
- Keeps acceptance criteria fresh as a connected PR evolves

**Usage:**

Expand All @@ -95,17 +94,12 @@ Connects Claude Code to Aviator via MCP for spec submission to [Runbooks](https:

- An Aviator account at https://app.aviator.co
- Repository connected to Aviator
- The `aviator` CLI installed (`go install github.com/aviator-co/aviator-cli/cmd/aviator@latest`) and configured with an API token (`AVIATOR_API_TOKEN` or `~/.config/aviator/config.yaml`)

**Self-hosted / On-prem:** The plugin connects to `https://app.aviator.co/mcp` by default. To point it at a self-hosted instance, set the `AVIATOR_MCP_URL` environment variable:
**Self-hosted / On-prem:** The CLI talks to `https://api.aviator.co` by default. To point it at a self-hosted instance, set `AVIATOR_API_HOST` (or `apiHost` in `~/.config/aviator/config.yaml`):

```bash
export AVIATOR_MCP_URL=https://aviator.your-company.com/mcp
```

Alternatively, you can configure the MCP server manually:

```bash
claude mcp add --transport http aviator https://aviator.your-company.com/mcp
export AVIATOR_API_HOST=https://aviator.your-company.com
```

## Installation
Expand Down
8 changes: 0 additions & 8 deletions aviator/.mcp.json

This file was deleted.

25 changes: 17 additions & 8 deletions aviator/commands/create-runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Create an Aviator Runbook from the current Claude Code session context. **Aviato

> Writing the code yourself and just want Aviator to verify it against intent + acceptance criteria? Use `/verify-submit` instead — it captures intent and AC with no implementation steps.

**Load the `spec-submission` skill** (Skill tool → `aviator:spec-submission`) before you start — it carries the shared mechanics this flow relies on: how the message reads, the Acceptance Criteria review loop, the `specSubmit` call, and the PR directive. This command file only covers what's specific to a Runbook.
**Load the `spec-submission` skill** (Skill tool → `aviator:spec-submission`) before you start — it carries the shared mechanics this flow relies on: how the message reads, the Acceptance Criteria review loop, the `aviator` CLI submission, and the PR directive. This command file only covers what's specific to a Runbook.

## Arguments

Expand All @@ -26,7 +26,7 @@ If a plan file exists from plan mode (check the plan file path mentioned in the

Similarly, if a spec file already exists in the conversation — one the user wrote, one generated earlier, or one provided via `$ARGUMENTS` — use it as-is. Do not restructure, reformat, or rewrite an existing spec. When the spec comes from a file, preserve the original filename.

If no existing spec is available, generate one. Keep it **free-form** — there's no required structure or fixed set of sections. Write whatever best conveys the change to the agent that will implement it: the intent and the implementation approach or steps, shaped to the task rather than forced into headings. (The acceptance criteria are passed separately as the `acceptance_criteria` argument — you don't hand-embed them; the backend folds them into the spec the agent works from.)
If no existing spec is available, generate one. Keep it **free-form** — there's no required structure or fixed set of sections. Write whatever best conveys the change to the agent that will implement it: the intent and the implementation approach or steps, shaped to the task rather than forced into headings. (The acceptance criteria are passed separately via `--criteria`/`--criteria-file` — you don't hand-embed them; the backend folds them into the spec the agent works from.)

## Step 2: Review with the user

Expand All @@ -40,14 +40,23 @@ A simple "yes" or "go ahead" is enough to submit.

## Step 3: Create Runbook

Submit via the `specSubmit` call described in the `spec-submission` skill. For a Runbook:
Submit with `aviator runbook`, following the CLI mechanics in the `spec-submission` skill (preflight, repo derivation, criteria-file guidance, result parsing). What's specific to a Runbook:

- `submission_type`: **`"runbook"`** — the agent writes the code from the spec. This is the default, but pass it explicitly.
- `acceptance_criteria`: the confirmed AC as a JSON array of strings (optional but recommended — they're folded into the spec the agent works from).
- `spec_files`: include only if a spec was generated; always a single file.
- `target_branch`: **optional** — the base branch the runbook builds on and checks out; the generated PR opens against it. Omit for the repo default (trunk). (Runbook mode generates its own PR, so there's no `working_branch` to connect here.)
- `--intent`: **required** — the confirmed intent: what the runbook should accomplish and why. Keep it short and human-friendly; the implementation detail travels in the spec, and the intent is stored and displayed on the session as-is.
- `--spec` (optional): the spec file — include only if one was generated or already existed; always a single file.
- `--criteria` / `--criteria-file` (optional but recommended): the confirmed AC. The backend folds them into the spec the agent works from. Prefer `--criteria-file` for more than 2–3. Make sure the spec itself carries no "Acceptance Criteria" section when you pass these — the backend rejects that combination rather than guess which list wins.
- `--target-branch` (optional): the base branch the runbook builds on and checks out; the generated PR opens against it. Omit for the repo default (trunk). (Runbook mode generates its own PR, so there's no working branch to connect here.)
- `--title` (optional but worth setting): a short deliberate title for the runbook. Left unset, the backend derives one from the intent — currently by truncation, which reads poorly for multi-sentence intents.

Then return the Runbook URL and set the PR directive, both per the `spec-submission` skill. The expected PR body shape:
```bash
aviator runbook \
--repo acme/web \
--intent "Migrate the settings page to the new design system" \
--spec /path/to/spec.md \
--criteria-file /path/to/criteria.txt
```

On success the command prints `✓ Runbook created: <url>` and a `Runbook #<n>` line. Then return the Runbook URL and set the PR directive, both per the `spec-submission` skill. The expected PR body shape:

```
Runbook: <runbook-url>
Expand Down
41 changes: 26 additions & 15 deletions aviator/commands/verify-submit.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Submit a Verify spec to Aviator from the current Claude Code session. Aviator Ve
- **Key decisions & architecture** — a free-form record of the decisions made and the shape of the change, written so a reviewer can understand the PR without reading every line.
- **Acceptance Criteria** — the concrete, observable behaviors the change must satisfy, verified independently against the code.

**Load the `spec-submission` skill** (Skill tool → `aviator:spec-submission`) before you start — it carries the shared mechanics this flow relies on: how the message reads, the Acceptance Criteria review loop, the `specSubmit` call, and the PR directive. This command file only covers what's specific to Verify.
**Load the `spec-submission` skill** (Skill tool → `aviator:spec-submission`) before you start — it carries the shared mechanics this flow relies on: how the message reads, the Acceptance Criteria review loop, the `aviator` CLI submission, and the PR directive. This command file only covers what's specific to Verify.

## Arguments

Expand All @@ -20,8 +20,8 @@ $ARGUMENTS - Optional additional context or instructions for the Verify submissi

The code is the ground truth for a Verify submission. Before generating anything:

- Identify the **working branch** — the branch the in-flight work lives on (typically the current git branch). You'll pass this as `working_branch` so Verify tracks the PR opened from it.
- Identify the **repository** in `owner/repo` form (e.g. from `git remote get-url origin`) — you'll pass this as `repo_name`.
- Identify the **working branch** — the branch the in-flight work lives on (typically the current git branch). You'll pass this as `--working-branch` so Verify tracks the PR opened from it.
- Identify the **repository** in `owner/repo` form, following the repo-derivation procedure in the `spec-submission` skill (don't just read `origin`) — you'll pass this as `--repo`.
- Read the **actual current changes** end-to-end (the diff against the base branch, and the modified files in full — not just the hunks). Understand what the code does: what behavior each change introduces, what invariants it preserves, what it exposes, what failure modes it handles, what it replaces.

Everything below is drawn from what the code actually does, cross-checked against `$ARGUMENTS` and any spec/plan already in the session — never from imagination.
Expand All @@ -47,7 +47,7 @@ Aim for the altitude of "what a thoughtful reviewer needs to not be surprised, a

### Assembling the spec file

Generate a single spec file (name it `spec.md`, or preserve the original filename if a spec already exists in the session — use it as-is, don't restructure it). The spec body is **intent + key decisions** — the acceptance criteria are **not** in the spec; they're passed as the `acceptance_criteria` argument at submit. Use these sections:
Generate a single spec file (name it `spec.md`, or preserve the original filename if a spec already exists in the session — use it as-is, don't restructure it). The spec body is **intent + key decisions** — the acceptance criteria are **not** in the spec; they're passed through the `--criteria`/`--criteria-file` flags at submit. Use these sections:

```
## Intent
Expand All @@ -61,29 +61,40 @@ Intent always belongs. Include Key Decisions & Architecture whenever the change

## Step 3: Review Acceptance Criteria with the user

Run the Acceptance Criteria review loop from the `spec-submission` skill — iterate until the user explicitly confirms.
Run the Acceptance Criteria review loop from the `spec-submission` skill — iterate until the user explicitly confirms. (Running non-interactively with no user available? Use that skill's non-interactive provision instead of stalling.)

One thing specific to Verify: show the user the **intent** line and the **Acceptance Criteria** — not the full spec body (Key Decisions & Architecture is submitted as supporting context, not what the user confirms). If they want to see the rest, they'll ask.

## Step 4: Submit for Verify

Submit via the `specSubmit` call described in the `spec-submission` skill. For Verify:

- `submission_type`: **`"verify"`** — this is the argument that makes it a Verify submission (intent + AC over human-authored code, no step generation). Do not omit it; the default is `"runbook"`.
- `acceptance_criteria`: **required** — the confirmed AC as a JSON array of strings.
- `working_branch`: **required** — the branch the in-flight work lives on (from Step 1), passed by name, so Verify tracks the PR you open from that branch.
Submit with `aviator verify`, following the CLI mechanics in the `spec-submission` skill (preflight, repo derivation, criteria-file guidance, result parsing). What's specific to Verify:

- `--intent`: **required** — the confirmed intent.
- `--criteria` / `--criteria-file`: **required** — the confirmed AC (`aviator verify` seeds its structured criteria set from these). Prefer `--criteria-file` for more than 2–3.
- `--working-branch`: **required for this flow** — the branch the in-flight work lives on (from Step 1), passed by name, so Verify tracks the PR you open from that branch. (The CLI marks the flag optional; a Verify submission still needs it, since without it no PR ever binds to the session.)
- `--spec` (optional): the spec file (intent + key decisions) from Step 2.
- `--target-branch` (optional): the base branch to verify against; omit for the repo default.

```bash
aviator verify \
--repo acme/web \
--intent "Gate the new banner behind the beta flag" \
--criteria-file /path/to/criteria.txt \
--working-branch feature/banner \
--spec /path/to/spec.md
```

Then return the Runbook URL and set the PR directive, both per the `spec-submission` skill.
On success the command prints `✓ Verify submission created: <url>` and a `Runbook #<n>` line. Then return the Runbook URL and set the PR directive, both per the `spec-submission` skill.

## Step 5: Keep Acceptance Criteria fresh as the PR evolves

Verify AC are a living contract, not a one-time snapshot. As you keep pushing commits to the connected PR, the code drifts from the AC the user originally signed off on — new behavior appears, scope shifts, an edge case gets handled differently. **Stale AC verify the wrong thing.**

So, after a meaningful push to the connected PR in this session (a new behavior, a changed contract, a dropped or added piece of scope — not a typo fix):
So, after a meaningful change to the work on this branch — pushed or still local — in this session (a new behavior, a changed contract, a dropped or added piece of scope — not a typo fix):

1. Re-read the current AC and the runbook's version: `getRunbook(url, fields=['acceptance_criteria'])` — note the returned `runbook_version` (an int).
1. Re-read the current AC and the runbook's version: `aviator results r/<n> --json` — note the `runbook_version` field in the output (an int). (`aviator show r/<n> --json` returns the full session; `results` is the lighter call.)
2. Compare the AC against the **current** diff. If the code now does something the AC don't cover, or an AC no longer matches what the code does, the AC are stale.
3. Refresh them: `editRunbook(runbook_url, expected_version=<the version you just read>, payload={"acceptance_criteria": [<complete new list, in order>]})`. The payload is the COMPLETE new list — including unchanged items — and expresses add/update/remove/reorder in one atomic edit. If the edit fails with a stale-version error, someone else moved the runbook; re-read and retry.
3. Refresh them with `aviator edit r/<n> --expected-version <the version you just read> --criteria-file <path>` (or repeated `--criteria` flags). The edit **replaces the entire criteria list**, so the file must hold the COMPLETE new list — including unchanged items, in order — expressing add/update/remove/reorder in one atomic edit. If it fails with a stale-version error (409), someone else moved the runbook; re-read the version and retry — a stale edit writes nothing.
4. Keep the same quality bar as Step 2 — observable outcomes, no implementation detail — and keep the user in the loop on non-trivial AC changes rather than silently rewriting their signed-off list.

Do not re-run `specSubmit` to refresh AC — that creates a new runbook. Use `editRunbook` to update the existing one.
Do not re-run `aviator verify` to refresh AC — that creates a new runbook. Use `aviator edit` to update the existing one.
Loading