Repository navigation
✨ Workflow models: simulate an issue/PR against a model - #17688
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of No conflicting open PRs or same-file overlaps were found. |
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. @clubanderson this PR is held at Failing checks:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. Review findings from Read the diff against Medium — coverage gate fails on two packages this PR adds code to. Job 130620704436:
Table-driven tests for What I verified as correct:
Not a duplicate of #17685 (phase 4 backend); the two share only route registration in |
There was a problem hiding this comment.
Decision needed: @clubanderson should decide whether the simulate route's repo+number fill (which drives the hive's App token at any caller-chosen owner/repo) should be owner-gated like ingest or restricted to the hive's configured repositories, before it ships open to every logged-in role.
Read the Go against the tree at 392d1de (api_workflows_simulate.go, github/workflow_item.go, deps.go, cmd/hive/main.go, and api_workflows_ingest.go / api_agent_control.go for the existing gating). The engine side (SimulateRef, Explain, the Facts mirror and lane simulation) looks sound; the one thing I would not settle alone is a permission boundary.
Medium — any-role route drives the hive's GitHub token at an arbitrary repository.
api_workflows_simulate.go:171-199: handleWorkflowSimulate has no requireOwnerRole, and when facts is absent it passes the caller's repo and number straight to s.deps.WorkflowItemSnapshot, which is wired in cmd/hive/main.go:1094 to b.ghClient.WorkflowItemSnapshot. workflow_item.go:36-73 then does Issues.Get, PullRequests.Get and SummarizeCommitChecks on that owner/repo with the hive's credential. The only validation is strings.Contains(req.Repo, "/") (:179); nothing checks the repo against the hive's configured set. Mechanism: a dashboard user with a non-owner role can read labels, author, state, head SHA, base ref, mergeability and check summary of any issue/PR in any repository the hive's App installation can see — including private repos the hive is not configured to work — and the 502 body at :192 echoes the forge error verbatim. The existing precedent for "caller-chosen repo via the hive's forge client" is POST /api/workflows/ingest, which is requireOwnerRole-gated (api_workflows_ingest.go:37) for the same read. The body states "read-only, any logged-in role" as intended, so this is a deliberate choice — but it widens a trust boundary and should be an explicit owner decision, not an incidental one. Two cheap options: gate only the fetch path (hand-set facts can stay any-role), or refuse repos outside the configured list.
Also verified, no action needed:
- Facts-present path never touches the forge (
:172), andkindis validated bySimulateRef(stageengine/simulate.go). storePRDetail(workflow_item.go:58) populates the shared PR-detail cache keyed byrepo/number, so a simulated foreign repo cannot shadow a configured one.
Observed: the coverage check is red on this head; I did not dig into the log.
Sequencing: overlaps api_workflows.go, dashboard/openapi.json and docs/design/workflow-models.md with #17685 (phase 4). Different phases, not duplicates; whichever lands second needs a rebase.
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
Mutation testing⏱️ The 900s mutation budget ran out, so 2 package(s) were not fully mutated and the results below do not cover them:
❌ gremlins did not produce a report for 1 package(s), so no verdict can be derived from this run (see the job log):
Verdict: |
Add POST /api/workflows/{name}/{version}/simulate. It loads a stored
model, compiles it, converts a JSON Facts mirror into stageengine.Facts
and runs the same deriver the engine uses, returning the decision
(state, column, stage, reason) and a per-rule trace from the new
stageengine.Explain. repo+number without facts best-effort fills from
the forge and reports which facts were filled vs unknown.
Refs #17620
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Settings → Workflows → model detail grows a Simulate section: pick a real owner/repo#N to fetch its facts, or describe one with a form generated from WF_SIM_FIELDS, then Run to see the decision chip (state · column · stage · reason), the ordered rule trace with the matched rule highlighted and each predicate's verdict, and the resulting state marked in the diagram. Refs #17620 Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Refs #17620 Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Refs #17620 Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Cover Explain (builtin issue/PR, every rules predicate op, unknown facts, skipped rules), the Parse*/name helpers, SimulatedLane/SimulateRef, and github WorkflowItemSnapshot plus small untested helpers (ChecksForRequired, required-check resolver, ReporterTrust hold summary/provenance, IssueTitleState, PRConflictCandidates, ReviewDeltaDiff, CI activity and retry helpers) so the coverage gate clears its floors. Refs #17620 Signed-off-by: Andrew Anderson <andy@clubanderson.com>
3b9f898 to
90e077e
Compare
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
RFC phase 3c — Simulate of the workflow-models RFC (#17620), on v5, follows #17681 (merged).
What
API —
POST /api/workflows/{name}/{version}/simulate(read-only, any logged-in role)Request:
{ "kind": "issue"|"pr", "facts": { ...SimulateFacts... }, "repo": "owner/repo", "number": 123 }factsis a lowerCamel JSON mirror ofstageengine.Facts(labels, open, merged, draft, origin, author{login,kind,selfAuthored,role}, park{holds,needsHuman,…}, issue{bucket,…}, pr{checks,review,dco,conflicting,changesRequested,humanApprover,lane,…}). Enumerations use the engine's name helpers (author.kindhive|clanker|human|unknown,pr.checksgreen|red|pending|unknown,pr.reviewapproved|changes_requested|pending|unavailable|requires_human,pr.dcoyes|no|unknown); new exportedstageengine.ParseAuthorKind/ParseChecks/ParseReview/ParseDCOare their inverses, so a response'sfactscan be pasted back as a request.repo+numberwithoutfacts→ best-effort fill through the hive's GitHub client (newgithub.Client.WorkflowItemSnapshot: issue, PR, mergeable state, head check summary).kindmay be omitted (taken from the item; mismatch → 400). No client wired → 422, never invented.Response:
{ "decision": { "state", "column", "stage", "label", "reason", "abstain", "covered", "terminal" }, "trace": [ { "id", "state", "kind", "evaluated", "matched", "skipped", "builtin", "reason", "predicates": [ { "fact", "op", "value", "holds", "known", "actual" } ] } ], "facts": { …normalized echo… }, "filled": [...], "unknown": [...], "notes": [...] }stageengine.Explain(profile, kind, facts) (Decision, []RuleTrace)sharesholds()with the rules evaluator;Deriveis untouched. Rules mode: every rule in order, other-kind rulesskipped, evaluated until the first match (predicate verdicts + actual value), the restevaluated:false; uncovered →abstain:true, covered:false, reason "workflow: not covered…". Builtin mode: single{builtin:true, state, reason}entry.stageengine.SimulatedLanesynthesizes one from the other facts and says so innotes(passpr.laneto override).UI — Simulate panel on the model detail page
Collapsed Simulate
<details>in Settings → Workflows → detail:owner/repo#123picker + Fetch facts, issue/PR radio, facts form generated from the declarativeWF_SIM_FIELDStable, Run. Result: decision chip (state · column · stage/NNN · reason, or Not covered), the trace as an ordered list with the matched rule highlighted and each predicate asfact op value → ✓/✗/?, fetched/unknown lists, and the resulting state's chip highlighted in the diagram (.wf-dg-state.simulated). Tokens-only CSS, no raw px/colours, no downloads, notoLocaleString.Guards
dashboard/openapi.jsonpath +WorkflowSimulateRequest/Facts/Resultschemas (route-parity test).TestWorkflowSimulateHiveDefaultPR,TestWorkflowSimulateRulesTrace,TestWorkflowSimulateErrors,TestWorkflowSimulateFetchesItem(dashboard);TestExplainRulesTrace,TestExplainBuiltinTrace(stageengine).TestWorkflowsSimulateWiring,TestWorkflowsSimulateNode(form kind-awareness + round-trip, matched-rule highlight, predicate verdicts, Not covered, diagram marking).pkg/dashboard,pkg/dashboard/webstatic,pkg/stageengine,pkg/workflowmodel/...green locally (apart from the known macOS-onlyTestConfigExportReportsPersistedPath/TestKnowledgeDrainLegacyMemory); style/download/timezone/import ratchets unchanged;webstatic/gen -checkclean.Docs
docs/design/workflow-models.md→ new "Simulate (RFC phase 3c)" section (API shape, trace semantics, best-effort fill caveats).changelog.d/added-workflowmodel-simulate.md.Refs #17620