LLM format: deterministic ordering, caveats, glossary, scoped hints - #164
Merged
Conversation
- categoryBreakdown ties broke on map iteration order and the ByConclusion max-pick flapped between runs on identical data — both now tie-break lexicographically (pinned by run-50-times tests) - new '## Data Caveats' section narrates result diagnostics: an LLM acting on the numbers must know when the dataset was truncated - new '## Glossary' defines volatility thresholds, persistence, q-value, and billable-minute semantics (previously only in the table legend, which the LLM format never showed) - volatile-step hints keyed by (workflow, job) — workflow A's docker step was attributable to workflow B's same-named job - [COST] priorities use the same PriorityScore >= 50 bar as the suggestions section instead of unconditional top-3 (ROADMAP U5)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes ROADMAP U5.
categoryBreakdownties and theByConclusionmax-pick both depended on map iteration order — output flapped between runs on identical data (would flake any golden test, and diff-based consumers see phantom changes). Lexicographic tie-breaks, pinned by run-50-times tests (red first).## Data Caveats: result diagnostics narrated in the briefing — an LLM acting on the numbers must see "3 runs exceed 50 jobs; extra entries were not fetched" (completes the D3 chain into the LLM format).## Glossary: volatility thresholds, persistence semantics, q-value meaning, billable-minute rules — previously only in the table legend, which--format llmnever rendered, so the LLM had to guess what "spiky" means.buildVolatileStepIndexkeyed by (workflow, job) — same-named jobs across workflows misattributed volatile steps (red test with two "build" jobs).[COST]gate: priorities now require PriorityScore ≥ 50, same bar as the suggestions section — top-3-regardless included 2-min/day workflows as "priority findings".Live: cli/cli
--format llmrenders Priority → Summaries → Pipelines → Steps → Suggestions → Data Caveats → Glossary → Raw Data.