From 8dedc335dfc6ca03ce2e3c848f44a250ceeb621f Mon Sep 17 00:00:00 2001 From: Hugh Grigg Date: Fri, 28 Aug 2026 19:41:33 +0100 Subject: [PATCH 1/2] feat: read a narrowed question back off the stored summaries A site that narrows a question said the same thing in three places. The `requests` prop on `RollupQueries`, the `requests` prop on `RollupSummaries`, and the options on every command line reading the answers back. The third copy had nowhere to read from, so whoever typed the command repeated what the deployment already declared, and a deployment that changed its narrowing left every shell alias behind. A run that names none of the narrowing options now takes them from the summaries it read, and standard error says which it took. An option somebody typed is still theirs, and `answersSomethingElse` still stops a run whose filters no stored summary covers. Telling a filter nobody typed from one that happens to equal the default is the whole problem. `rollupRequest` fills in a value for every field of every question, so by the time a `RollupRequest` exists the two are the same thing. `RollupAsked.named` records which options arrived, read off the command line before the defaults go in. Where the summaries of a span were computed more than one way, the command names the option that would settle it and stops. That is what a change to the `requests` prop leaves in the bucket, and taking one of two narrowings would answer part of the span under a question the rest never asked. `docs/rollups/#narrowing-a-saved-copy` now says that the two `requests` props take the same shape, which is the declaration a site can already share. --- docs/rollups/README.md | 51 ++++++++- src/cli/rollup-options.ts | 32 ++++++ src/cli/summary-adoption.ts | 180 +++++++++++++++++++++++++++++++ src/cli/summary-answer.test.ts | 119 +++++++++++++++++++- src/cli/summary-answer.ts | 77 ++++++------- src/cli/summary-covering.test.ts | 114 ++++++++++++++++++++ src/cli/summary-help.ts | 6 ++ src/cli/summary-narrowings.ts | 80 ++++++++++++++ src/cli/summary-question.ts | 36 ++++++- src/cli/summary-refusals.ts | 38 ++++++- src/cli/summary-report.ts | 28 +++++ 11 files changed, 707 insertions(+), 54 deletions(-) create mode 100644 src/cli/summary-adoption.ts create mode 100644 src/cli/summary-narrowings.ts diff --git a/docs/rollups/README.md b/docs/rollups/README.md index e918578..3eab324 100644 --- a/docs/rollups/README.md +++ b/docs/rollups/README.md @@ -234,6 +234,29 @@ for it. An entry takes whatever `RollupRequest` carries, minus the range and the dataset. A field added to the request arrives here on its own. +`RollupSummaries` takes the same shape on its own `requests` prop. A site narrowing a question for +both writes the narrowing once and passes the constant to each: + +```typescript +const searches = { paths: ["/liju/search/", "/cidian/search/"], param: "term" }; + +new RollupQueries(this, "RainlyticsRollups", { + table, + workgroup, + requests: { searches }, +}); +new RollupSummaries(this, "RainlyticsSummaries", { + table, + workgroup, + requests: { searches }, +}); +``` + +A saved query and a stored summary that drift apart answer two questions under one name. The +constant keeps them in step, and +[reading a precomputed answer](#a-summary-answers-the-question-it-was-computed-with) is where the +command line picks the same narrowing up. + A fact that belongs to every question, such as the host of one site on a distribution serving several, is a variable spread into each entry. Every key has to name a rollup being saved, and a mistyped one fails at synthesis. The alternative is a deployed query still counting whatever it @@ -550,7 +573,18 @@ count every combination of them. `RollupSummaries` computes the unfiltered form and [`requests`](../summary-schedule/) is where a deployment adds a narrowed one under a name of its own. -A run whose filters no stored summary matches is told what was stored: +A run that names none of those options takes the ones the summaries were computed with. The +deployment declared its narrowing once and the command reads that copy back. A shell alias never has +to carry a second one. Standard error says which filters the run took: + +```text +Took --path /liju/search/ /cidian/search/ from the summaries. Those options +were left off this command line, and the answer covers the narrowing the +deployment computes. +``` + +An option somebody typed stays theirs. A run whose filters no stored summary matches is told what +was stored: ```text The stored pageviews summaries answer a different question. @@ -560,9 +594,20 @@ on RollupSummaries is where a narrowed one is added. --query answers this run from Athena at the cost a query reports. ``` +A change to `requests` leaves both narrowings in the bucket. Over a span holding some of each, the +command names the option that would settle it and stops: + +```text +The stored pageviews summaries over that span were not all computed the same +way, and this run named nothing to settle it with. + --path: some windows computed with /guides/, others with the whole distribution +``` + +Typing the one you want settles it. So does a span on one side of the change. + `--limit` is the one option where the two only have to overlap. A summary computed with the top -hundred paths holds the top twenty inside it. The reverse loses rows nobody counted, and it is -refused. +hundred paths holds the top twenty inside it. The reverse loses rows nobody counted. A run that +typed the row count is refused, and a run that typed none takes the stored one. ### Several windows add up, and the ranking is approximate diff --git a/src/cli/rollup-options.ts b/src/cli/rollup-options.ts index 028a094..d415b0d 100644 --- a/src/cli/rollup-options.ts +++ b/src/cli/rollup-options.ts @@ -19,6 +19,8 @@ import { summaryBucketFrom, } from "./option-values.js"; import { defaultLimit, defaultParam } from "./rollup-help.js"; +import type { NarrowingOption } from "./summary-question.js"; +import { narrowingOptions } from "./summary-question.js"; /** What one rollup was asked for, read off the command line. */ export interface RollupAsked { @@ -34,6 +36,20 @@ export interface RollupAsked { */ readonly range: TimeRange; + /** + * The narrowing options this command line actually named. + * + * Read before the defaults go in, and this is the only place the two can + * still be told apart. `rollupRequest` fills in a value for every field of + * every question. A `RollupRequest` carrying `limit: 20` says nothing about + * whether anybody typed `--limit 20`. + * + * `summary-adoption.ts` is what needs the difference. A filter nobody named + * is one the stored summaries can supply, and a filter somebody named is one + * they have to match. + */ + readonly named: ReadonlySet; + /** The bucket holding the precomputed answers, where one is known. */ readonly summaries: string | undefined; @@ -50,6 +66,21 @@ export interface RollupAsked { readonly region: string | undefined; } +/** + * The narrowing options that arrived on one command line. + * + * `narrowingOptions` spells each one as a reader types it, and the parser keys + * on the long name without the dashes. An option nobody gave is absent from + * the values, whatever default the request will carry for it. + */ +function namedOn(context: CommandContext): ReadonlySet { + return new Set( + narrowingOptions.filter( + (option) => context.options[option.slice(2)] !== undefined, + ), + ); +} + /** * One command line, read as a rollup request. * @@ -67,6 +98,7 @@ export function requestFrom( return { database, range, + named: namedOn(context), workgroup: chosen(context.options["workgroup"]) ?? defaultWorkgroupName, region: chosen(context.options["region"]), summaries: summaryBucketFrom(context.options["summaries"]), diff --git a/src/cli/summary-adoption.ts b/src/cli/summary-adoption.ts new file mode 100644 index 0000000..c270d17 --- /dev/null +++ b/src/cli/summary-adoption.ts @@ -0,0 +1,180 @@ +// Settling the question a run is answered under, and stopping the runs the +// bucket cannot answer. +// +// A site that narrows a question used to say so three times. Once on +// `RollupQueries`, once on `RollupSummaries`, and again on every command line +// reading the answers back. The third copy is the one with nowhere to read +// from, and a deployment that changes its narrowing leaves every shell alias +// behind. A stored summary records the question it was computed with. The +// command reads that copy back, and the third one goes away. +// +// An option nobody named is the only kind taken. `rollupRequest` fills in a +// default for every field of every question, and by the time a +// `RollupRequest` exists a default and a value somebody chose are the same +// thing. `RollupAsked.named` carries the difference, read off the command line +// before the defaults go in. +// +// A named option that disagrees with the stored summaries is refused here too. +// Adoption fills gaps and never overrides one, and `refuseAnotherQuestion` at +// the foot of this file is the check that makes that safe to say. + +import type { RollupSummary, SummaryQuestion } from "../rollup-summaries.js"; +import type { Rollup } from "../rollups.js"; +import type { + NarrowingOption, + StoredDisagreement, +} from "./summary-question.js"; +import { narrowedBy, narrowingText } from "./summary-narrowings.js"; +import { narrowingOptions, questionDifferences } from "./summary-question.js"; +import { + answersSomethingElse, + computedMoreThanOneWay, +} from "./summary-refusals.js"; + +/** What a run settled on, once the stored narrowing had been read. */ +export interface AdoptedQuestion { + /** The question the stored summaries are then checked against. */ + readonly question: SummaryQuestion; + + /** + * The filters taken from them, as the command line that would have asked. + * + * In the order help prints the options, and empty on a run that narrowed + * everything the summaries narrow. + */ + readonly adopted: readonly string[]; +} + +/** + * The question asked, with the filters nobody named taken from the summaries. + * + * A command line that narrows nothing is answered under the narrowing its + * deployment declared, and {@link AdoptedQuestion.adopted} is what standard + * error then says it took. A command line that narrows something keeps what it + * was given, and `refuseAnotherQuestion` decides whether the summaries hold + * that answer. + * + * @throws {Error} where the summaries answer a filter nobody named two ways. + * Taking one of the two would answer part of the span under a narrowing the + * rest was never computed with, and the question has to be settled before + * anything can be checked against it. + */ +export function adoptedQuestion( + rollup: Rollup, + asked: SummaryQuestion, + named: ReadonlySet, + summaries: readonly RollupSummary[], +): AdoptedQuestion { + const [first] = summaries; + + if (first === undefined) { + return { question: asked, adopted: [] }; + } + + const elsewhere = storedElsewhere(rollup, asked, summaries); + const wanted = narrowingOptions.filter( + (option) => !named.has(option) && elsewhere.has(option), + ); + const disagreements = disagreedOn(rollup, wanted, first, summaries); + + if (disagreements.length > 0) { + throw computedMoreThanOneWay(rollup, disagreements); + } + + let question = asked; + + for (const option of wanted) { + question = narrowedBy(option, question, first.question); + } + + return { + question, + adopted: wanted.map((option) => narrowingText(option, first.question)), + }; +} + +/** The options any stored summary was computed with differently. */ +function storedElsewhere( + rollup: Rollup, + asked: SummaryQuestion, + summaries: readonly RollupSummary[], +): ReadonlySet { + return new Set( + summaries + .flatMap((summary) => + questionDifferences(rollup, asked, summary.question), + ) + .map((difference) => difference.option), + ); +} + +/** + * The options among `wanted` that the summaries answer two ways. + * + * Every summary is compared against `first`, both ways round, since `--limit` + * reports a difference in one direction alone. A window computed with the top + * hundred rows covers a run asking for twenty, and a window computed with the + * top one does not. Comparing `first` against itself finds nothing, which is + * why it stays in the list it is compared against. + */ +function disagreedOn( + rollup: Rollup, + wanted: readonly NarrowingOption[], + first: RollupSummary, + summaries: readonly RollupSummary[], +): readonly StoredDisagreement[] { + const found = new Map>(); + + for (const other of summaries) { + const between = [ + ...questionDifferences(rollup, first.question, other.question), + ...questionDifferences(rollup, other.question, first.question), + ]; + + for (const difference of between) { + const values = found.get(difference.option) ?? new Set(); + + found.set( + difference.option, + values.add(difference.asked).add(difference.computed), + ); + } + } + + return wanted.flatMap((option) => { + const values = found.get(option); + + return values === undefined ? [] : [{ option, computed: [...values] }]; + }); +} + +/** + * Stops a run whose filters no stored summary was computed with. + * + * Every window is checked rather than the first alone. A deployment whose + * question changed halfway through the span has summaries of both, and the + * older half answers something the command line never asked. + * + * The question checked is the settled one. A filter the command line named is + * still its own, and a filter taken from the summaries matches them by + * construction. + * + * @throws {Error} naming every option a reader would have to change. + */ +export function refuseAnotherQuestion( + rollup: Rollup, + question: SummaryQuestion, + summaries: readonly RollupSummary[], +): void { + const differences = new Map( + summaries + .flatMap((summary) => + questionDifferences(rollup, question, summary.question), + ) + .map((difference) => [difference.option, difference]), + ); + + if (differences.size > 0) { + throw answersSomethingElse(rollup, [...differences.values()]); + } +} diff --git a/src/cli/summary-answer.test.ts b/src/cli/summary-answer.test.ts index d283349..4838b3c 100644 --- a/src/cli/summary-answer.test.ts +++ b/src/cli/summary-answer.test.ts @@ -567,26 +567,137 @@ describe("the named questions, answered from stored summaries", () => { expect(Number(ratio?.["hit_percent"])).toBe(100); }); - it("refuses a row count the stored summary cannot reach", async () => { - // Given a deployment computing the top row alone. + it("takes the row count the summaries were computed with", async () => { + // Given a deployment computing the top row alone, and two pages looked + // at in the hour it computed. + const deployed = await deployAnalytics({ + requests: { pageviews: { limit: 1 } }, + }); + await putDelivered(deployed, anHour, [ + aRecord(anHour), + aRecord(anHour), + aRecord(anHour, { "cs-uri-stem": "/grammar/" }), + ]); + await untilTheScheduleFires(deployed); + + // When a run names no row count. + const run = await cli([ + "pageviews", + "--last", + "2h", + "--summaries", + deployed.summariesBucketName, + ]); + + // Then the stored row count answers, and standard error says where it + // came from. Refusing this would have made every shell alias carry the + // number its deployment already declared. + expect(run.code).toBe(0); + expect(run.rows).toStrictEqual([{ path: "/", views: "2" }]); + expect(run.error).toContain("Took --limit 1 from the summaries"); + }); + + it("refuses a row count somebody typed that the summary cannot reach", async () => { + // Given the same deployment computing the top row alone. const deployed = await deployAnalytics({ requests: { pageviews: { limit: 1 } }, }); await putDelivered(deployed, anHour, [aRecord(anHour)]); await untilTheScheduleFires(deployed); - // When a run asks for twenty. + // When a run asks for twenty of them. const run = await cli([ "pageviews", "--last", "2h", + "--limit", + "20", "--summaries", deployed.summariesBucketName, ]); // Then it says so. Nineteen rows nobody counted cannot be recovered from - // a stored answer holding one. + // a stored answer holding one, and a row count somebody typed is theirs + // rather than a gap to fill. expect(run.code).toBe(1); expect(run.error).toContain("--limit"); }); + + it("takes the sections a deployment narrowed to", async () => { + // Given a deployment that computes pageviews under one section, and + // traffic on both sides of that narrowing. + const deployed = await deployAnalytics({ + requests: { pageviews: { paths: ["/grammar/"] } }, + }); + await putDelivered(deployed, anHour, [ + aRecord(anHour), + aRecord(anHour, { "cs-uri-stem": "/grammar/" }), + ]); + await untilTheScheduleFires(deployed); + + // When the question is asked with no --path. + const run = await cli([ + "pageviews", + "--last", + "2h", + "--summaries", + deployed.summariesBucketName, + ]); + + // Then the narrowed answer comes back and standard error names the + // filter it took. The deployment declared that list once, and nobody has + // to type it again to read what it computed. + expect(run.code).toBe(0); + expect(run.rows).toStrictEqual([{ path: "/grammar/", views: "1" }]); + expect(run.error).toContain("Took --path /grammar/ from the summaries"); + }); + + it("takes a bot filter as the flag that would have set it", async () => { + // Given a deployment counting crawlers, and an hour holding one. + const deployed = await deployAnalytics({ + requests: { pageviews: { includeBots: true } }, + }); + await putDelivered(deployed, anHour, [ + aRecord(anHour), + aRecord(anHour, { "cs(User-Agent)": "ClaudeBot/1.0" }), + ]); + await untilTheScheduleFires(deployed); + + // When the question is asked with no --include-bots. + const run = await cli([ + "pageviews", + "--last", + "2h", + "--summaries", + deployed.summariesBucketName, + ]); + + // Then the crawler is in the count and the flag is named on its own. A + // flag takes no value, and the line is what a reader would have typed. + expect(run.code).toBe(0); + expect(run.rows).toStrictEqual([{ path: "/", views: "2" }]); + expect(run.error).toContain("Took --include-bots from the summaries"); + }); + + it("says nothing about filters where the command line matched", async () => { + // Given a deployment left to the defaults. That is what + // RollupSummaries computes where nobody narrows it. + const deployed = await deployAnalytics(); + await putDelivered(deployed, anHour, [aRecord(anHour)]); + await untilTheScheduleFires(deployed); + + // When the question is asked with nothing narrowing it either. + const run = await cli([ + "pageviews", + "--last", + "2h", + "--summaries", + deployed.summariesBucketName, + ]); + + // Then standard error carries no line about what was taken. A run that + // asked what the bucket holds took nothing from it. + expect(run.code).toBe(0); + expect(run.error).not.toContain("from the summaries"); + }); }); diff --git a/src/cli/summary-answer.ts b/src/cli/summary-answer.ts index 76bdf18..cc3844a 100644 --- a/src/cli/summary-answer.ts +++ b/src/cli/summary-answer.ts @@ -7,22 +7,23 @@ // without anybody choosing it, so every case that cannot be answered from the // bucket says what it found and names `--query`. // -// The steps are in three files around this one. `summary-coverage.ts` picks -// the windows, `summary-covering.ts` fetches them and decides what a gap -// means, and `summary-question.ts` decides whether what came back answers the -// question that was asked. +// The steps are in the files around this one. `summary-coverage.ts` picks the +// windows, `summary-covering.ts` fetches them and decides what a gap means, +// `summary-question.ts` compares what came back against what was asked, and +// `summary-adoption.ts` settles the question a run is answered under and stops +// the runs no stored summary covers. -import type { RollupSummary } from "../rollup-summaries.js"; +import type { RollupSummary, SummaryQuestion } from "../rollup-summaries.js"; import type { Rollup } from "../rollups.js"; import { summaryCoverage } from "../summary-coverage.js"; import { totalledRows } from "../summary-totals.js"; import type { CliIo } from "./io.js"; import type { CommandResult } from "./output/result.js"; import type { RollupAsked } from "./rollup-options.js"; +import { adoptedQuestion, refuseAnotherQuestion } from "./summary-adoption.js"; import { summaryCovering } from "./summary-covering.js"; -import { askedQuestion, questionDifferences } from "./summary-question.js"; +import { askedQuestion } from "./summary-question.js"; import { - answersSomethingElse, doesNotAdd, noWholeWindow, nowhereToRead, @@ -32,11 +33,17 @@ import { summaryReport } from "./summary-report.js"; /** * One question, answered from the bucket a schedule writes to. * + * A run that named no filters of its own is answered under the narrowing the + * summaries were computed with, and standard error says which filters it took. + * A site declares its narrowing on `RollupSummaries` and the command line + * reads that copy back. A shell alias never has to carry a second one. + * * @throws {UsageError} where nothing says which bucket to read, or where the * span asked for holds no whole stored window. * @throws {Error} where the windows were never computed, where the stored - * summaries answer a different question, or where several windows answered - * a question whose rows do not add. + * summaries answer a different question, where the span was computed more + * than one way, or where several windows answered a question whose rows do + * not add. */ export async function summaryRows( rollup: Rollup, @@ -61,8 +68,14 @@ export async function summaryRows( question, windows, ); + const settled = adoptedQuestion( + rollup, + question, + asked.named, + covering.summaries, + ); - refuseAnotherQuestion(rollup, question, covering.summaries); + refuseAnotherQuestion(rollup, settled.question, covering.summaries); io.error( summaryReport({ @@ -71,26 +84,27 @@ export async function summaryRows( summaries: covering.summaries, missing: covering.missing, gets: covering.gets, + adopted: settled.adopted, isRanked: rollup.isRanked, at: new Date(), }), ); - return answerFrom(rollup, asked, covering.summaries); + return answerFrom(rollup, settled.question, covering.summaries); } /** * The rows of one window, or the windows added together. * - * One window comes back in the order it was written, cut to the row count - * that was asked for. Nothing is grouped or re-ordered, so a pipeline reading - * the JSON sees what the query would have answered. The cut matters where a - * deployment computes deeper than a command asks. The top twenty of a stored - * hundred are the top twenty. + * One window comes back in the order it was written, cut to the row count the + * settled question carries. Nothing is grouped or re-ordered, so a pipeline + * reading the JSON sees what the query would have answered. The cut matters + * where a deployment computes deeper than a command asks. The top twenty of a + * stored hundred are the top twenty. */ function answerFrom( rollup: Rollup, - asked: RollupAsked, + question: SummaryQuestion, summaries: readonly RollupSummary[], ): CommandResult { // Every window of one question names the same columns, so the set of them @@ -101,7 +115,7 @@ function answerFrom( if (summaries.length === 1) { return { columns, - rows: rollup.isRanked ? rows.slice(0, asked.request.limit) : rows, + rows: rollup.isRanked ? rows.slice(0, question.limit) : rows, }; } @@ -115,33 +129,8 @@ function answerFrom( ...totalledRows( summaries, rollup.totals, - rollup.isRanked ? asked.request.limit : undefined, + rollup.isRanked ? question.limit : undefined, ), ], }; } - -/** - * Stops a run whose filters no stored summary was computed with. - * - * Every window is checked rather than the first alone. A deployment whose - * question changed halfway through the span has summaries of both, and the - * older half answers something the command line never asked. - */ -function refuseAnotherQuestion( - rollup: Rollup, - question: ReturnType, - summaries: readonly RollupSummary[], -): void { - const differences = new Map( - summaries - .flatMap((summary) => - questionDifferences(rollup, question, summary.question), - ) - .map((difference) => [difference.option, difference]), - ); - - if (differences.size > 0) { - throw answersSomethingElse(rollup, [...differences.values()]); - } -} diff --git a/src/cli/summary-covering.test.ts b/src/cli/summary-covering.test.ts index 031a302..10ce552 100644 --- a/src/cli/summary-covering.test.ts +++ b/src/cli/summary-covering.test.ts @@ -17,6 +17,7 @@ import { summarySpan } from "../summary-windows.js"; import { rainlyticsCommands } from "./command.js"; import { runCli } from "./run.js"; import { summaryRows } from "./summary-answer.js"; +import type { NarrowingOption } from "./summary-question.js"; /* * A bucket of summaries put there by hand, and the command line reading it. @@ -232,6 +233,118 @@ describe("reading a bucket of summaries", () => { expect(run.error).toContain("1 window in the range asked for has"); }); + it("takes every filter a narrowed question was computed with", async () => { + // Given summaries of a site's two search boxes, computed under the + // parameter that site's boxes use and the status its exact match + // answers with. That narrowing lives in the deployment's stack. + const deployed = await aBucket(); + await putSummary( + deployed, + anHourOn("2026-08-24", 8), + [{ term: "happy", searches: "3" }], + { + question: { + ...question, + name: "searches", + paths: ["/liju/search/", "/cidian/search/"], + param: "term", + redirectStatuses: ["301", "302"], + }, + columns: ["term", "searches"], + }, + ); + + // When the question is asked with none of them typed. + const run = await cli([ + "searches", + "--last", + "2h", + "--summaries", + deployed.bucket, + ]); + + // Then all three come off the summary and standard error names them as + // the command line that would have asked. The deployment declared the + // list once, and nothing repeats it in a shell alias. + expect(run.code).toBe(0); + expect(run.rows).toStrictEqual([{ term: "happy", searches: "3" }]); + expect(run.error).toContain( + "Took --path /liju/search/ /cidian/search/ --param term" + + " --redirect-status 301,302 from the summaries", + ); + }); + + it("refuses a span its deployment computed two ways", async () => { + // Given two hours computed either side of a change to the requests + // prop. The bucket holds one narrowed hour and one unnarrowed one. + const deployed = await aBucket(); + await putSummary( + deployed, + anHourOn("2026-08-24", 7), + [{ path: "/", views: "1" }], + { question: { ...question, paths: ["/grammar/"] } }, + ); + await putSummary(deployed, anHourOn("2026-08-24", 8), [ + { path: "/", views: "1" }, + ]); + + // When both are asked about with nothing narrowing the run. + const run = await cli([ + "pageviews", + "--last", + "3h", + "--summaries", + deployed.bucket, + ]); + + // Then it names the option that would settle it and stops. Taking one of + // the two would answer half the span under a narrowing the other half + // was never computed with, and nothing in the rows would say which half. + expect(run.code).toBe(1); + expect(run.error).toContain("not all computed the same way"); + expect(run.error).toContain( + "--path: some windows computed with /grammar/, others with the whole" + + " distribution", + ); + expect(run.error).toContain("--query"); + }); + + it("takes one filter and still refuses another that was typed", async () => { + // Given summaries computed for one site on the distribution, under one + // section of it. + const deployed = await aBucket(); + await putSummary( + deployed, + anHourOn("2026-08-24", 8), + [{ path: "/grammar/", views: "1" }], + { + question: { + ...question, + host: "docs.example.com", + paths: ["/grammar/"], + }, + }, + ); + + // When a run names a different section and leaves the host out. + const run = await cli([ + "pageviews", + "--last", + "2h", + "--path", + "/liju/", + "--summaries", + deployed.bucket, + ]); + + // Then the section is refused and the host is not mentioned. A filter + // somebody typed is theirs whatever else the summaries could have + // supplied. + expect(run.code).toBe(1); + expect(run.error).toContain("--path: asked for /liju/"); + expect(run.error).not.toContain("--host"); + }); + it("names a bot filter the stored summaries were not computed with", async () => { // Given a summary computed with crawlers left out, which is the default // every schedule takes. @@ -408,6 +521,7 @@ describe("reading a bucket of summaries", () => { dataset: defaultLogDataset, }), range: { from: new Date(now.getTime() - 10_800_000), to: now }, + named: new Set(), summaries: deployed.bucket, runsTheQuery: false, database: defaultLogDataset.databaseName, diff --git a/src/cli/summary-help.ts b/src/cli/summary-help.ts index eba547c..253a5e0 100644 --- a/src/cli/summary-help.ts +++ b/src/cli/summary-help.ts @@ -56,6 +56,12 @@ form, and its \`requests\` prop is where a deployment adds a narrowed one. A run whose filters no stored summary matches says what was stored and stops, so nothing quietly falls back to a query nobody chose. +A run that names none of those options takes the ones the summaries were +computed with, and standard error says which. The deployment declared its +narrowing on RollupSummaries and this command reads that copy back. A shell +alias never has to carry a second one. An option somebody typed is still +theirs, and still refused where no stored summary matches it. + A range is covered by whole UTC windows, days wherever a whole day fits and hours at the two edges. The part hours at either end have no stored window, so a span always reads a little short of the one asked for. Standard error names diff --git a/src/cli/summary-narrowings.ts b/src/cli/summary-narrowings.ts new file mode 100644 index 0000000..bf021bc --- /dev/null +++ b/src/cli/summary-narrowings.ts @@ -0,0 +1,80 @@ +// What each narrowing option does to the question a summary records. +// +// The join between an option a reader types and a field of `SummaryQuestion`. +// `summary-adoption.ts` walks it to take the filters a command line left out, +// and to spell each one back the way it would have arrived. +// +// Written out one option at a time rather than derived. The fields are named +// here. A field that leaves `RollupRequest` stops this compiling, and a reader +// can see what `--redirect-status` does to a question without following a +// lookup. + +import type { SummaryQuestion } from "../rollup-summaries.js"; +import type { NarrowingOption } from "./summary-question.js"; + +/** How one option reaches the question a summary records. */ +interface Narrowing { + /** The asked question, with this option taken from a stored one. */ + readonly taken: ( + asked: SummaryQuestion, + stored: SummaryQuestion, + ) => SummaryQuestion; + + /** What a reader would have given the option. Absent for a flag. */ + readonly value?: (stored: SummaryQuestion) => string | undefined; +} + +/** Each option, as the field it sets and the value a reader would type. */ +const narrowings: Readonly> = { + "--host": { + taken: (asked, stored) => ({ ...asked, host: stored.host }), + value: (stored) => stored.host, + }, + "--path": { + taken: (asked, stored) => ({ ...asked, paths: stored.paths }), + value: (stored) => stored.paths?.join(" "), + }, + "--include-bots": { + taken: (asked, stored) => ({ ...asked, includeBots: stored.includeBots }), + }, + "--param": { + taken: (asked, stored) => ({ ...asked, param: stored.param }), + value: (stored) => stored.param, + }, + "--redirect-status": { + taken: (asked, stored) => ({ + ...asked, + redirectStatuses: stored.redirectStatuses, + }), + value: (stored) => stored.redirectStatuses.join(","), + }, + "--limit": { + taken: (asked, stored) => ({ ...asked, limit: stored.limit }), + value: (stored) => String(stored.limit), + }, +}; + +/** One question, with one option's value taken from a stored summary. */ +export function narrowedBy( + option: NarrowingOption, + asked: SummaryQuestion, + stored: SummaryQuestion, +): SummaryQuestion { + return narrowings[option].taken(asked, stored); +} + +/** + * One stored option, as the command line that would have asked for it. + * + * `--path /liju/search/ /cidian/search/` for a list, and `--include-bots` on + * its own for a flag. A reader copies the line onto a command of their own, + * and a reporter prints it without knowing how any option is spelled. + */ +export function narrowingText( + option: NarrowingOption, + stored: SummaryQuestion, +): string { + const value = narrowings[option].value?.(stored); + + return value === undefined ? option : `${option} ${value}`; +} diff --git a/src/cli/summary-question.ts b/src/cli/summary-question.ts index a3e0965..dfd5f60 100644 --- a/src/cli/summary-question.ts +++ b/src/cli/summary-question.ts @@ -14,10 +14,33 @@ import type { SummaryQuestion } from "../rollup-summaries.js"; import type { Rollup, RollupRequest } from "../rollups.js"; +/** + * The options that narrow a question, spelled as a reader types them. + * + * The list every reader of a narrowing walks. `questionDifferences` compares + * these fields, `summary-adoption.ts` takes the ones a command line left out, + * and `rollup-options.ts` records which of them arrived. An option added to a + * rollup command reaches all three from here. + * + * `--last` is left out. The window is the range a summary answers over, and a + * span is what a reader is choosing when they type it. + */ +export const narrowingOptions = [ + "--host", + "--path", + "--include-bots", + "--param", + "--redirect-status", + "--limit", +] as const; + +/** One option that narrows a question. */ +export type NarrowingOption = (typeof narrowingOptions)[number]; + /** One way a stored summary answers something else. */ export interface QuestionDifference { /** The option a reader would change, as they would type it. */ - readonly option: string; + readonly option: NarrowingOption; /** What this run asked for. */ readonly asked: string; @@ -26,6 +49,15 @@ export interface QuestionDifference { readonly computed: string; } +/** One filter the stored summaries of a span were not all computed with. */ +export interface StoredDisagreement { + /** The option that would settle it. */ + readonly option: NarrowingOption; + + /** The values found among the summaries, as a sentence names them. */ + readonly computed: readonly string[]; +} + /** The question one command line asks, as a summary would record it. */ export function askedQuestion( rollup: Rollup, @@ -96,7 +128,7 @@ export function questionDifferences( /** The difference between two values, where there is one. */ function differing( - option: string, + option: NarrowingOption, asked: string, computed: string, ): readonly QuestionDifference[] { diff --git a/src/cli/summary-refusals.ts b/src/cli/summary-refusals.ts index d7f00de..4694170 100644 --- a/src/cli/summary-refusals.ts +++ b/src/cli/summary-refusals.ts @@ -15,7 +15,10 @@ import type { SummaryWindow } from "../summary-windows.js"; import { summarySpan } from "../summary-windows.js"; import { UsageError } from "./failure.js"; import { summaryBucketVariable } from "./summary-help.js"; -import type { QuestionDifference } from "./summary-question.js"; +import type { + QuestionDifference, + StoredDisagreement, +} from "./summary-question.js"; /** What a reader is told where nothing says which bucket holds the answers. */ export function nowhereToRead(rollup: Rollup): UsageError { @@ -66,6 +69,39 @@ export function answersSomethingElse( ); } +/** + * What a reader is told where the stored summaries were narrowed two ways. + * + * A command line naming no filters takes the ones its deployment declared, and + * this is the span where there is more than one answer to take. A deployment + * that changed its `requests` halfway through has summaries of both questions, + * and an answer assembled from them would cover part of the span under a + * narrowing the rest was never computed with. + * + * The reader settles it by typing the one they want. Every window the run + * still disagrees with is then named by {@link answersSomethingElse}. That is + * the same conversation from the other end. + */ +export function computedMoreThanOneWay( + rollup: Rollup, + disagreements: readonly StoredDisagreement[], +): Error { + const lines = disagreements.map( + (disagreement) => + ` ${disagreement.option}: some windows computed with` + + ` ${disagreement.computed.join(", others with ")}`, + ); + + return new Error( + `The stored ${rollup.name} summaries over that span were not all` + + ` computed the same way, and this run named nothing to settle it` + + ` with.\n${lines.join("\n")}\nA deployment that changed its requests` + + ` prop leaves both questions in the bucket. Name the one you want on` + + ` the command line, ask about a span on one side of the change, or run` + + ` it with --query.`, + ); +} + /** What a reader is told where a question's rows cannot be added up. */ export function doesNotAdd(rollup: Rollup, windows: number): Error { return new Error( diff --git a/src/cli/summary-report.ts b/src/cli/summary-report.ts index 52ff61d..a2378c8 100644 --- a/src/cli/summary-report.ts +++ b/src/cli/summary-report.ts @@ -35,6 +35,9 @@ export interface SummaryRead { /** How many objects were asked for, which is what the read cost. */ readonly gets: number; + /** The filters this run took from the summaries rather than the line. */ + readonly adopted: readonly string[]; + /** Whether the question answers with a ranked list. */ readonly isRanked: boolean; @@ -52,6 +55,7 @@ export interface SummaryRead { export function summaryReport(read: SummaryRead): string { return [ coverageLine(read), + ...adoptedLines(read), freshnessLine(read), ...missingLines(read), ...rankingLines(read), @@ -70,6 +74,30 @@ function coverageLine(read: SummaryRead): string { ); } +/** + * The filters this run took from the summaries. + * + * Part of what answered rather than a report of its own, and this is the + * paragraph saying which question the rows below belong to. A reader who typed + * four words and got an answer about one section of their site has to be able + * to see where that section came from, and the line is what they copy onto the + * command line to ask a narrower question of the same summaries. + * + * Printed on the runs that answered. A run refused for a filter somebody typed + * needs the refusal and not a paragraph about the filters it would have taken. + */ +function adoptedLines(read: SummaryRead): readonly string[] { + if (read.adopted.length === 0) { + return []; + } + + return [ + `Took ${read.adopted.join(" ")} from the summaries. Those options were` + + ` left off this command line, and the answer covers the narrowing the` + + ` deployment computes.\n`, + ]; +} + /** How old the newest of them is, and what the whole read cost. */ function freshnessLine(read: SummaryRead): string { const newest = newestComputedAt(read.summaries); From 9540e74e029c42c71886bc5ca00cd92f7fca82cf Mon Sep 17 00:00:00 2001 From: Hugh Grigg Date: Fri, 28 Aug 2026 19:52:52 +0100 Subject: [PATCH 2/2] fix: keep the command's own row count where the summaries go deeper A row count is not a narrowing, and treating it as one made a deployment computing the top hundred print a hundred rows for a bare `rainlytics pageviews`. The documented default is twenty, and the stored hundred holds them. `--limit` now sits apart from the five options deciding which requests were counted. Those five are taken from the stored summaries where a command line named none of them. The row count keeps the command's own default, cut to the shallowest stored window where one of them holds fewer rows than that. A row count somebody typed is refused as before. Two windows computed to different depths therefore hold one answer between them rather than a disagreement, and only the five can be computed two ways over one span. Typing a filter no longer settles a span narrowed two ways. The windows computed the other way refuse it through `answersSomethingElse`, so the refusal offers a shorter span and `--query` and stops advising the option. The help text and `docs/rollups/` now name `--redirect-status` alongside the other four, and say what happens to the row count. --- docs/rollups/README.md | 23 ++++--- src/cli/summary-adoption.ts | 100 +++++++++++++++---------------- src/cli/summary-answer.test.ts | 33 ++++++++++ src/cli/summary-covering.test.ts | 37 ++++++++++++ src/cli/summary-help.ts | 27 +++++---- src/cli/summary-narrowings.ts | 53 ++++++++++++---- src/cli/summary-question.ts | 25 ++++++-- src/cli/summary-refusals.ts | 17 +++--- 8 files changed, 219 insertions(+), 96 deletions(-) diff --git a/docs/rollups/README.md b/docs/rollups/README.md index 3eab324..ef1501d 100644 --- a/docs/rollups/README.md +++ b/docs/rollups/README.md @@ -568,12 +568,12 @@ that answered and how old it is. ### A summary answers the question it was computed with -`--path`, `--host`, `--include-bots` and `--param` each change the answer, and a schedule cannot -count every combination of them. `RollupSummaries` computes the unfiltered form of each question, -and [`requests`](../summary-schedule/) is where a deployment adds a narrowed one under a name of its -own. +`--path`, `--host`, `--include-bots`, `--param` and `--redirect-status` each decide which requests +were counted, and a schedule cannot count every combination of them. `RollupSummaries` computes the +unfiltered form of each question, and [`requests`](../summary-schedule/) is where a deployment adds a +narrowed one under a name of its own. -A run that names none of those options takes the ones the summaries were computed with. The +A run that names none of those five takes the ones the summaries were computed with. The deployment declared its narrowing once and the command reads that copy back. A shell alias never has to carry a second one. Standard error says which filters the run took: @@ -603,11 +603,16 @@ way, and this run named nothing to settle it with. --path: some windows computed with /guides/, others with the whole distribution ``` -Typing the one you want settles it. So does a span on one side of the change. +Typing one of them settles nothing, since the windows computed the other way then refuse it. A span +on one side of the change reads from stored summaries, and `--query` answers one covering both. -`--limit` is the one option where the two only have to overlap. A summary computed with the top -hundred paths holds the top twenty inside it. The reverse loses rows nobody counted. A run that -typed the row count is refused, and a run that typed none takes the stored one. +`--limit` is apart from those five. A row count decides how much of a ranked answer is printed and +leaves what was counted where it was, so a deployment computing the top hundred paths still answers +`rainlytics pageviews` with the top twenty. The stored hundred holds them. + +A summary computed with fewer rows than the command asks for is the other way round, and those rows +were never counted. A run that typed the count is refused. A run that typed none is cut to what the +stored windows hold, and standard error names the count it answered with. ### Several windows add up, and the ranking is approximate diff --git a/src/cli/summary-adoption.ts b/src/cli/summary-adoption.ts index c270d17..04773c7 100644 --- a/src/cli/summary-adoption.ts +++ b/src/cli/summary-adoption.ts @@ -21,11 +21,16 @@ import type { RollupSummary, SummaryQuestion } from "../rollup-summaries.js"; import type { Rollup } from "../rollups.js"; import type { + CountingOption, NarrowingOption, StoredDisagreement, } from "./summary-question.js"; -import { narrowedBy, narrowingText } from "./summary-narrowings.js"; -import { narrowingOptions, questionDifferences } from "./summary-question.js"; +import { + narrowedBy, + narrowingText, + storedRowCount, +} from "./summary-narrowings.js"; +import { countingOptions, questionDifferences } from "./summary-question.js"; import { answersSomethingElse, computedMoreThanOneWay, @@ -48,13 +53,13 @@ export interface AdoptedQuestion { /** * The question asked, with the filters nobody named taken from the summaries. * - * A command line that narrows nothing is answered under the narrowing its - * deployment declared, and {@link AdoptedQuestion.adopted} is what standard - * error then says it took. A command line that narrows something keeps what it - * was given, and `refuseAnotherQuestion` decides whether the summaries hold - * that answer. + * A command line that counts nothing in particular is answered under the + * narrowing its deployment declared, and {@link AdoptedQuestion.adopted} is + * what standard error then says it took. A command line that narrows something + * keeps what it was given, and `refuseAnotherQuestion` decides whether the + * summaries hold that answer. * - * @throws {Error} where the summaries answer a filter nobody named two ways. + * @throws {Error} where the summaries counted a filter nobody named two ways. * Taking one of the two would answer part of the span under a narrowing the * rest was never computed with, and the question has to be settled before * anything can be checked against it. @@ -71,8 +76,16 @@ export function adoptedQuestion( return { question: asked, adopted: [] }; } - const elsewhere = storedElsewhere(rollup, asked, summaries); - const wanted = narrowingOptions.filter( + // The options some stored summary was computed with differently, which are + // the only ones there is anything to take. + const elsewhere = new Set( + summaries + .flatMap((summary) => + questionDifferences(rollup, asked, summary.question), + ) + .map((difference) => difference.option), + ); + const wanted = countingOptions.filter( (option) => !named.has(option) && elsewhere.has(option), ); const disagreements = disagreedOn(rollup, wanted, first, summaries); @@ -87,64 +100,45 @@ export function adoptedQuestion( question = narrowedBy(option, question, first.question); } - return { - question, - adopted: wanted.map((option) => narrowingText(option, first.question)), - }; -} + const cut = storedRowCount(rollup, asked, named, summaries); + const adopted = wanted.map((option) => narrowingText(option, first.question)); -/** The options any stored summary was computed with differently. */ -function storedElsewhere( - rollup: Rollup, - asked: SummaryQuestion, - summaries: readonly RollupSummary[], -): ReadonlySet { - return new Set( - summaries - .flatMap((summary) => - questionDifferences(rollup, asked, summary.question), - ) - .map((difference) => difference.option), - ); + return cut === undefined + ? { question, adopted } + : { + question: { ...question, limit: cut }, + adopted: [...adopted, `--limit ${String(cut)}`], + }; } /** * The options among `wanted` that the summaries answer two ways. * - * Every summary is compared against `first`, both ways round, since `--limit` - * reports a difference in one direction alone. A window computed with the top - * hundred rows covers a run asking for twenty, and a window computed with the - * top one does not. Comparing `first` against itself finds nothing, which is - * why it stays in the list it is compared against. + * Every summary is compared against `first`. Comparing `first` against itself + * finds nothing, which is why it stays in the list it is compared against. + * + * `--limit` never reaches here. Two windows computed to different depths hold + * one answer between them, being the shallower of the two, and + * {@link storedRowCount} takes it. */ function disagreedOn( rollup: Rollup, - wanted: readonly NarrowingOption[], + wanted: readonly CountingOption[], first: RollupSummary, summaries: readonly RollupSummary[], ): readonly StoredDisagreement[] { - const found = new Map>(); - - for (const other of summaries) { - const between = [ - ...questionDifferences(rollup, first.question, other.question), - ...questionDifferences(rollup, other.question, first.question), - ]; - - for (const difference of between) { - const values = found.get(difference.option) ?? new Set(); - - found.set( - difference.option, - values.add(difference.asked).add(difference.computed), - ); - } - } + const between = summaries.flatMap((other) => + questionDifferences(rollup, first.question, other.question), + ); return wanted.flatMap((option) => { - const values = found.get(option); + const values = new Set( + between + .filter((difference) => difference.option === option) + .flatMap((difference) => [difference.asked, difference.computed]), + ); - return values === undefined ? [] : [{ option, computed: [...values] }]; + return values.size === 0 ? [] : [{ option, computed: [...values] }]; }); } diff --git a/src/cli/summary-answer.test.ts b/src/cli/summary-answer.test.ts index 4838b3c..580297f 100644 --- a/src/cli/summary-answer.test.ts +++ b/src/cli/summary-answer.test.ts @@ -597,6 +597,39 @@ describe("the named questions, answered from stored summaries", () => { expect(run.error).toContain("Took --limit 1 from the summaries"); }); + it("keeps its own row count where the summaries go deeper", async () => { + // Given a deployment computing the top hundred paths, and two of them + // looked at in the hour it computed. + const deployed = await deployAnalytics({ + requests: { pageviews: { limit: 100 } }, + }); + await putDelivered(deployed, anHour, [ + aRecord(anHour), + aRecord(anHour, { "cs-uri-stem": "/grammar/" }), + ]); + await untilTheScheduleFires(deployed); + + // When a run names no row count. + const run = await cli([ + "pageviews", + "--last", + "2h", + "--summaries", + deployed.summariesBucketName, + ]); + + // Then the command's own default of twenty answers and nothing is + // reported as taken. A row count decides how much of the answer is + // printed, and a deployment computing deeper does not make a bare + // command print a hundred rows. + expect(run.code).toBe(0); + expect(run.rows).toStrictEqual([ + { path: "/", views: "1" }, + { path: "/grammar/", views: "1" }, + ]); + expect(run.error).not.toContain("--limit"); + }); + it("refuses a row count somebody typed that the summary cannot reach", async () => { // Given the same deployment computing the top row alone. const deployed = await deployAnalytics({ diff --git a/src/cli/summary-covering.test.ts b/src/cli/summary-covering.test.ts index 10ce552..45af338 100644 --- a/src/cli/summary-covering.test.ts +++ b/src/cli/summary-covering.test.ts @@ -274,6 +274,43 @@ describe("reading a bucket of summaries", () => { ); }); + it("cuts to the shallowest window where two were computed differently", async () => { + // Given two hours computed to different depths, which is what a change + // to the requests prop leaves behind. Only the row count moved. + const deployed = await aBucket(); + await putSummary( + deployed, + anHourOn("2026-08-24", 7), + [{ path: "/", views: "2" }], + { question: { ...question, limit: 1 } }, + ); + await putSummary( + deployed, + anHourOn("2026-08-24", 8), + [ + { path: "/", views: "1" }, + { path: "/grammar/", views: "1" }, + ], + { question: { ...question, limit: 5 } }, + ); + + // When both are asked about with no row count typed. + const run = await cli([ + "pageviews", + "--last", + "3h", + "--summaries", + deployed.bucket, + ]); + + // Then the shallower window decides, and the two windows hold one answer + // between them rather than a disagreement. A row count is how much of a + // ranked answer is printed, and every window here reaches one row. + expect(run.code).toBe(0); + expect(run.rows).toStrictEqual([{ path: "/", views: "3" }]); + expect(run.error).toContain("Took --limit 1 from the summaries"); + }); + it("refuses a span its deployment computed two ways", async () => { // Given two hours computed either side of a change to the requests // prop. The bucket holds one narrowed hour and one unnarrowed one. diff --git a/src/cli/summary-help.ts b/src/cli/summary-help.ts index 253a5e0..058b06e 100644 --- a/src/cli/summary-help.ts +++ b/src/cli/summary-help.ts @@ -50,17 +50,24 @@ per window. Standard error says which windows were read and how old they are. --query runs the question through Athena instead. A summary answers the question it was computed with. --path, --host, ---include-bots and --param each change the answer, and a schedule cannot -compute every combination of them. RollupSummaries computes the unfiltered -form, and its \`requests\` prop is where a deployment adds a narrowed one. A -run whose filters no stored summary matches says what was stored and stops, -so nothing quietly falls back to a query nobody chose. +--include-bots, --param and --redirect-status each decide which requests were +counted, and a schedule cannot compute every combination of them. +RollupSummaries computes the unfiltered form, and its \`requests\` prop is where +a deployment adds a narrowed one. A run whose filters no stored summary +matches says what was stored and stops, so nothing quietly falls back to a +query nobody chose. -A run that names none of those options takes the ones the summaries were -computed with, and standard error says which. The deployment declared its -narrowing on RollupSummaries and this command reads that copy back. A shell -alias never has to carry a second one. An option somebody typed is still -theirs, and still refused where no stored summary matches it. +A run that names none of those five takes the ones the summaries were computed +with, and standard error says which. The deployment declared its narrowing on +RollupSummaries and this command reads that copy back. A shell alias never has +to carry a second one. An option somebody typed is still theirs, and still +refused where no stored summary matches it. + +--limit is apart from those five. A row count decides how much of a ranked +answer is printed and leaves what was counted where it was, so a deployment +computing the top hundred still answers this command's own default. A summary +holding fewer rows than that cuts the answer down and says so, and a row count +somebody typed is refused instead. A range is covered by whole UTC windows, days wherever a whole day fits and hours at the two edges. The part hours at either end have no stored window, so diff --git a/src/cli/summary-narrowings.ts b/src/cli/summary-narrowings.ts index bf021bc..9d11f60 100644 --- a/src/cli/summary-narrowings.ts +++ b/src/cli/summary-narrowings.ts @@ -1,16 +1,21 @@ -// What each narrowing option does to the question a summary records. +// What each counting option does to the question a summary records. // // The join between an option a reader types and a field of `SummaryQuestion`. // `summary-adoption.ts` walks it to take the filters a command line left out, // and to spell each one back the way it would have arrived. // +// `--limit` has no entry in that table. It sets no filter on what was counted, +// and `storedRowCount` at the foot of this file works it out from the stored +// windows instead. +// // Written out one option at a time rather than derived. The fields are named // here. A field that leaves `RollupRequest` stops this compiling, and a reader // can see what `--redirect-status` does to a question without following a // lookup. -import type { SummaryQuestion } from "../rollup-summaries.js"; -import type { NarrowingOption } from "./summary-question.js"; +import type { RollupSummary, SummaryQuestion } from "../rollup-summaries.js"; +import type { Rollup } from "../rollups.js"; +import type { CountingOption, NarrowingOption } from "./summary-question.js"; /** How one option reaches the question a summary records. */ interface Narrowing { @@ -25,7 +30,7 @@ interface Narrowing { } /** Each option, as the field it sets and the value a reader would type. */ -const narrowings: Readonly> = { +const narrowings: Readonly> = { "--host": { taken: (asked, stored) => ({ ...asked, host: stored.host }), value: (stored) => stored.host, @@ -48,15 +53,11 @@ const narrowings: Readonly> = { }), value: (stored) => stored.redirectStatuses.join(","), }, - "--limit": { - taken: (asked, stored) => ({ ...asked, limit: stored.limit }), - value: (stored) => String(stored.limit), - }, }; /** One question, with one option's value taken from a stored summary. */ export function narrowedBy( - option: NarrowingOption, + option: CountingOption, asked: SummaryQuestion, stored: SummaryQuestion, ): SummaryQuestion { @@ -71,10 +72,42 @@ export function narrowedBy( * and a reporter prints it without knowing how any option is spelled. */ export function narrowingText( - option: NarrowingOption, + option: CountingOption, stored: SummaryQuestion, ): string { const value = narrowings[option].value?.(stored); return value === undefined ? option : `${option} ${value}`; } + +/** + * The row count a run that named none answers with, where it is not the + * command's own default. + * + * The smallest of the stored windows, and nothing where every one of them + * reaches the default. A row count decides how much of a ranked answer is + * printed and leaves what was counted where it was. A deployment computing the + * top hundred still answers a bare `rainlytics pageviews` with the top twenty, + * and the stored hundred holds them. + * + * A window holding fewer rows than that is the case worth reporting. Nineteen + * rows nobody counted cannot be recovered from a stored answer holding one, + * and the honest answer is the one row with a line saying so. A run that typed + * the count is refused instead, by `refuseAnotherQuestion`. + */ +export function storedRowCount( + rollup: Rollup, + asked: SummaryQuestion, + named: ReadonlySet, + summaries: readonly RollupSummary[], +): number | undefined { + if (!rollup.isRanked || named.has("--limit")) { + return undefined; + } + + const deepest = Math.min( + ...summaries.map((summary) => summary.question.limit), + ); + + return deepest < asked.limit ? deepest : undefined; +} diff --git a/src/cli/summary-question.ts b/src/cli/summary-question.ts index dfd5f60..317360f 100644 --- a/src/cli/summary-question.ts +++ b/src/cli/summary-question.ts @@ -15,25 +15,38 @@ import type { SummaryQuestion } from "../rollup-summaries.js"; import type { Rollup, RollupRequest } from "../rollups.js"; /** - * The options that narrow a question, spelled as a reader types them. + * The options deciding which requests a question counted. * * The list every reader of a narrowing walks. `questionDifferences` compares - * these fields, `summary-adoption.ts` takes the ones a command line left out, - * and `rollup-options.ts` records which of them arrived. An option added to a - * rollup command reaches all three from here. + * these fields and `summary-adoption.ts` takes the ones a command line left + * out. An option added to a rollup command reaches both from here. * * `--last` is left out. The window is the range a summary answers over, and a * span is what a reader is choosing when they type it. */ -export const narrowingOptions = [ +export const countingOptions = [ "--host", "--path", "--include-bots", "--param", "--redirect-status", - "--limit", ] as const; +/** One option deciding which requests were counted. */ +export type CountingOption = (typeof countingOptions)[number]; + +/** + * Those, and the row count. + * + * `--limit` is apart from the five above and stays apart the whole way down. + * The five decide which requests were counted, and a run naming none of them + * takes what the summaries counted. A row count decides how much of a ranked + * answer is printed and leaves what was counted where it was. A run naming + * none of it keeps the command's own default, cut to what the stored windows + * hold. + */ +export const narrowingOptions = [...countingOptions, "--limit"] as const; + /** One option that narrows a question. */ export type NarrowingOption = (typeof narrowingOptions)[number]; diff --git a/src/cli/summary-refusals.ts b/src/cli/summary-refusals.ts index 4694170..0690f60 100644 --- a/src/cli/summary-refusals.ts +++ b/src/cli/summary-refusals.ts @@ -78,9 +78,10 @@ export function answersSomethingElse( * and an answer assembled from them would cover part of the span under a * narrowing the rest was never computed with. * - * The reader settles it by typing the one they want. Every window the run - * still disagrees with is then named by {@link answersSomethingElse}. That is - * the same conversation from the other end. + * Typing one of the two settles nothing. The windows computed the other way + * then refuse it through {@link answersSomethingElse}, which is the older half + * of the span saying the same thing from the other end. What answers is a + * shorter span or a query. */ export function computedMoreThanOneWay( rollup: Rollup, @@ -94,11 +95,11 @@ export function computedMoreThanOneWay( return new Error( `The stored ${rollup.name} summaries over that span were not all` + - ` computed the same way, and this run named nothing to settle it` + - ` with.\n${lines.join("\n")}\nA deployment that changed its requests` + - ` prop leaves both questions in the bucket. Name the one you want on` + - ` the command line, ask about a span on one side of the change, or run` + - ` it with --query.`, + ` computed the same way, and this run named no filter of its` + + ` own.\n${lines.join("\n")}\nA deployment that changed its requests` + + ` prop leaves both questions in the bucket, and typing one of them is` + + ` refused by the windows computed with the other. Ask about a span on` + + ` one side of the change, or run it with --query.`, ); }