feat: let a deployment hold no personal data

docs/log-bucket/README.md

@@ -59,8 +59,9 @@ Shortening it discards history that cannot be recovered afterwards.
 
 ### It is also how long the addresses are kept
 
-The delivered field set includes `c-ip`. What expires here is therefore a record of people as well
-as of requests. Rainlytics counts unique visitors by hashing the viewer's address and their user
+The delivered field set includes `c-ip` by default. What expires here is therefore a record of
+people as well as of requests, until a site delivers
+[the field set without it](../visitors/#running-without-a-visitor-count). Rainlytics counts unique visitors by hashing the viewer's address and their user
 agent under a salt that rotates daily, and a scheduled rollup computes that hash from the address
 the object already carries. The [log delivery](../log-delivery/) page covers the field set, and
 [Counting visitors](../visitors/) covers the hash.

docs/log-delivery/README.md

@@ -93,18 +93,23 @@ last exactly as long as the log objects do. On the defaults that is 365 days, pl
 superseded version survives, and the [log bucket](../log-bucket/) page has both numbers and how to
 change them.
 
-A site that would rather not keep addresses can deliver everything else:
+A site that would rather not keep addresses delivers the named set without it:
 
 ```typescript
-import { deliveredLogFieldNames } from "@kensio/rainlytics";
+import { logFieldNamesWithoutAddress } from "@kensio/rainlytics";
 
 new CloudFrontLogDelivery(this, "RainlyticsDelivery", {
   distributionId: "E1EXAMPLE1234",
   logBucket: logs.bucket,
-  fields: deliveredLogFieldNames.filter((field) => field !== "c-ip"),
+  fields: logFieldNamesWithoutAddress,
 });
 ```
 
+That is the only line a site changes. The [log table](../log-table/) declares the columns the
+delivery asks for and gets no `c_ip`. The [summary schedule](../summary-schedule/) reads the table,
+computes the same five questions with the visitor count off, and is granted no permission to read
+the salt. The SSM parameter a default deployment needs before its first run never has to exist.
+
 Pageviews, referrers, devices, status codes and geography all carry on. The visitor count is the one
 thing that stops being computable, and no later job can recover it for the days the field was
 absent. Geography survives because CloudFront resolves `c-country` at the edge from an address the

docs/log-table/README.md

@@ -137,12 +137,16 @@ and none of that has been tested here.
 
 ## `c_ip` is the viewer's address
 
-The table has a column for it because the delivery asks for one. Rainlytics counts unique visitors
-as a daily-rotating hash of the address and the user agent, and the rollup computing that hash reads
-this column. The [log delivery](../log-delivery/) page says what the field set holds, and the [log
-bucket](../log-bucket/) page says for how long.
+The table has a column for it where the delivery asks for one, which the default field set does.
+Rainlytics counts unique visitors as a daily-rotating hash of the address and the user agent, and
+the rollup computing that hash reads this column. The [log delivery](../log-delivery/) page says
+what the field set holds, and the [log bucket](../log-bucket/) page says for how long.
 
-Three things follow for anything querying this table.
+A delivery configured with `logFieldNamesWithoutAddress` produces a table with no such column. A
+[summary schedule](../summary-schedule/) over that table computes no visitor count, and every other
+column and question is the same.
+
+Three things follow for anything querying a table that has the column.
 
 Records delivered before the field was added carry no address. Athena answers `null` for them and
 reports success, and a visitor count over those days comes back low.

docs/visitors/README.md

@@ -136,8 +136,8 @@ A run that meets no parameter fails and says so, naming the parameter and printi
 
 `pageviews` alone, and it is one of the five questions a deployment gets when it passes no `rollups`
 of its own. A default deployment therefore reads the salt parameter, and the secret has to be there
-before its first run. A deployment that wants none passes `rollups` without a question that counts
-visitors, and needs no parameter.
+before its first run. [Running without a visitor count](#running-without-a-visitor-count) has the
+deployment that reads no parameter at all.
 
 A rollup says it counts with `countsVisitors`:
 
@@ -161,11 +161,51 @@ recomputing two windows, come to 250 queries a day and about 38 cents a month. T
 `pageviews` adds 50 of those, which is about 8 cents. The
 [summary schedule](../summary-schedule/#what-it-costs) page has the arithmetic.
 
+## Running without a visitor count
+
+A site that delivers no viewer address counts no visitors, and nothing else about it changes.
+
+```typescript
+import { logFieldNamesWithoutAddress } from "@kensio/rainlytics";
+
+new CloudFrontLogDelivery(this, "RainlyticsDelivery", {
+  distributionId: "E1EXAMPLE1234",
+  logBucket: logs.bucket,
+  fields: logFieldNamesWithoutAddress,
+});
+```
+
+That is the only line a site changes. The [log table](../log-table/) describes what the delivery
+writes and the [summary schedule](../summary-schedule/) reads the table. Both follow. The schedule
+computes the same five questions with the count off, needs no salt parameter, and is granted no
+`ssm:GetParameter`. Summaries carry no `visitors` field, which a reader tells apart from a count of
+zero.
+
+A deployment naming its own questions says so per question:
+
+```typescript
+import { pageviews, referrers, withoutVisitorCount } from "@kensio/rainlytics";
+
+new RollupSummaries(this, "RainlyticsSummaries", {
+  table,
+  workgroup,
+  rollups: [withoutVisitorCount(pageviews), referrers],
+});
+```
+
+A question that counts visitors over a table with no address is refused at synthesis, naming the
+question. Left alone it would run once an hour against a column the table has never heard of.
+
+The choice sits on the delivery because the delivery is what writes the raw store. Turning the
+address off later leaves every address already written where it is, until the [log
+bucket](../log-bucket/) expiry reaches it.
+
 ## Where the addresses are
 
 The raw log bucket holds viewer addresses in the clear, for as long as it holds anything. That is
 the price [#53](https://github.com/KensioSoftware/rainlytics/issues/53) paid for a visitor count,
-and the [log bucket](../log-bucket/) page has the expiry that decides how long it lasts.
+and the [log bucket](../log-bucket/) page has the expiry that decides how long it lasts. A site
+running the field set above has none of them to keep.
 
 The salt protects the identifier and never the source. Anyone who can read the log bucket has the
 addresses themselves, at better resolution than any digest would give them.

src/cdk/computed-questions.ts

@@ -0,0 +1,81 @@
+// Which questions a deployment computes, given what its table carries.
+//
+// Apart from `summary-configuration.ts` because it answers one question that
+// nothing else in the settling has to think about. Every other choice a
+// deployment makes is between what it was told and a default. This one is
+// between what it was told and what the delivered field set can support.
+
+import {
+  countsVisitorsFrom,
+  missingVisitorCountFields,
+} from "../log-fields.js";
+import { type Rollup, withoutVisitorCount } from "../rollups.js";
+import { rollups } from "../rollup-questions.js";
+import type { RollupSummariesProps } from "./summary-configuration.js";
+
+/**
+ * The questions this deployment computes, against the table it reads.
+ *
+ * A visitor count is the one question needing a field the delivery can be
+ * configured without. The shipped questions therefore follow the table. A
+ * deployment over a table carrying no viewer address gets the same five
+ * questions with the count off, and needs no salt for them.
+ *
+ * A caller that asked for the count by name gets an error instead. Dropping
+ * what it asked for would leave a deployment computing something other than
+ * what its code says.
+ *
+ * @throws {Error} where a named question counts visitors the table cannot
+ *   identify.
+ */
+export function computedQuestions(
+  props: RollupSummariesProps,
+): readonly Rollup[] {
+  if (countsVisitorsFrom(props.table.fields)) {
+    return props.rollups ?? rollups;
+  }
+
+  if (props.rollups === undefined) {
+    return rollups.map((rollup) => withoutVisitorCount(rollup));
+  }
+
+  assertNothingCountsVisitors(props.rollups, props.table.fields);
+
+  return props.rollups;
+}
+
+/**
+ * Refuses questions this table cannot answer.
+ *
+ * At synthesis, where somebody can still read it. The query would otherwise
+ * name a column the table has never heard of, and fail once an hour in a
+ * bucket nobody is watching.
+ *
+ * The message names the fields actually absent. A delivery keeping the
+ * address and dropping the user agent counts nobody, and being told to add
+ * the address would send its author looking at a field already there.
+ *
+ * @throws {Error} naming the questions, what is missing, and how to turn
+ *   their count off.
+ */
+function assertNothingCountsVisitors(
+  asked: readonly Rollup[],
+  fields: readonly string[],
+): void {
+  const counting = asked.filter((rollup) => rollup.countsVisitors === true);
+
+  if (counting.length === 0) {
+    return;
+  }
+
+  const named = counting.map((rollup) => rollup.name).join(", ");
+  const missing = missingVisitorCountFields(fields).join(" and ");
+
+  throw new Error(
+    `${named} counts visitors, and this deployment's delivery leaves out` +
+      ` ${missing}. A visitor is a hash of the viewer's address and their` +
+      ` user agent, and both have to be delivered for one to be counted.` +
+      ` Either add ${missing} to the delivered field set, or wrap the` +
+      ` question in withoutVisitorCount.`,
+  );
+}

src/cdk/log-table.test.ts

@@ -10,7 +10,10 @@ import { describe, expect, it } from "vitest";
 import { deployStacks, simStartedAt } from "#test/simulated-deployment.js";
 
 import { defaultLogDataset, qualifiedTableName } from "../dataset.js";
-import { deliveredLogColumnNames } from "../log-fields.js";
+import {
+  deliveredLogColumnNames,
+  logFieldNamesWithoutAddress,
+} from "../log-fields.js";
 import {
   partitionKeyNames,
   partitionLocationTemplate,
@@ -401,6 +404,22 @@ describe("the Glue table over delivered logs", () => {
     ]);
   });
 
+  it("describes no address where the delivery asked for none", async () => {
+    // Given a site delivering the field set that holds no personal data.
+    const table = catalogTable(
+      await deployTable({
+        delivery: { fields: [...logFieldNamesWithoutAddress] },
+      }),
+    );
+
+    // Then the table has no column an address could be read out of, while
+    // every other question still has the column it groups by.
+    const columns = table.columns.map((column) => column.Name);
+    expect(columns).not.toContain("c_ip");
+    expect(columns).toContain("cs_uri_stem");
+    expect(columns).toContain("cs_user_agent");
+  });
+
   describe("what it refuses to build", () => {
     /** A stack holding a log bucket, a delivery and whatever a case adds. */
     const synthesise = (

src/cdk/log-table.ts

@@ -115,6 +115,16 @@ export class LogTable extends Construct {
    */
   readonly logBucket: LogDeliveryBucket;
 
+  /**
+   * The fields the table describes, as the deliveries agreed on them.
+   *
+   * Here so that what reads the table can see what is in it. `RollupSummaries`
+   * asks whether the viewer's address is among these before it schedules a
+   * visitor count, because a query naming a column the table has no idea about
+   * fails once an hour in a bucket nobody is watching.
+   */
+  readonly fields: readonly string[];
+
   constructor(scope: Construct, id: string, props: LogTableProps) {
     super(scope, id);
 
@@ -134,6 +144,8 @@ export class LogTable extends Construct {
     const fields = deliveredLogFieldsNamed(delivery.fields);
     const format = logTableFormat(delivery.outputFormat, fields);
 
+    this.fields = delivery.fields;
+
     this.database = new CfnDatabase(this, "Database", {
       catalogId,
       databaseInput: {

src/cdk/rollup-summaries.test.ts

@@ -22,11 +22,16 @@ import { describe, expect, it } from "vitest";
 import { deployStacks, simStartedAt } from "#test/simulated-deployment.js";
 
 import { summaryEnvironment } from "../functions/summary-deployment.js";
+import { logFieldNamesWithoutAddress } from "../log-fields.js";
 import { partitionPrefix } from "../partitions.js";
 import { pageviews } from "../rollup-questions.js";
 import type { RollupSummary } from "../rollup-summaries.js";
 import { summarySchemaVersion } from "../rollup-summaries.js";
-import { defaultRedirectStatuses, windowPlaceholder } from "../rollups.js";
+import {
+  defaultRedirectStatuses,
+  windowPlaceholder,
+  withoutVisitorCount,
+} from "../rollups.js";
 import {
   defaultVisitorSaltParameter,
   visitorSaltPlaceholder,
@@ -51,6 +56,13 @@ describe("computing rollup summaries on a schedule", () => {
   const deployAnalytics = async (
     over: Partial<RollupSummariesProps> = {},
     inStack: (stack: Stack) => Partial<RollupSummariesProps> = () => ({}),
+    site: {
+      /** What the delivery asks CloudFront for, defaulting to the whole set. */
+      readonly fields?: readonly string[];
+
+      /** Whether the account holds a visitor salt at all. */
+      readonly salt?: boolean;
+    } = {},
   ) => {
     const logBucketName = `rainlytics-logs-${faker.string.uuid()}`;
     const summariesBucketName = `rainlytics-summaries-${faker.string.uuid()}`;
@@ -72,6 +84,7 @@ describe("computing rollup summaries on a schedule", () => {
         const delivery = new CloudFrontLogDelivery(stack, "Delivery", {
           distributionId: distribution.distributionId,
           logBucket: logs.bucket,
+          ...(site.fields === undefined ? {} : { fields: site.fields }),
         });
         const table = new LogTable(stack, "RainlyticsTable", {
           deliveries: [delivery],
@@ -97,18 +110,22 @@ describe("computing rollup summaries on a schedule", () => {
 
     // The salt secret, put where a site's operator puts it. Nothing in the
     // stack creates it, because CloudFormation writes no SecureString and a
-    // secret in a template is not one. `docs/visitors/` has the command.
-    await simAws
-      .region("us-east-1")
-      .account()
-      .ssm()
-      .putParameter({
-        input: {
-          Name: defaultVisitorSaltParameter,
-          Type: "SecureString",
-          Value: faker.string.hexadecimal({ length: 64, prefix: "" }),
-        },
-      });
+    // secret in a template is not one. `docs/visitors/` has the command. A
+    // case about a deployment counting no visitors leaves it out, which is
+    // the account a site running without one actually has.
+    if (site.salt !== false) {
+      await simAws
+        .region("us-east-1")
+        .account()
+        .ssm()
+        .putParameter({
+          input: {
+            Name: defaultVisitorSaltParameter,
+            Type: "SecureString",
+            Value: faker.string.hexadecimal({ length: 64, prefix: "" }),
+          },
+        });
+    }
 
     return {
       simAws,
@@ -448,6 +465,29 @@ describe("computing rollup summaries on a schedule", () => {
     await expect(summaryAt(deployed, closedHourKey)).resolves.toBeUndefined();
   });
 
+  it("summarises an hour with no salt where the address is undelivered", async () => {
+    // Given a site delivering the field set that holds no personal data, in
+    // an account where nobody has ever created a visitor salt.
+    const deployed = await deployAnalytics(
+      { rollups: [withoutVisitorCount(pageviews)] },
+      () => ({}),
+      { fields: logFieldNamesWithoutAddress, salt: false },
+    );
+    const { "c-ip": _address, ...asDelivered } = aRecord(theClosedHour);
+    await putDelivered(deployed, theClosedHour, [asDelivered, asDelivered]);
+
+    // When the schedule fires.
+    await deployed.simAws.clock().advanceBy({ minutes: 16 });
+
+    // Then the hour is summarised as usual and carries no visitor count. A
+    // reader sees the field absent rather than a zero, and the run needed no
+    // parameter to get there.
+    const summary = await summaryAt(deployed, closedHourKey);
+
+    expect(summary?.rows).toStrictEqual([{ path: "/", views: "2" }]);
+    expect(summary?.visitors).toBeUndefined();
+  });
+
   it("hands the visitor count to the schedule without the salt", async () => {
     // Given a deployment of the question as Rainlytics ships it, which counts
     // visitors.

src/cdk/rollup-summaries.ts

@@ -68,6 +68,7 @@ export class RollupSummaries extends Construct {
       workgroup: props.workgroup,
       bucket: this.bucket,
       windows: settled.windows,
+      countsVisitors: settled.countsVisitors,
       ...(props.visitorSaltParameter === undefined
         ? {}
         : { visitorSaltParameter: props.visitorSaltParameter }),