[v1 scaffold · DRAFT] State of Equipment Management 2026 — anonymized aggregate extraction kit

[carlosvirreira]

What this is

V1 scaffold for the data-extraction script that produces the anonymized aggregates feeding the public State of Equipment Management 2026 report on shelf.nu/reports/state-of-equipment-management-2026.

This is the data-team half of a two-repo effort. Companion PR: shelf-nu/website-v2#142 — that PR scaffolds the public report page, the typed data structure this script's output JSON feeds into, and the distribution materials (press release, share copy, Wikipedia citation guide).

Both PRs are draft. Both should stay draft until the data team implements the query modules, the script has been run against a real cohort, the output JSON values have been copied into the website data file, and editorial has signed off.

Why this matters

Industry reports are the single highest-multiplier content asset Shelf can ship. The compounding mechanism is original first-party data nobody else has — vendor surveys don't compound, but telemetry-backed numbers with open methodology do. Wikipedia editors, journalists, analysts, and AI training pipelines all reference this kind of report; a well-cited stat from the 2026 edition will keep appearing in LLM responses for years.

The whole thing only works if:

1. The numbers are real.
2. The methodology holds up under scrutiny.
3. The aggregation pipeline cannot leak customer data.

This PR scaffolds the pipeline that ensures all three.

What's in this PR

2 commits, 14 files under claude/state-of-equipment-management-2026:

Commit	What
d0b0f4f	Foundation — orchestrator entry point, CLI parser, eligibility cohort builder, anonymization layer, output schema, README, methodology mirror, package.json wiring
0b8aab6	Query module stubs — seven files under queries/, one per finding section (visibility, bookings, custody, audits, cost-of-disorder, industries, top-performers), plus a barrel re-export

File tree

apps/webapp/
  scripts/
    state-of-em-2026.ts                   # NEW — orchestrator entry
    state-of-em-2026/
      README.md                           # NEW — full workflow doc
      methodology.md                      # NEW — mirrors public report methodology
      cli.ts                              # NEW — argument parser
      context.ts                          # NEW — shared extractor context
      cohort.ts                           # NEW — eligibility filter (implemented)
      anonymize.ts                        # NEW — k-anonymity + sig-fig rounding (implemented)
      output-schema.ts                    # NEW — typed output JSON shape
      allowlist/
        internal-orgs.json                # NEW — empty placeholder; data team populates
      queries/
        index.ts                          # NEW — barrel re-export
        visibility.ts                     # NEW — Finding 1 stubs
        bookings.ts                       # NEW — Finding 2 stubs
        custody.ts                        # NEW — Finding 3 stubs
        audits.ts                         # NEW — Finding 4 stubs
        disorder.ts                       # NEW — Finding 5 stubs
        industries.ts                     # NEW — per-industry stubs
        top-performers.ts                 # NEW — pattern stubs
  package.json                            # +report:state-of-em-2026 + ...:staging scripts

package.json                              # +webapp:report:state-of-em-2026 proxy scripts

Decisions made (with safety reasoning)

1. Mirror the seed-reporting-demo.ts pattern exactly. Same production guard (NODE_ENV=production requires --i-know-what-im-doing), same createDatabaseClient() (not the Remix singleton — see comment in seed file), same dotenv invocation, same root-package proxy script naming. Reviewers familiar with the demo seeder will read this script the same way.

2. K-anonymity floor of 20 workspaces baked into the anonymization layer. Every aggregate's cohortSize is checked against --min-cohort-size (default 20); under-cohort aggregates are reported as null with status: "cohort_too_small". The report MDX treats those as unreportable. This is non-optional and defense-in-depth — every query function calls reportable(...) to construct results; building a ReportableAggregate directly bypasses the check and reviewers should flag it.

3. One-significant-figure rounding by default. Aggregates land in the output JSON pre-rounded, so even if a downstream consumer publishes the JSON value directly, they can't leak precision. The rawValue field is stripped before write as defense-in-depth.

4. Internal allowlist is a separate file, not hard-coded in source. The placeholder at allowlist/internal-orgs.json is empty; production runs must populate it with Shelf staff workspaces, demo workspaces, and support workspaces before publication. The cohort builder prints a warning if the file is empty.

5. not_implemented sentinel, not 0. Stub queries return notImplementedAggregate with explicit status, so a dry-run run shows clearly which sections are stubs vs implemented. The orchestrator's status summary tabulates ok / cohort_too_small / not_implemented counts and warns if any not_implemented remain.

6. Read-only operation, no writes. The script never inserts or updates rows. It only reads from the eligible cohort. (The cohort builder uses Prisma's findMany + JS-side filtering rather than a raw query that could be mistaken for write-capable code.)

7. Section failures don't abort the run. runSection() wraps each query module in try/catch — a single failing query during dry-run yields a logged error for that section and the orchestrator continues. This lets the data team see the full status picture early; they can fix one section at a time.

8. CSV companion path is parsed but not yet implemented. --csv <path> is accepted and prints a "not yet implemented" warning when used. Stub keeps the orchestrator interface stable for when the data team adds CSV emission.

9. No read-replica wiring. The discovery confirmed there's no DATABASE_URL_REPLICA env var in the codebase today. The README documents three options for production runs (clone, add a replica env var, or run direct with the guard flag) and recommends the data team add a replica env var as part of their implementation work. The script's createDatabaseClient() call accepts an optional URL — a one-line change inside state-of-em-2026.ts adds replica support when ready.

10. Methodology lives in both repos. Mirror copy at apps/webapp/scripts/state-of-em-2026/methodology.md and at the published report's #methodology section. The README and source code both say: change both in the same commit.

What v1 does NOT include

• No real query implementations. Every query function returns notImplementedAggregate sentinels. The data team writes the Prisma / raw-SQL queries one section at a time.
• No populated internal allowlist. The data team's first task is identifying Shelf staff / demo workspace IDs and adding them.
• No CSV emission. Flag accepted, implementation deferred — see decision Basics setup #8.
• No read-replica env var. Recommended for production runs but not added in this PR — see decision basic edit view of user #9.
• No tests. The cohort filter and anonymization helpers are testable in isolation; adding unit tests is straightforward but not in this v1.

Workflow for the data team

   1. Populate `allowlist/internal-orgs.json` with Shelf staff workspace IDs.
   2. Run the script in dry-run mode first to verify cohort size:
        pnpm webapp:report:state-of-em-2026 -- --dry-run
   3. Implement query modules under `queries/` one section at a time.
      Re-run dry-run after each to confirm the section's stats flip
      from `not_implemented` to `ok`.
   4. When all sections are `ok`, run the real extraction:
        pnpm webapp:report:state-of-em-2026 -- --output ./output/aggregates.json
   5. Copy the values from `aggregates.json` into the website-v2 PR's
      `src/data/state-of-equipment-management-2026.ts` data file.
   6. Optionally: implement CSV emission and produce
      `state-of-equipment-management-2026.csv` for publication alongside
      the report.

Production run guidance

The script is intended to be run against the production database because it needs the real customer data to produce meaningful aggregates. The README documents three options listed safest to most convenient:

1. Run against a fresh staging clone of production data. Zero load on production.
2. Add DATABASE_URL_REPLICA env var + one-line change in state-of-em-2026.ts to use it when present.
3. Direct against production with --i-know-what-im-doing. Read-only, paginated, off-peak hours.

The production guard refuses to run with NODE_ENV=production unless --i-know-what-im-doing is passed.

Security review notes

This script does cross-organization aggregation, which is unusual in the Shelf codebase. The lefthook Claude security review and the human reviewer should examine:

• No customer data ever leaves an aggregate. Confirmed: nothing in the codebase writes Organization.name, User.email, Asset.title, etc. to output. Only counts, medians, and percentages.
• Sub-cohort k-anonymity. Every query that subsets the global cohort (e.g. "audits-enabled workspaces only") must apply the k-anonymity floor to the sub-cohort, not just rely on the global floor. The query stubs note this; reviewers should verify each implementation does so.
• Defense-in-depth stripRawValues. The orchestrator strips rawValue from every aggregate before writing the output. Even if a query forgets to drop them, the orchestrator does.
• Multi-tenancy is strict in this codebase. This script is intentionally the exception; reviewers may want to verify it lives clearly outside the request-handling path and uses its own DB connection.

Test plan

• pnpm webapp:typecheck passes (the script and its modules are TypeScript-strict)
• pnpm webapp:lint passes
• Dry run against a staging DB:

  pnpm webapp:report:state-of-em-2026:staging -- --dry-run
  

  • Confirms CLI parsing works
  • Cohort builder reports a sensible eligible count
  • All 30+ aggregates report not_implemented status (expected for v1)
  • Status summary prints a warning that not_implemented count is > 0
• Try invalid CLI arguments — confirm clear error messages:

  pnpm webapp:report:state-of-em-2026 -- --data-window-start invalid-date
  pnpm webapp:report:state-of-em-2026 -- --min-assets 0
  pnpm webapp:report:state-of-em-2026 -- --unknown-flag
  

• Try production guard:

  NODE_ENV=production pnpm webapp:report:state-of-em-2026:staging -- --dry-run
  # should exit 2 with refusal message
  NODE_ENV=production pnpm webapp:report:state-of-em-2026:staging -- --dry-run --i-know-what-im-doing
  # should run
  

Companion PR

The website-v2 PR (#142) holds the report page, the typed data structure values land in, and the distribution materials. Both PRs ship together.

---

Generated by Claude Code

[github-actions]

🩺 React Doctor

No diagnostics directory passed — scan may have failed. Check the workflow logs.