Skip to content

feat: the picture by week and month, and All time from months kept for good (ADRs 0063, 0064) - #57

Merged
jagdish-kumawat merged 1 commit into
mainfrom
feat/2026-10-30-chart-by-week-and-month
Nov 5, 2026
Merged

jagdish-kumawat merged 1 commit into
mainfrom
feat/2026-10-30-chart-by-week-and-month

Conversation

@jagdish-kumawat

Copy link
Copy Markdown
Contributor

What

  • Weeks and months. The overview's picture can be read by days, weeks (Monday to Sunday) or
    months, in both views, with the population switch, Compare and the drill-down still right.
  • All time. The overview gains an All time period. Its cards and picture cover a website's
    whole history, by month or by year, from figures kept for every finished month.
  • No deleted days in comparisons. No comparison reaches into days that are no longer kept.

ADR 0063 covers the picture by day, week or month. ADR 0064 covers the months kept after the reports
they were counted from expire.

Why

Reports, verdicts and name confirmations expire after twelve months, so nothing on the dashboard
could look further back, and the picture was drawn only by the hour or the day.

The owner decided:

  • monthly figures only: nothing day by day outlives the twelve months;
  • All time lives on the overview;
  • a comparison that would reach into deleted days is hidden.

How

The picture by week and month (ADR 0063)

  • TimeGranularity.Week/Month. CalendarBuckets cuts and steps each bucket in the website's
    zone, and fills a series on its local dates before converting once.

    • This also fixes a bug on main: days repeated after a clock change that skips midnight
      (Santiago, Cairo, Havana, Beirut, the Azores).
  • The series endpoints accept week and month. RequestedWindow.LongestFor allows 31 days by the
    hour and 400 days otherwise.

  • Show the chart by offers:

    • Days;
    • Weeks, from four-week periods;
    • Months, from 90 days.

    A cut the period can't show moves to the shortest period that can. The choice travels in the
    address (by) and is remembered.

  • Bucket names, the table, the hover and the drill-down cover the days a bucket holds. Compare
    shifts by whole weeks or months so weekdays line up.

  • keptFrom is the first midnight in the website's zone on or after now − 12 months + 24 h. It is the
    same boundary as the engine's Retention.DetailFrom.

    • With no earlier stretch inside it, there is no change line and no Compare.
    • The chooser refuses older days, as too-old.

Months kept, and All time (ADR 0064)

  • Table. ClickHouse migration 0014_months_kept_after_their_reports.sql adds
    consolidated_months.
    • One row per website and month, as a ReplacingMergeTree(counted_at) with no TTL.
    • Each row holds page views, daily visitors, the busiest day, weekday and hour shares and the
      verdict tally. It holds the same for people, plus what the row was counted under.
  • Counting.
    • ConsolidationSqlCompiler counts with the dashboard's own fragments, so a kept month equals the
      overview of that month. CountingRules.Current is pinned by a hash of the compiled statements.
    • MonthsToCount picks only closed months whose reports are still held, oldest first.
  • The worker. ConsolidationWorker reads Dewiride:Consolidation: a 1-minute start delay, then
    every 15 minutes, 4 months a run, EventIds 5201–5207.
    • It never deletes.
    • It writes only while the website exists, under the per-website advisory lock that
      SiteDirectory.RemoveAsync and ClosedAccountPurge take.
    • Removal and closure purge the table, as they purge every website-scoped table.
  • First day. FirstDays is the one rule for a website's first day, shared by counting and
    reading. KeepingOrigin reads from the migration ledger when keeping began.
  • Reading. KeptHistory merges kept months with the months not yet kept, which are answered live
    from reports.
    • All time is never behind the overview, and works before the worker's first run.
    • A month never counted (missing) and one kept before judging finished (unjudged) are named
      apart, never drawn as zero.
  • Endpoints. GET /api/sites/{id}/all-time, /all-time/series and /all-time/traffic/series
    (KeptHistoryEndpoints; QueryVocabulary extracted from SiteEndpoints).
  • Overview, under All time:
    • cards and picture by month or year;
    • "Since {date}" under the period;
    • one sentence saying the lists cover the last 12 months;
    • months older than the kept detail can't be pressed;
    • the chooser suggests All time for older days, only where All time reaches further.

Retention, stated plainly

The retention envelope widens. Each website's monthly totals are kept until the website is removed,
or until its closed account is deleted thirty days after closing. Everything about a single visit
still expires after twelve months.

The hosted service's privacy pages and release runbook change in the private repository, in the same
release.

Releasing

  • Migration 0014 is applied as the engine starts. Back up first.
  • Going back to an older engine leaves consolidated_months rows behind for websites removed while
    it runs. The private runbook says how to find them.
  • The first runs after release count every finished month still held, four a run.
    • Until a month is counted, All time answers it live.

Checks

  • Both editions build with no warnings.
  • Suites:
    • Commercial edition: Ee (finishing locally), Ee.Integration (finishing locally).
    • Free edition: Testing 11, Domain 117, Application 504, Classification 354, Architecture 27,
      Api 19, SqlTests 303, Mail 27, Integration (finishing locally; CI runs it).
    • pnpm verify: web 1783, site 78, edition 124, tracker 82.
  • Review:
    • three adversarial review rounds for the picture and seven for the kept months, every confirmed
      finding fixed;
    • the first-day rule broken on purpose (its tests went red) and its cases swept in simulation.
  • Screens photographed in light and dark, at phone and desktop width, in every state All time can be
    in.

🤖 Generated with Claude Code

…r good (ADRs 0063, 0064)

The overview's picture could be drawn only by the hour or the day, and nothing about a website
outlived the twelve months its reports are kept. Both change here, all of it in the free product.

The picture by week and month (ADR 0063; revises 0042 and 0049, extends 0028, 0040 and 0041):
- TimeGranularity gains Week (ISO, Monday first) and Month. CalendarBuckets cuts and steps every
  bucket in the website's zone and fills a series on its local dates, converting once outside. That
  also ends days repeating after a clock change that skips midnight (Santiago, Cairo, Havana, Beirut,
  the Azores), which main got wrong.
- The series endpoints accept `week` and `month`; RequestedWindow.LongestFor allows 31 days by the
  hour and 400 otherwise.
- "Show the chart by" offers Days, Weeks from four-week periods and Months from 90 days. A cut the
  period cannot show moves to the shortest period that can. The choice travels in the address (`by`)
  and is remembered. Bucket names, the table, the hover and the drill-down cover the days a bucket
  holds, and the earlier stretch shifts by whole weeks or months so weekdays line up.
- No comparison reaches into days no longer kept, and no chosen stretch starts before them. keptFrom
  is the first midnight in the website's zone on or after now − 12 months + 24 hours, the same
  boundary as the engine's Retention.DetailFrom.

Months kept, and All time (ADR 0064; extends 0013, revises 0042, 0054 and 0063, follows 0025, 0041
and 0062):
- ClickHouse migration 0014 adds `consolidated_months`: one row per website and month,
  ReplacingMergeTree(counted_at), no TTL. This widens the retention envelope, which the ADR states in
  full. A row holds the month's page views, daily visitors, busiest day, weekday and hour shares and
  verdict tally, the same figures for people, and what it was counted under.
- ConsolidationSqlCompiler counts a month with the dashboard's own fragments; CountingRules is pinned
  by a hash of the compiled statements. MonthsToCount picks closed months still held, oldest first.
  ConsolidationWorker (Dewiride:Consolidation: every 15 minutes, 4 months a run, EventIds 5201–5207)
  never deletes. It writes only while the website exists, under the advisory lock that a removal and
  a closed account's purge take. Removal and closure purge the table.
- FirstDays is the one rule for a website's first day, shared by counting and reading. KeepingOrigin
  reads from the migration ledger when keeping began.
- KeptHistory merges kept months with the months not yet kept, which are answered live from reports,
  so All time is never behind the overview. A month never counted (`missing`) and one kept before
  judging finished (`unjudged`) are named apart and never drawn as zero.
- GET /api/sites/{id}/all-time, /all-time/series and /all-time/traffic/series come from
  KeptHistoryEndpoints, with QueryVocabulary extracted from SiteEndpoints. TenantScope carries when
  the website was added.
- The overview offers All time:
  - the cards and the picture cover the whole record, by month or by year;
  - "Since {date}" shows under the period;
  - one sentence says the lists cover the last 12 months;
  - the chooser points to All time for older days, only where All time reaches further.
- README; return lines in ADRs 0013, 0025, 0028, 0040, 0041, 0042, 0049, 0054 and 0062.

Tests:
- SqlTests goldens for weeks, months, kept months and the four counting statements.
- Application: FirstDays, KeptHistory, MonthsToCount, MonthlyConsolidation, DaysMeasured, Retention,
  ZoneDays, JudgingReach and BusiestDay.
- Api: ConsolidationWorker.
- Integration: MonthConsolidation, AllTimeRead, and series across clock changes in Kolkata,
  Kathmandu and London.
- The first-day rule was broken on purpose and its tests went red. Its cases were also swept in
  simulation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jagdish-kumawat
jagdish-kumawat merged commit 503b56e into main Nov 5, 2026
7 checks passed
@jagdish-kumawat
jagdish-kumawat deleted the feat/2026-10-30-chart-by-week-and-month branch November 5, 2026 04:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant