Repository navigation
feat: the picture by week and month, and All time from months kept for good (ADRs 0063, 0064) - #57
Merged
Conversation
…r good (ADRs 0063, 0064)
The overview's picture could be drawn only by the hour or the day, and nothing about a website
outlived the twelve months its reports are kept. Both change here, all of it in the free product.
The picture by week and month (ADR 0063; revises 0042 and 0049, extends 0028, 0040 and 0041):
- TimeGranularity gains Week (ISO, Monday first) and Month. CalendarBuckets cuts and steps every
bucket in the website's zone and fills a series on its local dates, converting once outside. That
also ends days repeating after a clock change that skips midnight (Santiago, Cairo, Havana, Beirut,
the Azores), which main got wrong.
- The series endpoints accept `week` and `month`; RequestedWindow.LongestFor allows 31 days by the
hour and 400 otherwise.
- "Show the chart by" offers Days, Weeks from four-week periods and Months from 90 days. A cut the
period cannot show moves to the shortest period that can. The choice travels in the address (`by`)
and is remembered. Bucket names, the table, the hover and the drill-down cover the days a bucket
holds, and the earlier stretch shifts by whole weeks or months so weekdays line up.
- No comparison reaches into days no longer kept, and no chosen stretch starts before them. keptFrom
is the first midnight in the website's zone on or after now − 12 months + 24 hours, the same
boundary as the engine's Retention.DetailFrom.
Months kept, and All time (ADR 0064; extends 0013, revises 0042, 0054 and 0063, follows 0025, 0041
and 0062):
- ClickHouse migration 0014 adds `consolidated_months`: one row per website and month,
ReplacingMergeTree(counted_at), no TTL. This widens the retention envelope, which the ADR states in
full. A row holds the month's page views, daily visitors, busiest day, weekday and hour shares and
verdict tally, the same figures for people, and what it was counted under.
- ConsolidationSqlCompiler counts a month with the dashboard's own fragments; CountingRules is pinned
by a hash of the compiled statements. MonthsToCount picks closed months still held, oldest first.
ConsolidationWorker (Dewiride:Consolidation: every 15 minutes, 4 months a run, EventIds 5201–5207)
never deletes. It writes only while the website exists, under the advisory lock that a removal and
a closed account's purge take. Removal and closure purge the table.
- FirstDays is the one rule for a website's first day, shared by counting and reading. KeepingOrigin
reads from the migration ledger when keeping began.
- KeptHistory merges kept months with the months not yet kept, which are answered live from reports,
so All time is never behind the overview. A month never counted (`missing`) and one kept before
judging finished (`unjudged`) are named apart and never drawn as zero.
- GET /api/sites/{id}/all-time, /all-time/series and /all-time/traffic/series come from
KeptHistoryEndpoints, with QueryVocabulary extracted from SiteEndpoints. TenantScope carries when
the website was added.
- The overview offers All time:
- the cards and the picture cover the whole record, by month or by year;
- "Since {date}" shows under the period;
- one sentence says the lists cover the last 12 months;
- the chooser points to All time for older days, only where All time reaches further.
- README; return lines in ADRs 0013, 0025, 0028, 0040, 0041, 0042, 0049, 0054 and 0062.
Tests:
- SqlTests goldens for weeks, months, kept months and the four counting statements.
- Application: FirstDays, KeptHistory, MonthsToCount, MonthlyConsolidation, DaysMeasured, Retention,
ZoneDays, JudgingReach and BusiestDay.
- Api: ConsolidationWorker.
- Integration: MonthConsolidation, AllTimeRead, and series across clock changes in Kolkata,
Kathmandu and London.
- The first-day rule was broken on purpose and its tests went red. Its cases were also swept in
simulation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
months, in both views, with the population switch, Compare and the drill-down still right.
whole history, by month or by year, from figures kept for every finished month.
ADR 0063 covers the picture by day, week or month. ADR 0064 covers the months kept after the reports
they were counted from expire.
Why
Reports, verdicts and name confirmations expire after twelve months, so nothing on the dashboard
could look further back, and the picture was drawn only by the hour or the day.
The owner decided:
How
The picture by week and month (ADR 0063)
TimeGranularity.Week/Month.CalendarBucketscuts and steps each bucket in the website'szone, and fills a series on its local dates before converting once.
main: days repeated after a clock change that skips midnight(Santiago, Cairo, Havana, Beirut, the Azores).
The series endpoints accept
weekandmonth.RequestedWindow.LongestForallows 31 days by thehour and 400 days otherwise.
Show the chart by offers:
A cut the period can't show moves to the shortest period that can. The choice travels in the
address (
by) and is remembered.Bucket names, the table, the hover and the drill-down cover the days a bucket holds. Compare
shifts by whole weeks or months so weekdays line up.
keptFromis the first midnight in the website's zone on or after now − 12 months + 24 h. It is thesame boundary as the engine's
Retention.DetailFrom.too-old.Months kept, and All time (ADR 0064)
0014_months_kept_after_their_reports.sqladdsconsolidated_months.ReplacingMergeTree(counted_at)with no TTL.verdict tally. It holds the same for people, plus what the row was counted under.
ConsolidationSqlCompilercounts with the dashboard's own fragments, so a kept month equals theoverview of that month.
CountingRules.Currentis pinned by a hash of the compiled statements.MonthsToCountpicks only closed months whose reports are still held, oldest first.ConsolidationWorkerreadsDewiride:Consolidation: a 1-minute start delay, thenevery 15 minutes, 4 months a run, EventIds 5201–5207.
SiteDirectory.RemoveAsyncandClosedAccountPurgetake.FirstDaysis the one rule for a website's first day, shared by counting andreading.
KeepingOriginreads from the migration ledger when keeping began.KeptHistorymerges kept months with the months not yet kept, which are answered livefrom reports.
missing) and one kept before judging finished (unjudged) are namedapart, never drawn as zero.
GET /api/sites/{id}/all-time,/all-time/seriesand/all-time/traffic/series(
KeptHistoryEndpoints;QueryVocabularyextracted fromSiteEndpoints).Retention, stated plainly
The retention envelope widens. Each website's monthly totals are kept until the website is removed,
or until its closed account is deleted thirty days after closing. Everything about a single visit
still expires after twelve months.
The hosted service's privacy pages and release runbook change in the private repository, in the same
release.
Releasing
consolidated_monthsrows behind for websites removed whileit runs. The private runbook says how to find them.
Checks
Api 19, SqlTests 303, Mail 27, Integration (finishing locally; CI runs it).
pnpm verify: web 1783, site 78, edition 124, tracker 82.finding fixed;
in.
🤖 Generated with Claude Code