feat(analytics): add dashboard model filter#861
Conversation
roborev: Combined Review (
|
|
Applied the chunking fix in I left the mixed-model behavior session-scoped on purpose. The issue asks for filtering the main dashboard by model, and specifically calls out seeing model usage in sessions. This PR keeps the dashboard scoped to sessions that contain at least one matching model, which is the contract already described in the PR body. That keeps session-level cards, projects, tools, skills, and top sessions on one consistent population instead of partially rewriting some of them into message-level metrics. A message-scoped model-usage dashboard still seems useful, I just see that as a separate slice from this session dashboard filter. |
roborev: Combined Review (
|
|
The trends paths now clear I also added mixed-model coverage for both backends, plus a PostgreSQL query-shape test that keeps the model predicate on the outer message rows. |
roborev: Combined Review (
|
|
I also added a focused frontend test for that request path, and |
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
|
Fixed on 8a23705. Session-shape now derives length buckets and autonomy from the same filtered per-session message stats that the model-scoped analytics views already use in internal/db/analytics.go, internal/postgres/analytics.go, and internal/duckdb/analytics_usage.go, so mixed-model sessions stop leaking off-model message volume and off-model user turns into the selected model's shape view. Velocity complexity now uses those filtered message counts as well, which keeps model-filtered sessions in the correct 1-15 or 16-60 bucket instead of classifying them by the full session totals. I also added mixed-model regressions for SQLite and DuckDB, plus matching PostgreSQL pgtests for the same shape and complexity cases. |
roborev: Combined Review (
|
roborev: Combined Review (
|
|
I traced the last roborev report back to the model-scoped message loaders rather than any single panel. The shared filtered-message helpers, the velocity loaders, and the trend scans were all treating This rework keeps the model filter anchored on the selected assistant turn, but it now buffers empty-model user rows and attributes them to that assistant exchange when the reply matches the active model. I’m applying that rule in SQLite, PostgreSQL, and DuckDB, and I switched the existing mixed-model regressions over to empty-model user rows so the tests exercise the real parser shape that roborev flagged. |
roborev: Combined Review (
|
|
Follow-up pushed in |
roborev: Combined Review (
|
|
Applied the tool analytics fix. Model-scoped tool counts now resolve each call against its own message timestamp, with session fallback only when the tool row has no message timestamp, so |
roborev: Combined Review (
|
|
Looking at this. Trying to figure out why this PR is so big so will see if there is some refactoring that can be done to make the change simpler |
|
Tests are more than half the diff. Most of the rest comes from three backends (SQLite/PostgreSQL/DuckDB) that don't share a query abstraction, so the model filter had to be threaded through each one independently. The rest grew from iterative edge-case fixes during review. I can split future work like this into stacked PRs if that's easier to review. A shared analytics query layer would also cut this kind of cross-backend feature down significantly; that felt like a bigger refactor to take on inside this PR. |
|
No worries, I get it! I have agents looking to see what refactoring could be done now to help with this |
|
I'm working on this, will update the PR in place |
9f4b1ab to
7ec8ae5
Compare
|
Sorry, just saw the comment. I'll leave it for you to work on. The force push was just to replace the old head with the rebased one. I resolved conflicts in |
roborev: Combined Review (
|
|
Okay, I'll force push over what is here once my agents are done working, they are in the middle of a large superpowers plan |
7ec8ae5 to
367846b
Compare
roborev: Combined Review (
|
roborev: Combined Review (
|
|
Thanks for this. The single-owner approach with thin adapters eliminates the scope leaks at the source. I'll dive deeper into this to make sure future work is aligned along these principles so extraction opportunities are caught before the code gets written. Appreciate you taking the time to do it right. |
|
All good! I was hoping it might be possible to make the change smaller but at least architecture wise this appears to be a tidier way to go about it. I'll work on getting this merged here today |
6644408 to
3c41559
Compare
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
roborev: Combined Review (
|
Add a model filter to the analytics dashboard so the stats, velocity, signals, activity, trends, and hour-of-week panels can be scoped to a selected model's user/assistant exchanges. Session Health stays session-scoped, counting whole sessions that used the model rather than recomputing its rollups from only that model's messages. Model-scoped message pairing is centralized in internal/db message-scope helpers (a streaming reducer, a day/hour predicate, and stats and timing projections) consumed by the SQLite, PostgreSQL, and DuckDB backends, so observable output is identical across all three. The reducer's grouping contract is collation-independent, so PostgreSQL ordering matches SQLite and DuckDB, and the timing projection restores ordinal order so velocity pairs each prompt with its response. The usage guide documents the control and its dashboard-only scope. Co-Authored-By: Wes McKinney <wesmckinn+git@gmail.com>
e52dbe6 to
f20b071
Compare
roborev: Combined Review (
|
Add a regression test asserting the SQLite model-filtered top-sessions path caps at ten for the messages and output_tokens metrics. The model-scoped re-sort drops the SQL LIMIT and ranks every matching session, so the caller's len > 10 truncation is what keeps the result at the top ten. Verified to fail (returns twelve) when that cap is removed.
roborev: Combined Review (
|
The analytics dashboard can already narrow by project, machine, agent, and time,
but still rolls every chart and summary together across models. This adds a
model-name filter to the main dashboard. A comma-separated
modelvalue isthreaded through the analytics route layer, the shared
AnalyticsFilter, theanalytics/trends/filtered-model lookup helpers, and the dashboard request state.
When a model filter is active, the dashboard first scopes to sessions that contain
at least one matching model message, then derives message, token, tool-call,
trend-term, session-shape, and velocity metrics from the matching model rows within
that session set. Summary totals, activity, heatmap, projects, tools, skills, top
sessions, signals, trend terms, session shape, and velocity stay aligned across the
SQLite, PostgreSQL, and DuckDB backends, including mixed-model sessions and hour/day
slices.
modelis the first message-grain analytics filter — a session spans many models —so it can't ride the session-grain WHERE builders the other filters use. Each panel
instead needs a user->assistant pairing pass over candidate message rows. Rather
than copy that pairing into every panel, the model membership, user-turn pairing,
and day/hour match now live in one shared streaming reducer (the
internal/dbmessage-scope helpers); each backend keeps its own candidate-row SQL (dialect,
placeholders, driver) and feeds rows through the shared reducer and its stats/timing
projections. Analytics and trends across all three backends share one implementation
instead of six near-duplicate pairing loops. The reducer requires only that rows
arrive grouped by session with ascending per-session ordinal — what
ORDER BY session_id, ordinalyields under any collation — so PostgreSQL'scollation-dependent ordering stays correct alongside SQLite and DuckDB.
On the frontend, the analytics store gains model-filter state, request params,
clear/toggle helpers, and active-filter chips, and the toolbar gains a model
dropdown that keeps known models stable across refreshes and filtered reloads.
One known limitation: the summary aggregation (median, p90, concentration) is still
computed per backend rather than shared; folding it into the same shared path is a
pre-existing cleanup left out here to keep this change scoped to model filtering.
Reviewers: the shared reducer (
internal/db/messagescope.go,messagescope_reducer.go) and the per-backend candidate-row resolvers(
internal/{db,postgres,duckdb}/analytics_scope.go) are the core; the rest isroute, filter, and frontend wiring. New analytics tests cover the filter across all
three backends.
Fixes #633