feat(inkless): add partition fan-in commit metrics#675
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds new FileCommitter commit “fan-in” observability by tracking (1) how many distinct topic-partitions appear in a committed file and (2) how many batches each partition contributes, helping quantify batch coalescing opportunity.
Changes:
- Add two new per-broker histograms to
FileCommitterMetrics:PartitionsPerCommitandBatchesPerPartitionPerCommit. - Record the new metrics during
FileCommitter.commit(...)using a single linear pass over already partition-grouped requests. - Add unit tests for the new histograms and regenerate
docs/inkless/metrics.rst.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| storage/inkless/src/main/java/io/aiven/inkless/produce/FileCommitterMetrics.java | Introduces two new histograms and computes partition fan-in from grouped commit requests. |
| storage/inkless/src/main/java/io/aiven/inkless/produce/FileCommitter.java | Records the new fan-in metrics when a file is successfully committed. |
| storage/inkless/src/test/java/io/aiven/inkless/produce/FileCommitterMetricsTest.java | Adds focused tests validating fan-in counting behavior and accumulation across commits. |
| docs/inkless/metrics.rst | Documents the two new FileCommitter metrics in the metrics reference. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Adds two per-broker histograms on FileCommitter: PartitionsPerCommit (distinct topic-partitions in a committed file) and BatchesPerPartitionPerCommit (a partition's batch fan-in). Batches-per-partition is the workload dimension that drives batch coalescing, so it surfaces the per-partition coalescing opportunity. Computed in a single linear pass over the committed requests, which are already grouped by topic-partition (BatchBuffer.close sorts them) — no intermediate grouping map. Regenerates metrics.rst. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jeqo
force-pushed
the
jeqo/kc-290-commit-fan-in-metrics
branch
from
7a7b359 to
df85d10
Compare
viktorsomogyi
approved these changes
Jun 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds two per-broker histograms on FileCommitter: PartitionsPerCommit (distinct topic-partitions in a committed file) and BatchesPerPartitionPerCommit (a partition's batch fan-in). Batches-per-partition is the workload dimension that drives batch coalescing, so it surfaces the per-partition coalescing opportunity.
Computed in a single linear pass over the committed requests, which are already grouped by topic-partition (BatchBuffer.close sorts them) — no intermediate grouping map. Regenerates metrics.rst.