feat(inkless:switch): implement AlterDisklessSwitch and tooling [KC-97]#665
Open
EelisK wants to merge 9 commits into
Open
feat(inkless:switch): implement AlterDisklessSwitch and tooling [KC-97]#665EelisK wants to merge 9 commits into
EelisK wants to merge 9 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an operator-facing override path for classic→diskless switch sealing by introducing a new AlterDisklessSwitch Admin/Controller RPC and a kafka-topic-switch seal command, plus metadata merge semantics to distinguish “tag absent” from “explicit -1”.
Changes:
- Introduces
AlterDisklessSwitchrequest/response protocol + wiring (API key, parsing, JSON conversion, broker forwarding, controller handling). - Adds AdminClient support (
Admin,KafkaAdminClient, forwarding/wrappers) and a CLI “seal” command with validation + dry-run. - Updates metadata decoding/merge behavior (start offset tag presence) and expands unit/integration test coverage + documentation.
Reviewed changes
Copilot reviewed 29 out of 29 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/src/test/java/org/apache/kafka/tools/TopicSwitchCommandTest.java | Adds tests for seal command offset selection, validation, and dry-run behavior. |
| tools/src/main/java/org/apache/kafka/tools/TopicSwitchCommand.java | Implements seal subcommand and calls new Admin API to override seal state. |
| test-common/test-common-runtime/src/main/java/org/apache/kafka/common/test/MockController.java | Adds stub implementation for new controller method in test controller. |
| streams/integration-tests/src/test/java/org/apache/kafka/streams/integration/TestingMetricsInterceptingAdminClient.java | Forwards the new Admin API in the test AdminClient wrapper. |
| server/src/main/java/org/apache/kafka/network/RequestConvertToJson.java | Adds JSON conversion support for the new request/response types. |
| metadata/src/test/java/org/apache/kafka/metadata/PartitionRegistrationTest.java | Adds merge tests for explicit -1 and -2 start offset updates. |
| metadata/src/test/java/org/apache/kafka/controller/ReplicationControlManagerTest.java | Adds controller-level tests for sealing/abort/re-arm and validation errors. |
| metadata/src/main/java/org/apache/kafka/metadata/PartitionRegistration.java | Adjusts merge logic to distinguish “tag absent” vs “tag present” for start offset. |
| metadata/src/main/java/org/apache/kafka/metadata/InitDisklessLogFields.java | Adds decodeClassicToDisklessStartOffsetIfPresent to preserve “absent tag” semantics. |
| metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java | Adds controller handler to write the override change record and leader-epoch bump on re-arm. |
| metadata/src/main/java/org/apache/kafka/controller/QuorumController.java | Wires controller interface call to the replication control manager via write event. |
| metadata/src/main/java/org/apache/kafka/controller/Controller.java | Extends controller interface with alterDisklessSwitch. |
| docs/inkless/CLASSIC_TO_DISKLESS_SWITCH.md | Documents explicit -1 serialization for operator abort and the new operator override flow. |
| core/src/main/scala/kafka/server/KafkaApis.scala | Forwards ALTER_DISKLESS_SWITCH from broker to controller. |
| core/src/main/scala/kafka/server/ControllerApis.scala | Handles ALTER_DISKLESS_SWITCH on the controller listener with authorization. |
| clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java | Adds request/response roundtrip test generation for new API. |
| clients/src/test/java/org/apache/kafka/clients/admin/MockAdminClient.java | Adds stub override for new Admin API in mock client. |
| clients/src/main/resources/common/message/AlterDisklessSwitchResponse.json | Defines protocol schema for the response. |
| clients/src/main/resources/common/message/AlterDisklessSwitchRequest.json | Defines protocol schema for the request. |
| clients/src/main/java/org/apache/kafka/common/requests/AlterDisklessSwitchResponse.java | Adds request class wrapper for response parsing/throttling/error counts. |
| clients/src/main/java/org/apache/kafka/common/requests/AlterDisklessSwitchRequest.java | Adds request class wrapper including error response mapping. |
| clients/src/main/java/org/apache/kafka/common/requests/AbstractResponse.java | Registers response parsing for new API key. |
| clients/src/main/java/org/apache/kafka/common/requests/AbstractRequest.java | Registers request parsing for new API key. |
| clients/src/main/java/org/apache/kafka/common/protocol/ApiKeys.java | Adds new API key definition and listener/forwarding attributes. |
| clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java | Implements the new Admin call and maps top-level errors to exceptions. |
| clients/src/main/java/org/apache/kafka/clients/admin/ForwardingAdmin.java | Forwards the new Admin method through the delegating wrapper. |
| clients/src/main/java/org/apache/kafka/clients/admin/AlterDisklessSwitchResult.java | Adds result wrapper for the Admin call. |
| clients/src/main/java/org/apache/kafka/clients/admin/AlterDisklessSwitchOptions.java | Adds options type for the Admin call. |
| clients/src/main/java/org/apache/kafka/clients/admin/Admin.java | Adds new Admin API surface (default overload + options overload). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This change makes it possible to differentiate between absent `classicToDisklessStartOffset` and `disklessProducerStates` vs explicitly encoded values.
EelisK
force-pushed
the
EelisK/KC-97-seal
branch
2 times, most recently
from
bf82594 to
22c9b0b
Compare
EelisK
force-pushed
the
EelisK/KC-97-seal
branch
from
22c9b0b to
f099337
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.