KAFKA-20032: KIP-1265 Automatic detection of internal API usage

api-checker/README.md

@@ -0,0 +1,90 @@
+# Kafka API-checker plugins
+
+This is a separate Gradle build (composite-included from the root `settings.gradle` via
+`pluginManagement { includeBuild 'api-checker' }`) that produces the KIP-1265 API checkers.
+Living outside Kafka's main build keeps the Maven runtime dependencies off the main-build
+classpath and lets each published artifact carry only the classes its consumers need.
+
+| Subproject | Publishes | Audience |
+|---|---|---|
+| `:core` | `org.apache.kafka:kafka-api-checker-core` | Shared scanner + validator + reporter (ASM only). Both plugin jars depend on it. |
+| `:gradle-plugins` | `org.apache.kafka:kafka-internal-api-checker-gradle-plugin` + plugin markers for `org.apache.kafka.public-api-checker` and `org.apache.kafka.internal-api-checker` | The Kafka-internal producer-side checker and the published consumer-side Gradle checker. |
+| `:maven-plugin` | `org.apache.kafka:kafka-internal-api-checker-maven-plugin` | Maven equivalent of the consumer-side checker. |
+
+End-user documentation (Gradle/Maven snippets, `@SuppressKafkaInternalApiUsage`,
+audience-inheritance rules) lives at
+[`docs/apis/internal-api-checker.md`](../docs/apis/internal-api-checker.md). The notes
+below cover building, testing, and publishing the plugins themselves.
+
+## Build
+
+```bash
+./gradlew :api-checker:core:build :api-checker:gradle-plugins:build :api-checker:maven-plugin:build
+```
+
+## Test
+
+```bash
+./gradlew :api-checker:core:test :api-checker:gradle-plugins:test
+```
+
+`:gradle-plugins:test` includes a Gradle TestKit end-to-end test that applies the
+`org.apache.kafka.internal-api-checker` plugin to a synthetic consumer project.
+
+## Publish
+
+Each subproject's `publish` task stages to the URL passed via `-PmavenUrl` (with
+`-PmavenUsername` / `-PmavenPassword` for credentials). The version is read from the
+repo-root `gradle.properties`, so `release.py`'s existing `updateVersion` call sets it
+automatically. `-PkafkaPluginsVersion=…` overrides for one-off out-of-band publishes.
+
+```bash
+# Stage to ASF Nexus alongside the rest of an AK release
+./gradlew :api-checker:core:publish \
+         :api-checker:gradle-plugins:publish \
+         :api-checker:maven-plugin:publish \
+  -PmavenUrl=$ASF_NEXUS_STAGING_URL \
+  -PmavenUsername=$NEXUS_USER \
+  -PmavenPassword=$NEXUS_PASS
+
+# Local smoke-test
+./gradlew :api-checker:core:publishToMavenLocal \
+         :api-checker:gradle-plugins:publishToMavenLocal \
+         :api-checker:maven-plugin:publishToMavenLocal
+```
+
+The five published coordinates:
+
+- `org.apache.kafka:kafka-api-checker-core:$KAFKA_VERSION` — shared scanner library.
+- `org.apache.kafka:kafka-internal-api-checker-gradle-plugin:$KAFKA_VERSION` — Gradle plugin
+  implementation jar (consumed by the marker poms).
+- `org.apache.kafka.internal-api-checker:org.apache.kafka.internal-api-checker.gradle.plugin:$KAFKA_VERSION` —
+  marker pom for `plugins { id 'org.apache.kafka.internal-api-checker' }`.
+- `org.apache.kafka.public-api-checker:org.apache.kafka.public-api-checker.gradle.plugin:$KAFKA_VERSION` —
+  marker pom for the producer-side checker. (Kafka-internal use, but published from the
+  same module for consistency.)
+- `org.apache.kafka:kafka-internal-api-checker-maven-plugin:$KAFKA_VERSION` — Maven plugin
+  (packaging `maven-plugin`).
+
+## Layout
+
+```
+api-checker/
+├── settings.gradle           # declares the three subprojects
+├── build.gradle              # group / version / signing / common publishing config
+├── core/
+│   ├── build.gradle          # ASM dep; published as kafka-api-checker-core
+│   └── src/
+│       ├── main/java/.../apicheck/                # scanner, validators, reporter
+│       ├── test/java/.../apicheck/                # unit tests
+│       └── testFixtures/java/.../apicheck/        # AsmClassFactory, TempJarBuilder
+│                                                  # — re-used by :gradle-plugins' tests
+├── gradle-plugins/
+│   ├── build.gradle          # java-gradle-plugin; depends on :core
+│   └── src/{main,test}/java/.../gradle/           # Plugin/Task/Extension × 2
+└── maven-plugin/
+    ├── build.gradle          # Maven deps; templates plugin.xml at processResources
+    └── src/main/
+        ├── java/.../maven/KafkaInternalApiCheckerMojo.java
+        └── resources/META-INF/maven/plugin.xml
+```

api-checker/build.gradle

@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Shared configuration for the api-checker included build. Lives outside Kafka's main
+// settings tree so its Maven dependencies (in :maven-plugin) don't end up on every
+// developer's main-build classpath, and so each published artifact carries only the classes
+// its consumers need.
+
+// Default the published version to the same string as the rest of Kafka (gradle.properties
+// at the repo root). release.py's `updateVersion` task writes that file, so the included
+// build's artifacts ship under the AK release version automatically. `-PkafkaPluginsVersion=…`
+// overrides for one-off out-of-band publishes.
+def rootGradleProps = new Properties()
+def rootGradlePropsFile = file('../gradle.properties')
+if (rootGradlePropsFile.exists()) {
+    rootGradlePropsFile.withInputStream { rootGradleProps.load(it) }
+}
+def kafkaVersion = providers.gradleProperty('kafkaPluginsVersion')
+        .orElse(providers.provider { rootGradleProps.getProperty('version') })
+        .orElse('1.0.0-SNAPSHOT')
+
+allprojects {
+    group = 'org.apache.kafka'
+    version = kafkaVersion.get()
+
+    repositories {
+        gradlePluginPortal()
+        mavenCentral()
+    }
+}
+
+subprojects {
+    apply plugin: 'java-library'
+    apply plugin: 'maven-publish'
+    apply plugin: 'signing'
+    apply plugin: 'checkstyle'
+
+    java {
+        sourceCompatibility = JavaVersion.VERSION_11
+        targetCompatibility = JavaVersion.VERSION_11
+        // Maven Central requires sources + javadoc jars alongside every artifact.
+        withSourcesJar()
+        withJavadocJar()
+    }
+
+    test {
+        useJUnitPlatform()
+    }
+
+    checkstyle {
+        toolVersion = '12.3.1'
+        configDirectory = file("$rootDir/../checkstyle")
+        configProperties = [importControlFile: 'import-control-api-checker.xml']
+    }
+
+    publishing {
+        repositories {
+            // Mirrors the root Kafka project's publishing.repositories block — credentials
+            // come from ~/.gradle/gradle.properties via the same -PmavenUrl / -PmavenUsername /
+            // -PmavenPassword properties release.py uses.
+            maven {
+                url = project.hasProperty('mavenUrl') ? project.mavenUrl : ''
+                credentials {
+                    username = project.hasProperty('mavenUsername') ? project.mavenUsername : ''
+                    password = project.hasProperty('mavenPassword') ? project.mavenPassword : ''
+                }
+            }
+        }
+    }
+
+    def skipSigning = project.hasProperty('skipSigning') && skipSigning.toBoolean()
+    def shouldSign = !skipSigning && !version.toString().endsWith('SNAPSHOT')
+    if (shouldSign) {
+        signing {
+            sign publishing.publications
+        }
+    }
+}

api-checker/core/build.gradle

@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Shared scanner + validator + reporter for the KIP-1265 API checkers. Both the Gradle
+// plugin and the Maven plugin depend on this module — neither carries the bytecode
+// scanner classes themselves any more, so the published Gradle jar no longer drags Maven
+// classes (and vice versa).
+
+apply plugin: 'java-test-fixtures'
+
+dependencies {
+    // ASM is the entire production dependency footprint. No Gradle, no Maven.
+    //
+    // ASM versions cap the JDK class-file major version they can parse. 9.6 covers up to
+    // Java 21 (class-file 65). If Kafka starts compiling against a newer JDK whose major
+    // exceeds what this ASM release knows about, the scanner will throw
+    // `IllegalArgumentException: Unsupported class file major version N` on encountering
+    // a fresh bytecode file. Bump in lockstep with the project's max-supported compile JDK.
+    api 'org.ow2.asm:asm:9.6'
+    api 'org.slf4j:slf4j-api:2.0.16'
+
+    testImplementation 'org.junit.jupiter:junit-jupiter:5.9.2'
+    testImplementation 'org.junit.jupiter:junit-jupiter-engine:5.9.2'
+    testImplementation 'org.junit.platform:junit-platform-launcher:1.9.2'
+
+    // ASM-based class-file builders used by every checker test in the included build.
+    // Exposed as test-fixtures so the gradle-plugins module's tests can pull them in via
+    // `testImplementation(testFixtures(project(':core')))` without us re-publishing them
+    // as a separate "tests" classifier artifact.
+    testFixturesImplementation 'org.ow2.asm:asm:9.6'
+}
+
+publishing {
+    publications {
+        coreJar(MavenPublication) {
+            artifactId = 'kafka-api-checker-core'
+            from components.java
+
+            pom {
+                name = 'Apache Kafka API Checker Core'
+                description = 'Bytecode scanner and reporter shared by the KIP-1265 Gradle and Maven plugins'
+                url = 'https://kafka.apache.org'
+
+                licenses {
+                    license {
+                        name = 'The Apache License, Version 2.0'
+                        url = 'https://www.apache.org/licenses/LICENSE-2.0.txt'
+                    }
+                }
+                developers {
+                    developer {
+                        id = 'apache-kafka'
+                        name = 'Apache Kafka Team'
+                        email = 'dev@kafka.apache.org'
+                    }
+                }
+                scm {
+                    connection = 'scm:git:https://github.com/apache/kafka.git'
+                    developerConnection = 'scm:git:https://github.com/apache/kafka.git'
+                    url = 'https://github.com/apache/kafka'
+                }
+            }
+        }
+    }
+}