Overview
This issue is the StreamPark 3.0 Roadmap — the single source of truth for all planned breaking changes and major features targeted at the 3.0 release .
#
Theme
Summary
Design issue
Implementation PR
Status
1
JDK 11 Console minimum runtime JDK 11; compile baseline 11
#4409 #4417 ✅ Done (merged to dev)
2
Remove dev framework Drop Flink/Spark dev traits and connector wrapper modules
#4431 #4432 ✅ Done (merged to dev)
3
Remove Scala Java-only codebase; remove Scala toolchain
#4408 In progress
4
Flink 1.17+ / 2.x Drop Flink 1.12–1.16; baseline 1.17–1.20; add 2.0/2.1/2.2
#4421 Pending
5
Spark 3.5+ / 4.x Baseline Spark 3.5+; add Spark 4.x support
#4422 Pending
6
Frontend Naive UI New streampark-console-webapp-v2 on Naive UI + Vite
#4414 In progress
7
PyFlink First-class PyFlink job lifecycle (submit, config, packaging, UI)
(this roadmap)
[]
Pending
Current Baseline (2.x)
Area
Today
Console JDK
8 (compile & runtime)
Flink shims
1.12, 1.13, 1.14, 1.15, 1.16, 1.17, 1.18, 1.19, 1.20
Spark
3.1.2 (-Pspark profile)
Backend language
~343 Scala files + Java Console
Dev framework
FlinkStreaming / FlinkTable traits + connector wrappers
Frontend
Vue 3 + Vite + Ant Design Vue 3 + Pinia
PyFlink
Partial — .py upload validation, PythonDriver, basic PythonOptions in client; no dedicated UI flow
3.0 Target State
Area
3.0 Target
Status
Console JDK
11+ required✅ Done (#4417 )
Dev framework
Removed — users adopt native Flink/Spark APIs + official connectors✅ Done (#4432 )
Flink
1.17, 1.18, 1.19, 1.20, 2.0, 2.1, 2.2 Pending
Spark
3.5+ and 4.x (-Pspark profile)Pending
Backend
Java only , zero .scalaPending
Frontend
Vue 3 + Vite + Naive UI + Pinia (webapp-v2)
In progress
PyFlink
Full support: env registration, venv packaging, submit on YARN/K8s/standalone, Console UI
Pending
Delivery Tracks (in-flight PRs)
Track 1 — JDK 11 ✅ Completed (#4409 → #4417 )
Status: Merged to dev on 2026-07-15 (merge commit e777acd ).
Goal: Console runs and builds on JDK 11.
Delivered:
Maven compile baseline JDK 11; CI/Docker aligned to JDK 11
Console runtime JDK 11+ enforcement, javax.annotation-api, JVM --add-opens
ClassLoaderUtils JDK 9+ compatibilityDependency license registration for JDK 11 artifacts
Yarn E2E Docker images bundle Eclipse Temurin JDK 11 (StreamPark Console startup on JDK 11)
Track 2 — Remove dev framework ✅ Completed (#4431 → #4432 )
Status: Merged to dev on 2026-07-15 (merge commit 16f5fdc ).
Goal: Remove the StreamPark development framework and all connector wrapper modules; keep Console platform runtime (submit/cancel, SQL mode, validation, K8s, packaging).
Delivered:
Removed streampark-flink-core (FlinkStreaming, FlinkTable, FlinkStreamTable, DataStreamExt)
Removed entire streampark-flink-connector tree (Kafka, JDBC, ClickHouse, Redis, ES, HBase, HTTP, etc.)
Removed Spark dev traits (Spark, SparkBatch, SparkStreaming) and streampark-spark-connector
Removed Maven -Pconnector profile
Refactored Flink SqlClient to use FlinkTableInitializer + TableContext / StreamTableContext from shims
Added SparkSqlExecutor and refactored Spark SqlClient accordingly
Depends on: #4417 (JDK 11 baseline)Done .
Breaking changes (user jobs):
Jobs extending FlinkStreaming / FlinkTable / SparkStreaming / SparkBatch must migrate to native Flink/Spark APIs
Code using org.apache.streampark.flink.connector.* or org.apache.streampark.spark.connector.* must migrate to official connectors
Track 3 — Remove Scala (#4408 → #4418 )
Goal: Zero Scala source and zero Scala build toolchain.
Single PR migrates the remaining backend (common, Flink proxy/client/shims, k8s, packer, Spark modules) to Java and removes Scala Maven plugins.
Depends on: #4417 (JDK 11 baseline)Done . #4432 (Remove dev framework)Done — connector modules already removed, reducing migration scope.
Track 4 — Flink 1.17+ / 2.x (#4421 → #4419 )
Goal: Remove legacy Flink versions; add Flink 2.x shims.
Drop: streampark-flink-shims_flink-1.12 through 1.16, related UI/E2E referencesKeep: 1.17, 1.18, 1.19, 1.20Add: streampark-flink-shims-base-v2, shims_flink-2.0/2.1/2.2 (Java, no _2.12 suffix)
Depends on: #4418 (Remove Scala).
Track 5 — Spark 3.5+ / 4.x (#4422 → #4420 )
Goal: Bump Spark baseline; add Spark 4.x support.
Bump spark.version to 3.5.x
SparkVersion / SparkEnvUtils for 3.5+ and 4.x (Spark 4 requires JDK 17+ for job runtime)Update Spark client/shims for API compatibility
Depends on: #4418 (Remove Scala). Independent of #4419 (Flink).
Track 6 — Frontend Naive UI (#4414 → #4415 )
Goal: Deliver a new Naive UI console as streampark-console-webapp-v2.
Strategy: new webapp directory + full page migration on a feature branch, not incremental page-by-page PRs against the old Ant Design app.
Parallel with backend tracks after scaffolding is in place.
Track 7 — PyFlink (TBD)
Goal: Production-ready PyFlink support in Console.
Existing (partial):
.py file validation in FlinkApplicationActionServiceImplPythonDriver / PythonOptions in Flink clientvenv.zip workspace path in Constants
Gaps to close: submit path hardening, packaging pipeline, Console UI (target Naive UI in webapp-v2), env validation, monitoring, E2E, docs.
Dependency: PyFlink UI should land in Track 6 (webapp-v2).
Cross-Track Dependency Graph
#4417 JDK 11 baseline ✅ (merged)
↓
#4432 Remove dev framework ✅ (merged)
↓
#4418 Remove Scala (full backend Java migration)
├──→ #4419 Flink 1.17+ / 2.x
└──→ #4420 Spark 3.5+ / 4.x
#4415 Frontend Naive UI (webapp-v2) ─── parallel, relatively independent
Track 7 PyFlink ─── backend early; UI after #4415 scaffolding
Recommended Merge Order
#4417 — JDK 11 baseline → devDone #4432 — Remove dev framework → devDone #4418 — Remove Scala → dev ← next #4419 and #4420 — can proceed in parallel after [Build] Remove Scala for StreamPark 3.0 #4418 #4415 — Frontend Naive UI (can review in parallel; cutover timing TBD)Track 7 — PyFlink (follow-up within 3.0 or 3.0.x)Release notes + upgrade guide
3.0 Breaking Changes Summary
Change
User impact
JDK 11+ required for Console
Upgrade deployment JVM
StreamPark dev framework removed
Migrate FlinkStreaming/FlinkTable/Spark traits to native Flink/Spark APIs
Connector wrapper modules removed
Use official Flink/Spark connectors instead of org.apache.streampark.*.connector.*
Flink 1.12–1.16 no longer supported
Re-register env on 1.17+
Scala development API removed
Rewrite apps with Java API
Spark 3.5+ only (4.x supported)
Upgrade Spark clusters
Ant Design Vue UI replaced by Naive UI (webapp-v2)
Visual change; feature parity expected
PyFlink
New capability (additive)
Related Issues
Overview
This issue is the StreamPark 3.0 Roadmap — the single source of truth for all planned breaking changes and major features targeted at the 3.0 release.
dev)dev)streampark-console-webapp-v2on Naive UI + ViteCurrent Baseline (2.x)
-Psparkprofile)FlinkStreaming/FlinkTabletraits + connector wrappers.pyupload validation,PythonDriver, basicPythonOptionsin client; no dedicated UI flow3.0 Target State
-Psparkprofile).scalawebapp-v2)Delivery Tracks (in-flight PRs)
Track 1 — JDK 11 ✅ Completed (#4409 → #4417)
Status: Merged to
devon 2026-07-15 (merge commite777acd).Goal: Console runs and builds on JDK 11.
Delivered:
javax.annotation-api, JVM--add-opensClassLoaderUtilsJDK 9+ compatibilityTrack 2 — Remove dev framework ✅ Completed (#4431 → #4432)
Status: Merged to
devon 2026-07-15 (merge commit16f5fdc).Goal: Remove the StreamPark development framework and all connector wrapper modules; keep Console platform runtime (submit/cancel, SQL mode, validation, K8s, packaging).
Delivered:
streampark-flink-core(FlinkStreaming,FlinkTable,FlinkStreamTable,DataStreamExt)streampark-flink-connectortree (Kafka, JDBC, ClickHouse, Redis, ES, HBase, HTTP, etc.)Spark,SparkBatch,SparkStreaming) andstreampark-spark-connector-PconnectorprofileSqlClientto useFlinkTableInitializer+TableContext/StreamTableContextfrom shimsSparkSqlExecutorand refactored SparkSqlClientaccordinglyDepends on:
#4417 (JDK 11 baseline)✅ Done.Breaking changes (user jobs):
FlinkStreaming/FlinkTable/SparkStreaming/SparkBatchmust migrate to native Flink/Spark APIsorg.apache.streampark.flink.connector.*ororg.apache.streampark.spark.connector.*must migrate to official connectorsTrack 3 — Remove Scala (#4408 → #4418)
Goal: Zero Scala source and zero Scala build toolchain.
Single PR migrates the remaining backend (common, Flink proxy/client/shims, k8s, packer, Spark modules) to Java and removes Scala Maven plugins.
Depends on:
#4417 (JDK 11 baseline)✅ Done.#4432 (Remove dev framework)✅ Done — connector modules already removed, reducing migration scope.Track 4 — Flink 1.17+ / 2.x (#4421 → #4419)
Goal: Remove legacy Flink versions; add Flink 2.x shims.
streampark-flink-shims_flink-1.12through1.16, related UI/E2E referencesstreampark-flink-shims-base-v2,shims_flink-2.0/2.1/2.2(Java, no_2.12suffix)Depends on: #4418 (Remove Scala).
Track 5 — Spark 3.5+ / 4.x (#4422 → #4420)
Goal: Bump Spark baseline; add Spark 4.x support.
spark.versionto 3.5.xSparkVersion/SparkEnvUtilsfor 3.5+ and 4.x (Spark 4 requires JDK 17+ for job runtime)Depends on: #4418 (Remove Scala). Independent of #4419 (Flink).
Track 6 — Frontend Naive UI (#4414 → #4415)
Goal: Deliver a new Naive UI console as
streampark-console-webapp-v2.Strategy: new webapp directory + full page migration on a feature branch, not incremental page-by-page PRs against the old Ant Design app.
Parallel with backend tracks after scaffolding is in place.
Track 7 — PyFlink (TBD)
Goal: Production-ready PyFlink support in Console.
Existing (partial):
.pyfile validation inFlinkApplicationActionServiceImplPythonDriver/PythonOptionsin Flink clientvenv.zipworkspace path inConstantsGaps to close: submit path hardening, packaging pipeline, Console UI (target Naive UI in
webapp-v2), env validation, monitoring, E2E, docs.Dependency: PyFlink UI should land in Track 6 (
webapp-v2).Cross-Track Dependency Graph
Recommended Merge Order
#4417 — JDK 11 baseline →✅ Donedev#4432 — Remove dev framework →✅ Donedevdev← next3.0 Breaking Changes Summary
FlinkStreaming/FlinkTable/Spark traits to native Flink/Spark APIsorg.apache.streampark.*.connector.*webapp-v2)Related Issues