Skip to content

chore: upgrade to Python 3.14 - #775

Open
jtligon wants to merge 11 commits into
jumpstarter-dev:mainfrom
jtligon:upgrade-python-3.14
Open

chore: upgrade to Python 3.14#775
jtligon wants to merge 11 commits into
jumpstarter-dev:mainfrom
jtligon:upgrade-python-3.14

Conversation

@jtligon

@jtligon jtligon commented Jun 8, 2026

Copy link
Copy Markdown

Add Python 3.14 Support

Summary

Adds support for Python 3.14 while maintaining backward compatibility with Python 3.12 and 3.13.

Changes

  • Updated requires-python from >=3.11 to >=3.12 (65 packages)
  • Updated ruff target-version to py312
  • Updated GitHub Actions CI to test with Python 3.12, 3.13, and 3.14
  • Updated uv.lock with Python 3.14 compatible dependencies

Dependencies

This required upgrading to newer versions of packages with Python 3.14 support:

  • pydantic-core: 2.33.2 → 2.46.4 (has Python 3.14 wheels)
  • pydantic: 2.11.7 → 2.13.4

Additional Fixes

Fixed pre-existing issues identified by CodeRabbit:

  • Added missing Apache-2.0 license to jumpstarter-driver-iscsi
  • Removed duplicate pytest-asyncio dependency in jumpstarter-driver-http
  • Fixed jumpstarter-driver-opendal entry point group (adapters → drivers)

Build Requirements

Building with Python 3.14 requires:

  • Environment variable: PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 for Rust-based Python extensions
  • macOS:
    • brew install gcc (provides gfortran for scipy)
    • brew install openblas (required for scipy)
    • export PKG_CONFIG_PATH="/opt/homebrew/opt/openblas/lib/pkgconfig:$PKG_CONFIG_PATH"
  • Linux: Install equivalent packages via system package manager

Testing

  • ✅ Successfully synced all packages with uv sync --all-packages --all-extras
  • ✅ Verified jmp CLI works correctly on Python 3.14.5
  • ✅ Backward compatible: supports Python 3.12, 3.13, and 3.14

Notes

Python 3.14 was released recently, so some packages in the ecosystem are still catching up. The forward compatibility flag allows PyO3-based packages to build using the stable ABI. Python 3.12 and 3.13 continue to work without any special configuration.

Checklist

  • All pyproject.toml files updated
  • CI configuration updated to test all versions
  • Dependencies resolved and lockfile updated
  • Local testing completed on Python 3.14
  • Backward compatibility maintained
  • CI passes

@coderabbitai

coderabbitai Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Repository baseline is updated to Python 3.12: local pin, CI pytest matrix (3.12–3.14), Ruff target, example projects, and package manifests across core packages and all driver packages adjust requires-python to >=3.12. CI gains Python 3.14 Pillow build dependency steps. HTTP driver dev dependency and iSCSI license metadata are updated.

Changes

Python 3.12 baseline rollout

Layer / File(s) Summary
CI and tooling baseline set to 3.12–3.14
.py-version, .github/workflows/python-tests.yaml, python/pyproject.toml
Repository Python pin updated to 3.14, GitHub Actions pytest matrix expanded to 3.12–3.14 with OS-specific Pillow build dependency installation for Python 3.14, and Ruff linter target bumped to py312.
Examples and core packages requires-python updates
python/examples/*, python/packages/jumpstarter-all/*, python/packages/jumpstarter-cli*/*, python/packages/jumpstarter-kubernetes/*, python/packages/jumpstarter-imagehash/*, python/packages/jumpstarter-mcp/*, python/packages/jumpstarter-protocol/*, python/packages/jumpstarter-testing/*, python/packages/jumpstarter/pyproject.toml, python/packages/hatch-pin-jumpstarter/*
Example projects and core packages update requires-python from >=3.11 to >=3.12 across the repository.
Driver packages and template requires-python updates with metadata adjustments
python/packages/jumpstarter-driver-*/pyproject.toml, python/__templates__/driver/pyproject.toml.tmpl
All driver packages and driver template raise requires-python to >=3.12. HTTP driver bumps pytest-asyncio from >=0.0.0 to >=0.24.0; iSCSI driver adds license = "Apache-2.0" metadata.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

Suggested reviewers

  • NickCao

A rabbit skipped through version lines,
From 3.11 to 3.12 with design,
CI and drivers all aligned,
Pillow builds for 3.14 refined. 🐰
The baseline hops toward better times!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: upgrading Python support to version 3.14, which is the primary objective of this PR.
Description check ✅ Passed The description comprehensively explains the changes, including the Python version updates, dependency upgrades, build requirements, testing results, and bug fixes identified during the upgrade process.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
python/packages/jumpstarter-driver-iscsi/pyproject.toml (1)

1-8: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Add required Apache-2.0 license metadata to this package.

[project] is missing license = "Apache-2.0", which violates the repository packaging policy.

Suggested fix
 [project]
 name = "jumpstarter-driver-iscsi"
 dynamic = ["version", "urls"]
 description = "Exporter ISCSI service driver"
 readme = "README.md"
+license = "Apache-2.0"
 authors = [{ name = "Benny Zlotnik", email = "bzlotnik@redhat.com" }]
 requires-python = ">=3.14"

As per coding guidelines, “All packages must use Apache-2.0 license as specified in pyproject.toml”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/packages/jumpstarter-driver-iscsi/pyproject.toml` around lines 1 - 8,
Add the required Apache-2.0 license metadata to the pyproject.toml [project]
metadata: update the project table in "pyproject.toml" (the [project] block for
the package jumpstarter-driver-iscsi) to include license = "Apache-2.0" so the
package metadata complies with repository packaging policy; place the new
license field alongside existing keys like name, dynamic, description, readme,
authors, and requires-python.

Source: Coding guidelines

python/packages/jumpstarter-driver-http/pyproject.toml (1)

41-42: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Remove duplicate pytest-asyncio constraints in dev dependencies.

Having both pytest-asyncio>=0.0.0 and pytest-asyncio>=0.24.0 is redundant and can confuse dependency resolution intent; keep only the stricter constraint.

Suggested cleanup
 [dependency-groups]
 dev = [
     "pytest-cov>=6.0.0",
     "pytest>=8.3.3",
-    "pytest-asyncio>=0.0.0",
     "pytest-asyncio>=0.24.0",
 ]
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/packages/jumpstarter-driver-http/pyproject.toml` around lines 41 - 42,
Remove the duplicate pytest-asyncio entries from the dev dependencies in
pyproject.toml by deleting the weaker constraint "pytest-asyncio>=0.0.0" and
keeping the stricter "pytest-asyncio>=0.24.0" (ensure the duplicate string
entries under the dev-dependencies section are consolidated so only the >=0.24.0
constraint remains).
python/packages/jumpstarter-driver-opendal/pyproject.toml (1)

18-20: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Register this package under jumpstarter.drivers (or rename/scope it if it is adapter-only).

For a jumpstarter-driver-* package, the required project.entry-points."jumpstarter.drivers" registration is missing.

As per coding guidelines, “Drivers must register via jumpstarter.drivers entry point in pyproject.toml”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/packages/jumpstarter-driver-opendal/pyproject.toml` around lines 18 -
20, The pyproject.toml registers the package under
project.entry-points."jumpstarter.adapters" but for a jumpstarter-driver-*
package it must register under project.entry-points."jumpstarter.drivers";
update the entry-point group to "jumpstarter.drivers" (or if this package is
strictly an adapter, rename/scope the package accordingly) and ensure the entry
maps the OpendalAdapter symbol
(jumpstarter_driver_opendal.adapters:OpendalAdapter) under that group so the
driver is discoverable.

Sources: Coding guidelines, Learnings

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@python/packages/jumpstarter-driver-http/pyproject.toml`:
- Around line 41-42: Remove the duplicate pytest-asyncio entries from the dev
dependencies in pyproject.toml by deleting the weaker constraint
"pytest-asyncio>=0.0.0" and keeping the stricter "pytest-asyncio>=0.24.0"
(ensure the duplicate string entries under the dev-dependencies section are
consolidated so only the >=0.24.0 constraint remains).

In `@python/packages/jumpstarter-driver-iscsi/pyproject.toml`:
- Around line 1-8: Add the required Apache-2.0 license metadata to the
pyproject.toml [project] metadata: update the project table in "pyproject.toml"
(the [project] block for the package jumpstarter-driver-iscsi) to include
license = "Apache-2.0" so the package metadata complies with repository
packaging policy; place the new license field alongside existing keys like name,
dynamic, description, readme, authors, and requires-python.

In `@python/packages/jumpstarter-driver-opendal/pyproject.toml`:
- Around line 18-20: The pyproject.toml registers the package under
project.entry-points."jumpstarter.adapters" but for a jumpstarter-driver-*
package it must register under project.entry-points."jumpstarter.drivers";
update the entry-point group to "jumpstarter.drivers" (or if this package is
strictly an adapter, rename/scope the package accordingly) and ensure the entry
maps the OpendalAdapter symbol
(jumpstarter_driver_opendal.adapters:OpendalAdapter) under that group so the
driver is discoverable.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f69689ae-69d2-4fbc-8ba7-999be9c428c7

📥 Commits

Reviewing files that changed from the base of the PR and between be78b3b and 3282c04.

⛔ Files ignored due to path filters (1)
  • python/uv.lock is excluded by !**/*.lock
📒 Files selected for processing (65)
  • .github/workflows/python-tests.yaml
  • .py-version
  • python/examples/android-emulator/pyproject.toml
  • python/examples/automotive/pyproject.toml
  • python/examples/soc-pytest/pyproject.toml
  • python/examples/xcp-ecu/pyproject.toml
  • python/packages/hatch-pin-jumpstarter/pyproject.toml
  • python/packages/jumpstarter-all/pyproject.toml
  • python/packages/jumpstarter-cli-admin/pyproject.toml
  • python/packages/jumpstarter-cli-common/pyproject.toml
  • python/packages/jumpstarter-cli-driver/pyproject.toml
  • python/packages/jumpstarter-cli/pyproject.toml
  • python/packages/jumpstarter-driver-adb/pyproject.toml
  • python/packages/jumpstarter-driver-androidemulator/pyproject.toml
  • python/packages/jumpstarter-driver-ble/pyproject.toml
  • python/packages/jumpstarter-driver-can/pyproject.toml
  • python/packages/jumpstarter-driver-composite/pyproject.toml
  • python/packages/jumpstarter-driver-corellium/pyproject.toml
  • python/packages/jumpstarter-driver-doip/pyproject.toml
  • python/packages/jumpstarter-driver-dut-network/pyproject.toml
  • python/packages/jumpstarter-driver-dutlink/pyproject.toml
  • python/packages/jumpstarter-driver-energenie/pyproject.toml
  • python/packages/jumpstarter-driver-esp32/pyproject.toml
  • python/packages/jumpstarter-driver-flashers/pyproject.toml
  • python/packages/jumpstarter-driver-gpiod/pyproject.toml
  • python/packages/jumpstarter-driver-http-power/pyproject.toml
  • python/packages/jumpstarter-driver-http/pyproject.toml
  • python/packages/jumpstarter-driver-iscsi/pyproject.toml
  • python/packages/jumpstarter-driver-mitmproxy/pyproject.toml
  • python/packages/jumpstarter-driver-network/pyproject.toml
  • python/packages/jumpstarter-driver-noyito-relay/pyproject.toml
  • python/packages/jumpstarter-driver-opendal/pyproject.toml
  • python/packages/jumpstarter-driver-pi-pico/pyproject.toml
  • python/packages/jumpstarter-driver-power/pyproject.toml
  • python/packages/jumpstarter-driver-probe-rs/pyproject.toml
  • python/packages/jumpstarter-driver-pyserial/pyproject.toml
  • python/packages/jumpstarter-driver-qemu/pyproject.toml
  • python/packages/jumpstarter-driver-renode/pyproject.toml
  • python/packages/jumpstarter-driver-ridesx/pyproject.toml
  • python/packages/jumpstarter-driver-sdwire/pyproject.toml
  • python/packages/jumpstarter-driver-shell/pyproject.toml
  • python/packages/jumpstarter-driver-snmp/pyproject.toml
  • python/packages/jumpstarter-driver-someip/pyproject.toml
  • python/packages/jumpstarter-driver-ssh-mitm/pyproject.toml
  • python/packages/jumpstarter-driver-ssh-mount/pyproject.toml
  • python/packages/jumpstarter-driver-ssh/pyproject.toml
  • python/packages/jumpstarter-driver-stlink-msd/pyproject.toml
  • python/packages/jumpstarter-driver-tasmota/pyproject.toml
  • python/packages/jumpstarter-driver-tftp/pyproject.toml
  • python/packages/jumpstarter-driver-tmt/pyproject.toml
  • python/packages/jumpstarter-driver-uboot/pyproject.toml
  • python/packages/jumpstarter-driver-uds-can/pyproject.toml
  • python/packages/jumpstarter-driver-uds-doip/pyproject.toml
  • python/packages/jumpstarter-driver-uds/pyproject.toml
  • python/packages/jumpstarter-driver-ustreamer/pyproject.toml
  • python/packages/jumpstarter-driver-vnc/pyproject.toml
  • python/packages/jumpstarter-driver-xcp/pyproject.toml
  • python/packages/jumpstarter-driver-yepkit/pyproject.toml
  • python/packages/jumpstarter-imagehash/pyproject.toml
  • python/packages/jumpstarter-kubernetes/pyproject.toml
  • python/packages/jumpstarter-mcp/pyproject.toml
  • python/packages/jumpstarter-protocol/pyproject.toml
  • python/packages/jumpstarter-testing/pyproject.toml
  • python/packages/jumpstarter/pyproject.toml
  • python/pyproject.toml

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/packages/jumpstarter-driver-shell/pyproject.toml`:
- Line 7: Update the Python driver template's pyproject setting: locate the
requires-python entry in python/__templates__/driver/pyproject.toml.tmpl (used
by python/__templates__/create_driver.sh) and change requires-python = ">=3.11"
to requires-python = ">=3.12" so newly generated drivers match existing
packages; verify no other template copies still use ">=3.11".
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 1902221a-cbd7-4ae7-b4ed-a776da798944

📥 Commits

Reviewing files that changed from the base of the PR and between 3282c04 and 9b320c1.

📒 Files selected for processing (65)
  • .github/workflows/python-tests.yaml
  • PR_DESCRIPTION.md
  • python/examples/android-emulator/pyproject.toml
  • python/examples/automotive/pyproject.toml
  • python/examples/soc-pytest/pyproject.toml
  • python/examples/xcp-ecu/pyproject.toml
  • python/packages/hatch-pin-jumpstarter/pyproject.toml
  • python/packages/jumpstarter-all/pyproject.toml
  • python/packages/jumpstarter-cli-admin/pyproject.toml
  • python/packages/jumpstarter-cli-common/pyproject.toml
  • python/packages/jumpstarter-cli-driver/pyproject.toml
  • python/packages/jumpstarter-cli/pyproject.toml
  • python/packages/jumpstarter-driver-adb/pyproject.toml
  • python/packages/jumpstarter-driver-androidemulator/pyproject.toml
  • python/packages/jumpstarter-driver-ble/pyproject.toml
  • python/packages/jumpstarter-driver-can/pyproject.toml
  • python/packages/jumpstarter-driver-composite/pyproject.toml
  • python/packages/jumpstarter-driver-corellium/pyproject.toml
  • python/packages/jumpstarter-driver-doip/pyproject.toml
  • python/packages/jumpstarter-driver-dut-network/pyproject.toml
  • python/packages/jumpstarter-driver-dutlink/pyproject.toml
  • python/packages/jumpstarter-driver-energenie/pyproject.toml
  • python/packages/jumpstarter-driver-esp32/pyproject.toml
  • python/packages/jumpstarter-driver-flashers/pyproject.toml
  • python/packages/jumpstarter-driver-gpiod/pyproject.toml
  • python/packages/jumpstarter-driver-http-power/pyproject.toml
  • python/packages/jumpstarter-driver-http/pyproject.toml
  • python/packages/jumpstarter-driver-iscsi/pyproject.toml
  • python/packages/jumpstarter-driver-mitmproxy/pyproject.toml
  • python/packages/jumpstarter-driver-network/pyproject.toml
  • python/packages/jumpstarter-driver-noyito-relay/pyproject.toml
  • python/packages/jumpstarter-driver-opendal/pyproject.toml
  • python/packages/jumpstarter-driver-pi-pico/pyproject.toml
  • python/packages/jumpstarter-driver-power/pyproject.toml
  • python/packages/jumpstarter-driver-probe-rs/pyproject.toml
  • python/packages/jumpstarter-driver-pyserial/pyproject.toml
  • python/packages/jumpstarter-driver-qemu/pyproject.toml
  • python/packages/jumpstarter-driver-renode/pyproject.toml
  • python/packages/jumpstarter-driver-ridesx/pyproject.toml
  • python/packages/jumpstarter-driver-sdwire/pyproject.toml
  • python/packages/jumpstarter-driver-shell/pyproject.toml
  • python/packages/jumpstarter-driver-snmp/pyproject.toml
  • python/packages/jumpstarter-driver-someip/pyproject.toml
  • python/packages/jumpstarter-driver-ssh-mitm/pyproject.toml
  • python/packages/jumpstarter-driver-ssh-mount/pyproject.toml
  • python/packages/jumpstarter-driver-ssh/pyproject.toml
  • python/packages/jumpstarter-driver-stlink-msd/pyproject.toml
  • python/packages/jumpstarter-driver-tasmota/pyproject.toml
  • python/packages/jumpstarter-driver-tftp/pyproject.toml
  • python/packages/jumpstarter-driver-tmt/pyproject.toml
  • python/packages/jumpstarter-driver-uboot/pyproject.toml
  • python/packages/jumpstarter-driver-uds-can/pyproject.toml
  • python/packages/jumpstarter-driver-uds-doip/pyproject.toml
  • python/packages/jumpstarter-driver-uds/pyproject.toml
  • python/packages/jumpstarter-driver-ustreamer/pyproject.toml
  • python/packages/jumpstarter-driver-vnc/pyproject.toml
  • python/packages/jumpstarter-driver-xcp/pyproject.toml
  • python/packages/jumpstarter-driver-yepkit/pyproject.toml
  • python/packages/jumpstarter-imagehash/pyproject.toml
  • python/packages/jumpstarter-kubernetes/pyproject.toml
  • python/packages/jumpstarter-mcp/pyproject.toml
  • python/packages/jumpstarter-protocol/pyproject.toml
  • python/packages/jumpstarter-testing/pyproject.toml
  • python/packages/jumpstarter/pyproject.toml
  • python/pyproject.toml
✅ Files skipped from review due to trivial changes (11)
  • PR_DESCRIPTION.md
  • python/packages/jumpstarter-driver-someip/pyproject.toml
  • python/examples/android-emulator/pyproject.toml
  • python/packages/hatch-pin-jumpstarter/pyproject.toml
  • python/packages/jumpstarter-protocol/pyproject.toml
  • python/packages/jumpstarter-driver-androidemulator/pyproject.toml
  • python/packages/jumpstarter-driver-xcp/pyproject.toml
  • python/packages/jumpstarter-driver-esp32/pyproject.toml
  • python/packages/jumpstarter-driver-ssh-mount/pyproject.toml
  • python/packages/jumpstarter-driver-composite/pyproject.toml
  • python/packages/jumpstarter-kubernetes/pyproject.toml
🚧 Files skipped from review as they are similar to previous changes (22)
  • python/packages/jumpstarter-driver-vnc/pyproject.toml
  • python/packages/jumpstarter-driver-noyito-relay/pyproject.toml
  • python/packages/jumpstarter-cli-admin/pyproject.toml
  • python/packages/jumpstarter-cli-common/pyproject.toml
  • python/packages/jumpstarter-driver-network/pyproject.toml
  • python/packages/jumpstarter-driver-mitmproxy/pyproject.toml
  • python/packages/jumpstarter-driver-ustreamer/pyproject.toml
  • python/packages/jumpstarter-driver-corellium/pyproject.toml
  • python/examples/automotive/pyproject.toml
  • python/packages/jumpstarter/pyproject.toml
  • python/packages/jumpstarter-driver-ridesx/pyproject.toml
  • python/packages/jumpstarter-driver-flashers/pyproject.toml
  • python/packages/jumpstarter-driver-ssh/pyproject.toml
  • python/packages/jumpstarter-driver-uds-can/pyproject.toml
  • python/packages/jumpstarter-driver-renode/pyproject.toml
  • python/packages/jumpstarter-cli-driver/pyproject.toml
  • python/packages/jumpstarter-driver-uds/pyproject.toml
  • python/packages/jumpstarter-testing/pyproject.toml
  • python/packages/jumpstarter-driver-doip/pyproject.toml
  • python/packages/jumpstarter-driver-tasmota/pyproject.toml
  • python/examples/soc-pytest/pyproject.toml
  • python/packages/jumpstarter-driver-pyserial/pyproject.toml

Comment thread python/packages/jumpstarter-driver-shell/pyproject.toml Outdated
Comment thread PR_DESCRIPTION.md Outdated

@mangelajo mangelajo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a couple of tiny comments. Thanks @jtligon !

Comment thread python/packages/jumpstarter-driver-opendal/pyproject.toml Outdated
Comment thread python/packages/jumpstarter-driver-shell/pyproject.toml Outdated
Comment thread PR_DESCRIPTION.md Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/packages/jumpstarter-driver-opendal/pyproject.toml`:
- Line 18: Add the missing entry point group for the driver so the Opendal
driver is discoverable: in the pyproject.toml for the jumpstarter-driver-opendal
package, add a [project.entry-points."jumpstarter.drivers"] section that
registers the Opendal driver class (the concrete driver symbol used in this
package) alongside or in addition to the existing
[project.entry-points."jumpstarter.adapters"] registration; alternatively, if
the package intentionally does not expose a driver, add a short comment or
README note in the package explaining why no
[project.entry-points."jumpstarter.drivers"] entry exists so maintainers know
this is deliberate.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f56550fc-29ec-425d-9816-eec8940497ea

📥 Commits

Reviewing files that changed from the base of the PR and between 9b320c1 and cb5f521.

📒 Files selected for processing (2)
  • python/__templates__/driver/pyproject.toml.tmpl
  • python/packages/jumpstarter-driver-opendal/pyproject.toml
✅ Files skipped from review due to trivial changes (1)
  • python/templates/driver/pyproject.toml.tmpl

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/packages/jumpstarter-driver-opendal/pyproject.toml`:
- Line 18: Add the missing entry point group for the driver so the Opendal
driver is discoverable: in the pyproject.toml for the jumpstarter-driver-opendal
package, add a [project.entry-points."jumpstarter.drivers"] section that
registers the Opendal driver class (the concrete driver symbol used in this
package) alongside or in addition to the existing
[project.entry-points."jumpstarter.adapters"] registration; alternatively, if
the package intentionally does not expose a driver, add a short comment or
README note in the package explaining why no
[project.entry-points."jumpstarter.drivers"] entry exists so maintainers know
this is deliberate.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f56550fc-29ec-425d-9816-eec8940497ea

📥 Commits

Reviewing files that changed from the base of the PR and between 9b320c1 and cb5f521.

📒 Files selected for processing (2)
  • python/__templates__/driver/pyproject.toml.tmpl
  • python/packages/jumpstarter-driver-opendal/pyproject.toml
✅ Files skipped from review due to trivial changes (1)
  • python/templates/driver/pyproject.toml.tmpl
🛑 Comments failed to post (1)
python/packages/jumpstarter-driver-opendal/pyproject.toml (1)

18-18: ⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Check entry point registration patterns across all driver packages

# Find all driver package pyproject.toml files and show their entry points
fd -t f 'pyproject.toml' 'python/packages/jumpstarter-driver-*' --exec sh -c '
  echo "=== {} ==="
  rg -A2 "project.entry-points" {} || echo "No entry-points found"
  echo
'

Repository: jumpstarter-dev/jumpstarter

Length of output: 195


Fix missing jumpstarter.drivers entry point in jumpstarter-driver-opendal

  • python/packages/jumpstarter-driver-opendal/pyproject.toml (line 18) registers only [project.entry-points."jumpstarter.adapters"] and has no corresponding [project.entry-points."jumpstarter.drivers"], despite this being a driver package.
  • Either add the missing jumpstarter.drivers entry point so the Opendal driver is discoverable (while keeping adapter registration if needed), or explicitly document why this package intentionally does not use jumpstarter.drivers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/packages/jumpstarter-driver-opendal/pyproject.toml` at line 18, Add
the missing entry point group for the driver so the Opendal driver is
discoverable: in the pyproject.toml for the jumpstarter-driver-opendal package,
add a [project.entry-points."jumpstarter.drivers"] section that registers the
Opendal driver class (the concrete driver symbol used in this package) alongside
or in addition to the existing [project.entry-points."jumpstarter.adapters"]
registration; alternatively, if the package intentionally does not expose a
driver, add a short comment or README note in the package explaining why no
[project.entry-points."jumpstarter.drivers"] entry exists so maintainers know
this is deliberate.

Source: Coding guidelines

@jtligon
jtligon requested a review from mangelajo June 9, 2026 12:42
@jtligon
jtligon force-pushed the upgrade-python-3.14 branch from cb5f521 to d114931 Compare June 10, 2026 12:33
@mangelajo
mangelajo enabled auto-merge (squash) June 10, 2026 13:38
@jtligon

jtligon commented Jun 11, 2026

Copy link
Copy Markdown
Author

Pillow Build Failure on Python 3.14

The CI is failing because Pillow doesn't have pre-built wheels for Python 3.14 yet, so it's trying to build from source but the runners are missing required system libraries.

Proposed Solution: Install System Dependencies

Add the required build dependencies to the CI workflow for Python 3.14:

Ubuntu:

- name: Install system dependencies for Pillow (Python 3.14)
  if: matrix.python-version == '3.14'
  run: |
    sudo apt-get update
    sudo apt-get install -y libjpeg-dev zlib1g-dev libtiff-dev libfreetype6-dev liblcms2-dev libwebp-dev libopenjp2-7-dev

macOS:

- name: Install system dependencies for Pillow (Python 3.14)
  if: matrix.python-version == '3.14'
  run: |
    brew install libjpeg libtiff little-cms2 webp

This is the most robust approach and ensures Pillow builds correctly from source until official wheels are available.

Error from CI:

The headers or library files could not be found for jpeg,
a required dependency when compiling Pillow from source.

@mangelajo

Copy link
Copy Markdown
Member

Pillow Build Failure on Python 3.14

The CI is failing because Pillow doesn't have pre-built wheels for Python 3.14 yet, so it's trying to build from source but the runners are missing required system libraries.

Proposed Solution: Install System Dependencies

Add the required build dependencies to the CI workflow for Python 3.14:

Ubuntu:

- name: Install system dependencies for Pillow (Python 3.14)
  if: matrix.python-version == '3.14'
  run: |
    sudo apt-get update
    sudo apt-get install -y libjpeg-dev zlib1g-dev libtiff-dev libfreetype6-dev liblcms2-dev libwebp-dev libopenjp2-7-dev

macOS:

- name: Install system dependencies for Pillow (Python 3.14)
  if: matrix.python-version == '3.14'
  run: |
    brew install libjpeg libtiff little-cms2 webp

This is the most robust approach and ensures Pillow builds correctly from source until official wheels are available.

Error from CI:

The headers or library files could not be found for jpeg,
a required dependency when compiling Pillow from source.

sound good, doing it as part of this PR should work.

@mangelajo
mangelajo self-requested a review June 15, 2026 07:34
auto-merge was automatically disabled June 15, 2026 13:22

Head branch was pushed to by a user without write access

@jtligon

jtligon commented Jun 15, 2026

Copy link
Copy Markdown
Author

Checking CI with scipy build dependencies added

@jtligon

jtligon commented Jun 15, 2026

Copy link
Copy Markdown
Author

Build Dependencies Added

Added system build dependencies for Python 3.14 to both test workflows:

Python Tests (python-tests.yaml)

  • Ubuntu & macOS (conditional on Python 3.14):
    • Pillow: libjpeg-dev, zlib1g-dev, libtiff-dev, libfreetype6-dev, liblcms2-dev, libwebp-dev, libopenjp2-7-dev
    • scipy: gfortran, libopenblas-dev, liblapack-dev, pkg-config (Ubuntu) / gcc, openblas (macOS)

E2E Tests (e2e.yaml)

  • All jobs using .py-version (which is now 3.14):
    • e2e-tests
    • e2e-compat-old-controller
    • e2e-compat-old-client
    • Same Pillow + scipy dependencies as above

Once a maintainer approves the workflow runs, CI should pass. Both Pillow and scipy currently lack Python 3.14 wheels, so they must build from source.

Tracking: Issue #795 and draft PR #796 are set up to remove these dependencies once wheels become available.

@mangelajo
mangelajo enabled auto-merge (squash) June 16, 2026 07:55
@mangelajo

Copy link
Copy Markdown
Member

Comment thread .py-version Outdated
@mangelajo mangelajo added this to the 0.9.0 milestone Jun 23, 2026
@jtligon

jtligon commented Jun 23, 2026

Copy link
Copy Markdown
Author

macOS gfortran fix applied

Added a symlink to make gfortran discoverable by scipy's meson build on macOS. Homebrew's gcc provides version-specific binaries like gfortran-14, but scipy expects gfortran.

Latest commit: 435565b

Note: macOS 3.12 test failure

The macOS 3.12 job is failing on test_drain_exits_when_deadline_exceeded_before_select - this is a flaky timing test unrelated to the Python 3.14 upgrade. The same test passes on macOS 3.13 and Ubuntu 3.12/3.13/3.14.

@jtligon
jtligon force-pushed the upgrade-python-3.14 branch 3 times, most recently from 6e823e2 to 9be0da5 Compare June 23, 2026 15:24
Comment thread python/packages/jumpstarter/jumpstarter/exporter/hooks_test.py Outdated

@mangelajo mangelajo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jtligon

jtligon commented Jul 8, 2026

Copy link
Copy Markdown
Author

@mangelajo 0.9.0 is out, what did we need to figure out here? is the mac still failing somewhere?

@mangelajo

Copy link
Copy Markdown
Member

@mangelajo 0.9.0 is out, what did we need to figure out here? is the mac still failing somewhere?

At this point, if you can rebase the PR on main, and let's see if it keeps crashing

@jtligon
jtligon force-pushed the upgrade-python-3.14 branch from 7f2967a to c6f48ca Compare July 8, 2026 15:14
@mangelajo
mangelajo self-requested a review July 8, 2026 15:23
@jtligon

jtligon commented Jul 8, 2026

Copy link
Copy Markdown
Author

Python 3.14 CI Status

Fixed:

  • ✅ Added build dependencies for Pillow, scipy, cffi to all workflows
  • ✅ Added UV_LOCK_TIMEOUT for source builds
  • ✅ Fixed .py-version format (3.14.6 required by e2e scripts)

Remaining failure:

  • jumpstarter-driver-snmp tests fail with RuntimeError: There is no current event loop in thread 'MainThread'

This is a Python 3.14 asyncio behavior change. Python 3.14 requires explicit event loop creation - asyncio.get_event_loop() no longer automatically creates one in sync contexts.

The SNMP driver code needs to be updated to handle Python 3.14's stricter asyncio requirements.

@mangelajo

Copy link
Copy Markdown
Member

Python 3.14 CI Status

Fixed:

  • ✅ Added build dependencies for Pillow, scipy, cffi to all workflows
  • ✅ Added UV_LOCK_TIMEOUT for source builds
  • ✅ Fixed .py-version format (3.14.6 required by e2e scripts)

Remaining failure:

  • jumpstarter-driver-snmp tests fail with RuntimeError: There is no current event loop in thread 'MainThread'

This is a Python 3.14 asyncio behavior change. Python 3.14 requires explicit event loop creation - asyncio.get_event_loop() no longer automatically creates one in sync contexts.

The SNMP driver code needs to be updated to handle Python 3.14's stricter asyncio requirements.

please prompt your agent to fix that and to look for other instances of asyncio.get_event_loop() to verify if a similar fix could be necessary.

@jtligon

jtligon commented Jul 9, 2026

Copy link
Copy Markdown
Author

Python 3.14 asyncio Compatibility Fix

Fixed Issues

jumpstarter-driver-snmp: Made on() and off() methods async to fix Python 3.14 compatibility.

Root cause: Python 3.14 requires explicit event loop management:

  • asyncio.Event() can only be created when an event loop exists
  • asyncio.get_event_loop() no longer auto-creates loops in sync contexts
  • asyncio.get_running_loop() must be used in async contexts

Solution: Converted the driver to use async methods, which is an established pattern in Jumpstarter (http, mitmproxy, ble drivers all use async).

Breaking change: Callers must now await server.on() and await server.off(). However, this aligns with other async drivers in the project.

Potential Future Issues

jumpstarter-driver-tftp creates asyncio.Event() in __init__ (sync context). Currently works because it sets the event loop first, but this is fragile. May need similar async conversion in the future if Python becomes stricter.

jumpstarter-driver-shell uses asyncio.get_event_loop().time() but only in async contexts, so it's safe.

self.logger.info(f"Sending power {state.name} command to {self.host}")
loop, created_loop = self._setup_event_loop()

# Get the running event loop (Python 3.14+ requires this)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we will need to test this in the lab exporters to make sure it doesn't break or verify there's good coverage in testing @bennyz WDYT?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, i can do a test next week

@mangelajo

Copy link
Copy Markdown
Member

I checked if there are more places that needed update. We had kind of a mess, I am proposing this one https://github.com/jtligon/jumpstarter/pull/1/changes on top of this PR, if you approve/merge it it will be merged in your PR.

@mangelajo

Copy link
Copy Markdown
Member
image

@mangelajo

Copy link
Copy Markdown
Member
image

@jtligon

jtligon commented Jul 10, 2026

Copy link
Copy Markdown
Author

✅ Merged @mangelajo's additional asyncio fixes (PR #1 to my fork):

  • Shell driver: Replaced deprecated get_event_loop() with get_running_loop()
  • TFTP driver: Replaced manual event loop management with asyncio.run() and moved TftpServer construction into async context (fixes the Event() creation issue)
  • Mitmproxy example: Replaced deprecated ensure_future() with create_task()

All Python 3.14 asyncio compatibility issues in our code are now fixed. The remaining CI failures are third-party library issues (beartype/sphinx and pydantic-core) that we cannot fix.

jtligon and others added 11 commits July 20, 2026 10:49
Upgrade project Python version from 3.11/3.12 to 3.14.

Changes:
- Update .py-version to 3.14
- Update all requires-python fields to >=3.14 across all packages
- Update ruff target-version to py314
- Update CI to test with Python 3.14
- Update uv.lock with Python 3.14 compatible dependencies

This required upgrading pydantic-core to 2.46.4 which has Python 3.14
support via prebuilt wheels.

Note: Building with Python 3.14 requires:
- PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 for Rust extensions
- OpenBLAS (macOS: brew install openblas)
- gfortran (macOS: brew install gcc)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add missing Apache-2.0 license to jumpstarter-driver-iscsi
- Remove duplicate pytest-asyncio dependency in jumpstarter-driver-http
- Fix jumpstarter-driver-opendal entry point group from adapters to drivers

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Revert jumpstarter-driver-opendal entry point back to jumpstarter.adapters
- Update driver template to use Python 3.12+ (was 3.11)
- Remove PR_DESCRIPTION.md file

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The .py-version file was updated to 3.14, so e2e tests also need
build dependencies for Pillow and scipy since they don't have
pre-built wheels for Python 3.14 yet.

Added the same dependency installation steps to all e2e jobs that
use .py-version:
- e2e-tests
- e2e-compat-old-controller
- e2e-compat-old-client

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This empty commit triggers CI to test the Python 3.14 build dependencies.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Pillow, scipy, and cffi need system libraries when building from source for Python 3.14.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add build dependencies to type-check-python workflow
- Fix .py-version to use full version 3.14.6 (e2e scripts require X.Y.Z format)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Python 3.14 requires explicit event loop management and doesn't allow
creating asyncio.Event() or calling get_event_loop() in sync contexts.

Changes:
- Made on() and off() methods async
- Simplified _snmp_set() to be async and use get_running_loop()
- Removed manual event loop creation/management code
- Updated all tests to use pytest.mark.asyncio
- Removed redundant event loop mocking in tests

This is a breaking API change - callers must now await on()/off().
However, many other Jumpstarter drivers already use async methods
(http, mitmproxy, ble, etc.), so this pattern is established.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Remove unused Tuple import
- Remove unused loop variable assignment

Fixes ruff linting errors.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…ompatibility

- TFTP driver: replace new_event_loop()/set_event_loop()/run_until_complete()
  with asyncio.run(), and move TftpServer construction into async context
  so asyncio.Event() objects are created with a running loop (required by 3.14)
- Shell driver: replace get_event_loop() with get_running_loop() (called
  from async context, but get_event_loop() is deprecated since 3.10)
- mitmproxy example: replace ensure_future() with create_task()
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jtligon
jtligon force-pushed the upgrade-python-3.14 branch from 798cdd6 to 0e611c1 Compare July 20, 2026 15:38
@jtligon

jtligon commented Jul 20, 2026

Copy link
Copy Markdown
Author

Rebase & status update

Rebased on main — no conflicts. The branch went from 50 commits down to 11 (the rest were already merged).

Test results after rebase

  • Lint: passes
  • Core jumpstarter tests (584 tests): all pass on Python 3.14
  • Driver package tests: all pass except jumpstarter-driver-http

jumpstarter-driver-http failures (pre-existing, not related to this PR)

10 of 16 tests fail because HttpServer defaults host=None, which calls get_ip_address()socket.gethostbyname(hostname). On machines with Tailscale (or similar VPN), this resolves to the VPN interface IP (e.g. 100.x.x.x) instead of a locally-bindable address, and aiohttp.TCPSite throws OSError: could not bind.

Verified this fails identically on main — it's not a regression from this PR.

The one test that explicitly passes host="127.0.0.1" (test_driver_close_with_running_loop) passes fine. The fix would be to use 127.0.0.1 in test fixtures, but that's a separate issue.

maboras-rh pushed a commit to maboras-rh/jumpstarter that referenced this pull request Jul 22, 2026
)

## Summary

Audited the entire codebase for deprecated asyncio event loop patterns
that emit `DeprecationWarning` on Python 3.12/3.13 and will break on
Python 3.14. Found and fixed three locations (the SNMP driver had the
same issue and was already fixed in jumpstarter-dev#775).

## Compatibility matrix

| Deprecated pattern | 3.12 | 3.13 | 3.14 | Replacement (since 3.7) |
|---|---|---|---|---|
| `get_event_loop()` (no running loop) | DeprecationWarning |
DeprecationWarning | **RuntimeError** | `get_running_loop()` |
| `set_event_loop()` | DeprecationWarning | DeprecationWarning |
**No-op** | `asyncio.run()` |
| `asyncio.Event()` (no running loop) | DeprecationWarning |
DeprecationWarning | **RuntimeError** | Create inside `async def` |
| `ensure_future()` | DeprecationWarning | DeprecationWarning |
DeprecationWarning | `create_task()` |

## Changes

### 1. TFTP driver (HIGH — will crash on 3.14)

`jumpstarter-driver-tftp/jumpstarter_driver_tftp/driver.py`

`_start_server()` ran in a thread using `new_event_loop()` +
`set_event_loop()` + `run_until_complete()`, and constructed
`TftpServer` (which creates `asyncio.Event()` objects) before the loop
was running.

Replaced with `asyncio.run()` and moved `TftpServer` construction into
an async method so `asyncio.Event()` objects are created with a running
loop.

### 2. Shell driver (MEDIUM — deprecation warnings on 3.12/3.13)

`jumpstarter-driver-shell/jumpstarter_driver_shell/driver.py`

Replaced `asyncio.get_event_loop().time()` with
`asyncio.get_running_loop().time()` (called from async context, so
trivial swap).

### 3. mitmproxy example (LOW — deprecated but functional)

`jumpstarter-driver-mitmproxy/examples/addons/data_stream_websocket.py`

Replaced `asyncio.ensure_future()` with `asyncio.create_task()`.

## Audit scope

Searched all Python packages for `get_event_loop`, `new_event_loop`,
`set_event_loop`, `run_until_complete`, `ensure_future`, and
`asyncio.Event()`/`Queue()`/`Lock()` created outside async contexts. The
rest of the codebase already uses the correct modern APIs.
@mangelajo

Copy link
Copy Markdown
Member

hey @jtligon sorry to keep you hanging in there.

We addressed some of the asyncio warnings/issues with compatibiity. Can you rebase on top and clear the conflicts? then I think we're ready to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants