Skip to content

Add Python wrapper package#47

Open
gustavorubim wants to merge 1 commit into
astroautomata:mainfrom
gustavorubim:python-wrapper-for-pysr
Open

Add Python wrapper package#47
gustavorubim wants to merge 1 commit into
astroautomata:mainfrom
gustavorubim:python-wrapper-for-pysr

Conversation

@gustavorubim

@gustavorubim gustavorubim commented Jun 28, 2026

Copy link
Copy Markdown

Summary

  • Add a symbolic_regression_py PyO3/maturin package exposing symbolic_regression_rs.search(...).
  • Rename the Python distribution to pysr-rust-backend while keeping the import module symbolic_regression_rs for the PySR backend boundary.
  • Keep the Python boundary deliberately narrow: dense float32/float64 NumPy arrays plus plain Python dict/list options in, hall-of-fame dict output out.
  • Add wrapper CI that installs, tests, builds, and smoke-tests wheel and sdist artifacts.
  • Add a manual publish workflow for TestPyPI/PyPI wheels and sdist, with per-wheel smoke tests before upload.
  • Document the wrapper package in the workspace README and package README.

Context

This is the wrapper-first step for exposing the Rust engine to PySR as discussed in astroautomata/PySR#1197. Once this package is accepted and published, PySR can depend on a normal pysr-rust-backend package from its optional Rust extra instead of vendoring the Rust workspace.

Follow-up to review comments

  • Propagate the seed option into Options::seed.
  • Reject unknown option keys and unknown nested mutation_weights keys instead of silently ignoring them.
  • Validate operator arity from the operators={arity: [names]} mapping.
  • Document the wrapper as an experimental PySR backend wire package rather than a stable standalone Python API.
  • Document unsupported feature coverage explicitly.
  • Smoke-test publish workflow wheels after build and before upload.

Related PR

Validation

  • cargo fmt --check -p symbolic_regression_rs_py
  • cargo check -p symbolic_regression_rs_py
  • cargo test
  • python -m pytest symbolic_regression_py/tests -q: 12 passed
  • Parsed .github/workflows/ci-python-wrapper.yml and .github/workflows/publish-python-wrapper.yml with PyYAML

@MilesCranmer

Copy link
Copy Markdown
Member

@MilesCranmerBot please review this PR. In particular it would be useful to know if you think this is the right design. Also please check feature coverage of this.

@gustavorubim thanks for your effort on this. Small initial comment is we probably want a different name for the PyPI package to indicate it is the rust backend.

@MilesCranmerBot MilesCranmerBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together — I think the wrapper-first direction is the right one for getting PySR onto a normal optional dependency instead of vendoring the Rust workspace. The narrow PyO3 boundary also seems sensible for an experimental backend.

That said, I would avoid publishing this API as stable until a few contract issues are fixed:

  1. Blocker: seed is accepted/documented but ignored. symbolic_regression_py/src/lib.rs maps many option keys in apply_options, but never assigns options.seed, even though the README/tests pass "seed": 0 and the Rust search uses Options::seed for RNG initialization. This will silently break reproducibility.

  2. Blocker: unknown options are silently ignored. Because apply_options only reads a whitelist, typos or PySR-forwarded-but-unsupported options become no-ops. For this boundary I’d strongly prefer a validated wire schema: either reject every unknown key, or report unsupported keys explicitly. Ideally this should reuse/generate from the existing Rust options spec / WasmOptionsShim pattern rather than maintaining a second manual mapping.

  3. Operator API shape is misleading. The Python API accepts operators={2: [...]}, but the Rust code ignores the dict keys and flattens all values. Either take a flat list, or validate arity and expose unary_operators/binary_operators-style fields. As written, {1: ["+"]} and {2: ["+"]} behave the same.

  4. Feature coverage is fine for an MVP, but should be explicit. The current wrapper covers dense float32/float64, single-output, builtin operators, scalar early-stop threshold, and a lot of scalar search options. Major PySR/Rust-engine features still missing include sample weights, non-MSE loss selection, per-variable/per-operator complexities, constraints/nested constraints, custom operators/losses, warm starts, and richer output. I’d call those unsupported loudly rather than allowing them through the options dict.

  5. Package/API naming should be settled before release. symbolic-regression-rs as the PyPI distribution does signal “Rust backend” reasonably well, but the distribution/import/crate/directory names are all slightly different. If the package is intended as a public standalone Python API, I’d add defaults and a small typed result layer. If it is mainly a PySR wire backend, I’d document the instability/private nature more explicitly.

  6. Release workflow: CI tests Ubuntu editable/wheel/sdist builds, but the publish workflow builds Windows/macOS/Linux wheels without smoke-testing each artifact before upload. I’d add at least an import + tiny search smoke test after each wheel build before publishing.

Overall: design direction 👍, but I’d request changes before merge/release for at least (1) seed propagation, (2) unknown-option rejection, and (3) a clearer operator/options contract. For feature coverage, the current subset is OK as a staged MVP if PySR’s adapter rejects unsupported features up front.

Local checks I ran:

  • cargo fmt --check -p symbolic_regression_rs_py
  • cargo check -p symbolic_regression_rs_py
  • editable install into a fresh uv venv + python -m unittest discover symbolic_regression_py/tests

Upstream check observed: pre-commit.ci is passing.

@gustavorubim

Copy link
Copy Markdown
Author

Updated this PR in response to the package naming and contract feedback:

  • Renamed the distribution to pysr-rust-backend while keeping the import module as symbolic_regression_rs.
  • Propagated seed into Options::seed.
  • Added strict rejection for unknown option keys and nested mutation_weights keys.
  • Added operator arity validation for the operators={arity: [names]} mapping.
  • Documented the current feature coverage and unsupported features explicitly.
  • Added publish-workflow smoke tests for built wheels before upload.

Validation rerun:

  • cargo fmt --check -p symbolic_regression_rs_py
  • cargo check -p symbolic_regression_rs_py
  • cargo test
  • python -m pytest symbolic_regression_py/tests -q: 12 passed
  • Workflow YAML parse check

I also opened the in-PySR connector comparison draft here: astroautomata/PySR#1251

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants