Skip to content

[cpyrt] Spell out the CPython names behind the Python 2 aliases - #128

Merged
guitargeek merged 1 commit into
compiler-research:mainfrom
guitargeek:python2_aliases
Oct 1, 2026
Merged

guitargeek merged 1 commit into
compiler-research:mainfrom
guitargeek:python2_aliases

Conversation

@guitargeek

@guitargeek guitargeek commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

cpyrt.h still carried the cpyrt_PyText_*, PyInt_*, cpyrt_PyCapsule_* and CPPJIT__*__ aliases that let the sources build against both Python 2 and 3. With Python >= 3.12 required they are plain renames of the PyUnicode_, PyLong_ and PyCapsule_* APIs and the Python 3 dunder names, so use those directly and drop the aliases, together with the unused cpyrt_PyText_GetSize, PyIntObject, PyBuffer_Type and dict_lookup_func.

cpyrt_PyText_AsStringAndSize stays: unlike PyUnicode_AsUTF8AndSize it also accepts bytes.

The preprocessed sources are unchanged except in two places. PyObjectDir27.inc tested PyType_Check(obj) || PyClass_Check(obj), which collapses to PyType_Check(obj). CPPJIT_INITIALIZE_STRING stringifies its argument without expanding it, so gDiv held "CPPJIT__div__" rather than "__truediv__"; it now holds "__truediv__". Nothing reads gDiv.

cpyrt.h still carried the cpyrt_PyText_*, PyInt_*, cpyrt_PyCapsule_*
and CPPJIT__*__ aliases that let the sources build against both Python
2 and 3. With Python >= 3.12 required they are plain renames of the
PyUnicode_*, PyLong_* and PyCapsule_* APIs and the Python 3 dunder
names, so use those directly and drop the aliases, together with the
unused cpyrt_PyText_GetSize, PyIntObject, PyBuffer_Type and
dict_lookup_func.

cpyrt_PyText_AsStringAndSize stays: unlike PyUnicode_AsUTF8AndSize it
also accepts bytes.

The preprocessed sources are unchanged except in two places.
PyObjectDir27.inc tested PyType_Check(obj) || PyClass_Check(obj), which
collapses to PyType_Check(obj). CPPJIT_INITIALIZE_STRING stringifies its
argument without expanding it, so gDiv held "CPPJIT__div__" rather than
"__truediv__"; it now holds "__truediv__". Nothing reads gDiv.

@aaronj0 aaronj0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see that we no longer need the cpyrt_* helpers which were quite annoying when writing CPython extension code. Can you take a look at the concern on PyInt_FromLong -> PyLong_FromLong call sites. Otherwise looks good.

Comment thread src/cpyrt/Converters.cxx
// special case to be used with arrays: return a Python int instead of str
// (following the same convention as module array.array)
return PyInt_FromLong((long)*((signed char*)address));
return PyLong_FromLong((long)*((signed char*)address));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this special case is supposed to retain the return as PyInt: https://docs.python.org/3/library/array.html

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was literally defined to be the same in cpyrt.h:
#define PyInt_FromLong PyLong_FromLong

So where is the concern exactly? Maybe you're hinting at something else that is not directly related to this change?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wow, that looks wrong.. Then yes it's not directly related to your change. Was PyInt_FromLong only in Python2? It just looked like a functional change

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes correct, with Python 3 all the PyInt_* functions got replaced by PyLong_* equivalents.

Comment thread src/cpyrt/Converters.cxx
// special case to be used with arrays: return a Python int instead of str
// (following the same convention as module array.array)
return PyInt_FromLong((long)*((unsigned char*)address));
return PyLong_FromLong((long)*((unsigned char*)address));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

Comment thread src/cpyrt/cpyrt.h
#define cpyrt_PyText_AsString PyUnicode_AsUTF8
#define cpyrt_PyText_AsStringChecked PyUnicode_AsUTF8
#define cpyrt_PyText_GetSize PyUnicode_GetSize
#define cpyrt_PyText_GET_SIZE PyUnicode_GET_LENGTH

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good quality of life improvement :)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For human and machine 🙂

@aaronj0 aaronj0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I'd also annotate [NFC] in the commit message when merging

@guitargeek
guitargeek merged commit 4ba4a0d into compiler-research:main Oct 1, 2026
25 checks passed
@guitargeek
guitargeek deleted the python2_aliases branch October 1, 2026 12:07
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

CppJIT CI has detected a new failure on workflow Test running on main while building 4ba4a0d at job "macos-26-intel/llvm22/py3.12/c++20 / run".

Full details are available at: https://github.com/compiler-research/cppjit/actions/runs/36859698686
Here is the relevant piece of the build log for the reference

Step "Build and test" failure:
  Python2_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.12.10/x64
  Python3_ROOT_DIR: /Users/runner/hostedtoolcache/Python/3.12.10/x64
  LLVM_DIR: /Users/runner/work/cppjit/cppjit/install/lib/cmake/llvm
  Clang_DIR: /Users/runner/work/cppjit/cppjit/install/lib/cmake/clang
##[endgroup]
(node:15118) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
With the provided path, there will be 1 file uploaded
Artifact name is valid!
Root directory input is valid!
(node:15118) [DEP0169] DeprecationWarning: `url.parse()` behavior is not standardized and prone to errors that have security implications. Use the WHATWG URL API instead. CVEs are not issued for `url.parse()` vulnerabilities.
Beginning upload of artifact content to blob storage
Uploaded bytes 224
Finished uploading artifact content to blob storage!
SHA256 digest of uploaded artifact zip is 67356d31449782477d22b87cc119ec2dee3e15e77d23ce0ba4a716c300c54bc0
Finalizing artifact upload
Artifact result-macos-26-intel-llvm22-py3.12-cxx20.zip successfully finalized. Artifact ID 11161292468
Artifact result-macos-26-intel-llvm22-py3.12-cxx20 has been successfully uploaded! Final size is 224 bytes. Artifact ID is 11161292468
Artifact download URL: https://github.com/compiler-research/cppjit/actions/runs/36859698686/artifacts/11161292468
Post job cleanup.
Post job cleanup.
[command]/usr/local/bin/git version
git version 2.55.0
Copying '/Users/runner/.gitconfig' to '/Users/runner/work/_temp/daef6b3f-e602-4aba-b8c0-9f43a9351f73/.gitconfig'
Temporarily overriding HOME='/Users/runner/work/_temp/daef6b3f-e602-4aba-b8c0-9f43a9351f73' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/usr/local/bin/git config --global --add safe.directory /Users/runner/work/cppjit/cppjit
[command]/usr/local/bin/git config --local --name-only --get-regexp core\.sshCommand
[command]/usr/local/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
[command]/usr/local/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
http.https://github.com/.extraheader
[command]/usr/local/bin/git config --local --unset-all http.https://github.com/.extraheader
[command]/usr/local/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
[command]/usr/local/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir:
[command]/usr/local/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
Evaluate and set job outputs
Set output 'summary'
Cleaning up orphan processes
##[warning]Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v4, actions/setup-python@v5, actions/upload-artifact@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/

@vgvassilev

Copy link
Copy Markdown
Contributor

Not arguing against this change. The advantage of the wrappers could be that we have a central place to check for errors of different CRuntime operations and have better diagnostics or ignore logic.

@guitargeek

Copy link
Copy Markdown
Collaborator Author

Sure, that's always the advantages of wrappers in general. But that's a whole different topic from having isolated aliases for a random tiny subset from the CPtthon API just because of Python 2 compatibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants