Skip to content

gh-152666: Avoid reference counting of code objects#152955

Open
brijkapadia wants to merge 8 commits into
python:mainfrom
brijkapadia:code-obj
Open

gh-152666: Avoid reference counting of code objects#152955
brijkapadia wants to merge 8 commits into
python:mainfrom
brijkapadia:code-obj

Conversation

@brijkapadia

@brijkapadia brijkapadia commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Functions keep a list of past code objects so that a frame only needs borrowed references to them.

@brijkapadia brijkapadia marked this pull request as ready for review July 4, 2026 17:06
@brijkapadia brijkapadia requested a review from markshannon as a code owner July 4, 2026 17:06

@markshannon markshannon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation of the old codes list looks good.
I'm wondering if the behavior, not the implementation, could be tested (or is it already?)

Also, can you make f_executable a PyObject * and always borrow it, to minimize overhead?

struct _PyInterpreterFrame {
_PyStackRef f_executable; /* Borrowed reference (code object or None) */
/* Borrowed reference (code object or None) */
/* Strong reference for generators */

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a borrowed reference for generators as well.
The function object still keeps the code object alive, regardless of whether it is a generator function or not.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if there is a trivial way for a generator to hold onto a borrowed reference since for a generator g, you can always access the code object by doing g.gi_code. The function can get freed but the code object is still needed. See, for example, this test:

def test_generator_resurrect(self):
# Test that a resurrected generator still has a valid gi_code
resurrected = []
# Resurrect a generator in a finalizer
exec(textwrap.dedent("""
def gen():
try:
yield
except:
resurrected.append(g)
g = gen()
next(g)
"""), {"resurrected": resurrected})
support.gc_collect()
self.assertEqual(len(resurrected), 1)
self.assertIsInstance(resurrected[0].gi_code, types.CodeType)

Comment thread Include/internal/pycore_interpframe_structs.h Outdated
def h():
pass

old_codes = _testcapi.function_get_old_codes(f)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old codes list is an implementation detail.
Would it be possible to check the behavior rather than the implementation?

Specifically, test that:

  • reassigning the code object for a running function still works
  • reassigning many code objects doesn't leak memory

@bedevere-app

bedevere-app Bot commented Jul 6, 2026

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants