Skip to content

gh-162273: Acquire strong reference to fillvalue upfront in itertools.zip_longest - #162274

Closed
espressolee wants to merge 1 commit into
python:mainfrom
espressolee:gh-162273-zip-longest-fillvalue
Closed

espressolee wants to merge 1 commit into
python:mainfrom
espressolee:gh-162273-zip-longest-fillvalue

Conversation

@espressolee

Copy link
Copy Markdown
Contributor

zip_longest_new() in Modules/itertoolsmodule.c previously obtained fillvalue from kwds as a borrowed reference via PyDict_GetItemWithError() and kept the raw pointer on the C stack across PyTuple_New(), PyObject_GetIter() on each input iterable, and type->tp_alloc() before calling Py_NewRef(fillvalue).

If kwds is cleared or mutated while PyObject_GetIter() runs (for example when zip_longest is called through a PyObject_Call() path that forwards kwargs without copying, such as _thread.start_new_thread(), or under concurrent mutation), fillvalue can be deallocated before lz->fillvalue = Py_NewRef(fillvalue) is reached.

Acquire a strong reference upfront using PyDict_GetItemRef() (or Py_NewRef(Py_None) when no keyword argument is provided), release it with Py_DECREF(fillvalue) on error paths, and transfer ownership directly into lz->fillvalue.

…rtools.zip_longest

zip_longest_new() borrowed fillvalue from kwds via PyDict_GetItemWithError() and only promoted it with Py_NewRef() after calling PyObject_GetIter() on input iterables. Use PyDict_GetItemRef() upfront and decref fillvalue on error paths so clearing or mutating kwds during iterator initialization cannot free fillvalue before ownership is stored on the zip_longest object.
@eendebakpt

Copy link
Copy Markdown
Contributor

The issue is addressed in #156384 which is a more fundamental solution (the kwds argument should not mutate at all)

@espressolee

Copy link
Copy Markdown
Contributor Author

Thanks @eendebakpt!

I built the branch from #156384 (tested at commit 62f89d3) and ran the reentrancy reproducer against it. The reproducer passes cleanly without crashes, with no local changes to itertoolsmodule.c.

The kwargs-copying change in the PyObject_Call() path addresses this scenario, so I’m closing this PR in favor of #156384.

If useful, please feel free to reuse test_zip_longest_reentrant_kwds_clear from commit 40a7b64 as an additional regression test in #156384.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants