Skip to content

gh-161694: Check for integer overflow in _PyObject_VAR_SIZE() - #161911

Open
vstinner wants to merge 7 commits into
python:mainfrom
vstinner:object_var_size
Open

vstinner wants to merge 7 commits into
python:mainfrom
vstinner:object_var_size

Conversation

@vstinner

@vstinner vstinner commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Check also for integer overflow in _PyType_AllocNoTrack().

Remove redundant (and incomplete) integer overflow from tuple_alloc() and _PyTuple_Resize().

Check also for integer overflow in _PyType_AllocNoTrack().

Remove redundant (and incomplete) integer overflow from tuple_alloc()
and _PyTuple_Resize().
@bedevere-app bedevere-app Bot added the type-feature A feature request or enhancement label Nov 12, 2026
@vstinner vstinner removed the type-feature A feature request or enhancement label Nov 12, 2026
@vstinner

Copy link
Copy Markdown
Member Author

I wrote a first PR #161908 to detect integer overflow in PyObject_GC_NewVar() and PyObject_GC_Resize().

Then I searched for _PyObject_VAR_SIZE in PyPI top 15,000 projects (at 2026-09-17) and I found 7 projects using _PyObject_VAR_SIZE():

  • cython (3.3.0)
  • jpype1 (1.7.1)
  • nuitka (4.2.1)
  • numpy (2.5.3)
  • orjson (3.12.0)
  • pywinpty (3.0.5)
  • recordclass (0.24.1)

So instead of changing PyObject_GC_NewVar() and PyObject_GC_Resize(), I wrote this PR which is more generic to change _PyObject_VAR_SIZE(). It fixes PyObject_GC_NewVar() and PyObject_GC_Resize(), but also all projects calling directly _PyObject_VAR_SIZE().

In debug mode, check also that nitems >= 0 in _PyType_AllocNoTrack()
and _PyObject_GC_Resize().
@bedevere-app bedevere-app Bot added the type-feature A feature request or enhancement label Nov 12, 2026
Fix compiler warning:

    Include/cpython/objimpl.h:31:5: error: ISO C90 forbids mixed
    declarations and code [-Werror=declaration-after-statement]
Don't compute nitems * itemsize if tp_itemsize is zero.
@vstinner

Copy link
Copy Markdown
Member Author

test_run_coroutine_threadsafe of test_asyncio.test_free_threading failed with a timeout Timeout (0:10:00)! on Windows (free-threading) / Build and test (arm64, switch-case). I re-run the failed job.

Comment thread Include/cpython/objimpl.h
if (type->tp_itemsize != 0) {
size_t nitems = _Py_STATIC_CAST(size_t, nitems_signed);
size_t itemsize = _Py_STATIC_CAST(size_t, type->tp_itemsize);
if (nitems > ((size_t)PY_SSIZE_T_MAX - basicsize) / itemsize) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a runtime check that could reduce performance (it hits tuples of size > 20, or small tuples if the freelist is exhausted). Before the check was in tuple_alloc, but there it is a compile-time constant.

Faster (but a bit more code)?

const size_t half = (size_t)1 << (sizeof(size_t) * 4 - 1);
    if ((nitems | itemsize) >= half) {
        if (itemsize != 0
            && nitems > ((size_t)PY_SSIZE_T_MAX - basicsize) / itemsize) {
            return SIZE_MAX;
        }
    }

I did a quick check, the performance regression does not seem large, but needs to be measured a bit more carefully.

@vstinner

Copy link
Copy Markdown
Member Author

This is a runtime check that could reduce performance (...)
I did a quick check, the performance regression does not seem large, but needs to be measured a bit more carefully.

I wrote a micro-benchmark on PyTuple_New(1):

I ran ./python bench.py --rigorous on Python built with gcc -O3 on Fedora 45 (./configure && make).

Result: Mean +- std dev: [ref] 14.5 ns +- 0.2 ns -> [detect_overflow] 13.1 ns +- 0.2 ns: 1.11x faster.

I'm surprised that my change makes PyTuple_New(1) faster. I expected the same performance or worse performance. So I'm not sure about my benchmark result.

The benchmark calls 2500x PyTuple_New(1) and store results to make sure that the tuple freelist is empty for the main part of the benchmark: call PyTuple_New(1) followed by Py_DECREF() in a loop.

@vstinner

Copy link
Copy Markdown
Member Author

I re-ran the benchmark with CPU isolation and Python built with PGO and Clang (./configure --enable-optimizations --with-lto CC=clang LD=clang). Also, I ran the benchmark with -p100 to spawn 100 processes sequentially instead of 20 (by default): ./python ~/bench.py -p100 -o ref.json.

Result: Mean +- std dev: [ref] 13.0 ns +- 0.8 ns -> [detect_overflow] 12.3 ns +- 0.5 ns: 1.05x faster.

It's still faster, but now the difference is less than a nanosecond which is more realistic :-)

@vstinner

Copy link
Copy Markdown
Member Author

@eendebakpt: Since I didn't measure any performance slowdown, I don't think that your if ((nitems | itemsize) >= half) { optimization is needed.

@eendebakpt

Copy link
Copy Markdown
Contributor

@eendebakpt: Since I didn't measure any performance slowdown, I don't think that your if ((nitems | itemsize) >= half) { optimization is needed.

Thanks for taking the time to look at this.

On my system performance varies a bit (but all cases in a 0.95x to 1.05x range). This is due to statistical noise and binary layout effects (for particular the alignment of tp_alloc).

The divsion in the check is rather slow, but with PGO on my system the code compiles effectively to

if (itemsize == 8)
    quotient = x >> 3;      // dividing by 8 is a shift
else
    quotient = x / itemsize; // general case, kept for correctness

So performance wise we are good I would say.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting core review type-feature A feature request or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants