Repository navigation
Conversation
Check also for integer overflow in _PyType_AllocNoTrack(). Remove redundant (and incomplete) integer overflow from tuple_alloc() and _PyTuple_Resize().
|
I wrote a first PR #161908 to detect integer overflow in PyObject_GC_NewVar() and PyObject_GC_Resize(). Then I searched for
So instead of changing PyObject_GC_NewVar() and PyObject_GC_Resize(), I wrote this PR which is more generic to change |
In debug mode, check also that nitems >= 0 in _PyType_AllocNoTrack() and _PyObject_GC_Resize().
Fix compiler warning:
Include/cpython/objimpl.h:31:5: error: ISO C90 forbids mixed
declarations and code [-Werror=declaration-after-statement]
Don't compute nitems * itemsize if tp_itemsize is zero.
|
test_run_coroutine_threadsafe of |
| if (type->tp_itemsize != 0) { | ||
| size_t nitems = _Py_STATIC_CAST(size_t, nitems_signed); | ||
| size_t itemsize = _Py_STATIC_CAST(size_t, type->tp_itemsize); | ||
| if (nitems > ((size_t)PY_SSIZE_T_MAX - basicsize) / itemsize) { |
There was a problem hiding this comment.
This is a runtime check that could reduce performance (it hits tuples of size > 20, or small tuples if the freelist is exhausted). Before the check was in tuple_alloc, but there it is a compile-time constant.
Faster (but a bit more code)?
const size_t half = (size_t)1 << (sizeof(size_t) * 4 - 1);
if ((nitems | itemsize) >= half) {
if (itemsize != 0
&& nitems > ((size_t)PY_SSIZE_T_MAX - basicsize) / itemsize) {
return SIZE_MAX;
}
}
I did a quick check, the performance regression does not seem large, but needs to be measured a bit more carefully.
I wrote a micro-benchmark on I ran Result: I'm surprised that my change makes PyTuple_New(1) faster. I expected the same performance or worse performance. So I'm not sure about my benchmark result. The benchmark calls 2500x PyTuple_New(1) and store results to make sure that the tuple freelist is empty for the main part of the benchmark: call PyTuple_New(1) followed by Py_DECREF() in a loop. |
|
I re-ran the benchmark with CPU isolation and Python built with PGO and Clang ( Result: It's still faster, but now the difference is less than a nanosecond which is more realistic :-) |
|
@eendebakpt: Since I didn't measure any performance slowdown, I don't think that your |
Thanks for taking the time to look at this. On my system performance varies a bit (but all cases in a 0.95x to 1.05x range). This is due to statistical noise and binary layout effects (for particular the alignment of The divsion in the check is rather slow, but with PGO on my system the code compiles effectively to So performance wise we are good I would say. |
Check also for integer overflow in _PyType_AllocNoTrack().
Remove redundant (and incomplete) integer overflow from tuple_alloc() and _PyTuple_Resize().
PyTupleWriterAPI #161694