You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Seemingly unnecessary complexification of foo(**kw) #86199
Following bpo-39320 the highly specialised bytecode for vararg calls were replaced by simpler ones, but there seems to be at least one area where the generated bytecode regressed for possibly no reason?
The PR 18141 does not seem to change the implementation of CALL_FUNCTION_EX so I would expect that if it was fine with taking the var arbitrary mapping before it stil is now, and the extra two opcodes (and creation of a dict) is unnecessary?
Have you observed any slowdown or incorrect behaviour?
The 3.8 bytecode looks incorrect to me.
The C-API documentation doesn't prohibit callables from mutating the dictionary they receive.
Unless a copy is made, then a callee could mutate var.
I have not noticed anything, I was just looking at the bytecode changes and stumbled upon this oddity. Though I would expect a small slowdown as every fn(**kw) would now incur an extra dict copy, unless there’s something in call_function_ex which copies the input dict iff its ref count is not one?
For whatever that’s worth, the 3.8 bytecode has been there since call_function_ex was added in 3.6 and before that call_function_kw looks identical (load_global foo, load_local var, call_function_kw)
$ python3.8 -m timeit -s "a = {'a': 1}""dict(**a)"
2000000 loops, best of 5: 113 nsec per loop
$ python3.9 -m timeit -s "a = {'a': 1}""dict(**a)"
2000000 loops, best of 5: 181 nsec per loop
Reacted by Josh {*()} Rosenberg
MojoVampire commented on Dec 16, 2020
MojoVampiremannequin
Mannequin
More actions
Even if making a copy is necessary when the underlying function receives the dict "raw", preemptively performing the copy (before knowing if the function being called is a Vectorcall function) means that when it's a Vectorcall function (e.g. all user-defined functions, right?), instead of just copying from the original dict to the unpacked stack for vectorcall, it makes an intermediate copy, then copies from that copy to the unpacked stack later on; the copy is otherwise completely unused.
The extra bytecode isn't even defending against "dict-like" kwargs, because CALL_FUNCTION_EX itself already copies to a true dict for anything that's not an exact dict (that defense shouldn't even be there if the bytecode compiler is already guaranteeing a true dict).
Seems like, if preventing the caller's dict from being passed directly to the underlying function is necessary and intended, it should be done in PyObject_Call (which can avoid the copy entirely when call a Vectorcall function and when the reference count on the dict is 1), not at the bytecode interpreter layer. As is, PyObject_Call is already violating the documented behavior by *not* matching the behavior of callable(*args, **kwargs) (see bpo-42629), so moving it to PyObject_Call would fix that problem and improve performance passing a single kwargs.
I've investigated fixing this. It's pretty easy to special-case the compiler to not emit the BUILD_MAP 0/DICT_MERGE when there's only one source of keyword args and it's being ** unpacked.
The downside to this is that, per #86795, PyObject_Call does nothing to prevent a dict that is both passed as **kwargs and received as a keywords dict (via non-vectorcall APIs) from being modified by the callee. User-defined functions (at the Python level) are all vectorcall now (right?), so that's fine, but without the BUILD_MAP 0/DICT_MERGE, any METH_VARARGS | METH_KEYWORDS C level function will called with an alias of the original dict, and any modifications made will affect the caller's dict.
I see four options for fixing this:
Disable BUILD_MAP 0+DICT_MERGE and leave everything else unchanged:
Pro: Most performant, never makes unnecessary copies
Can: Running callable(**kwargs) where callable isn't vectorcall will allow callable to change the caller's kwargs (do we make explicit guarantees here?)
Disable BUILD_MAP 0+DICT_MERGE and modify CALL_FUNCTION_EX to apply the copying technique for inexact dicts for dicts with refcnt > 1:
Pro: Running callable(**kwargs) will never modify caller's kwargs
Con: A wasted copy when callable turns out to be vectorcall (equivalent to the wasted copy from BUILD_MAP 0+DICT_MERGE, but removing those two opcodes, which would probably make it a bit faster)
Disable BUILD_MAP 0+DICT_MERGE and modify PyObject_Call to apply copy any incoming dicts when following the non-vectorcall path
Pros: Running callable(**kwargs) will never modify caller's kwargs. Even C API users can rely on this guaranteed copy. No copies made when following vectorcall path
Con: C API users who want to pass kwargs through (with the intent of either handing off ownership or expecting the callee to mutate it for beneficial reasons) can't do so; the decision to copy is made for them.
Leave the BUILD_MAP 0+DICT_MERGE in place, and remove the special-case for inexact dicts in CALL_FUNCTION_EX (I suspect we could also drop the special-case for inexact tuples too, for the same reason, but I haven't checked thoroughly)
Pro: Shrink the eval loop; the inexact dict case can never be hit with the guaranteed BUILD_MAP involved, so we can dump half of the code in CALL_FUNCTION_EX (if it's just the dict case we can eliminate) or almost all of it (if we can eliminate the tuple special casing too).
Con: All user passed **kwargs will be a little bit slower in the common case where they'd ultimately reach a vectorcall function
Performance-wise, options #2 and #4 are probably the worst (since they preserve an unnecessary copy in the common case where the callable is vectorcall). #1 is the best performance, but might constitute a behavior regression and violate expectations (guarantees?). #3 slows things down in a rare case (C API callers calling non-vectorcall functions), but arguably makes things more consistent (making the documentation claim that PyObject_Call(callable, args, kwds) is equivalent to callable(*args, **kwargs) fully correct).
I've submitted a PR for this and #86795 (since it's not possible to fix this issue without fixing #86795 too). A tiny patch to the compiler makes it not perform the BUILD_MAP+DICT_MERGE when the only source of keyword arguments is a single **kwargs. Tests added to ensure no future regression causes the dict to ever reach a callee without being copied.
Performance-wise, using Serhiy's case, we save about 30% on overhead:
$ ./python_unmodified_20220429 -m timeit -s "a = {'a': 1}" "dict(**a)"
2000000 loops, best of 5: 120 nsec per loop
$ ./python3.dictoptimize -m timeit -s "a = {'a': 1}" "dict(**a)"
5000000 loops, best of 5: 88.7 nsec per loop
Same behavior seen when a = {} (30% faster). When any other keyword arguments are passed (meaning the new dict has to be built regardless), no savings are observed (on repeated runs, passing b=1 either before or after **a, the times are all within 3%, and there's no pattern to which one wins):
$ ./python_unmodified_20220429 -m timeit -s "a = {'a': 1}" "dict(b=1, **a)"
2000000 loops, best of 5: 167 nsec per loop
$ ./python3.dictoptimize -m timeit -s "a = {'a': 1}" "dict(b=1, **a)"
2000000 loops, best of 5: 160 nsec per loop
$ ./python_unmodified_20220429 -m timeit -s "a = {'a': 1}" "dict(**a, b=1)"
2000000 loops, best of 5: 191 nsec per loop
$ ./python3.dictoptimize -m timeit -s "a = {'a': 1}" "dict(**a, b=1)"
2000000 loops, best of 5: 198 nsec per loop
The winner swaps from run to run, so I'm guessing it's unrelated to the change (which changed nothing in the code path followed by this use case).
This does involve a small regression in the time required to call non-vectorcall functions from the C layer (required to make PyObject_Call adhere to the documented equivalence to callable(*args, **kwargs), which never allows the original kwargs to reach the callee). It's hard to test without my _testcapi addition though. Rather than continue the microbenchmarks, I'm going to try running the pyperformance benchmarks to check behavior on more varied code.
pyperformance numbers. orig was built from the base checkout from April 30th that the PR is based on, dictoptimize is with my changes. Both builds were made with ./configure --enable-optimizations --with-lto, compiled using gcc 10.3.1, run in an Alpine Linux 3.15 container running under WSLv2 on Windows:
So it looks like a grand total of one benchmark got slower (unpickle), 15 were unchanged, and 43 got at least 1% faster, overall gain of 3% across all the benchmarks.
Activity
masklinn commentedon Oct 14, 2020
Following bpo-39320 the highly specialised bytecode for vararg calls were replaced by simpler ones, but there seems to be at least one area where the generated bytecode regressed for possibly no reason?
In Python 3.8, foo(**var) compiles to:
0 LOAD_GLOBAL 0 (foo)
2 BUILD_TUPLE 0
4 LOAD_FAST 2 (var)
6 CALL_FUNCTION_EX 1
In Python 3.9, it compiles to:
0 LOAD_GLOBAL 0 (foo)
2 BUILD_TUPLE 0
4 BUILD_MAP 0
6 LOAD_FAST 2 (var)
8 DICT_MERGE 1
0 CALL_FUNCTION_EX 1
The PR 18141 does not seem to change the implementation of CALL_FUNCTION_EX so I would expect that if it was fine with taking the
vararbitrary mapping before it stil is now, and the extra two opcodes (and creation of a dict) is unnecessary?markshannon commentedon Oct 15, 2020
Have you observed any slowdown or incorrect behaviour?
The 3.8 bytecode looks incorrect to me.
The C-API documentation doesn't prohibit callables from mutating the dictionary they receive.
Unless a copy is made, then a callee could mutate
var.https://docs.python.org/3/c-api/call.html
masklinn commentedon Oct 15, 2020
I have not noticed anything, I was just looking at the bytecode changes and stumbled upon this oddity. Though I would expect a small slowdown as every fn(**kw) would now incur an extra dict copy, unless there’s something in call_function_ex which copies the input dict iff its ref count is not one?
For whatever that’s worth, the 3.8 bytecode has been there since call_function_ex was added in 3.6 and before that call_function_kw looks identical (load_global foo, load_local var, call_function_kw)
serhiy-storchaka commentedon Oct 15, 2020
MojoVampire commentedon Dec 16, 2020
Even if making a copy is necessary when the underlying function receives the dict "raw", preemptively performing the copy (before knowing if the function being called is a Vectorcall function) means that when it's a Vectorcall function (e.g. all user-defined functions, right?), instead of just copying from the original dict to the unpacked stack for vectorcall, it makes an intermediate copy, then copies from that copy to the unpacked stack later on; the copy is otherwise completely unused.
The extra bytecode isn't even defending against "dict-like" kwargs, because CALL_FUNCTION_EX itself already copies to a true dict for anything that's not an exact dict (that defense shouldn't even be there if the bytecode compiler is already guaranteeing a true dict).
Seems like, if preventing the caller's dict from being passed directly to the underlying function is necessary and intended, it should be done in PyObject_Call (which can avoid the copy entirely when call a Vectorcall function and when the reference count on the dict is 1), not at the bytecode interpreter layer. As is, PyObject_Call is already violating the documented behavior by *not* matching the behavior of callable(*args, **kwargs) (see bpo-42629), so moving it to PyObject_Call would fix that problem and improve performance passing a single kwargs.
MojoVampire commentedon Apr 30, 2022
I've investigated fixing this. It's pretty easy to special-case the compiler to not emit the
BUILD_MAP 0/DICT_MERGEwhen there's only one source of keyword args and it's being**unpacked.The downside to this is that, per #86795,
PyObject_Calldoes nothing to prevent adictthat is both passed as**kwargsand received as a keywordsdict(via non-vectorcall APIs) from being modified by the callee. User-defined functions (at the Python level) are all vectorcall now (right?), so that's fine, but without theBUILD_MAP 0/DICT_MERGE, anyMETH_VARARGS | METH_KEYWORDSC level function will called with an alias of the originaldict, and any modifications made will affect the caller'sdict.I see four options for fixing this:
Disable
BUILD_MAP 0+DICT_MERGEand leave everything else unchanged:Pro: Most performant, never makes unnecessary copies
Can: Running
callable(**kwargs)wherecallableisn't vectorcall will allowcallableto change the caller'skwargs(do we make explicit guarantees here?)Disable
BUILD_MAP 0+DICT_MERGEand modifyCALL_FUNCTION_EXto apply the copying technique for inexactdicts fordicts with refcnt > 1:Pro: Running
callable(**kwargs)will never modify caller'skwargsCon: A wasted copy when
callableturns out to be vectorcall (equivalent to the wasted copy fromBUILD_MAP 0+DICT_MERGE, but removing those two opcodes, which would probably make it a bit faster)Disable
BUILD_MAP 0+DICT_MERGEand modifyPyObject_Callto apply copy any incomingdicts when following the non-vectorcall pathPros: Running
callable(**kwargs)will never modify caller'skwargs. Even C API users can rely on this guaranteed copy. No copies made when following vectorcall pathCon: C API users who want to pass
kwargsthrough (with the intent of either handing off ownership or expecting the callee to mutate it for beneficial reasons) can't do so; the decision to copy is made for them.Leave the
BUILD_MAP 0+DICT_MERGEin place, and remove the special-case for inexactdicts inCALL_FUNCTION_EX(I suspect we could also drop the special-case for inexacttuples too, for the same reason, but I haven't checked thoroughly)Pro: Shrink the eval loop; the inexact
dictcase can never be hit with the guaranteedBUILD_MAPinvolved, so we can dump half of the code inCALL_FUNCTION_EX(if it's just thedictcase we can eliminate) or almost all of it (if we can eliminate thetuplespecial casing too).Con: All user passed
**kwargswill be a little bit slower in the common case where they'd ultimately reach a vectorcall functionPerformance-wise, options
#2and#4are probably the worst (since they preserve an unnecessary copy in the common case where the callable is vectorcall).#1is the best performance, but might constitute a behavior regression and violate expectations (guarantees?).#3slows things down in a rare case (C API callers calling non-vectorcall functions), but arguably makes things more consistent (making the documentation claim thatPyObject_Call(callable, args, kwds)is equivalent tocallable(*args, **kwargs)fully correct).Any thoughts?
MojoVampire commentedon Apr 30, 2022
I'm considering submitting a PR for option
#3during the upcoming sprints, so please let me know ASAP if there is a problem with it.Fix python#86199 and python#86795 by:
Fix python#86199 and python#86795 by:
MojoVampire commentedon May 2, 2022
I've submitted a PR for this and #86795 (since it's not possible to fix this issue without fixing #86795 too). A tiny patch to the compiler makes it not perform the
BUILD_MAP+DICT_MERGEwhen the only source of keyword arguments is a single**kwargs. Tests added to ensure no future regression causes thedictto ever reach a callee without being copied.Performance-wise, using Serhiy's case, we save about 30% on overhead:
Same behavior seen when
a = {}(30% faster). When any other keyword arguments are passed (meaning the newdicthas to be built regardless), no savings are observed (on repeated runs, passingb=1either before or after**a, the times are all within 3%, and there's no pattern to which one wins):The winner swaps from run to run, so I'm guessing it's unrelated to the change (which changed nothing in the code path followed by this use case).
This does involve a small regression in the time required to call non-vectorcall functions from the C layer (required to make
PyObject_Calladhere to the documented equivalence tocallable(*args, **kwargs), which never allows the originalkwargsto reach the callee). It's hard to test without my_testcapiaddition though. Rather than continue the microbenchmarks, I'm going to try running thepyperformancebenchmarks to check behavior on more varied code.MojoVampire commentedon May 2, 2022
pyperformance numbers.
origwas built from the base checkout from April 30th that the PR is based on,dictoptimizeis with my changes. Both builds were made with./configure --enable-optimizations --with-lto, compiled using gcc 10.3.1, run in an Alpine Linux 3.15 container running under WSLv2 on Windows:MojoVampire commentedon May 2, 2022
So it looks like a grand total of one benchmark got slower (
unpickle), 15 were unchanged, and 43 got at least 1% faster, overall gain of 3% across all the benchmarks.3 remaining items
pythongh-86199: Simplify by adopting the pythonGH-92192 approach
pythongh-86199: Use PyDict_Copy instead of a copy helper
pythongh-86199: Enter recursive call before copying kwargs; trim comm…
pythongh-86199: Lock the kwargs dict in _PyStack_UnpackDict
pythongh-86199: Add free-threading regression test for f(**kwargs) calls
pythongh-86199: Target the C-API unpack path in the FT regression test
pythongh-86199: Rename the FT kwargs test and tighten its names
pythongh-86199: Avoid an unnecessary dict copy for f(**kwargs) calls
pythongh-86199: Fix CI: move kwargs conversion out of the instruction…
pythongh-86199: Keep &dupkey out of DICT_MERGE too (MSVC tail-call bu…