Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Doc/c-api/frame.rst
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ See also :ref:`Reflection <reflection>`.

.. versionadded:: 3.11

.. versionchanged:: 3.16
May return ``NULL`` for a coroutine frame before its first suspension;
see :attr:`frame.f_generator`.


.. c:function:: PyObject* PyFrame_GetGlobals(PyFrameObject *frame)

Expand Down
55 changes: 46 additions & 9 deletions Doc/library/dis.rst
Original file line number Diff line number Diff line change
Expand Up @@ -820,19 +820,33 @@ not have to be) the original ``STACK[-2]``.
Implements ``STACK[-1] = get_awaitable(STACK[-1])``, where ``get_awaitable(o)``
returns ``o`` if ``o`` is a coroutine object or a generator object with
the :data:`~inspect.CO_ITERABLE_COROUTINE` flag, or resolves
``o.__await__``.
``o.__await__``. Then pushes ``NULL`` and ``None``, which start the
:opcode:`SEND` loop that follows.

If the ``where`` operand is nonzero, it indicates where the instruction
occurs:
If ``where >> 1`` is nonzero, it indicates where the instruction occurs:

* ``1``: After a call to ``__aenter__``
* ``2``: After a call to ``__aexit__``
* ``1``: After a call to ``__aenter__``
* ``2``: After a call to ``__aexit__``

If the low bit of ``where`` is set, the awaited object is the result of an
awaited :opcode:`CALL`, :opcode:`CALL_KW` or :opcode:`CALL_FUNCTION_EX`,
and the instruction also pops the extra slot that was pushed below that
call's callable. If the call ran a coroutine function without creating the
coroutine object, that slot tells how the body ended, and
:opcode:`GET_AWAITABLE` pushes the result or the yielded value in place of
``None``, with a ``NULL`` receiver if the body returned, or with the new
coroutine and ``None`` in place of ``NULL`` if it suspended (see
:opcode:`SEND`).

.. versionadded:: 3.5

.. versionchanged:: 3.11
Previously, this instruction did not have an oparg.

.. versionchanged:: next
Pushes ``NULL`` and ``None`` for the following :opcode:`SEND`;
the low bit of ``where`` marks an awaited call.


.. opcode:: GET_AITER

Expand Down Expand Up @@ -1615,7 +1629,8 @@ iterations of the loop.
* ``self`` or ``NULL``
* The remaining positional arguments

``argc`` is the total of the positional arguments, excluding ``self``.
``argc >> 1`` is the total of the positional arguments, excluding ``self``.
The low bit of ``argc`` is set when the result is awaited right away.

``CALL`` pops all arguments and the callable object off the stack,
calls the callable object with those arguments, and pushes the return value
Expand All @@ -1629,6 +1644,10 @@ iterations of the loop.
.. versionchanged:: 3.13
Calls with keyword arguments are now handled by :opcode:`CALL_KW`.

.. versionchanged:: next
The number of arguments is now ``argc >> 1``; the low bit marks an
awaited call.


.. opcode:: CALL_KW (argc)

Expand All @@ -1641,7 +1660,8 @@ iterations of the loop.
* The named arguments
* A :class:`tuple` of keyword argument names

``argc`` is the total of the positional and named arguments, excluding ``self``.
``argc >> 1`` is the total of the positional and named arguments, excluding
``self``. The low bit of ``argc`` is set when the result is awaited right away.
The length of the tuple of keyword argument names is the number of named arguments.

``CALL_KW`` pops all arguments, the keyword names, and the callable object
Expand All @@ -1650,21 +1670,29 @@ iterations of the loop.

.. versionadded:: 3.13

.. versionchanged:: next
The number of arguments is now ``argc >> 1``; the low bit marks an
awaited call.


.. opcode:: CALL_FUNCTION_EX (flags)

Calls a callable object with variable set of positional and keyword
arguments. If the lowest bit of *flags* is set, the top of the stack
contains a mapping object containing additional keyword arguments.
arguments. The top of the stack contains a mapping object with additional
keyword arguments, or ``NULL``.
Before the callable is called, the mapping object and iterable object
are each "unpacked" and their contents passed in as keyword and
positional arguments respectively.
``CALL_FUNCTION_EX`` pops all arguments and the callable object off the stack,
calls the callable object with those arguments, and pushes the return value
returned by the callable object.
The lowest bit of *flags* is set when the result is awaited right away.

.. versionadded:: 3.6

.. versionchanged:: next
The lowest bit of *flags* marks an awaited call.


.. opcode:: PUSH_NULL

Expand Down Expand Up @@ -1840,12 +1868,21 @@ iterations of the loop.
push the exception's ``value`` attribute, and increment the bytecode counter
by *delta*.

After an awaited call (see :opcode:`GET_AWAITABLE`), the receiver may be
``NULL``: then ``STACK[-1]`` is already the result and the bytecode counter
is incremented by *delta*. If ``STACK[-2]`` is ``None``, ``STACK[-1]`` has
already been yielded by the receiver: ``STACK[-2]`` is set to ``NULL`` and
execution continues with the next instruction.

.. versionadded:: 3.11

.. versionchanged:: 3.15
The receiver is now at ``STACK[-3]``, followed by the value pushed with
it by :opcode:`GET_ITER`.

.. versionchanged:: next
Handles the results of awaited calls.


.. opcode:: HAVE_ARGUMENT

Expand Down
4 changes: 3 additions & 1 deletion Doc/library/inspect.rst
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,9 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
| | | coroutine object that |
| | | owns this frame, or |
| | | ``None`` if the frame is |
| | | of a regular function |
| | | of a regular function or |
| | | the generator is not |
| | | materialized yet |
+-----------------+-------------------+---------------------------+
| | f_trace | tracing function for this |
| | | frame, or ``None`` |
Expand Down
14 changes: 13 additions & 1 deletion Doc/reference/datamodel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1736,10 +1736,22 @@ Special read-only attributes

* - .. attribute:: frame.f_generator
- The :term:`generator` or :term:`coroutine` object that owns this frame,
or ``None`` if the frame is a normal function.
or ``None`` if the frame is a normal function or if the generator
is not materialized yet.

.. impl-detail::

A coroutine that is awaited as soon as it is called
(``await coro_func()``) may run without a coroutine object until
it first suspends, so this attribute is ``None`` before that.
Use :attr:`~codeobject.co_flags` of :attr:`frame.f_code` to tell
coroutine frames apart from normal function frames.

.. versionadded:: 3.14

.. versionchanged:: 3.16
May be ``None`` for a coroutine frame before its first suspension.

.. index::
single: f_trace (frame attribute)
single: f_trace_lines (frame attribute)
Expand Down
41 changes: 41 additions & 0 deletions Doc/whatsnew/3.16.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1147,6 +1147,19 @@ datetime and time
(Contributed by Paul Ganssle in :gh:`158252`, based on work by
Stan Ulbrych.)

Interpreter
-----------

* A coroutine that is awaited as soon as it is called
(``await coro_func()``) now runs without creating a coroutine object
unless it suspends, making awaits that complete immediately about 40%
faster in a micro-benchmark (``await noop()``: 48 ns to 27 ns on x86-64).
This does not apply while coroutine origin tracking is enabled or inside
code compiled by the experimental JIT.
Until it first suspends, :attr:`frame.f_generator` is ``None`` for such a
coroutine; see :ref:`whatsnew316-porting`.
(Contributed by Andrew Svetlov in :gh:`161170`.)

re
--

Expand Down Expand Up @@ -1405,12 +1418,23 @@ New deprecations
.. include:: ../deprecations/pending-removal-in-future.rst


.. _whatsnew316-porting:

Porting to Python 3.16
======================

This section lists previously described changes and other bugfixes
that may require changes to your code.

* :attr:`frame.f_generator` and :c:func:`PyFrame_GetGenerator` may now
return ``None``/``NULL`` for a coroutine frame before its first
suspension, because a coroutine that is awaited as soon as it is called
(``await coro_func()``) runs without a coroutine object until then.
Code that used ``frame.f_generator is None`` to detect frames of normal
functions should check ``frame.f_code.co_flags`` for
:data:`~inspect.CO_COROUTINE` instead.
(Contributed by Andrew Svetlov in :gh:`161170`.)

* In :mod:`tkinter`, the *name* parameter of the
:meth:`~tkinter.Misc.wait_variable`, :meth:`~tkinter.Misc.setvar` and
:meth:`~tkinter.Misc.getvar` methods and the *value* parameter of
Expand Down Expand Up @@ -1488,6 +1512,23 @@ that may require changes to your code.
(Contributed by An Long in :gh:`86768`.)


CPython bytecode changes
========================

* The oparg of :opcode:`CALL` and :opcode:`CALL_KW` is now the number of
arguments shifted left by one. The low bit of their oparg, and of the
oparg of :opcode:`CALL_FUNCTION_EX`, marks a call whose result is awaited
right away.
(Contributed by Andrew Svetlov in :gh:`161170`.)

* :opcode:`GET_AWAITABLE` now also pushes the ``NULL`` and ``None`` that
start the :opcode:`SEND` loop following it, so the compiler no longer
emits :opcode:`PUSH_NULL` and :opcode:`LOAD_COMMON_CONSTANT` after it.
After an awaited call, the low bit of its oparg is set and it also pops
a slot that the compiler pushes below the call's callable.
(Contributed by Andrew Svetlov in :gh:`161170`.)


Build changes
=============

Expand Down
6 changes: 6 additions & 0 deletions Include/internal/pycore_ceval.h
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,12 @@ extern int _PyEval_EnsureBuiltinsWithModule(
PyObject **p_builtins);

PyAPI_FUNC(PyObject *)_Py_MakeCoro(PyFunctionObject *func);
// Lazily created coroutines (see RETURN_GENERATOR in bytecodes.c).
// Exported for _testinternalcapi and external JIT support.
PyAPI_FUNC(PyObject *) _PyCoro_MaterializeLazy(_PyInterpreterFrame *frame);
PyAPI_FUNC(_PyInterpreterFrame *) _PyEval_MoveLazyCoroFrame(PyThreadState *tstate,
PyGenObject *gen,
_PyInterpreterFrame *frame);

/* Handle signals, pending calls, GIL drop request
and asynchronous exception */
Expand Down
4 changes: 4 additions & 0 deletions Include/internal/pycore_interpframe.h
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,8 @@ static inline void _PyFrame_Copy(_PyInterpreterFrame *src, _PyInterpreterFrame *
dest->stackpointer = dest->localsplus + stacktop;
// visited is GC bookkeeping for the current stack walk, not frame state.
dest->visited = 0;
// A lazily started coroutine stops being lazy once its frame is copied.
dest->lazy_coro = 0;
#ifdef Py_DEBUG
dest->stackpointer_valid = src->stackpointer_valid;
dest->lltrace = src->lltrace;
Expand Down Expand Up @@ -208,6 +210,7 @@ _PyFrame_Initialize(
frame->return_offset = 0;
frame->owner = FRAME_OWNED_BY_THREAD;
frame->visited = 0;
frame->lazy_coro = 0;
#ifdef Py_DEBUG
frame->stackpointer_valid = 1;
frame->lltrace = 0;
Expand Down Expand Up @@ -441,6 +444,7 @@ _PyFrame_PushTrampolineUnchecked(PyThreadState *tstate, PyCodeObject *code, int
#endif
frame->owner = FRAME_OWNED_BY_THREAD;
frame->visited = 0;
frame->lazy_coro = 0;
#ifdef Py_DEBUG
frame->stackpointer_valid = 1;
frame->lltrace = 0;
Expand Down
3 changes: 2 additions & 1 deletion Include/internal/pycore_interpframe_structs.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,10 @@ struct _PyInterpreterFrame {
char owner;
uint8_t visited:1;
uint8_t external:1; /* Set when the frame was pushed by something other than the VM */
uint8_t lazy_coro:1;
#ifdef Py_DEBUG
uint8_t stackpointer_valid:1;
uint8_t lltrace:5;
uint8_t lltrace:4;
#endif
/* Locals and stack */
_PyStackRef localsplus[1];
Expand Down
3 changes: 2 additions & 1 deletion Include/internal/pycore_magic_number.h
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,7 @@ Known values:
Python 3.16a1 3704 (Replace DELETE_ATTR with PUSH_NULL; STORE_ATTR)
Python 3.16a1 3705 (Add INTRINSIC_ADD_CONDITIONAL_ANNOTATION)
Python 3.16a1 3706 (Add INTRINSIC_MATCH_CLASS_ISINSTANCE)
Python 3.16a1 3707 (Lazily materialize coroutine objects for awaited calls)

Python 3.17 will start with 3750

Expand All @@ -313,7 +314,7 @@ Known values:

*/

#define PYC_MAGIC_NUMBER 3706
#define PYC_MAGIC_NUMBER 3707
/* This is equivalent to converting PYC_MAGIC_NUMBER to 2 bytes
(little-endian) and then appending b'\r\n'. */
#define PYC_MAGIC_NUMBER_TOKEN \
Expand Down
Loading
Loading