Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Doc/c-api/frame.rst
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ See also :ref:`Reflection <reflection>`.

.. versionadded:: 3.11

.. versionchanged:: 3.16
May return ``NULL`` for a coroutine frame before its first suspension;
see :attr:`frame.f_generator`.


.. c:function:: PyObject* PyFrame_GetGlobals(PyFrameObject *frame)

Expand Down
18 changes: 18 additions & 0 deletions Doc/library/dis.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1630,6 +1630,24 @@ iterations of the loop.
.. versionadded:: 3.13


.. opcode:: ASYNC_CALL (argc)

Like :opcode:`CALL`, for a call whose result is awaited at once, as in
``await f(...)``, ``async with`` and asynchronous comprehensions.
It is always followed by :opcode:`GET_AWAITABLE`, :opcode:`PUSH_NULL`,
:opcode:`LOAD_COMMON_CONSTANT` (``None``) and :opcode:`SEND`.

.. versionadded:: 3.16


.. opcode:: ASYNC_CALL_KW (argc)

Like :opcode:`CALL_KW`, for a call whose result is awaited at once.
See :opcode:`ASYNC_CALL`.

.. versionadded:: 3.16


.. opcode:: CALL_FUNCTION_EX (flags)

Calls a callable object with variable set of positional and keyword
Expand Down
4 changes: 3 additions & 1 deletion Doc/library/inspect.rst
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,9 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
| | | coroutine object that |
| | | owns this frame, or |
| | | ``None`` if the frame is |
| | | of a regular function |
| | | of a regular function or |
| | | the generator is not |
| | | materialized yet |
+-----------------+-------------------+---------------------------+
| | f_trace | tracing function for this |
| | | frame, or ``None`` |
Expand Down
14 changes: 13 additions & 1 deletion Doc/reference/datamodel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1736,10 +1736,22 @@ Special read-only attributes

* - .. attribute:: frame.f_generator
- The :term:`generator` or :term:`coroutine` object that owns this frame,
or ``None`` if the frame is a normal function.
or ``None`` if the frame is a normal function or if the generator
is not materialized yet.

.. impl-detail::

A coroutine that is awaited as soon as it is called
(``await coro_func()``) may run without a coroutine object until
it first suspends, so this attribute is ``None`` before that.
Use :attr:`~codeobject.co_flags` of :attr:`frame.f_code` to tell
coroutine frames apart from normal function frames.

.. versionadded:: 3.14

.. versionchanged:: 3.16
May be ``None`` for a coroutine frame before its first suspension.

.. index::
single: f_trace (frame attribute)
single: f_trace_lines (frame attribute)
Expand Down
31 changes: 31 additions & 0 deletions Doc/whatsnew/3.16.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1015,6 +1015,17 @@ zlib
Optimizations
=============

Interpreter
-----------

* A coroutine function that is awaited as soon as it is called
(``await coro_func()``, ``async with``) no longer allocates a coroutine
object unless its body suspends, once the call has been specialized.
Until it first suspends, :attr:`frame.f_generator` is ``None`` for such a
coroutine; see :ref:`whatsnew316-porting`.
This does not apply while coroutine origin tracking is enabled.
(Contributed by Andrew Svetlov in :gh:`161170`.)

datetime and time
-----------------

Expand Down Expand Up @@ -1273,12 +1284,23 @@ New deprecations
.. include:: ../deprecations/pending-removal-in-future.rst


.. _whatsnew316-porting:

Porting to Python 3.16
======================

This section lists previously described changes and other bugfixes
that may require changes to your code.

* :attr:`frame.f_generator` and :c:func:`PyFrame_GetGenerator` may now
return ``None``/``NULL`` for a coroutine frame before its first
suspension, because a coroutine that is awaited as soon as it is called
(``await coro_func()``) may run without a coroutine object until then.
Code that used ``frame.f_generator is None`` to detect frames of normal
functions should check ``frame.f_code.co_flags`` for
:data:`~inspect.CO_COROUTINE` instead.
(Contributed by Andrew Svetlov in :gh:`161170`.)

* In :mod:`tkinter`, the *name* parameter of the
:meth:`~tkinter.Misc.wait_variable`, :meth:`~tkinter.Misc.setvar` and
:meth:`~tkinter.Misc.getvar` methods and the *value* parameter of
Expand Down Expand Up @@ -1334,6 +1356,15 @@ that may require changes to your code.
(Contributed by An Long in :gh:`86768`.)


CPython bytecode changes
========================

* Added :opcode:`ASYNC_CALL` and :opcode:`ASYNC_CALL_KW`, which the compiler
emits instead of :opcode:`CALL` and :opcode:`CALL_KW` for a call whose
result is awaited at once.
(Contributed by Andrew Svetlov in :gh:`161170`.)


Build changes
=============

Expand Down
6 changes: 6 additions & 0 deletions Include/internal/pycore_ceval.h
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,12 @@ extern int _PyEval_EnsureBuiltinsWithModule(
PyObject **p_builtins);

PyAPI_FUNC(PyObject *)_Py_MakeCoro(PyFunctionObject *func);
// Lazily created coroutines (see _ASYNC_CALL_LAZY_FRAME in bytecodes.c).
// Exported for _testinternalcapi and external JIT support.
PyAPI_FUNC(PyObject *) _PyCoro_MaterializeLazy(_PyInterpreterFrame *frame);
PyAPI_FUNC(_PyInterpreterFrame *) _PyEval_MoveLazyCoroFrame(PyThreadState *tstate,
PyGenObject *gen,
_PyInterpreterFrame *frame);

/* Handle signals, pending calls, GIL drop request
and asynchronous exception */
Expand Down
28 changes: 28 additions & 0 deletions Include/internal/pycore_code.h
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ extern "C" {
#endif

#include "pycore_backoff.h" // _Py_BackoffCounter
#include "pycore_opcode_utils.h" // CONSTANT_NONE
#include "opcode_ids.h" // GET_AWAITABLE
#include "pycore_structs.h" // _Py_CODEUNIT
#include "pycore_tstate.h" // _PyThreadStateImpl

Expand Down Expand Up @@ -128,6 +130,8 @@ typedef struct {

#define INLINE_CACHE_ENTRIES_CALL CACHE_ENTRIES(_PyCallCache)
#define INLINE_CACHE_ENTRIES_CALL_KW CACHE_ENTRIES(_PyCallCache)
#define INLINE_CACHE_ENTRIES_ASYNC_CALL CACHE_ENTRIES(_PyCallCache)
#define INLINE_CACHE_ENTRIES_ASYNC_CALL_KW CACHE_ENTRIES(_PyCallCache)

typedef struct {
_Py_BackoffCounter counter;
Expand Down Expand Up @@ -317,6 +321,10 @@ PyAPI_FUNC(void) _Py_Specialize_Call(_PyStackRef callable, _PyStackRef self_or_n
_Py_CODEUNIT *instr, int nargs);
PyAPI_FUNC(void) _Py_Specialize_CallKw(_PyStackRef callable, _Py_CODEUNIT *instr,
int nargs);
PyAPI_FUNC(void) _Py_Specialize_AsyncCall(_PyStackRef callable, _Py_CODEUNIT *instr,
int nargs);
PyAPI_FUNC(void) _Py_Specialize_AsyncCallKw(_PyStackRef callable, _Py_CODEUNIT *instr,
int nargs);
PyAPI_FUNC(void) _Py_Specialize_BinaryOp(_PyStackRef lhs, _PyStackRef rhs, _Py_CODEUNIT *instr,
int oparg, _PyStackRef *locals);
PyAPI_FUNC(void) _Py_Specialize_CompareOp(_PyStackRef lhs, _PyStackRef rhs,
Expand Down Expand Up @@ -542,6 +550,26 @@ PyAPI_FUNC(int) _Py_Instrument(PyCodeObject *co, PyInterpreterState *interp);
// Export for '_testinternalcapi' shared extension
PyAPI_FUNC(_Py_CODEUNIT) _Py_GetBaseCodeUnit(PyCodeObject *code, int offset);

PyAPI_DATA(const uint8_t) _PyOpcode_Deopt[256];

/* The SEND that awaits the result of the ASYNC_CALL or ASYNC_CALL_KW whose
* cache ends at `tail`, or NULL if the instructions after the call are not
* the ones the compiler emits for an await. */
static inline _Py_CODEUNIT *
_PyCode_AwaitedCallSend(_Py_CODEUNIT *tail)
{
/* None of the first three instructions can end the code. */
if (tail[0].op.code == GET_AWAITABLE
&& tail[1].op.code == PUSH_NULL
&& tail[2].op.code == LOAD_COMMON_CONSTANT
&& tail[2].op.arg == CONSTANT_NONE
&& _PyOpcode_Deopt[tail[3].op.code] == SEND)
{
return &tail[3];
}
return NULL;
}

extern int _PyInstruction_GetLength(PyCodeObject *code, int offset);

extern PyObject *_PyInstrumentation_BranchesIterator(PyCodeObject *code);
Expand Down
4 changes: 4 additions & 0 deletions Include/internal/pycore_interpframe.h
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,8 @@ static inline void _PyFrame_Copy(_PyInterpreterFrame *src, _PyInterpreterFrame *
dest->stackpointer = dest->localsplus + stacktop;
// visited is GC bookkeeping for the current stack walk, not frame state.
dest->visited = 0;
// A lazily started coroutine stops being lazy once its frame is copied.
dest->lazy_coro = 0;
#ifdef Py_DEBUG
dest->stackpointer_valid = src->stackpointer_valid;
dest->lltrace = src->lltrace;
Expand Down Expand Up @@ -208,6 +210,7 @@ _PyFrame_Initialize(
frame->return_offset = 0;
frame->owner = FRAME_OWNED_BY_THREAD;
frame->visited = 0;
frame->lazy_coro = 0;
#ifdef Py_DEBUG
frame->stackpointer_valid = 1;
frame->lltrace = 0;
Expand Down Expand Up @@ -441,6 +444,7 @@ _PyFrame_PushTrampolineUnchecked(PyThreadState *tstate, PyCodeObject *code, int
#endif
frame->owner = FRAME_OWNED_BY_THREAD;
frame->visited = 0;
frame->lazy_coro = 0;
#ifdef Py_DEBUG
frame->stackpointer_valid = 1;
frame->lltrace = 0;
Expand Down
6 changes: 5 additions & 1 deletion Include/internal/pycore_interpframe_structs.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,13 @@ struct _PyInterpreterFrame {
char owner;
uint8_t visited:1;
uint8_t external:1; /* Set when the frame was pushed by something other than the VM */
/* Set for a thread-owned frame running a coroutine body whose coroutine
* object has not been created yet (see RETURN_GENERATOR in bytecodes.c).
* Its own _PyErr_StackItem lives on the data stack right after it. */
uint8_t lazy_coro:1;
#ifdef Py_DEBUG
uint8_t stackpointer_valid:1;
uint8_t lltrace:5;
uint8_t lltrace:4;
#endif
/* Locals and stack */
_PyStackRef localsplus[1];
Expand Down
3 changes: 2 additions & 1 deletion Include/internal/pycore_magic_number.h
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,7 @@ Known values:
Python 3.16a1 3704 (Replace DELETE_ATTR with PUSH_NULL; STORE_ATTR)
Python 3.16a1 3705 (Add INTRINSIC_ADD_CONDITIONAL_ANNOTATION)
Python 3.16a1 3706 (Add INTRINSIC_MATCH_CLASS_ISINSTANCE)
Python 3.16a2 3707 (Add ASYNC_CALL and ASYNC_CALL_KW)

Python 3.17 will start with 3750

Expand All @@ -313,7 +314,7 @@ Known values:

*/

#define PYC_MAGIC_NUMBER 3706
#define PYC_MAGIC_NUMBER 3707
/* This is equivalent to converting PYC_MAGIC_NUMBER to 2 bytes
(little-endian) and then appending b'\r\n'. */
#define PYC_MAGIC_NUMBER_TOKEN \
Expand Down
Loading
Loading