Skip to content

Free-threaded: format() with a spec doesn't scale across threads (PyObject_Format refcounts the shared __format__) #162227

Open

Description

@mikeckennedy

Bug report

CPython versions: 3.15.2 and 3.14.10, free-threading builds (uv's python-build-standalone, Clang 23.1.3), macOS arm64. PyObject_Format and _PyObject_LookupSpecial read the same on main today.

On the free-threaded build, format(x, spec) (and so f"{x:spec}" and str.format with a spec) gets slower per call as threads are added, even though each thread formats only its own objects. At 6 threads it is about as slow as on a GIL build, where threads take turns. The same loops without a spec (str(x), f"{x}") scale.

Reproduction

repro.py (below, pure Python, no dependencies): each of N threads runs the same loop over its own ints, e.g. for i in range(n): format(i, "08d"). It reports ns per call per thread, median of 5 rounds.

3.15.2 free-threading build | GIL enabled: False
case                 1 thr      2 thr      4 thr      6 thr  Slowdown
with_spec          70.9 ns   120.0 ns   280.5 ns   443.5 ns     6.26x
fstring_spec       68.5 ns   120.0 ns   285.8 ns   465.3 ns     6.80x
dunder_spec        60.2 ns    63.8 ns    64.4 ns    64.0 ns     1.06x
without_spec       26.8 ns    31.8 ns    32.0 ns    34.4 ns     1.28x
fstring_plain      27.3 ns    30.7 ns    31.2 ns    31.9 ns     1.17x

3.14.10 free-threading build | GIL enabled: False
with_spec          65.9 ns   114.4 ns   290.7 ns   440.9 ns     6.69x
fstring_spec       72.5 ns   124.4 ns   288.8 ns   471.2 ns     6.50x
dunder_spec        58.8 ns    58.2 ns    58.6 ns    63.5 ns     1.08x
without_spec       25.1 ns    26.4 ns    26.7 ns    28.2 ns     1.13x
fstring_plain      24.9 ns    26.3 ns    26.4 ns    27.1 ns     1.09x

3.12.16 (GIL, for comparison)
with_spec          64.5 ns   130.6 ns   258.6 ns   406.5 ns     6.30x
without_spec       23.8 ns    48.1 ns    96.1 ns   144.7 ns     6.08x

(18-core machine, load average about 2.8 to 3.0 during the run.)

repro.py
"""format() with a spec doesn't scale across threads on free-threaded CPython.

Run: python3.14t repro.py   (also python3.15t; compare with a GIL build)

Each thread formats its own ints; nothing is shared but the type's __format__.
Per-call time should stay flat as threads are added. It grows with the thread
count for format(x, spec) / f"{x:.2f}", and stays flat for str(x) / f"{x}".

dunder_spec calls int.__format__(i, "08d") directly: the same formatting code,
reached through the interpreter's lookup instead of PyObject_Format's. It stays
flat, which puts the cost in PyObject_Format's lookup, not in the formatting.
"""

import statistics
import sys
import threading
import time

N = 200_000
THREADS = (1, 2, 4, 6)
ROUNDS = 5


def with_spec(n: int) -> None:
    for i in range(n):
        format(i, "08d")


def fstring_spec(n: int) -> None:
    for i in range(n):
        f"{i:08d}"


def dunder_spec(n: int) -> None:
    for i in range(n):
        int.__format__(i, "08d")


def without_spec(n: int) -> None:
    for i in range(n):
        str(i)


def fstring_plain(n: int) -> None:
    for i in range(n):
        f"{i}"


def run(fn, threads: int) -> float:
    barrier = threading.Barrier(threads + 1)

    def work() -> None:
        barrier.wait()
        fn(N)

    pool = [threading.Thread(target=work) for _ in range(threads)]
    for t in pool:
        t.start()
    barrier.wait()
    start = time.perf_counter()
    for t in pool:
        t.join()
    return (time.perf_counter() - start) / N * 1e9  # ns per call per thread


def main() -> None:
    gil = getattr(sys, "_is_gil_enabled", lambda: True)()
    print(sys.version.replace("\n", " "), "| GIL enabled:", gil)
    fns = (with_spec, fstring_spec, dunder_spec, without_spec, fstring_plain)
    print(f"{'case':<15}" + "".join(f"{f'{n} thr':>11}" for n in THREADS) + f"{'Slowdown':>10}")
    for fn in fns:
        cells = []
        for threads in THREADS:
            samples = [run(fn, threads) for _ in range(ROUNDS)]
            cells.append(statistics.median(samples))
        change = cells[-1] / cells[0]
        print(f"{fn.__name__:<15}" + "".join(f"{f'{c:.1f} ns':>11}" for c in cells) + f"{f'{change:.2f}x':>10}")
    print("ns per call per thread, median of", ROUNDS, "rounds; slowdown =", THREADS[-1], "threads / 1 thread")


if __name__ == "__main__":
    main()

Where the time goes

A macOS sample profile at 6 threads (3.15t) puts most of PyObject_Format's time in _PyObject_LookupSpecial, with _Py_DecRefShared and method_get under it, and _Py_MergeZeroLocalRefcount / meth_dealloc after the call.

In Objects/abstract.c, PyObject_Format has a fast path only for an empty spec on an exact str or int. Otherwise it calls _PyObject_LookupSpecial(obj, &_Py_ID(__format__)). In Objects/typeobject.c, that function:

  • takes a strong reference to the type's __format__ descriptor with _PyType_LookupRef. It's one object, shared by every thread formatting an int.
  • calls its tp_descr_get, building a new bound method object for each call.

Both are dropped after the call. So every call does atomic refcount operations on the same shared descriptor, plus an allocation and a free of a temporary method object. The threads contend on the descriptor's shared refcount.

The descriptor is created with deferred refcounting (descr_new), so _PyType_LookupStackRefAndVersion returns it without touching the count. _PyType_LookupRef then turns that stack reference into a real one with Py_NewRef, which is the shared increment, and _PyObject_LookupSpecial drops it again (_Py_DecRefShared in the profile).

The dunder_spec case in the table isolates this. It calls int.__format__(i, "08d") directly, which runs the same formatting code but finds __format__ through the interpreter's own lookup (a stack reference, no bound method). It stays flat (1.06x at 6 threads on 3.15t, 1.08x on 3.14t) while format(i, "08d") slows down about 6.5x. The formatting itself scales, and the cost is in PyObject_Format's lookup.

Possible direction

_PyObject_LookupSpecialMethod, defined right below _PyObject_LookupSpecial, already avoids both. It looks the method up as a stack reference (_PyType_LookupStackRefAndVersion), and for a method descriptor it skips the temporary method object and passes self separately. PyObject_Format could use it, or the same pattern, and so could other hot _PyObject_LookupSpecial callers. A fast path for exact int, float and str with a non-empty spec would also cover the common case. This is only a suggestion; the maintainers will know the right shape.

How it was found

We were profiling a template engine that renders pages on several threads at once. Formatting numbers with a spec was the part that stopped scaling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions