Skip to content

Windows ARM64 tail-call interpreter: compiler and commit performance effects #159521

Open

Description

@diegorusso

Update: The results below used Release builds without PGO or LTCG. See the [PGO+LTCG follow-up](#159521 (comment) for updated results and the maintained Gist for reproduction instructions.

While testing and measuring the tail-call interpreter on Windows ARM64, with the goal of enabling it by default on WoA, I noticed a performance change across commit ad1513a, "Reduce the work done to spill and reload the stack around calls".
The change avoids redundant stores and reloads of the interpreter's stack pointer around calls, keeping that pointer live across calls and increasing register pressure. Changing the compiler from MSVC 14.51 to 14.52 also affects performance.

I wanted to separate these effects before quoting a tail-call speedup.
The detailed results below are native ARM64 on a Windows Dev Kit 2023, pinned to performance-class CPU.

In terms of commits, both revisions are 3.16.0a0:

1. Effect of the commit and compiler version on the switch interpreter

Put all four switch based builds on one common scale: before-commit MSVC 14.51 = 100. Higher means faster.

Source revision MSVC 14.51 speed index MSVC 14.52 speed index
Before the commit 100.00 100.43
After the commit 92.49 96.80

Before the commit, the compilers produce almost equally fast builds. After the commit, both switch builds slow down: by 7.51% with MSVC 14.51 and 3.62% with MSVC 14.52, each relative to its own before-commit build.
MSVC 14.52 partly mitigates the switch interpreter regression introduced by the commit. Its larger lead after the commit reflects losing less performance across that change. We have measured this interaction; we have not yet established which difference in generated machine code explains it.

Emulated x64. Testing x64 binaries under Windows emulation on the same ARM64 device also shows a switch slowdown across the commit with both compilers. In those runs, 14.52 does not provide the mitigation observed on native ARM64.

The overall result varies across individual benchmarks: asyncio_websockets shows a measured speed loss of about 56% with 14.52 on both revisions.

Most affected benchmarks when changing the compiler: 14.52 versus 14.51

Each row below compares the switch build made with MSVC 14.52 against the switch build made with MSVC 14.51 at the same source revision. Positive means faster; negative means slower. † marks high variability (coefficient of variation above 10% in either compared build).

Source revision Largest measured gains Largest measured losses
Before the commit regex_effbot (+6.87%); base85_large (+5.88%); nqueens (+5.21%) asyncio_websockets (-55.85%); tornado_http† (-33.75%); ascii85_large (-4.71%)
After the commit generators (+30.37%); unpack_sequence (+28.93%); nbody (+16.72%) asyncio_websockets (-55.68%); tornado_http† (-22.28%); unpickle_list (-4.45%)
Most affected benchmarks across the commit: after versus before

Each row below compares the after-commit switch build against its before-commit switch build, keeping the compiler unchanged. Positive means faster; negative means slower. † marks high variability (coefficient of variation above 10% in either compared build).

Compiler Largest measured gains Largest measured losses
MSVC 14.51 python_startup_no_site† (+1.65%); gc_traversal (+0.93%); regex_dna (+0.45%) generators (-29.42%); tornado_http† (-25.86%); nbody (-25.79%)
MSVC 14.52 unpack_sequence (+2.76%); create_gc_cycles (+0.88%); gc_traversal (+0.59%) spectral_norm (-17.16%); nbody (-15.87%); tornado_http† (-13.03%)

2. Tail-call versus switch: MSVC 14.52

Change only the interpreter. Both builds use MSVC 14.52. For each revision, the baseline is the switch build at that same revision.

Source revision Tail-call build, compared with switch build using the same source and compiler
Before the commit 4.43% faster
After the commit 9.36% faster

Tail-call is faster than switch on both revisions: by 4.43% before the commit and 9.36% after it. These are two comparisons against the switch build at the corresponding revision. The previous table shows that the commit slows switch, which contributes to the larger after-commit tail-call advantage.
A claim of "9.36% faster" should therefore specify the MSVC 14.52 after-commit switch baseline. This is context for reporting relative speedups whilst keeping the goal to enabling tail-call by default on WoA.

Emulated x64. With MSVC 14.52 under x64 emulation, tail-call's advantage over switch at the same revision grows from 11.47% before the commit to 17.43% afterwards, mainly because switch slows down while tail-call speed changes little. These are relative comparisons within emulation; the percentages need confirmation on native x64 hardware.

Most affected benchmarks: three largest gains and losses per comparison

These use the same baseline as the table above. Positive means faster; negative means slower. † marks high variability (coefficient of variation above 10% in either compared build).

Source revision Largest measured gains Largest measured losses
Before the commit tornado_http† (+55.63%); hexiom (+19.41%); deltablue (+16.50%) dulwich_log (-0.17%); base85_large (-0.08%); ascii85_large (-0.07%)
After the commit tornado_http† (+112.05%); spectral_norm (+30.13%); nbody (+28.81%) pathlib† (-7.23%); asyncio_tcp (-2.25%); gc_traversal (-0.75%)

The tail-call interpreter should be enabled by default on Windows on Arm. Speedup claims should be precise and explicitly state the baseline, including the CPython revision, compiler version, and interpreter configuration.

I'd also welcome someone repeating these experiments on native x86-64 (x64) hardware running Windows, as I don't currently have access to such a machine. :)

The scripts and current reproduction instructions are maintained in the CPython Windows ARM64 PGO benchmark Gist. Download its four files together. To reproduce the PGO+LTCG follow-up, use the command there with --pgo; omitting --pgo selects the original non-PGO/no-LTCG configuration.

Linked PRs

Activity

  1. diegorusso commented on Oct 16, 2026

    @diegorusso
    ContributorAuthor

    Some feedback I received: we should build and test PGO builds. The above results are without it. I've put the scripts in a gist for now that should address the PGO and the plots (everyone likes plots).

    Here's the link: https://gist.github.com/diegorusso/3a0cfca565ad919ff094c88778cc17bd

    I'm re-running the benchmarks again and update the above test with the new results.

  2. diegorusso commented on Oct 20, 2026

    @diegorusso
    ContributorAuthor

    PGO+LTCG follow-up: native Windows ARM64

    Following the build-configuration feedback, I repeated the six ARM64 configurations with PGO+LTCG, keeping the source revisions, compiler toolsets and Dev Kit CPU affinity unchanged from the original experiment. This repeat covers ARM64 only.

    PGO+LTCG overview: switch performance across the commit and same-revision tail-call speedups
    Image

    1. Effect of the commit and compiler version on the switch interpreter

    The two speed-index columns share one baseline: before-commit MSVC 14.51 switch = 100. Higher means faster. The final column compares compilers at the same revision, using that row's 14.51 switch build as its baseline.

    Source revision MSVC 14.51 speed index MSVC 14.52 speed index 14.52 versus 14.51 at this revision
    Before the commit 100.00 101.60 +1.60%
    After the commit 89.76 97.83 +9.00%

    Across the commit, switch performance falls by 10.24% with 14.51 and 3.71% with 14.52, each relative to its own compiler's before-commit switch build. The compiler's lead therefore grows from 1.60% to 9.00%. The switch slowdown and its partial mitigation by 14.52 persist with PGO+LTCG.

    Most affected benchmarks: compiler and commit comparisons

    Each comparison names its baseline below. † marks a coefficient of variation above 10% in either compared build. These are ranked observed changes, not claims of statistical significance.

    For compiler comparisons, the baseline is the same-revision 14.51 switch build, and the changed build uses 14.52.

    Revision Largest measured gains Largest measured losses
    Before urlsafe_base64_small (+25.43%); base64_large (+23.71%); base16_small (+23.25%) json_loads (−5.88%); scimark_fft (−3.61%); asyncio_tcp (−3.34%)
    After tornado_http† (+64.84%); generators (+29.81%); richards (+24.28%) unpickle_list (−8.29%); pathlib† (−7.36%); json_loads (−3.10%)

    For commit comparisons, the baseline is the same compiler's before-commit switch build, and the changed build is after the commit.

    Compiler Largest measured gains Largest measured losses
    14.51 base64_large (+24.28%); base32_large (+22.59%); base16_large (+18.90%) tornado_http† (−36.20%); generators (−32.66%); logging_silent (−26.66%)
    14.52 asyncio_tcp (+10.28%); tornado_http† (+3.25%); connected_components (+1.60%) logging_silent (−14.19%); generators (−14.00%); richards (−11.39%)

    2. Tail-call versus switch with MSVC 14.52

    For each row, the baseline is the PGO+LTCG switch build at that same revision, using the same 14.52 toolset. Only interpreter selection changes.

    Source revision Tail-call speedup over the matching switch build
    Before the commit +5.25%
    After the commit +11.56%

    The larger after-commit advantage combines the switch build's 3.71% loss with the tail-call build's own 2.06% gain across the commit. These effects combine multiplicatively; 11.56% compares interpreter modes at the after revision.

    Most affected benchmarks when enabling tail-call

    The baseline is the same-revision 14.52 PGO+LTCG switch build; the changed build enables tail-call. † has the same variability definition as above.

    Revision Largest measured gains Largest measured losses
    Before coroutines (+21.04%); tomli_loads (+21.01%); tornado_http† (+20.21%) html5lib† (−18.87%); json_dumps (−12.21%); bench_mp_pool† (−11.39%)
    After nbody (+35.69%); coroutines (+35.17%); generators (+34.12%) asyncio_tcp (−4.06%); regex_dna (−1.31%); base16_large (−1.10%)

    The following bench_runner plots compare tail-call with the same-revision 14.52 switch baseline. Ratios above 1 mean faster. Red violins identify interpreter-heavy benchmarks. The violins use bench_runner's significance tests and outlier handling; the ALL row pools displayed ratios. The full-suite geomean in each title is the 121-measurement metric used in the tables above.

    Before the commit:

    Before the commit: MSVC 14.52 PGO+LTCG tail-call versus switch, with the switch build as baseline
    Image

    After the commit:

    After the commit: MSVC 14.52 PGO+LTCG tail-call versus switch, with the switch build as baseline.
    Image

    The maintained Gist contains the reproduction command; use --architectures ARM64 --pgo for this repeat.

    Overall summary

    Moving to PGO+LTCG changes the pattern of benchmark gains and losses, while preserving the main findings: the commit still slows the switch interpreter, and MSVC 14.52 partly mitigates that slowdown. In these runs, tail-call's advantage over the matching 14.52 switch build increases from 4.39% to 5.25% before the commit, and from 9.32% to 11.56% after it.

  3. chris-eibl commented on Oct 20, 2026

    @chris-eibl
    Member

    For people interested in even more data, see my gist for

    • Arm64 Cobalt 100
    • x64 Intel i5-4570
    • x64 Ryzen 5625U

    Cobalt 100 results are similar to @diegorusso's findings here: tail-calling is a clear win for Arm64:

    Before the commit: MSVC 14.52 PGO+LTCG tail-call versus switch, with the switch build as baseline

    Image

    After the commit: MSVC 14.52 PGO+LTCG tail-call versus switch, with the switch build as baseline.

    Image

    x64 as well still shows big wins of tail-calling over switch-case like back then in #139922 (gist).

    The "peculiarities" around the before / after commit wrt MSVC version are far less pronounced on x64, though.

    x64 Intel i5-4570

    Image

    x64 Ryzen 5625U

    Image

  4. diegorusso commented on Oct 20, 2026

    @diegorusso
    ContributorAuthor

    I guess the following can be claimed:

    On the Windows Dev Kit 2023, enabling the tail-call interpreter with MSVC 14.52 and PGO+LTCG delivered an 11.6% geometric-mean speedup across 121 successful pyperformance measurements, compared with the same-revision switch interpreter.

  5. pinned and unpinned a comment on Oct 20, 2026
  6. zware commented on Oct 21, 2026

    @zware
    Member

    My results such as they are can be found at https://gist.github.com/zware/bce63986f3765c731c5628b140013ec2

  7. added 2 commits that reference this issue on Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions