Skip to content

free-threading GC collect penalty on ARM MacOS and other ARM platform due to paired load #162248

Open

Description

@johng

During GC we can use this struct to buffer in traversed objects if total object count is over 200k.

typedef struct {
unsigned int in;
unsigned int out;
_PyObjectStack stack;
gc_span_stack_t spans;
PyObject *buffer[BUFFER_SIZE];
bool use_prefetch;
} gc_mark_args_t;

On M5 and other arm platforms a paired load is emitted due to the location of the in and out integer fields for tracking the buffers positions. If we instead adjust the struct such that it those two fields no longer sit next to each other like below. We instead have a single load as we iterate through the GC heap and store + load in. This seems to benefit from various load predictions vs the previous paired load.

--- a/Python/gc_free_threading.c
+++ b/Python/gc_free_threading.c
@@ -592,11 +592,15 @@ typedef struct {
 } gc_span_stack_t;

 typedef struct {
+    // Keep 'in' and 'out' apart.  If they are adjacent, AArch64 compilers
+    // read both with one paired load (ldp)
     unsigned int in;
-    unsigned int out;
     _PyObjectStack stack;
     gc_span_stack_t spans;
     PyObject *buffer[BUFFER_SIZE];
+    unsigned int out;
     bool use_prefetch;
 } gc_mark_args_t;

This can be shown with this micro benchmark timing gc collect.

import gc, statistics, time

gc.disable()
heap = [[i] * 5 for i in range(1_000_000)]
gc.collect()  # the first collection doesn't use the prefetch buffer yet
times = []
for _ in range(9):
    t0 = time.perf_counter()
    gc.collect()
    times.append(time.perf_counter() - t0)
print(f"{statistics.median(times) * 1e3:.1f} ms per gc.collect()")
CPU Platform Compiler main (ms) fix (ms) fix / main
Apple M5 macOS 27 Apple clang 21 19.2 12.5 0.65
Apple M5 macOS 27 Apple clang 21, PGO+LTO 18.6 13.2 0.71
Neoverse N3 (GCP n4a-standard-4) Ubuntu 24.04 clang 18.1 47.9 35.2 0.74
Neoverse N3 (GCP n4a-standard-4) Ubuntu 24.04 GCC 13.3 32.4 30.6 0.94
Neoverse V2 (GCP c4a-standard-4) Ubuntu 24.04 clang 18.1 33.8 33.3 0.98
Neoverse V2 (GCP c4a-standard-4) Ubuntu 24.04 GCC 13.3 30.7 31.5 1.03
Neoverse N1 (GCP t2a-standard-8) Ubuntu 24.04 clang 18.1 49.5 48.0 0.97
Neoverse N1 (GCP t2a-standard-8) Ubuntu 24.04 GCC 13.3 43.8 43.7 1.00

For M5 the improvement was clear from the load speculation of the in counters as the tree is traversed as it possible to explicitly disable that optimisation and see any performance gain be removed.

For the other Arm CPUs on the cloud platforms I tested with I was able to see benefit from the same patch but seemed be to different root cause than for M5, I'm unable yet to find a concrete conclusion.

For completeness some other workloads were tested which seemed to optimise better under different scenarios.

Workload Apple M5
Apple clang 21
Neoverse N3
clang 18
Neoverse N3
GCC 13
Neoverse V2
clang 18
Neoverse V2
GCC 13
Neoverse N1
clang 18
Neoverse N1
GCC 13
inst: linked instances 0.72 0.79 0.90 0.97 1.00 1.01 1.00
dict: small dicts 0.73 0.78 0.96 0.97 0.91 1.00 0.98
list: small lists 0.68 0.74 0.94 0.99 1.03 0.97 1.00
shuffled: dicts in random order 0.89 0.87 0.62 0.99 0.56 1.02 1.00
inst, 100k objects (below the 200k threshold) 0.97 0.99 1.00 0.99 1.00 0.98 1.00
  • workload being gc collecting a heap filled with X type of objects.

Wondering if there's any other insight into this and if it's worth this small change. x86 doesn't emit paired in load instructions so is unaffected either way.

Note I did try one real benchmark in building Python's Sphinx docs which did show GC improvements but total time was within noise due to already low amount.

main fix fix / main
Total GC pause (median) 3.51 s 3.29 s 0.94
Total GC pause (range) 3.46–3.51 s 3.28–3.32 s
Total build time (median) 38.6 s 38.2 s 0.99 (within noise)
GC share of build time 9.1% 8.6%

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions