You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
JIT shim object drops GNU property notes (CET/BTI/PAC) from output binaries #149697
When building with --enable-experimental-jit the jit_shim.o object lacks GNU property notes for hardware security features. The linker removes all such properties from the output binaries when any input object lacks them.
The regression was introduced with 9633c52 which makes the shim being compiled as a regular object file.
You can reproduce it by using ./configure --enable-experimental-jit=yes-off CFLAGS="-fcf-protection" && make -j.
Before the aforementioned commit a readelf -n python | grep 'x86 feature' would show Properties: x86 feature: IBT, SHSTK, whereas now nothing.
This is the case as well for BTI/PAC/GCS for aarch64 when building with #130864 and -mbranch-protection=standard.
My testing included GCC15.2.1 for the interpreter and Clang 21.1.8 for the jit, on Fedora 43.
Digging into it more there are a couple more layers here beyond the property notes.
The stencils need landing pad instructions (endbr64 on x86_64 / BTI or aarch64) on their entry points because JIT code is entered via indirect branches. When CET/BTI is active the CPU will reject the branch if the target of an indirect branch lacks a landing pad.
This doesn't crash now with -fcf-protection (endbr64) or -mbranch-protection(bti), as the missing property note stops the kernel from enabling the hardware protections (if they are there). That is for x86_64, for aarch64 is the same but won't manifest till #130864 is resolved as well.
So just fixing the property note of the shim would make the JIT crash :(
For x86_64 if I pass -fcf-protection via CFLAGS_JIT (e.g. from extracting it via CFLAGS), things work fine, the stenctils get the endbr64 instruction and the shim the correct property note, all tests pass as well, but there is a cost of 4 bytes per stencil and one extra cycle per stencil at runtime (only the first stencil actually needs the instruction since it goes there from an indirect branch, for the rest the instruction becomes NOP).
Now for aarch64, passing -mbranch-protection=standard/bti from CFLAGS_JIT fails with KeyError: '_JIT_ENTRY', the stencil assembly optimizer doesn't handle properly the .note.gnu.property and the .Lsec_end0 label that is added to the assembly by clang. Same for the .ARM.attributes section type.
In summary, the x86_64 fix would be simple, but there may be a performance tradeoff, no idea how much.
diff --git a/Tools/jit/_optimizers.py b/Tools/jit/_optimizers.py
index f192783a559..ff104b2cdd2 100644
--- a/Tools/jit/_optimizers.py+++ b/Tools/jit/_optimizers.py@@ -573,6 +573,18 @@ class OptimizerAArch64(Optimizer): # pylint: disable = too-few-public-methods
# https://developer.arm.com/documentation/ddi0602/2025-09/Base-Instructions/RET--Return-from-subroutine-
_re_return = re.compile(r"\s*ret\b")
+ def _preprocess(self, text: str) -> str:+ text = super()._preprocess(text)+ # Strip .note.gnu.property section and aeabi build attributes emitted+ # by clang with -mbranch-protection. The note section contains a+ # .Lsec_end0: label that breaks the block structure of the optimizer.+ text = re.sub(r"^\s*\.aeabi_\w+.*\n", "", text, flags=re.MULTILINE)+ text = re.sub(+ r"^\s*\.section\s+\.note\.gnu\.property.*?^\.Lsec_end\d+:\n",+ "", text, flags=re.MULTILINE | re.DOTALL,+ )+ return text+
supports_small_constants = True
_re_small_const_1 = re.compile(
r"\s*(?P<instruction>adrp)\s+(?P<register>x\d\d?),.*(?P<value>_JIT_OP(ARG|ERAND(0|1))_(16|32)).*"
diff --git a/Tools/jit/_targets.py b/Tools/jit/_targets.py
index ceee383ea68..3d7b2b5110a 100644
--- a/Tools/jit/_targets.py+++ b/Tools/jit/_targets.py@@ -569,6 +569,7 @@ def _handle_section(
"SHT_NULL",
"SHT_STRTAB",
"SHT_SYMTAB",
+ "Unknown",
}, section_type
def _handle_relocation(
diff --git a/configure b/configure
index ecdd6095669..7c6c8f066d8 100755
--- a/configure+++ b/configure@@ -11311,6 +11311,19 @@ then :
else case e in #(
e) as_fn_append CFLAGS_NODIST " $jit_flags"
+ if ${CFLAGS_JIT:+false} :+then :++ for _cflag in $CFLAGS $CFLAGS_NODIST; do+ case $_cflag in #(+ -mbranch-protection=*|-fcf-protection|-fcf-protection=*) :+ as_fn_append CFLAGS_JIT " $_cflag" ;; #(+ *) :+ ;;+esac+ done++fi
REGEN_JIT_COMMAND="\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"$CFLAGS_JIT\" --llvm-version=\"$LLVM_VERSION\" --llvm-tools-install-dir=\"$LLVM_TOOLS_INSTALL_DIR\""
if test "x$Py_DEBUG" = xtrue
then :
diff --git a/configure.ac b/configure.ac
index 9863afe5bba..dad035e6387 100644
--- a/configure.ac+++ b/configure.ac@@ -2893,6 +2893,17 @@ AS_VAR_IF([jit_flags],
[],
[],
[AS_VAR_APPEND([CFLAGS_NODIST], [" $jit_flags"])
+ dnl If CFLAGS_JIT was not explicitly set, populate it with flags from+ dnl CFLAGS/CFLAGS_NODIST that affect GNU property notes. The linker+ dnl uses AND-merge for these properties, so every object in the link+ dnl must carry them or they are dropped from the output binary.+ AS_VAR_IF([CFLAGS_JIT], [], [+ for _cflag in $CFLAGS $CFLAGS_NODIST; do+ AS_CASE([$_cflag],+ [-mbranch-protection=*|-fcf-protection|-fcf-protection=*],+ [AS_VAR_APPEND([CFLAGS_JIT], [" $_cflag"])])+ done+ ])
AS_VAR_SET([REGEN_JIT_COMMAND],
["\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"$CFLAGS_JIT\" --llvm-version=\"$LLVM_VERSION\" --llvm-tools-install-dir=\"$LLVM_TOOLS_INSTALL_DIR\""])
AS_VAR_IF([Py_DEBUG],
I don't understand the JIT infra code well enough to be confident on this code but I tested it on both x86_64 and aarch64 on top of #130864 with enabling the respective hardware protections and the issue is resolved, JIT works, hardware protections are enabled and the final binary/libpython contain the proper notes.
I suppose it might break the approach for folks who build JIT with CFLAGS_JIT set. Should it append the flags if CFLAGS_JIT is set instead?
True, appending these flags unconditionally there is the solution (unconditionally as in whatever CFLAGS_JIT is set to, still the flags would be appended only if they are found).
Activity
stratakis commentedon May 11, 2026
cc @diegorusso
stratakis commentedon May 12, 2026
Digging into it more there are a couple more layers here beyond the property notes.
The stencils need landing pad instructions (endbr64 on x86_64 / BTI or aarch64) on their entry points because JIT code is entered via indirect branches. When CET/BTI is active the CPU will reject the branch if the target of an indirect branch lacks a landing pad.
This doesn't crash now with -fcf-protection (endbr64) or -mbranch-protection(bti), as the missing property note stops the kernel from enabling the hardware protections (if they are there). That is for x86_64, for aarch64 is the same but won't manifest till #130864 is resolved as well.
So just fixing the property note of the shim would make the JIT crash :(
For x86_64 if I pass
-fcf-protectionviaCFLAGS_JIT(e.g. from extracting it via CFLAGS), things work fine, the stenctils get the endbr64 instruction and the shim the correct property note, all tests pass as well, but there is a cost of 4 bytes per stencil and one extra cycle per stencil at runtime (only the first stencil actually needs the instruction since it goes there from an indirect branch, for the rest the instruction becomes NOP).Now for aarch64, passing
-mbranch-protection=standard/btifromCFLAGS_JITfails withKeyError: '_JIT_ENTRY', the stencil assembly optimizer doesn't handle properly the .note.gnu.property and the .Lsec_end0 label that is added to the assembly by clang. Same for the .ARM.attributes section type.In summary, the x86_64 fix would be simple, but there may be a performance tradeoff, no idea how much.
The aarch64 fix would be a lot more involved.
stratakis commentedon May 12, 2026
Fed it into Claude, Opus 4.6 and this came out:
diff
I don't understand the JIT infra code well enough to be confident on this code but I tested it on both x86_64 and aarch64 on top of #130864 with enabling the respective hardware protections and the issue is resolved, JIT works, hardware protections are enabled and the final binary/libpython contain the proper notes.
hroncok commentedon May 14, 2026
I suppose it might break the approach for folks who build JIT with CFLAGS_JIT set. Should it append the flags if CFLAGS_JIT is set instead?
stratakis commentedon May 14, 2026
True, appending these flags unconditionally there is the solution (unconditionally as in whatever CFLAGS_JIT is set to, still the flags would be appended only if they are found).
00504: Do not link the clang-built JIT shim into libpython
00504: Do not link the clang-built JIT shim into libpython