Skip to content

fix: lock the translator list when adding the local exception translator - #6208

Merged
rwgk merged 5 commits into
pybind:masterfrom
YukaSadaoka:fix-translator-race
Oct 24, 2026
Merged

rwgk merged 5 commits into
pybind:masterfrom
YukaSadaoka:fix-translator-race

Conversation

@YukaSadaoka

Copy link
Copy Markdown
Contributor

Description

Fixes #6207.

  • Fixing a data race on a module's first import with another thread fetching internals under free threaded runtime.
  • check_internals_local_exception_translator() reads and pushes onto internals.registered_exception_translators without holding exception_translator_mutex. This PR adds the mutex under Py_GIL_DISABLED as a fix.
  • It locks the mutex directly rather than using with_exception_translators() because that calls get_internals() which would run the hook again.
  • The hook only exists when __GLIBCXX__ isn't defined. Hence, libstdc++ builds are unaffected.

Test

  • test_local_translator_registration_vs_internals_fetch keeps calling ensure_internals() from pybind11_tests on a thread while the main thread first-imports a new module, mod_concurrent_import.
  • Ran test_exceptions.py locally with clang 19 + libc++ + -fsanitize=thread against a TSan-built free-threaded CPython 3.14.7.

Suggested changelog entry:

  • Fixed a data race on free-threaded Python when a module's local exception translator was registered while another thread was fetching internals.

@YukaSadaoka
YukaSadaoka requested a review from henryiii as a code owner October 22, 2026 21:46
@rwgk

rwgk commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

@YukaSadaoka I merged master and added a19ef4f to guard the new lock to free-threaded Python 3.14.0rc1 and newer.

Older free-threaded builds use a nonrecursive mutex here, which can deadlock if an exception translator imports another pybind11 module. The guard preserves existing 3.13t behavior, including the original race, while fixing it on 3.14+. This keeps this PR moving without adding another 3.13t workaround (it'd be non-trivial); the intent is to formally drop 3.13t support before the next release.

@rwgk

rwgk commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

@YukaSadaoka I'm working on bit rot fixes (for some of the CI failures here) separately. I'll come back here after the fixes are merged.

@rwgk

rwgk commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

@YukaSadaoka I'm working on bit rot fixes (for some of the CI failures here) separately. I'll come back here after the fixes are merged.

The bit rot fixes were merged with #6209, then the new master was merged here.

@rwgk rwgk left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving based on manual review and review with GPT-6.1-Sol ultra.

@rwgk
rwgk merged commit d797942 into pybind:master Oct 24, 2026
151 of 152 checks passed
@github-actions github-actions Bot added the needs changelog Possibly needs a changelog entry label Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs changelog Possibly needs a changelog entry

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: Data race on the exception translator list when importing modules on free threaded Python

2 participants