Skip to content

ctypes CDLL search path issue on MacOS #88130

Open

Description

@vlazzarini
BPO 43964
Nosy @ronaldoussoren, @ned-deily, @vlazzarini

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-04-28.08:39:49.501>
labels = ['OS-mac', '3.9', '3.10', '3.11']
title = 'ctypes CDLL search path issue on MacOS'
updated_at = <Date 2021-06-26.19:26:05.420>
user = 'https://github.com/vlazzarini'

bugs.python.org fields:

activity = <Date 2021-06-26.19:26:05.420>
actor = 'ccabrera'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['macOS']
creation = <Date 2021-04-28.08:39:49.501>
creator = 'Victor.Lazzarini'
dependencies = []
files = []
hgrepos = []
issue_num = 43964
keywords = []
message_count = 5.0
messages = ['392173', '394592', '394782', '394788', '396555']
nosy_count = 4.0
nosy_names = ['ronaldoussoren', 'ned.deily', 'Victor.Lazzarini', 'ccabrera']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue43964'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Linked PRs

Activity

  1. added
    3.11only security fixes
    and removed on May 30, 2021
  2. vlazzarini commented on May 31, 2021

    vlazzarinimannequin
    MannequinAuthor

    Hi Ned,

    thanks for your detailed response. I have to say that the LD_LIBRARY_PATH partially worked in some cases, the library got loaded. However in cases where there were dependencies, further issues appeared particularly when the link (as revealed by otool -L) was prefixed by '@rpath'. I was able to fix these by editing these paths (which is possible in my use case), but we just need to note that it may cause difficulties for others.

    We also found a solution to avoid needing to set the environment var. We use the following code

    ct.CDLL(ctypes.util.find_library('mylib'))

    instead of just passing the library. That works well, and perhaps you may recommend it in the docs.

  3. ccabrera commented on Jun 26, 2021

    ccabreramannequin
    Mannequin

    Re option 3: relying on DYLD_LIBRARY_PATH or DYLD_FRAMEWORK_PATH isn't a great solution either because of SIP. It's can get impractical to use in many standard Makefile-based build systems, for example. (cf. qmk/qmk_cli#60, Homebrew/discussions#1737) I imagine it can get quite vexing to set these variables but find them to not have the effect you expected them to.

    One workaround for the example above would be to compile your own make program (or use one from a package manager) so that SIP doesn't sanitise your environment, but I imagine that's not an ideal solution either.

  4. transferred this issue fromon Apr 10, 2022
  5. added
    docsDocumentation in the Doc dir
    3.14bugs and security fixes
    3.15bugs and security fixes
    3.16new features, bugs and security fixes
    and removed
    3.11only security fixes
    on Oct 21, 2026
  6. mesubash commented on Nov 7, 2026

    @mesubash

    This still reproduces on macOS 27.2. I built a small test library that links only to libSystem, put it in ~/lib, and loaded it from three Pythons:

    Python CDLL("libgh88130.dylib") full path bare name + DYLD_LIBRARY_PATH=~/lib find_library("gh88130")
    python.org 3.15.2 OSError OK OK finds it
    Homebrew 3.14.0 OSError OK OK finds it
    Apple /usr/bin/python3 3.9.6 OSError OK OSError finds it

    With DYLD_PRINT_SEARCHING=1, the python.org build tries only the bare name, the cryptex prefix and /usr/lib (listed as "default fallback"). It never tries ~/lib or /usr/local/lib. Homebrew's Python is only ad-hoc signed, without the hardened runtime, and it doesn't try ~/lib either (it adds its own rpath, /opt/homebrew/lib). So this doesn't look specific to the hardened runtime. That matches what #160985 found.

    DYLD_LIBRARY_PATH works for the python.org build, which is signed with com.apple.security.cs.allow-dyld-environment-variables. It has no effect on Apple's /usr/bin/python3.

    Frameworks behave the same way. A test framework in ~/Library/Frameworks fails to load as CDLL("GH88130.framework/GH88130") in all three Pythons, though find_library finds it. With the framework in another folder, DYLD_FRAMEWORK_PATH makes the bare name work in the python.org and Homebrew builds, but not in Apple's.

    I'd like to write the docs note. My plan is a short note under CDLL in Doc/library/ctypes.rst: on macOS, a bare library name is only searched for in system locations, so pass a full path. It would also mention DYLD_LIBRARY_PATH and DYLD_FRAMEWORK_PATH, with the caveat that some binaries ignore them.

    One question: find_library() found the library in all three cases, but the docs now call it soft deprecated. Should the note mention it as an option, or leave it out?

  7. ned-deily commented on Nov 9, 2026

    @ned-deily
    Member

    @encukou Any thoughts on this issue and the related PR (#161624) ?

  8. encukou commented on Nov 9, 2026

    @encukou
    Member

    cc @ZeroIntensity

    Documenting what happens now is useful, but I'd frame it differently.

    I think the docs should more clearly imply that ctypes calls dlopen, and inherits its quirks -- current and future ones, i.e. if Apple changes the details, CDLL will follow, without deprecation. Ignoring DYLD_LIBRARY_PATH is a consequence, not CDLL-on-macOS's contract.

    I think the docs should say that all notes in man dlopen apply. (The earlier :manpage:`dlopen(3)` link doesn't go to a macOS-specific version, and there's a bunch of notes. Maybe we should link to the macOS man page, if Apple provides stable URL for it?)
    I'd also quote the note from man dlopen directly, and mention that the python.org builds are affected (because readers might not be aware how their CPython was built).

    BTW, the manpage suggests that CDLL("@rpath/libfoo.dylib") could work; does it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.14bugs and security fixes3.15bugs and security fixes3.16new features, bugs and security fixesOS-macdocsDocumentation in the Doc direasytopic-ctypes

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions