You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
thanks for your detailed response. I have to say that the LD_LIBRARY_PATH partially worked in some cases, the library got loaded. However in cases where there were dependencies, further issues appeared particularly when the link (as revealed by otool -L) was prefixed by '@rpath'. I was able to fix these by editing these paths (which is possible in my use case), but we just need to note that it may cause difficulties for others.
We also found a solution to avoid needing to set the environment var. We use the following code
ct.CDLL(ctypes.util.find_library('mylib'))
instead of just passing the library. That works well, and perhaps you may recommend it in the docs.
ccabrera commented on Jun 26, 2021
ccabreramannequin
Mannequin
More actions
Re option 3: relying on DYLD_LIBRARY_PATH or DYLD_FRAMEWORK_PATH isn't a great solution either because of SIP. It's can get impractical to use in many standard Makefile-based build systems, for example. (cf. qmk/qmk_cli#60, Homebrew/discussions#1737) I imagine it can get quite vexing to set these variables but find them to not have the effect you expected them to.
One workaround for the example above would be to compile your own make program (or use one from a package manager) so that SIP doesn't sanitise your environment, but I imagine that's not an ideal solution either.
This still reproduces on macOS 27.2. I built a small test library that links only to libSystem, put it in ~/lib, and loaded it from three Pythons:
Python
CDLL("libgh88130.dylib")
full path
bare name + DYLD_LIBRARY_PATH=~/lib
find_library("gh88130")
python.org 3.15.2
OSError
OK
OK
finds it
Homebrew 3.14.0
OSError
OK
OK
finds it
Apple /usr/bin/python3 3.9.6
OSError
OK
OSError
finds it
With DYLD_PRINT_SEARCHING=1, the python.org build tries only the bare name, the cryptex prefix and /usr/lib (listed as "default fallback"). It never tries ~/lib or /usr/local/lib. Homebrew's Python is only ad-hoc signed, without the hardened runtime, and it doesn't try ~/lib either (it adds its own rpath, /opt/homebrew/lib). So this doesn't look specific to the hardened runtime. That matches what #160985 found.
DYLD_LIBRARY_PATH works for the python.org build, which is signed with com.apple.security.cs.allow-dyld-environment-variables. It has no effect on Apple's /usr/bin/python3.
Frameworks behave the same way. A test framework in ~/Library/Frameworks fails to load as CDLL("GH88130.framework/GH88130") in all three Pythons, though find_library finds it. With the framework in another folder, DYLD_FRAMEWORK_PATH makes the bare name work in the python.org and Homebrew builds, but not in Apple's.
I'd like to write the docs note. My plan is a short note under CDLL in Doc/library/ctypes.rst: on macOS, a bare library name is only searched for in system locations, so pass a full path. It would also mention DYLD_LIBRARY_PATH and DYLD_FRAMEWORK_PATH, with the caveat that some binaries ignore them.
One question: find_library() found the library in all three cases, but the docs now call it soft deprecated. Should the note mention it as an option, or leave it out?
Documenting what happens now is useful, but I'd frame it differently.
I think the docs should more clearly imply that ctypes calls dlopen, and inherits its quirks -- current and future ones, i.e. if Apple changes the details, CDLL will follow, without deprecation. Ignoring DYLD_LIBRARY_PATH is a consequence, not CDLL-on-macOS's contract.
I think the docs should say that all notes in man dlopen apply. (The earlier :manpage:`dlopen(3)` link doesn't go to a macOS-specific version, and there's a bunch of notes. Maybe we should link to the macOS man page, if Apple provides stable URL for it?)
I'd also quote the note from man dlopen directly, and mention that the python.org builds are affected (because readers might not be aware how their CPython was built).
BTW, the manpage suggests that CDLL("@rpath/libfoo.dylib") could work; does it?
Activity
vlazzarini commentedon May 31, 2021
Hi Ned,
thanks for your detailed response. I have to say that the LD_LIBRARY_PATH partially worked in some cases, the library got loaded. However in cases where there were dependencies, further issues appeared particularly when the link (as revealed by otool -L) was prefixed by '@rpath'. I was able to fix these by editing these paths (which is possible in my use case), but we just need to note that it may cause difficulties for others.
We also found a solution to avoid needing to set the environment var. We use the following code
instead of just passing the library. That works well, and perhaps you may recommend it in the docs.
ccabrera commentedon Jun 26, 2021
Re option 3: relying on
DYLD_LIBRARY_PATHorDYLD_FRAMEWORK_PATHisn't a great solution either because of SIP. It's can get impractical to use in many standard Makefile-based build systems, for example. (cf. qmk/qmk_cli#60, Homebrew/discussions#1737) I imagine it can get quite vexing to set these variables but find them to not have the effect you expected them to.One workaround for the example above would be to compile your own
makeprogram (or use one from a package manager) so that SIP doesn't sanitise your environment, but I imagine that's not an ideal solution either.mesubash commentedon Nov 7, 2026
This still reproduces on macOS 27.2. I built a small test library that links only to libSystem, put it in
~/lib, and loaded it from three Pythons:CDLL("libgh88130.dylib")DYLD_LIBRARY_PATH=~/libfind_library("gh88130")OSErrorOSError/usr/bin/python33.9.6OSErrorOSErrorWith
DYLD_PRINT_SEARCHING=1, the python.org build tries only the bare name, the cryptex prefix and/usr/lib(listed as "default fallback"). It never tries~/libor/usr/local/lib. Homebrew's Python is only ad-hoc signed, without the hardened runtime, and it doesn't try~/libeither (it adds its own rpath,/opt/homebrew/lib). So this doesn't look specific to the hardened runtime. That matches what #160985 found.DYLD_LIBRARY_PATHworks for the python.org build, which is signed withcom.apple.security.cs.allow-dyld-environment-variables. It has no effect on Apple's/usr/bin/python3.Frameworks behave the same way. A test framework in
~/Library/Frameworksfails to load asCDLL("GH88130.framework/GH88130")in all three Pythons, thoughfind_libraryfinds it. With the framework in another folder,DYLD_FRAMEWORK_PATHmakes the bare name work in the python.org and Homebrew builds, but not in Apple's.I'd like to write the docs note. My plan is a short note under
CDLLinDoc/library/ctypes.rst: on macOS, a bare library name is only searched for in system locations, so pass a full path. It would also mentionDYLD_LIBRARY_PATHandDYLD_FRAMEWORK_PATH, with the caveat that some binaries ignore them.One question:
find_library()found the library in all three cases, but the docs now call it soft deprecated. Should the note mention it as an option, or leave it out?ned-deily commentedon Nov 9, 2026
@encukou Any thoughts on this issue and the related PR (#161624) ?
encukou commentedon Nov 9, 2026
cc @ZeroIntensity
Documenting what happens now is useful, but I'd frame it differently.
I think the docs should more clearly imply that ctypes calls
dlopen, and inherits its quirks -- current and future ones, i.e. if Apple changes the details,CDLLwill follow, without deprecation. IgnoringDYLD_LIBRARY_PATHis a consequence, notCDLL-on-macOS's contract.I think the docs should say that all notes in
man dlopenapply. (The earlier:manpage:`dlopen(3)`link doesn't go to a macOS-specific version, and there's a bunch of notes. Maybe we should link to the macOS man page, if Apple provides stable URL for it?)I'd also quote the note from
man dlopendirectly, and mention that the python.org builds are affected (because readers might not be aware how their CPython was built).BTW, the manpage suggests that
CDLL("@rpath/libfoo.dylib")could work; does it?