Skip to content

README.md

Latest commit

 

History

History
83 lines (66 loc) · 3.12 KB

README.md

File metadata and controls

83 lines (66 loc) · 3.12 KB
Raw

Finding a NumPy reference-counting bug with LLDB

Complete the reference-counting setup. This recreates the bug fixed by NumPy PR #23318. From the repository root, outside any other Pixi shell:

pixi shell --locked
git -C numpy-src apply ../debugger-tutorial/reintroduce-refcount-bug.patch
cd numpy-src
spin build
cd ../debugger-tutorial
python measure.py

measure.py should show about 100 extra references for 100 dtype constructions and 1000 for 1000, plus a small constant measurement offset.

Follow the writes

reproduce.py uses SIGUSR1 to stop after imports; its no-op handler lets us resume without terminating Python. On macOS, configure debugserver in this shell. The first command below finds the full path to the active environment's Python program, following symbolic links (shortcuts) to the actual file.

python_executable="$(python -c 'from pathlib import Path; import sys; print(Path(sys.executable).resolve())')"
pixi run --locked -e debugger lldb -- "$python_executable" "$PWD/reproduce.py"

Stop at the signal, then advance to the dtype allocation in the pinned NumPy source:

(lldb) process handle SIGUSR1 --stop true --notify true --pass true
(lldb) run
(lldb) breakpoint set --name arraydescr_new
(lldb) continue
(lldb) breakpoint delete 1
(lldb) thread until 2552
(lldb) watchpoint set expression -w write -s 8 -- &_PyRuntime.interpreters.main->object_state.reftotal
(lldb) watchpoint list -v
(lldb) continue
(lldb) bt
(lldb) continue
(lldb) bt

A hardware watchpoint uses CPU debug registers to catch writes without single-stepping. Here, -w write selects writes to the reference total, -s 8 sets the byte count, and & supplies the address. Check the hardware resources with watchpoint list -v; use the breakpoint/watchpoint IDs LLDB prints. Containers and VMs must allow debugging child processes and expose hardware watchpoints.

When creating or listing a watchpoint, Watchpoint 1 hit with hit_count = 0 shows the initial value; real stops may highlight the line after the write.

Compare the stacks: tp_alloc initializes the object through PyType_GenericAlloc, then PyObject_Init initializes it again:

PyType_GenericAlloc → arraydescr_new (descriptor.c:2552)
PyObject_Init       → arraydescr_new (descriptor.c:2557)

Both increment the debug total, but the object's reference count is reset to 1, so deleting it subtracts only once. Run watchpoint disable 1, continue twice (past the second signal), then quit.

Fix and verify

Remove PyObject_Init((PyObject *)descr, subtype); from arraydescr_new in ../numpy-src/numpy/_core/src/multiarray/descriptor.c, then rebuild:

cd ../numpy-src
spin build -j 4
cd ../debugger-tutorial
python measure.py

All batch sizes should now give the same small offset ([1, 1, 1, 1, 1]). For a Python API to hardware watchpoints on Linux, try libdebug.