[conrade-ctc]

This adds experimental support to debug JIT-compiled C++ at the source level with gdb, from Python and from VS Code. It is off by default: only a build with -DCPPJIT_ENABLE_DEBUGGER_SUPPORT=ON has it. Without the option, the build has no new code, exported symbols or package files, and python/cppjit does not change.

In a build with the support, CPPJIT_DEBUGGER_SUPPORT=1 starts a debug session:

• The JIT compiles with -g -O0 -fstandalone-debug. With Write interpreter inputs to files for debuggers (experimental) CppInterOp#1156, each C++ input goes to a file, so gdb shows the source and binds file:line breakpoints, also in headers that cppjit.include loads.
• cppjit.debugger adds Python line breakpoints and steps through sys.monitoring.
• A gdb extension (cppjit/debugger/gdb) shows the Python frames between the C++ frames, with the variables of both. It reads CPython's _Py_DebugOffsets, so CPython needs no debug information (3.13 or later).
• examples/vscode-gdb uses it from VS Code through gdb's DAP server (gdb 16 or later). A small DAP proxy adds Python breakpoints, and steps between Python and C++. The demo reads a JSON file with nlohmann/json, and loads a prebuilt library without its source. Its AGENTS.md has notes for whoever continues the work, for example on lldb support.

Notes for review:

• The C++ changes are behind CPPJIT_WITH_DEBUGGER_SUPPORT: a hook pair around each JIT call in WrapperCall, a few exported variables for the debugger, and the import of cppjit.debugger in the module init.
• The gdb extension also reads a few layouts that the debug offsets do not give: frame globals, dict entries, and the CPPInstance fields. I think that is acceptable for an experimental feature, but a change of these layouts needs an update here.
• In a DAP session, a value shows at most 10000 children, and the proxy cancels an evaluation after 5 s. Without these limits, a watch on an object before its constructor runs can block gdb.
• The debugger CI job builds with the option and runs test/test_debugging.py with gdb. It fails if a test skips. The other cells skip the gdb tests, but run the two decoder tests from the source tree. Locally (Linux, gdb 16.3, CPython 3.13), all 26 tests pass with the option, and so does the full suite.
• The CppInterOp pin points at the branch of Write interpreter inputs to files for debuggers (experimental) CppInterOp#1156, for CI. I will reset it before merge.
• The Bazel build has no debugger option, so test_debugging.py has no Bazel target.
• file:line breakpoints need the CppInterOp change and a pin that has it. Breakpoints by function name work without it.

🤖 Done with the help of Claude Code (Opus 5.5, human in the loop)

[aaronj0]

Nice! So that your changes are tested on the CI, can you add a commit like cc7edf7 pointing to your CppInterOp remote and branch? That would stack your changes and any iterations on compiler-research/CppInterOp#1156 will automatically reflect here

[Vipul-Cariappa]

I have only looked through some of the READMEs. I wanted to know why GDB? I believe LLDB provides better API support than GDB. With GDB, you are working with pipes and parsing the output as though you are running it in a shell.

[Vipul-Cariappa]

Can we have type hints for all the Python files in python/cppjit-debugger folder?
Will make it a bit easier to maintain and review the code.

[conrade-ctc]

Done: all the files in python/cppjit-debugger have type hints now, and mypy finds no issues. The files use from __future__ import annotations, because gdb can embed an older Python.

[conrade-ctc]

Thanks for the review! The extension does not use pipes. gdb embeds Python, so the API calls run in the gdb process: a frame filter shows the Python frames, and the commands and pretty printers read the memory of the inferior. One place parses text, the output of info types, to find the C++ type of a proxy. VS Code talks to gdb's built-in DAP server (gdb 16 and later) through stdin and stdout, with JSON messages. Only the tests run gdb in batch mode and read its output.

I chose gdb because it is the default debugger on Linux, and it has the JIT interface and DAP built in. I agree that LLDB has a good API. The CPython decoder (gdb/cpython.py) needs only a function that reads memory, so I think an LLDB front end can use it too. Maybe we can do that as a follow up.

[vgvassilev]

It is also worth looking at how https://github.com/vgvassilev/clad recently implemented the no-source debug support (especially for lldb). That might be helpful here, too.

[conrade-ctc]

It is also worth looking at how https://github.com/vgvassilev/clad recently implemented the no-source debug support (especially for lldb). That might be helpful here, too.

My plan is to do a second pass after this lands that adds separate support for lldb, and I'll take a look at how it's done in clad. I was actually hoping to ask another person to do this that is in interested in this project, and I would oversea how they do it to make sure it is sensible ;) That way, I'm getting someone else interested in the project, and can parallelize :)

Okay with you to do that as a separate PR after this lands?

[vgvassilev]

Okay with you to do that as a separate PR after this lands?

Yes, absolutely!