KLayout crash exposed by a libc++ 22 behavior change

Discovery and investigation

After updating Xcode on macOS, I rebuilt KLayout and ran the QA tests.
dbLibrariesTests:7_monsterlib crashed in db::Layout::cleanup()—a failure I had not encountered with the previous toolchain.

I investigated a debug build with LLDB, with assistance from ChatGPT. The investigation showed that a forward scan over equivalent std::multimap keys started at find(), assuming that it returned the first matching entry.
With the updated libc++, it returned an entry partway through the group.
As a result, unregistering a library proxy could miss its entry, leaving a stale registration after the cell was deleted.

The official libc++ 22 release notes describe this implementation change:

“This was never guaranteed by the Standard, but libc++ previously happened to always return the first equal element, as other implementations do.”

The release notes recommend lower_bound() or equal_range() when the first equivalent entry is required.
This implementation change exposes an existing assumption, not a change to the C++ standard.

Targeted fix

I changed three calls from find() to lower_bound(), retaining the existing loop boundary checks:

-  auto l = m_lib_proxy_map.find (key);
+  auto l = m_lib_proxy_map.lower_bound (key);

File	Function	Locations
`dbLayout.cc`	`Layout::unregister_lib_proxy()`	1
`dbLibraryManager.cc`	`LibraryManager::lib_by_name()`	2

The first change resolved the crash. The other two addressed library lookup failures in dbLibrariesTests:4. After all three changes, the selected library tests passed in both debug and release builds on macOS.

This was a targeted investigation; I have not audited the entire source tree for similar assumptions.

Verification on Ubuntu

To investigate outside macOS, I installed Clang 22 and libc++ 22.1.8 alongside the existing libstdc++ environment on Ubuntu 22.04.
A small program test_multimap.cpp inserted twelve entries with the same key into a multimap, with values from 1 through 12:

#include <iostream>
#include <map>

int main()
{
    std::multimap<int, int> m;
    for (int i = 1; i <= 12; ++i)
        m.emplace(0, i);

    std::cout << "find:        " << m.find(0)->second << '\n';
    std::cout << "lower_bound: " << m.lower_bound(0)->second << '\n';
}

Compiler / standard library	Value returned by `find()`	Value returned by `lower_bound()`
GCC / libstdc++	1	1
Clang 22 / libstdc++	1	1
Clang 22 / libc++ 22.1.8	4	1

These results demonstrate the library-dependent behavior for this example; the particular value returned by find() should not be relied upon.

I then built KLayout with Clang 22 and -stdlib=libc++, testing both the original and patched commits.

CC=clang-22 \
CXX=clang++-22 \
AR=ar \
OBJCOPY=objcopy \
CFLAGS="-O2 -DNDEBUG" \
CXXFLAGS="-O2 -DNDEBUG -stdlib=libc++" \
LDFLAGS="-stdlib=libc++" \
LIBS="" \
./build.sh -expert -option -j6

Before/after results

Environment	Before: `e71272c3b`	After: `19a9a645e`
macOS arm64 / updated Xcode / libc++ 22.1.6	`7_monsterlib`: SIGSEGV in `Layout::cleanup()`, address `0x0`	Library tests: all 16 executions passed in each of the debug and release builds
Ubuntu 22.04 x86_64 / Clang 22 / libc++ 22.1.8 / release	`7_monsterlib`: SIGSEGV in `Layout::cleanup()`, address `0x0`	Library tests: all 16 executions passed
Ubuntu 22.04 x86_64 / GCC / libstdc++ / release	Not tested in this comparison	Library tests: all 16 executions passed

The 16 executions cover eight tests in both editable and non-editable modes.

The Ubuntu reproduction matches the macOS failure in test name, failing function, and fault address.
Although the exact source line and variables were not inspected in the Ubuntu release build, the comparison provides a Linux reproduction and supports the targeted fix.

A separate macOS crash in gsiExpressionTests:16 remains under investigation; the patched Ubuntu/libc++ build passed all 36 executions of gsiExpressionTests.

Xcode 27 Release Notes from Apple

https://developer.apple.com/documentation/xcode-release-notes/xcode-27-release-notes?utm_source=chatgpt.com　

Follow-up: verification with the previous Xcode toolchain

I reinstalled Xcode 26.6 alongside Xcode 27.1 and rebuilt the unmodified baseline commit e71272c3b, before the three find() → lower_bound() changes.

The libc++ header versions were confirmed using _LIBCPP_VERSION:

Xcode	`_LIBCPP_VERSION`	libc++ version
26.6	210106	21.1.6
27.1	220106	22.1.6

With Xcode 26.6, the release build passed both tests below in non-editable and editable modes:

Test	Result
`dbLibrariesTests:7_monsterlib`	Both modes passed
`gsiExpressionTests:16`	Both modes passed

The 7_monsterlib result complements the earlier Ubuntu reproduction: the unmodified baseline passes with libc++ 21.1.6 on macOS, but crashes with libc++ 22.1.6 on macOS and libc++ 22.1.8 on Ubuntu. This supports the diagnosis that the newer find() behavior exposed an existing assumption in KLayout.

The separate gsiExpressionTests:16 failure also did not reproduce with Xcode 26.6. However, the root cause remains unresolved: the earlier macOS debug build reported a heap use-after-free, and passing with the older toolchain doesn't prove the underlying issue is fixed or related to the multimap changes.

For now, I will use Xcode 26.6 for regular builds while keeping Xcode 27.1 available for further investigation.

Switching Between Xcode Versions (for self)

Both versions are installed under /Applications:

Xcode 26.6: /Applications/Xcode-26.6.app
Xcode 27.1: /Applications/Xcode.app

Adjust the paths below if the applications are renamed.

Select Xcode 26.6

sudo xcode-select --switch /Applications/Xcode-26.6.app/Contents/Developer

Select Xcode 27.x

sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer

These commands change the system-wide selection used by command-line developer tools.

Check the Active Toolchain

xcode-select --print-path
xcrun clang++ --version

The compiler's InstalledDir should point to the selected Xcode installation.

Check the libc++ Header Version

printf '#include <version>\n' |
  xcrun clang++ -stdlib=libc++ -dM -E -x c++ - |
  grep '^#define _LIBCPP_VERSION '

Confirmed on this Mac:

Xcode	`_LIBCPP_VERSION`	libc++ version
26.6	210106	21.1.6
27.1	220106	22.1.6

Temporarily Select Xcode for One Command

Use DEVELOPER_DIR to override the system-wide selection without changing it. No sudo is required.

DEVELOPER_DIR=/Applications/Xcode-26.6.app/Contents/Developer \
  xcrun clang++ --version

For example, check the older libc++ headers:

printf '#include <version>\n' |
  DEVELOPER_DIR=/Applications/Xcode-26.6.app/Contents/Developer \
  xcrun clang++ -stdlib=libc++ -dM -E -x c++ - |
  grep '^#define _LIBCPP_VERSION '

If DEVELOPER_DIR has previously been exported in the current shell, it overrides xcode-select. Remove that override to use the system-wide selection:

unset DEVELOPER_DIR

Rebuilding KLayout After Switching

Use a clean build when comparing toolchains. Move aside the previous build output to avoid mixing object files compiled with different Xcode versions.

Switching Xcode does not rebuild existing binaries.

Important: Xcode 26.6 Cannot Be Used on macOS GoldenGate

Xcode 26.6 cannot run on macOS GoldenGate in my environment. Therefore, keeping an older Xcode installation does not provide a workaround on that OS.

Impact on the KLayout Investigation

On macOS Tahoe, I confirmed that the unmodified baseline commit e71272c3b, built with Xcode 26.6 and libc++ 21.1.6, passes both of these tests in editable and non-editable modes:

dbLibrariesTests:7_monsterlib
gsiExpressionTests:16

However, this fallback depends on retaining a compatible macOS environment. It won't carry over to GoldenGate with Xcode 26.6.

Revised Plan

Retain the Tahoe / Xcode 26.6 environment for comparison and temporary use.
Keep the three find() → lower_bound() fixes for compatibility with libc++ 22.
Track the separate GSI heap-use-after-free issue as unresolved.

Downgrading Xcode is a temporary workaround on Tahoe, not a long-term solution for newer macOS environments.

Scope of This Observation

This incompatibility concerns Xcode 26.6 in my GoldenGate environment. I have not individually verified other Xcode 26.x releases.

@Kazzz-S

Hi @Kazzz-S,

thanks for bringing this up.

This applies to many places. I don't know why C++ needs the find implementation then anyway if there are no warranties. So far maybe the behavior maybe was not guaranteed but C++, but it was always the first element of the sequence of elements with identical keys. There are many non-warranties that are useful and C++ should not change. I hope these C++ purists know that.

I had to change many more places. I assume that help, although I can't find ways to reproduce problems for these other cases.

Best regards,

Matthias

Turns out that lower_bound is not exactly a replacement for find. The exact replacement is:

//  old:
iterator i = multimap.find(key);
if (i != multimap.end ()) {
...
}

//  new
iterator i = multimap.lower_bound(key);
if (i != multimap.end () && i->first == key) {
...
}

While find returned end() if it could not find an object with the given key, lower_bound will return the iterator pointing to a element past the given key.

Matthias

@klayoutmatthias

Hi @klayoutmatthias ,

Thank you for looking into this issue.
I've tested the issue-2518 branch (06be194) on Golden Gate and attached a QAT log file.
==> QATest_06be19450_2026_1023_0944__qt5MP.build.macos-GoldenGate-C32-release-RsysPsys.macQAT.zip

It still appears there are some potential issues.

macos27{sekigawa} ~ (1)% clang++ --version
Apple clang version 21.0.0 (clang-2100.3.23.3)
Target: arm64-apple-darwin27.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

macos27{sekigawa} ~ (2)% printf '#include <version>\n' |
  xcrun clang++ -stdlib=libc++ -dM -E -x c++ - |
  grep '^#define _LIBCPP_VERSION '
#define _LIBCPP_VERSION 220106

Best regards,
Kazzz-S

@Kazzz-S

Hi @Kazzz-S,

thanks for this additional information. I don't think there is a relationship to the multimap problem. Maybe there is something else. I will try to reproduce the problem with my tools at hand.

Matthias

I can see the problem. It does not crash for me, but I can see it in valgrind.

I used clang 21.1.8 with libc++ 22.1.2.

However, I also see a crazy number of other fails - like assertions in the OASIS writer and such. That combination is definitely not stable. That needs a lot more debugging :(

Matthias

I have added two patches to the branch: the first one fixes the gsiExpressionTests:16 failure. The reason was improper invalidation of references to objects stored in tl::Variant and most likely a change in the destruction order of std::map triggered the problem.

The second patch fixes the OASIS writer assertion I got with clang 21.1.8. However, that is not an obvious bug in KLayout code. Instead a dynamic_cast fails while it shouldn't. I had similar trouble in the past when compilers had problems keeping the RTTI tables in sync between different shared objects.

Maybe the second problem is specific to Linux. In that particular case, the fix is to replace dynamic_cast by static_cast. This is possible here, but not in general and I think that triggers other issues I have not debugged yet. So my advise is not to use clang 21 at least on Linux.

Matthias

@klayoutmatthias

Hi @klayoutmatthias ,

Thank you for the fix 24896c9.
The QA tests are now complete.
I have attached the logs from macOS 27 GoldenGate.

==> QATest_24896c9ab_2026_1028_1920__qt5MP.build.macos-GoldenGate-C32-release-RsysPsys.macQAT.zip

Best regards,
Kazzz-S

😄 Improved with 4a17a36.
==> QATest_4a17a3675_2026_1029_1054__qt5MP.build.macos-GoldenGate-C32-release-RsysPsys.macQAT.zip

Golden Gage Env. (arm64)

macos27{sekigawa} ~ (1)% uname -a
Darwin macos27.shared 27.2.0 Darwin Kernel Version 27.2.0: Wed Oct 14 23:44:09 PDT 2026; root:xnu-13432.41.4~25/RELEASE_ARM64_VMAPPLE arm64

macos27{sekigawa} ~ (2)% clang++ --version
Apple clang version 21.0.0 (clang-2100.3.23.3)
Target: arm64-apple-darwin27.2.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

macos27{sekigawa} ~ (3)% printf '#include <version>\n' |
  xcrun clang++ -stdlib=libc++ -dM -E -x c++ - |
  grep '^#define _LIBCPP_VERSION '
#define _LIBCPP_VERSION 220106 <===

Best regards,
Kazzz-S

@Kazzz-S

Hi @Kazzz-S, thanks a lot for your test reports!

The dbRecursiveShapeIteratorTests:10 fail scares me a bit. Is that unique to this branch and/or clang 21/libc++ 22?

Thanks,

Matthias

Again with 4a17a36 ...

Tahoe Env. (arm64)

MacMiniKS{sekigawa} ~ (1)% uname -a
Darwin MacMiniKS.local 25.6.0 Darwin Kernel Version 25.6.0: Mon Oct 12 00:41:28 PDT 2026; root:xnu-12377.161.15.701.48~2/RELEASE_ARM64_T8132 arm64

MacMiniKS{sekigawa} ~ (2)% clang++ --version
Apple clang version 21.0.0 (clang-2100.1.1.101)
Target: arm64-apple-darwin25.6.0
Thread model: posix
InstalledDir: /Applications/Xcode-26.6.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

MacMiniKS{sekigawa} ~ (3)% printf '#include <version>\n' |
  xcrun clang++ -stdlib=libc++ -dM -E -x c++ - |
  grep '^#define _LIBCPP_VERSION '
#define _LIBCPP_VERSION 210106 <===

==> QATest_4a17a3675_2026_1030_0813__qt5MP.build.macos-Tahoe-C32-release-RsysPsys.macQAT.zip

Sequoia Env. (x86_64)

MacBookPro2{sekigawa} ~ (1)% uname -a
Darwin MacBookPro2.local 24.6.0 Darwin Kernel Version 24.6.0: Sun Oct 11 23:36:58 PDT 2026; root:xnu-11417.140.69.713.47~2/RELEASE_X86_64 x86_64

MacBookPro2{sekigawa} ~ (2)% clang++ --version
Apple clang version 17.0.0 (clang-1700.0.13.5)
Target: x86_64-apple-darwin24.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

MacBookPro2{sekigawa} ~ (3)% printf '#include <version>\n' |
  xcrun clang++ -stdlib=libc++ -dM -E -x c++ - |
  grep '^#define _LIBCPP_VERSION '
#define _LIBCPP_VERSION 190102 <===

==> QATest_4a17a3675_2026_1030_1017__qt5MP.build.macos-Sequoia-C32-release-RsysPsys.macQAT.zip

Tahoe Env. (arm64) with the latest Xcode 27.2

MacMiniKS{sekigawa} ~ (1)% uname -a
Darwin MacMiniKS.local 25.6.0 Darwin Kernel Version 25.6.0: Mon Oct 12 00:41:28 PDT 2026; root:xnu-12377.161.15.701.48~2/RELEASE_ARM64_T8132 arm64

MacMiniKS{sekigawa} ~ (2)% clang++ --version
Apple clang version 21.0.0 (clang-2100.4.1.1)
Target: arm64-apple-darwin25.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

MacMiniKS{sekigawa} ~ (3)% printf '#include <version>\n' |
  xcrun clang++ -stdlib=libc++ -dM -E -x c++ - |
  grep '^#define _LIBCPP_VERSION '
#define _LIBCPP_VERSION 220106

MacMiniKS{sekigawa} qt5MP.build.macos-Tahoe-C32-release-RsysPsys.macQAT (4)% grep -n dbRecursiveShapeIteratorTests:10 *.log
277:  dbRecursiveShapeIteratorTests:10
4208:Running dbRecursiveShapeIteratorTests:10
4300:    Test dbRecursiveShapeIteratorTests:10 failed (continued mode - see previous messages)
18219:Running dbRecursiveShapeIteratorTests:10
18311:    Test dbRecursiveShapeIteratorTests:10 failed (continued mode - see previous messages)
31263:        dbRecursiveShapeIteratorTests:10
31272:        dbRecursiveShapeIteratorTests:10

==> QATest_4a17a3675_2026_1031_0852__qt5MP.build.macos-Tahoe-C32-release-RsysPsys.macQAT.zip

I could not see the RecursiveShapeIterator problem on Linux with libc++ 22 and clang 21, so I think it is related to the ARM platform. It needs further investigation, but it can't be a fundamental problem as otherwise many more tests should fail. I'm closing this ticket and create a new one specifically for this problem as the original PR contains some important patches also for Linux.

Thanks for the feedback.

Matthias

Navigation Menu

Repository navigation

KLayout crash exposed by a libc++ 22 behavior change #2518

Discovery and investigation

Targeted fix

Verification on Ubuntu

Before/after results

Xcode 27 Release Notes from Apple

Follow-up: verification with the previous Xcode toolchain

Switching Between Xcode Versions (for self)

Select Xcode 26.6

Select Xcode 27.x

Check the Active Toolchain

Check the libc++ Header Version

Temporarily Select Xcode for One Command

Rebuilding KLayout After Switching

Important: Xcode 26.6 Cannot Be Used on macOS GoldenGate

Impact on the KLayout Investigation

Revised Plan

Scope of This Observation

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

KLayout crash exposed by a libc++ 22 behavior change #2518

Description

Discovery and investigation

Targeted fix

Verification on Ubuntu

Before/after results

Activity

Kazzz-S commented on Oct 22, 2026

Xcode 27 Release Notes from Apple

Kazzz-S commented on Oct 22, 2026

Follow-up: verification with the previous Xcode toolchain

Kazzz-S commented on Oct 22, 2026

Switching Between Xcode Versions (for self)

Select Xcode 26.6

Select Xcode 27.x

Check the Active Toolchain

Check the libc++ Header Version

Temporarily Select Xcode for One Command

Rebuilding KLayout After Switching

Kazzz-S commented on Oct 22, 2026

Important: Xcode 26.6 Cannot Be Used on macOS GoldenGate

Impact on the KLayout Investigation

Revised Plan

Scope of This Observation

klayoutmatthias commented on Oct 22, 2026

klayoutmatthias commented on Oct 22, 2026

Kazzz-S commented on Oct 23, 2026

klayoutmatthias commented on Oct 25, 2026

klayoutmatthias commented on Oct 25, 2026

klayoutmatthias commented on Oct 27, 2026

Kazzz-S commented on Oct 28, 2026

Kazzz-S commented on Oct 29, 2026

klayoutmatthias commented on Oct 29, 2026

Kazzz-S commented on Oct 29, 2026

klayoutmatthias commented on Nov 1, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions