Support Unicode identifiers and macOS GNU3 manglings - #8585
Conversation
| { | ||
| if (params.empty() || params.front().type->GetClass() == VarArgsTypeClass) | ||
| throw DemangleException(); | ||
| params.front().name = "this"; |
There was a problem hiding this comment.
In source code explicit object parameters look like template <typename T> void Foo(this T&& self) { … }. this is not the parameter name, but a qualifier that appears prior to the parameter. I'm not sure what the consequences are of using it as the parameter name like this.
| return ch >= 0x20 && ch <= 0x7e && ch != '\\' && ch != '"'; | ||
| })) | ||
| return _STD_STRING(name); | ||
| const auto escaped = fmt::format("{:?}", name); |
There was a problem hiding this comment.
Our MSVC demangler appears to use Unicode::ToEscapedString for this. Is there a reason to do something different here?
| // Escape raw identifier bytes only, after consuming their encoded length. fmt's | ||
| // debug string formatting preserves printable UTF-8 and escapes controls, | ||
| // invisible Unicode, malformed UTF-8, and literal backslashes unambiguously. | ||
| inline _STD_STRING EscapeDemangledName(std::string_view name) |
There was a problem hiding this comment.
Is this header the right place for this to live? It doesn't seem related to reading.
| namespace | ||
| { | ||
| // Mangling grammar uses ASCII regardless of the locale or identifier encoding. | ||
| constexpr bool IsAsciiDigit(char ch) { return ch >= '0' && ch <= '9'; } |
There was a problem hiding this comment.
demangled_template_simplifier.cpp still contains calls to isdigit that seem like they could be problematic for the same reason we're moving off isdigit here.
| bool allTypeArgs = false; | ||
| type = DemangleNestedName(&allTypeArgs); | ||
| if (!m_inLocalName && allTypeArgs) | ||
| PushType(type); |
There was a problem hiding this comment.
While following through how state is saved/restored around the new calls to DemangleName in this PR, I noticed this adds substitution entries in cases it shouldn't. Itanium never makes the name of the entity being mangled a substitution candidate, so every later S<n>_ ends up off by one.
Consider this output from c++filt:
_ZN1A1BINS_1CEE1fENS_1DES2_S3_
A::B<A::C>::f(A::D, A::B<A::C>, A::D)
_ZN1A1fINS_1CEEEvNS_1BIT_EES4_PS3_S5_
void A::f<A::C>(A::B<A::C>, A::B<A::C>, A::C*, A::C*)
_ZSt2swIiEvPT_S1_St1WIS0_ES3_
void std::sw<int>(int*, int*, std::W<int>, std::W<int>)
Where we give:
>>> print(demangle_gnu3(bv.arch, "_ZN1A1BINS_1CEE1fENS_1DES2_S3_")[0])
int64_t(struct A::B<A::C>* this, A::D, A::B<A::C>, A::B<A::C>::f)
>>> print(demangle_gnu3(bv.arch, "_ZN1A1fINS_1CEEEvNS_1BIT_EES4_PS3_S5_")[0])
void(struct A* this, A::B<A::C>, A::C, A::B*, A::B<A::C>)
>>> print(demangle_gnu3(bv.arch, "_ZSt2swIiEvPT_S1_St1WIS0_ES3_")[0])
void(int32_t*, int32_t, std::W<std::sw<int32_t> >, std::W)
This isn't a new issue. I just noticed it while reviewing how the Ts/Tu/Te cases interact with DemangleName.
There was a problem hiding this comment.
Oh good find I'll create a new issue to track.
Replace locale-dependent character classification with ASCII grammar checks and remove the shared printable-ASCII input restriction. Preserve printable UTF-8 identifiers while escaping controls, invalid UTF-8, and literal escape characters before names cross the type API. Validate floating-point payloads and reference-temporary sequence IDs instead of copying arbitrary bytes. Handle Apple's block literals and virtual-function-pointer thunk suffixes, C++23 explicit object parameters, local generic-lambda template scopes, and nested elaborated types. Preserve explicit parameter names when rendering local function signatures, and keep GNU3 initialization inside its exception handler. Fixes #8584.
Expose ReadHexString(size_t) alongside the reader's other bounded reads and use it for all four GNU3 floating-point literal encodings. Preserve the existing ASCII validation and string-view result.
Parse Ty, Tn, Tt, and Tp declarations before their template argument, preserving the argument or pack's template substitution slot. Apply the existing nesting limit to recursive declarations and qualified arguments.
ebb7eed to
dacd956
Compare
2 participants
GNU3 and MSVC demangling now accepts printable UTF-8 identifiers such as
AGX·PIwhile escaping control and invisible characters, malformed UTF-8, and literal escape characters in emitted names. Replace locale-dependent character classification with ASCII grammar checks, validate numeric and hex payloads, and removeValidatePrintableAscii. Move bounded hex reads intoDemangleReader::ReadHexString.Add GNU3 support for:
_vfpthunk_suffixes.Fixes #8584. Follows up on #8583.