From 15570369eb651f21abd6e6b3c1c9aba9386f78ce Mon Sep 17 00:00:00 2001 From: Colby McHenry Date: Tue, 6 Oct 2026 22:04:50 -0500 Subject: [PATCH 1/2] fix(cpp): a receiver declared as a std or other outside type with a lowercase name calls that type's own member A C++ call on a variable, parameter or member the calling function or its class declares as a namespace-qualified type the project does not declare, with a lowercase name (std::string, std::vector, absl::flat_hash_set, ...), no longer falls through to Strategy 3's guess by the method's name. leveldb's `std::string saved_key_; saved_key_.clear()` reached Slice::clear, protobuf's `std::string proto; proto.append(...)` LeftoverBuffer::append. The gate applies to `.` on a value and `->` through a raw pointer; `->` on a smart pointer, iterator or optional still reaches the element type. It skips declarations it cannot trust: read from outside the caller's function and class (including a class nested in the caller's), contradicted by the call (`.` on a pointer), shadowed by an unreadable re-declaration (`auto`, range-for, structured binding), unqualified, with unbalanced `<>` (the tail of a multi-line declaration), or a `_t` name (scalars, tags, traits like std::conditional_t). A `std::` name counts as the project's only when the project declares it in `std` (vendored googletest's testing::internal::string). Receiver inference now blanks comments before matching declarations, so `// ... non-null imm_` no longer types leveldb's imm_ as `null`. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 1 + __tests__/cpp-library-type-receiver.test.ts | 522 ++++++++++++++++++++ src/resolution/name-matcher.ts | 181 ++++++- 3 files changed, 680 insertions(+), 24 deletions(-) create mode 100644 __tests__/cpp-library-type-receiver.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index fe1144fe4a..87524b06e3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - In C, C++ and Objective-C, a struct or class member declared as a pointer, reference, array or function, like `SharedState* shared;`, `jv elements[];` or `virtual Status Put(…) = 0;`, no longer makes the struct look like it inherits from the member's type. Before, `codegraph_explore` and impact listed those types as base classes, every struct holding a pointer to a type as one of its subclasses, and a method of that struct as an override of the type's method with the same name. Real base classes like `class Derived : public Base`, Objective-C superclasses and Go's embedded fields are unchanged. Re-index C, C++ and Objective-C projects after upgrading. - In C++, a call on a variable, parameter or member declared through a `typedef` or `using` alias now links to the method of the type the alias names, with the alias looked up the way C++ does it: in the calling function, its class and the classes that class inherits from, then the namespaces around it. In google/leveldb, `table_.Insert(buf)` on a `Table table_;` declared next to `typedef SkipList Table;` now reaches `SkipList::Insert` instead of the unrelated `HandleTable::Insert`, and `Table::Iterator iter(&table_);` reaches `SkipList::Iterator`'s constructor. A class that only shares the alias's name is no longer taken for it, and when the aliased type has no such method, as with an alias of `std::vector` or of a template parameter, no method is guessed from the receiver's name. Re-index C++ projects after upgrading. - In C++, a base class like `Message` in `class DynamicMessage final : public Message` now links to the class C++ itself would pick: the one declared in the namespaces around the class, or brought in by the file's `using` declarations and namespace aliases, and the class a `typedef` or `using` alias names. Before, a base class was matched by its name alone, so it could land on a class of the same name in another namespace or another language: in protocolbuffers/protobuf, `DynamicMessage` derived from the JSON parser's internal `Message` class, and in google/leveldb and RocksDB, iterators derived from an unrelated nested `Iterator` class such as the skip list's. The wrong base showed up in `codegraph_explore`'s type hierarchy and in impact, and tied the base's methods to the wrong overrides. A template parameter used as a base no longer links to a class that shares its name, and a base written from the global scope, like `public ::testing::Test`, now links to its class. Re-index C++ projects after upgrading. +- In C++, a call on a variable, parameter or member declared as a standard-library or other outside type with a lowercase name, like `std::string`, `std::vector`, `std::set` or `absl::flat_hash_set`, no longer links to a project method that only shares the method's name. In google/leveldb, `saved_key_.clear()` on a `std::string saved_key_;` reached `Slice::clear`, and in protocolbuffers/protobuf, `proto.append(…)` on a `std::string` reached an unrelated `LeftoverBuffer::append`. The same goes for `->` through a raw pointer like `std::string* out`, while `->` through a smart pointer, iterator or optional still links to the type it holds. Comments also no longer pass for a variable's declaration, so more calls now link to the method of the type the variable is really declared with. Re-index C++ projects after upgrading. - Framework support now switches on when an app's `package.json` sits three or more folders deep, as with an ASP.NET solution's Angular or React app in `src/Web/ClientApp/`, or a React app in a workspace folder like `web/ui/mantine-ui/`. Before, only the project root and the first two folder levels were read, so frameworks such as Angular, React Router, Vue Router, SvelteKit and TanStack Router could go undetected there: an Angular app's routes, its `routerLink` and `router.navigate(…)` navigation and its template's event bindings were missing, so a handler used only from a `(click)` binding had no callers, and a React app's `navigate('/login')` calls didn't reach their route. Re-index projects laid out this way after upgrading. - In Go, an interface that embeds another, like gin's `IRouter` embedding `IRoutes`, now links to it, so `codegraph_explore` and impact show the embedded interface as its base. Before, every embedded interface was dropped. An embedded field written through another package of your project, like `*refresh.Discovery` or `storage.Appender`, or with type arguments, like `List[int]`, now links too. One from the standard library or another module, like `io.Closer` or `sync.Mutex`, links to nothing rather than to a project type that shares its name, and a type-set constraint like `~int | ~float64` is not an embedding. An embedded type written without a package now always links to its own package's type: before, etcd's `Client`, which embeds its own `Lease` interface, was linked to a `Lease` struct in another package. Re-index Go projects after upgrading. - In Go, working out which structs implement an interface now counts the methods of the interfaces it embeds, and the methods a struct gets from the types it embeds. Before, only each type's own methods counted: etcd's `AuthReadTx`, which embeds `UnsafeAuthReader` and adds `RLock` and `RUnlock`, was listed as implemented by every read-write lock in the project, an interface that only embeds others had no implementations at all, and a struct that gets its methods from an embedded base, like prometheus's service discoveries embedding `refresh.Discovery` or gin's `Engine` embedding `RouterGroup`, was missing. `codegraph_explore`, impact and the type hierarchy now list the right implementations, and a call through the interface reaches a method the embedding struct overrides, like gin's `Engine.Use`. Re-index Go projects after upgrading. diff --git a/__tests__/cpp-library-type-receiver.test.ts b/__tests__/cpp-library-type-receiver.test.ts new file mode 100644 index 0000000000..70d368dcd1 --- /dev/null +++ b/__tests__/cpp-library-type-receiver.test.ts @@ -0,0 +1,522 @@ +/** + * A C++ receiver declared as a type from outside the project with a lowercase + * name — `std::string`, `std::vector<…>`, `absl::flat_hash_set<…>` — calls + * that type's own member, not a project method. + * + * google/leveldb's `DBIter` declares `std::string saved_key_;` and calls + * `saved_key_.clear()`. Receiver inference read the type as `string`, which + * names no project class, and the check for a type from outside the project + * only caught capitalized names (`List`, `String`), so the call fell through + * to a guess by the method's name: `Slice::clear`, the one project `clear`. + * protocolbuffers/protobuf had about 900 such edges (`std::string proto; + * proto.append(…)` reached `LeftoverBuffer::append`). + * + * When the calling function or its class declares the receiver that way, a + * `.` call — or a `->` through a raw pointer, `std::string* out` — gets no + * project edge. A `->` on a smart pointer, iterator or optional still goes to + * the element type, and a call that contradicts its declaration (`.` on a + * pointer) keeps its guess, since that declaration is another variable's. + * Comments no longer pass for declarations in the scan: `// … non-null imm_` + * read as `imm_`'s declaration, with the type `null`. + */ +import { describe, it, expect, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { CodeGraph } from '../src'; +import type { Node } from '../src/types'; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); +}); + +async function indexed(files: Record): Promise { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cpp-library-')); + roots.push(root); + for (const [rel, content] of Object.entries(files)) { + fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true }); + fs.writeFileSync(path.join(root, rel), content); + } + return CodeGraph.init(root, { index: true }); +} + +function callable(cg: CodeGraph, qualifiedName: string): Node { + const node = [...cg.getNodesByKind('method'), ...cg.getNodesByKind('function')].find((n) => n.qualifiedName === qualifiedName); + if (!node) throw new Error(`no function or method ${qualifiedName}`); + return node; +} + +/** `calls` callees of a function or method, as `qualifiedName`. */ +function calls(cg: CodeGraph, caller: string): string[] { + return cg + .getCallees(callable(cg, caller).id) + .filter((r) => r.edge.kind === 'calls') + .map((r) => r.node.qualifiedName) + .sort(); +} + +const lines = (...l: string[]): string => [...l, ''].join('\n'); + +/** leveldb's Slice and its namesakes elsewhere: every method a library type's call could be mistaken for. */ +const PROJECT = { + 'include/leveldb/slice.h': lines( + 'namespace leveldb {', + 'class Slice {', + ' public:', + ' const char* data() const { return data_; }', + ' unsigned size() const { return size_; }', + ' void clear() { size_ = 0; }', + ' const char* begin() const { return data_; }', + ' void remove_prefix(unsigned n) { size_ -= n; }', + ' private:', + ' const char* data_;', + ' unsigned size_;', + '};', + '} // namespace leveldb', + ), + 'util/buffers.h': lines( + 'namespace leveldb {', + 'class LeftoverBuffer {', + ' public:', + ' void append(const char* s, unsigned n) {}', + '};', + 'class Arena {', + ' public:', + ' void insert(int block) {}', + '};', + 'class Cache {', + ' public:', + ' void reset() {}', + '};', + 'class Iterator {', + ' public:', + ' void SeekToFirst() {}', + '};', + '} // namespace leveldb', + ), +}; + +describe('a C++ receiver declared as a library type with a lowercase name', () => { + it('reproduction: `std::string saved_key_; saved_key_.clear()` is not Slice::clear', async () => { + const cg = await indexed({ + ...PROJECT, + 'db/db_iter.cc': lines( + '#include ', + '#include "include/leveldb/slice.h"', + 'namespace leveldb {', + 'class DBIter {', + ' public:', + ' void Next();', + ' private:', + ' std::string saved_key_; // == current key when direction_==kReverse', + '};', + 'void DBIter::Next() {', + ' saved_key_.clear();', + '}', + '} // namespace leveldb', + ), + }); + try { + expect(calls(cg, 'leveldb::DBIter::Next')).toEqual([]); + expect(cg.getCallers(callable(cg, 'leveldb::Slice::clear').id).filter((r) => r.edge.kind === 'calls')).toEqual([]); + } finally { + cg.close(); + } + }); + + it('a local, a parameter, a raw pointer, nested template arguments and an abseil container', async () => { + const cg = await indexed({ + ...PROJECT, + 'db/encode.cc': lines( + '#include ', + '#include ', + '#include "absl/container/flat_hash_set.h"', + '#include "include/leveldb/slice.h"', + '#include "util/buffers.h"', + 'namespace leveldb {', + 'void Encode(std::string* dst, const Slice& value) {', + ' dst->clear();', + ' dst->append(value.data(), value.size());', + '}', + 'std::string Render(const std::vector>& files) {', + ' std::string proto;', + ' proto.append("x", 1);', + ' absl::flat_hash_set seen;', + ' seen.insert(1);', + ' const char* first = proto.data();', + ' return files.begin() == files.end() ? proto : std::string(first);', + '}', + '} // namespace leveldb', + ), + }); + try { + // `value` is a Slice; `dst`, `proto`, `seen` and `files` are not. + expect(calls(cg, 'leveldb::Encode')).toEqual(['leveldb::Slice::data', 'leveldb::Slice::size']); + expect(calls(cg, 'leveldb::Render')).toEqual([]); + } finally { + cg.close(); + } + }); + + it('a `std::` name beside a project alias of the same name elsewhere', async () => { + // facebook/rocksdb vendors googletest, whose `typedef ::std::string + // string;` in `testing::internal` is a project type named `string`. + const cg = await indexed({ + ...PROJECT, + 'third_party/gtest.h': lines( + '#include ', + 'namespace testing {', + 'namespace internal {', + 'typedef ::std::string string;', + '} // namespace internal', + '} // namespace testing', + ), + 'tools/trim.cc': lines( + '#include ', + '#include "include/leveldb/slice.h"', + 'void Trim(std::string& text) {', + ' text.clear();', + '}', + ), + }); + try { + expect(calls(cg, 'Trim')).toEqual([]); + } finally { + cg.close(); + } + }); + + it('a smart pointer: `.` is its own member, `->` reaches the element type', async () => { + const cg = await indexed({ + ...PROJECT, + 'db/scan.cc': lines( + '#include ', + '#include "util/buffers.h"', + 'namespace leveldb {', + 'void Scan(Iterator* raw) {', + ' std::unique_ptr iter(raw);', + ' iter->SeekToFirst();', + ' iter.reset();', + '}', + '} // namespace leveldb', + ), + }); + try { + // `iter.reset()` is unique_ptr's, not Cache::reset. + expect(calls(cg, 'leveldb::Scan')).toEqual(['leveldb::Iterator::SeekToFirst']); + } finally { + cg.close(); + } + }); + + it('a std trait or member type names another type, maybe the project\'s', async () => { + // facebook/rocksdb's compression.cc: `std::conditional_t dict` is a Slice when the flag is off. + const cg = await indexed({ + ...PROJECT, + 'util/compression.cc': lines( + '#include ', + '#include ', + '#include "include/leveldb/slice.h"', + 'namespace leveldb {', + 'template ', + 'unsigned DictSize(std::conditional_t dict) {', + ' return dict.size();', + '}', + 'void Drop(const std::vector& slices) {', + ' std::vector::value_type first = slices[0];', + ' first.remove_prefix(1);', + '}', + '} // namespace leveldb', + ), + }); + try { + expect(calls(cg, 'leveldb::DictSize')).toEqual(['leveldb::Slice::size']); + expect(calls(cg, 'leveldb::Drop')).toEqual(['leveldb::Slice::remove_prefix']); + } finally { + cg.close(); + } + }); + + it('a project type with a lowercase name is still the project\'s', async () => { + const cg = await indexed({ + ...PROJECT, + 'util/arena.h': lines( + 'namespace util {', + 'class arena {', + ' public:', + ' void* allocate(unsigned n) { return 0; }', + '};', + '} // namespace util', + ), + 'util/use.cc': lines( + '#include "util/arena.h"', + 'void Use(util::arena& a) {', + ' a.allocate(8);', + '}', + ), + }); + try { + expect(calls(cg, 'Use')).toEqual(['util::arena::allocate']); + } finally { + cg.close(); + } + }); +}); + +describe('a declaration that is not the receiver\'s does not rule a guess out', () => { + it('a range-for variable, or an `auto` one, hides an earlier library-typed one of the same name', async () => { + const cg = await indexed({ + ...PROJECT, + 'db/shadow.cc': lines( + '#include ', + '#include ', + '#include "include/leveldb/slice.h"', + '#include "util/buffers.h"', + 'namespace leveldb {', + 'void Trim(const std::vector& slices) {', + ' std::string key;', + ' for (const Slice& key : slices) {', + ' key.remove_prefix(1);', + ' }', + '}', + 'void Flush(Cache* caches) {', + ' { std::string buf; buf.clear(); }', + ' auto buf = caches[0];', + ' buf.reset();', + '}', + '} // namespace leveldb', + ), + }); + try { + expect(calls(cg, 'leveldb::Trim')).toEqual(['leveldb::Slice::remove_prefix']); + // The block's `buf.clear()` is the string's; the `auto` one's `reset` is a guess left alone. + expect(calls(cg, 'leveldb::Flush')).toEqual(['leveldb::Cache::reset']); + } finally { + cg.close(); + } + }); + + it('a call that contradicts the declaration it found: `.` on what reads as a pointer', async () => { + // As in fmtlib/fmt's bigint, where `carry = (upper * bigits_[i] …` reads + // as a declaration `upper* bigits_[…]`: the product here reads as + // `absl::kWeightScale* key[…]`, but `key.size()` is a `.` call. + const cg = await indexed({ + ...PROJECT, + 'util/weight.cc': lines( + '#include "include/leveldb/slice.h"', + 'namespace leveldb {', + 'unsigned Weight(const Slice& key) {', + ' unsigned w = absl::kWeightScale * key[0];', + ' return w + key.size();', + '}', + '} // namespace leveldb', + ), + }); + try { + expect(calls(cg, 'leveldb::Weight')).toEqual(['leveldb::Slice::size']); + } finally { + cg.close(); + } + }); + + it('a constant of a project class read as a type, and a scalar', async () => { + const cg = await indexed({ + ...PROJECT, + 'util/mask.cc': lines( + '#include ', + '#include "include/leveldb/slice.h"', + '#define SLICE_AT(var, list, i) const Slice& var = list[i]', + 'namespace leveldb {', + 'unsigned Mask(const Slice& key) {', + // reads like `Slice::kMask& key[…]` + ' unsigned m = Slice::kMask & key[0];', + ' return m + key.size();', + '}', + 'void Visit(const Slice* slices) {', + ' std::size_t key = 0;', + ' {', + // the macro declares the `key` the call is made on + ' SLICE_AT(key, slices, 0);', + ' key.remove_prefix(1);', + ' }', + '}', + '} // namespace leveldb', + ), + }); + try { + expect(calls(cg, 'leveldb::Mask')).toEqual(['leveldb::Slice::size']); + expect(calls(cg, 'leveldb::Visit')).toEqual(['leveldb::Slice::remove_prefix']); + } finally { + cg.close(); + } + }); + + it('a member of another class in the file', async () => { + // google/leveldb's db/log_test.cc: StringDest's `std::string contents_` + // sits above StringSource, whose own `Slice contents_` comes after its methods. + const cg = await indexed({ + ...PROJECT, + 'db/log_test.cc': lines( + '#include ', + '#include "include/leveldb/slice.h"', + 'namespace leveldb {', + 'class StringDest {', + ' public:', + ' std::string contents_;', + '};', + 'class StringSource {', + ' public:', + ' void Skip(unsigned n) { contents_.remove_prefix(n); }', + ' private:', + ' Slice contents_;', + '};', + '} // namespace leveldb', + ), + }); + try { + expect(calls(cg, 'leveldb::StringSource::Skip')).toEqual(['leveldb::Slice::remove_prefix']); + } finally { + cg.close(); + } + }); + + it('the last line of a declaration that starts on an earlier one', async () => { + // facebook/rocksdb's BlobFileReader: `autovector>>& blob_reqs` starts the next. + const cg = await indexed({ + 'util/autovector.h': lines( + 'namespace rocksdb {', + 'template ', + 'class autovector {', + ' public:', + ' unsigned size() const { return 0; }', + '};', + '} // namespace rocksdb', + ), + 'db/blob/blob_file_reader.cc': lines( + '#include ', + '#include "util/autovector.h"', + 'namespace rocksdb {', + 'class BlobFileReader {', + ' public:', + ' unsigned MultiGetBlob(autovector>>& blob_reqs);', + '};', + 'unsigned BlobFileReader::MultiGetBlob(autovector>>& blob_reqs) {', + ' return blob_reqs.size();', + '}', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::BlobFileReader::MultiGetBlob')).toEqual(['rocksdb::autovector::size']); + } finally { + cg.close(); + } + }); + + it('a member of a class nested in the caller\'s', async () => { + // facebook/rocksdb's MultiScan: its nested MultiScanIterator declares a + // `scan_opts_` of its own, above MultiScan's. + const cg = await indexed({ + 'include/rocksdb/multi_scan.h': lines( + '#include ', + 'namespace rocksdb {', + 'struct ScanOptions {};', + 'class MultiScanArgs {', + ' public:', + ' int GetScanRanges() const { return 0; }', + '};', + 'class MultiScan {', + ' public:', + ' class MultiScanIterator {', + ' private:', + ' const std::vector& scan_opts_;', + ' };', + ' int begin() { return scan_opts_.GetScanRanges(); }', + ' private:', + ' const MultiScanArgs scan_opts_;', + '};', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::MultiScan::begin')).toEqual(['rocksdb::MultiScanArgs::GetScanRanges']); + } finally { + cg.close(); + } + }); +}); + +describe('comments in the declaration scan', () => { + it('a comment that mentions the receiver is not its declaration', async () => { + // google/leveldb's db_impl.cc: `// … since there is a non-null imm_` + // above `imm_->…` read as `imm_`'s declaration, with the type `null`. + const cg = await indexed({ + 'db/memtable.h': lines( + 'namespace leveldb {', + 'class MemTable {', + ' public:', + ' unsigned ApproximateMemoryUsage() { return 0; }', + '};', + 'class Arena {', + ' public:', + ' unsigned ApproximateMemoryUsage() { return 0; }', + '};', + '} // namespace leveldb', + ), + 'db/db_impl.h': lines( + '#include "db/memtable.h"', + 'namespace leveldb {', + 'class DBImpl {', + ' public:', + ' unsigned MemoryUsage();', + ' private:', + ' MemTable* imm_;', + '};', + '} // namespace leveldb', + ), + 'db/db_impl.cc': lines( + '#include "db/db_impl.h"', + 'namespace leveldb {', + 'unsigned DBImpl::MemoryUsage() {', + ' // Count the memtable being compacted, since there is a non-null imm_', + ' return imm_->ApproximateMemoryUsage();', + '}', + '} // namespace leveldb', + ), + }); + try { + expect(calls(cg, 'leveldb::DBImpl::MemoryUsage')).toEqual(['leveldb::MemTable::ApproximateMemoryUsage']); + } finally { + cg.close(); + } + }); + + it('a parameter after a leading `/* static */` is the caller\'s own', async () => { + const cg = await indexed({ + ...PROJECT, + 'util/status.cc': lines( + '#include ', + '#include "include/leveldb/slice.h"', + 'namespace leveldb {', + 'class Status {', + ' public:', + ' static bool Check(const std::string& msg);', + '};', + '/* static */ bool Status::Check(const std::string& msg) {', + ' return msg.begin() != nullptr;', + '}', + '} // namespace leveldb', + ), + }); + try { + expect(calls(cg, 'leveldb::Status::Check')).toEqual([]); + } finally { + cg.close(); + } + }); +}); diff --git a/src/resolution/name-matcher.ts b/src/resolution/name-matcher.ts index 08dc5575fd..b00d118eb9 100644 --- a/src/resolution/name-matcher.ts +++ b/src/resolution/name-matcher.ts @@ -14,7 +14,7 @@ import { SWIFT_TYPE_PATH_CALL, resolveSwiftTypePathCall } from './swift-type-vis import { dartImportPrefixes, dartLibrarySees, dartPrefixSees, inSameDartLibrary } from './dart-libraries'; import { isDartLocallyBound } from './dart-local-scope'; import { breakVbTie, isVbMemberInScope, isVbNestedTypeInScope, isVbTypeQualifiedBy, matchVbTypedCall, preferVbProject, sameVbProject } from './vbnet-receivers'; -import { cppAliasedTypeName, isCppPointerType, resolveCppAliasedType } from './cpp-type-aliases'; +import { cppAliasedTypeName, cppTypeSegments, isCppPointerType, resolveCppAliasedType } from './cpp-type-aliases'; import { isTestPath } from '../search/query-utils'; import { isMinifiedContent } from '../extraction/generated-detection'; import { getCargoWorkspaceCrateMap } from './frameworks/cargo-workspace'; @@ -8302,6 +8302,16 @@ interface CppReceiverDeclaration { pointer?: boolean; /** The type is one of the project's class templates, whose specializations may add members. */ classTemplate?: boolean; + /** The declared type as written (`std::vector*`). */ + raw?: string; + /** The declaration is the calling function's own, or a member of its class (isCppCallersDeclaration). */ + callers?: boolean; + /** + * On its way back to the declaration the scan passed one of the receiver + * it could not read a type from — `auto x = Make();`, `for (Foo& x : xs)`, + * `auto [x, y] = …` — so the declaration it found is an earlier variable's. + */ + shadowed?: boolean; } /** @@ -8335,37 +8345,136 @@ function cppDeclaredType( return aliased ? cppAliasedTypeName(aliased) : null; } +const CPP_CLASSES_IN_FILE = new WeakMap>(); + +/** The classes, structs and unions a C or C++ file declares. */ +function cppClassesIn(file: string, context: ResolutionContext): Node[] { + let memo = CPP_CLASSES_IN_FILE.get(context); + if (!memo) CPP_CLASSES_IN_FILE.set(context, (memo = new Map())); + let classes = memo.get(file); + if (!classes) { + classes = context.getNodesInFile(file).filter((n) => n.kind === 'class' || n.kind === 'struct' || n.kind === 'union'); + memo.set(file, classes); + } + return classes; +} + /** - * Is line `line` of `file` — where `text` declares a C++ receiver at column - * `at` — the calling function's own code, or a member declaration of the - * caller's class? Receiver inference reads back to the top of the file and - * through the header, so a declaration it finds elsewhere (an earlier - * function, another class in the header, a comment) may be another variable - * of the same name. + * Is line `line` of `file`, which declares a C++ receiver, the calling + * function's own code, or a member declaration of the caller's class? + * Receiver inference reads back to the top of the file and through the + * header, so a declaration it finds elsewhere — an earlier function, another + * class in the header, a class nested in the caller's (rocksdb's + * `MultiScan::MultiScanIterator` has a `scan_opts_` of its own) — may be + * another variable of the same name. (The scan has blanked comments.) */ -function isCppCallersDeclaration(file: string, line: number, text: string, at: number, ref: UnresolvedRef, context: ResolutionContext): boolean { - if (/^\s*(?:\/\/|\/\*|\*)/.test(text)) return false; - const comment = text.indexOf('//'); - if (comment >= 0 && comment < at) return false; +function isCppCallersDeclaration(file: string, line: number, ref: UnresolvedRef, context: ResolutionContext): boolean { const caller = context.getNodeById?.(ref.fromNodeId); if (!caller || (caller.kind !== 'method' && caller.kind !== 'function')) return false; if (file === ref.filePath && line >= caller.startLine && line <= ref.line) return true; const cut = caller.qualifiedName.lastIndexOf('::'); if (cut < 0) return false; - return context.getNodesByQualifiedName(caller.qualifiedName.slice(0, cut)).some((owner) => - (owner.kind === 'class' || owner.kind === 'struct' || owner.kind === 'union') && owner.filePath === file && - line >= owner.startLine && line <= (owner.endLine ?? owner.startLine)); + let innermost: Node | undefined; + for (const cls of cppClassesIn(file, context)) { + if (line < cls.startLine || line > (cls.endLine ?? cls.startLine)) continue; + if (!innermost || cls.startLine >= innermost.startLine) innermost = cls; + } + return innermost?.qualifiedName === caller.qualifiedName.slice(0, cut); } -/** Is the C++ member call at `ref` written `receiver.method(…)`, not through `->`? */ -function isCppDotCall(receiver: string, ref: UnresolvedRef, context: ResolutionContext): boolean { +/** + * The operator the C++ member call at `ref` is written with: `.` for + * `receiver.method(…)`, `->` through a pointer, iterator or smart pointer, or + * null when the source doesn't show it. + */ +function cppMemberOperator(receiver: string, ref: UnresolvedRef, context: ResolutionContext): '.' | '->' | null { const lines = context.getFileLines?.(ref.filePath) ?? context.readFile(ref.filePath)?.split(/\r?\n/); const at = lines?.[ref.line - 1]?.slice(ref.column); - if (!lines || !at?.startsWith(receiver)) return false; + if (!lines || !at?.startsWith(receiver)) return null; let rest = at.slice(receiver.length); // `symbols_by_parent_` ending its line, `.insert(…)` starting the next. for (let next = ref.line; !rest.trim() && next < Math.min(lines.length, ref.line + 3); next++) rest = lines[next]!; - return /^\s*\./.test(rest); + const op = /^\s*(\.|->)/.exec(rest)?.[1]; + return op === '.' || op === '->' ? op : null; +} + +/** Is the C++ member call at `ref` written `receiver.method(…)`, not through `->`? */ +function isCppDotCall(receiver: string, ref: UnresolvedRef, context: ResolutionContext): boolean { + return cppMemberOperator(receiver, ref, context) === '.'; +} + +/** Kinds that declare a C or C++ type name. */ +const CPP_TYPE_KINDS: ReadonlySet = new Set(['class', 'struct', 'union', 'enum', 'interface', 'type_alias']); + +/** The C and C++ types the project declares under `name`. */ +function cppTypesNamed(name: string, context: ResolutionContext): Node[] { + return context.getNodesByName(name).filter((n) => (n.language === 'cpp' || n.language === 'c') && CPP_TYPE_KINDS.has(n.kind)); +} + +/** + * Is a C++ declared type, as written (`std::vector*`, `const + * absl::flat_hash_set&`), one from outside the project with a lowercase + * name, as the standard library's and abseil's are? (isUndeclaredTypeName + * takes the capitalized ones.) The name is qualified, by namespaces that are + * no project type's (`Foo::kMask & key` reads like a declaration of `key`), + * and the project declares no type of that name: in `std` itself for a + * `std::` name (googletest's `testing::internal::string` is no + * `std::string`), anywhere for any other, since a namespace a macro opens + * (fmt's `FMT_BEGIN_NAMESPACE`) is in no qualified name. An unqualified name + * is not known to be std's: fmt's `using namespace std;` is inside `namespace + * adl { … }`, and its `basic_string_view` is fmt's own. Nor is a type whose + * `>`s outnumber its `<`s: the end of a declaration begun on an earlier line + * (`autovector>>& reqs`). A + * `_t` name has no members of its own: it is a scalar (`std::size_t`, so + * the declaration is another variable's), a tag, or a trait that names + * another type (rocksdb's `std::conditional_t + * dict`). + */ +function isCppLibraryType(raw: string, context: ResolutionContext): boolean { + if ((raw.match(//g)?.length ?? 0)) return false; + const segments = cppTypeSegments(raw); + if (!segments || segments.length < 2) return false; + const name = segments[segments.length - 1]!; + if (!/^[a-z]/.test(name) || /_t$/.test(name)) return false; + if (segments.slice(0, -1).some((s) => cppTypesNamed(s, context).length > 0)) return false; + const declared = cppTypesNamed(name, context); + if (segments[0] !== 'std') return declared.length === 0; + const spelled = segments.join('::'); + return !declared.some((n) => n.qualifiedName === spelled || n.qualifiedName.endsWith(`::${spelled}`)); +} + +/** + * A C++ source line as code, for reading declarations: null for a line of a + * comment (`// …`, ` * …`), else the line with its comments blanked, columns + * kept. Receiver inference read leveldb's `// … non-null imm_` as a + * declaration of `imm_` with the type `null`. + */ +function cppCodeOf(line: string): string | null { + if (/^\s*(?:\/\/|\*)/.test(line)) return null; + let code = line.replace(/\/\*.*?\*\//g, (c) => ' '.repeat(c.length)); + // A block comment opening here runs past the line. + const open = code.indexOf('/*'); + if (open >= 0) code = code.slice(0, open); + const comment = code.indexOf('//'); + return comment >= 0 ? code.slice(0, comment) : code; +} + +/** + * Does `line` declare the receiver in a form the declarator pattern doesn't + * read: a range-`for` variable (`for (const Foo& x : xs)`) or a structured + * binding (`auto [x, y] = …`)? + */ +function cppRebindsReceiver(line: string, escapedReceiver: string): boolean { + return new RegExp(String.raw`\bfor\s*\(.*\b${escapedReceiver}\s*:(?!:)`).test(line) || + new RegExp(String.raw`\bauto\s*&{0,2}\s*\[[^\]]*\b${escapedReceiver}\b[^\]]*\]`).test(line); +} + +/** Record the C++ declaration receiver inference settles on. */ +function noteCppDeclaration(found: CppReceiverDeclaration | undefined, raw: string, callers: boolean): void { + if (!found) return; + found.raw = raw; + found.callers = callers; + found.pointer = isCppPointerType(raw); } /** @@ -8398,7 +8507,9 @@ function inferCppReceiverType( const declaratorRegex = buildDeclaratorRegex(escapedReceiver); for (let i = callLineIndex; i >= 0; i--) { - const line = lines[i]; + const text = lines[i]; + if (!text || !receiverPattern.test(text)) continue; + const line = cppCodeOf(text); if (!line || !receiverPattern.test(line)) continue; const declaratorMatch = line.match(declaratorRegex); @@ -8409,11 +8520,16 @@ function inferCppReceiverType( // from the initializer (call return type / construction) (#645). const initType = inferCppAutoInitializerType(line, receiverName, ref, context, depth); if (initType) return initType; - // No usable initializer on this line — keep scanning earlier ones. + // No usable initializer on this line — keep scanning earlier ones, + // where what is declared is an earlier variable. + if (found) found.shadowed = true; } else if (normalized) { - const inCallerScope = isCppCallersDeclaration(ref.filePath, i + 1, line, declaratorMatch.index ?? 0, ref, context); + const inCallerScope = isCppCallersDeclaration(ref.filePath, i + 1, ref, context); + noteCppDeclaration(found, declaratorMatch[1]!, inCallerScope); return cppDeclaredType(declaratorMatch[1]!, normalized, inCallerScope, ref, context, found); } + } else if (found && cppRebindsReceiver(line, escapedReceiver)) { + found.shadowed = true; } } @@ -8431,13 +8547,16 @@ function inferCppReceiverType( if (!headerLines) continue; for (let i = 0; i < headerLines.length; i++) { - const line = headerLines[i]!; - if (!receiverPattern.test(line)) continue; + const text = headerLines[i]!; + if (!receiverPattern.test(text)) continue; + const line = cppCodeOf(text); + if (!line) continue; const declaratorMatch = line.match(declaratorRegex); if (!declaratorMatch) continue; const normalized = normalizeCppTypeName(declaratorMatch[1] ?? ''); if (normalized && normalized !== 'auto') { - const inCallerScope = isCppCallersDeclaration(headerPath, i + 1, line, declaratorMatch.index ?? 0, ref, context); + const inCallerScope = isCppCallersDeclaration(headerPath, i + 1, ref, context); + noteCppDeclaration(found, declaratorMatch[1]!, inCallerScope); return cppDeclaredType(declaratorMatch[1]!, normalized, inCallerScope, ref, context, found); } } @@ -8952,6 +9071,7 @@ export function clearNameMatcherMemos(context: ResolutionContext): void { CPP_NS_MACROS.delete(context); CPP_NS_FRAMES.delete(context); CPP_NS_ALIASES.delete(context); + CPP_CLASSES_IN_FILE.delete(context); SOLIDITY_SUPERS.delete(context); DECLARED_SUPERS.delete(context); INHERITED_METHODS.delete(context); @@ -10188,6 +10308,19 @@ export function matchMethodCall( // followed. if (isUndeclaredTypeName(cppDecl.written!, ref, context)) return null; } + // A C++ receiver the calling function or its class declares as a type + // from outside the project with a lowercase name calls that type's own + // member, through `.` or a raw pointer's `->`: leveldb's `std::string + // data_; data_.data()` went to `Slice::data`, protobuf's `std::string + // proto; proto.append(…)` to `LeftoverBuffer::append`. Not so for `->` + // on such a value — an iterator, smart pointer or optional hands the call + // to its element type — nor for a call that doesn't fit the declaration + // (`.` on a pointer), which then is another variable's. + if (ref.language === 'cpp' && cppDecl.callers && !cppDecl.aliased && !cppDecl.shadowed && + cppMemberOperator(objectOrClass!, ref, context) === (cppDecl.pointer ? '->' : '.') && + isCppLibraryType(cppDecl.raw!, context)) { + return null; + } } // Go 2-hop field chain `base.field.Method` (#1276): the base's type comes From 03721e77db907014c566926ffda148dd2dd0f8f8 Mon Sep 17 00:00:00 2001 From: Colby McHenry Date: Wed, 7 Oct 2026 08:02:34 -0500 Subject: [PATCH 2/2] fix(cpp): a `->` call on a smart pointer or optional reaches the type it holds A C++ call made with `->` on a receiver the calling function or its class declares as a `std::unique_ptr`, `std::shared_ptr` or `std::optional` now resolves on `T`: on the class C++ name lookup finds for it where the call is written, or on a class that class derives from. Before, receiver inference read the type as `unique_ptr`, found no method, and the call fell through to the guess by method name: rocksdb's `std::unique_ptr iter; iter->Valid()` reached `ArenaWrappedDBIter::Valid`. The held type goes through #2407's lookup (`cppClassWritten`, extracted from `matchCppSupertype`), not `resolveMethodOnType` on its last name: rocksdb has 22 classes named `Iterator`, and the one the calls mean inherits its methods from `IteratorBase`, so a last-name lookup reaches `MemTableRep::Iterator`. The method is taken from that class or its bases through the class's own extends edges (`getSupertypeNodes`). A template parameter, a library type or an alias of one gets no edge; a type the project doesn't declare, or an alias the lookup can't follow, leaves the call as it was. The namespace-macro helpers move from name-matcher.ts to cpp-namespaces.ts unchanged, so name-matcher can import cpp-supertypes without a cycle. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 1 + __tests__/cpp-smart-pointer-receiver.test.ts | 450 +++++++++++++++++++ src/resolution/cpp-namespaces.ts | 113 +++++ src/resolution/cpp-supertypes.ts | 113 +++-- src/resolution/cpp-type-aliases.ts | 17 + src/resolution/index.ts | 16 + src/resolution/name-matcher.ts | 193 ++++---- src/resolution/types.ts | 8 + 8 files changed, 776 insertions(+), 135 deletions(-) create mode 100644 __tests__/cpp-smart-pointer-receiver.test.ts create mode 100644 src/resolution/cpp-namespaces.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 7a944e1c6c..ea51f8da5e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -71,6 +71,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - In C++, a call on a variable, parameter or member declared through a `typedef` or `using` alias now links to the method of the type the alias names, with the alias looked up the way C++ does it: in the calling function, its class and the classes that class inherits from, then the namespaces around it. In google/leveldb, `table_.Insert(buf)` on a `Table table_;` declared next to `typedef SkipList Table;` now reaches `SkipList::Insert` instead of the unrelated `HandleTable::Insert`, and `Table::Iterator iter(&table_);` reaches `SkipList::Iterator`'s constructor. A class that only shares the alias's name is no longer taken for it, and when the aliased type has no such method, as with an alias of `std::vector` or of a template parameter, no method is guessed from the receiver's name. Re-index C++ projects after upgrading. - In C++, a base class like `Message` in `class DynamicMessage final : public Message` now links to the class C++ itself would pick: the one declared in the namespaces around the class, or brought in by the file's `using` declarations and namespace aliases, and the class a `typedef` or `using` alias names. Before, a base class was matched by its name alone, so it could land on a class of the same name in another namespace or another language: in protocolbuffers/protobuf, `DynamicMessage` derived from the JSON parser's internal `Message` class, and in google/leveldb and RocksDB, iterators derived from an unrelated nested `Iterator` class such as the skip list's. The wrong base showed up in `codegraph_explore`'s type hierarchy and in impact, and tied the base's methods to the wrong overrides. A template parameter used as a base no longer links to a class that shares its name, and a base written from the global scope, like `public ::testing::Test`, now links to its class. Re-index C++ projects after upgrading. - In C++, a call on a variable, parameter or member declared as a standard-library or other outside type with a lowercase name, like `std::string`, `std::vector`, `std::set` or `absl::flat_hash_set`, no longer links to a project method that only shares the method's name. In google/leveldb, `saved_key_.clear()` on a `std::string saved_key_;` reached `Slice::clear`, and in protocolbuffers/protobuf, `proto.append(…)` on a `std::string` reached an unrelated `LeftoverBuffer::append`. The same goes for `->` through a raw pointer like `std::string* out`, while `->` through a smart pointer, iterator or optional still links to the type it holds. Comments also no longer pass for a variable's declaration, so more calls now link to the method of the type the variable is really declared with. Re-index C++ projects after upgrading. +- In C++, a call made with `->` on a variable, parameter or member declared as a `std::unique_ptr`, `std::shared_ptr` or `std::optional`, like `iter->Valid()` on a `std::unique_ptr iter`, now links to that method of the type it holds, or of a class that type inherits from, with the type looked up the way C++ does it, so a class of the same name nested elsewhere is not mistaken for it. Before, the call went to whichever project method shared its name: in facebook/rocksdb, `iter->Valid()` reached `ArenaWrappedDBIter::Valid` and `fs->GetFileSize(…)` on a `std::shared_ptr` reached a test's stand-in file system, and calls like `it->Next()` were often not linked at all. When what it holds is a template parameter or a standard-library type, like `std::unique_ptr>`, the call no longer links to a project method. Re-index C++ projects after upgrading. - Framework support now switches on when an app's `package.json` sits three or more folders deep, as with an ASP.NET solution's Angular or React app in `src/Web/ClientApp/`, or a React app in a workspace folder like `web/ui/mantine-ui/`. Before, only the project root and the first two folder levels were read, so frameworks such as Angular, React Router, Vue Router, SvelteKit and TanStack Router could go undetected there: an Angular app's routes, its `routerLink` and `router.navigate(…)` navigation and its template's event bindings were missing, so a handler used only from a `(click)` binding had no callers, and a React app's `navigate('/login')` calls didn't reach their route. Re-index projects laid out this way after upgrading. - In Go, an interface that embeds another, like gin's `IRouter` embedding `IRoutes`, now links to it, so `codegraph_explore` and impact show the embedded interface as its base. Before, every embedded interface was dropped. An embedded field written through another package of your project, like `*refresh.Discovery` or `storage.Appender`, or with type arguments, like `List[int]`, now links too. One from the standard library or another module, like `io.Closer` or `sync.Mutex`, links to nothing rather than to a project type that shares its name, and a type-set constraint like `~int | ~float64` is not an embedding. An embedded type written without a package now always links to its own package's type: before, etcd's `Client`, which embeds its own `Lease` interface, was linked to a `Lease` struct in another package. Re-index Go projects after upgrading. - In Go, working out which structs implement an interface now counts the methods of the interfaces it embeds, and the methods a struct gets from the types it embeds. Before, only each type's own methods counted: etcd's `AuthReadTx`, which embeds `UnsafeAuthReader` and adds `RLock` and `RUnlock`, was listed as implemented by every read-write lock in the project, an interface that only embeds others had no implementations at all, and a struct that gets its methods from an embedded base, like prometheus's service discoveries embedding `refresh.Discovery` or gin's `Engine` embedding `RouterGroup`, was missing. `codegraph_explore`, impact and the type hierarchy now list the right implementations, and a call through the interface reaches a method the embedding struct overrides, like gin's `Engine.Use`. Re-index Go projects after upgrading. diff --git a/__tests__/cpp-smart-pointer-receiver.test.ts b/__tests__/cpp-smart-pointer-receiver.test.ts new file mode 100644 index 0000000000..842e9755e5 --- /dev/null +++ b/__tests__/cpp-smart-pointer-receiver.test.ts @@ -0,0 +1,450 @@ +/** + * A C++ `->` call on a receiver declared as a standard smart pointer or + * optional — `std::unique_ptr iter; iter->Valid()` — calls a member + * of the type it holds. + * + * facebook/rocksdb has about 14,000 of these calls on a declaration in the + * calling function or its class. Receiver inference read the declared type as + * `unique_ptr`, which has no project methods, so the call fell through to a + * guess by the method's name and the receiver's words: `iter->Valid()` went + * to `ArenaWrappedDBIter::Valid`, `db_copy->Get()` to a test's `Get`. + * + * The held type is looked up as C++ looks a name up where the call is + * written — the scopes around the call, the file's `using`s, aliases + * followed — and the method is taken from that class or the classes it + * derives from. Its last name alone is not enough: rocksdb's `Iterator` is + * also the name of `MemTableRep::Iterator` and other nested classes, while the + * `Iterator` the call means inherits `Valid` from `IteratorBase`. A template + * parameter, a library type and an alias of a type outside the project hold + * no project class, so those calls get no edge; neither does a call the held + * class has no method for (unless a class template's specialization may + * declare it). A held type the project doesn't declare — a generated message + * — leaves the call as it was. + */ +import { describe, it, expect, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { CodeGraph } from '../src'; +import type { Node } from '../src/types'; + +const roots: string[] = []; +afterEach(() => { + for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); +}); + +async function indexed(files: Record): Promise { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cpp-held-')); + roots.push(root); + for (const [rel, content] of Object.entries(files)) { + fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true }); + fs.writeFileSync(path.join(root, rel), content); + } + return CodeGraph.init(root, { index: true }); +} + +function callable(cg: CodeGraph, qualifiedName: string): Node { + const node = [...cg.getNodesByKind('method'), ...cg.getNodesByKind('function')].find((n) => n.qualifiedName === qualifiedName); + if (!node) throw new Error(`no function or method ${qualifiedName}`); + return node; +} + +/** `calls` callees of a function or method, as `qualifiedName`. */ +function calls(cg: CodeGraph, caller: string): string[] { + return cg + .getCallees(callable(cg, caller).id) + .filter((r) => r.edge.kind === 'calls') + .map((r) => r.node.qualifiedName) + .sort(); +} + +const lines = (...l: string[]): string => [...l, ''].join('\n'); + +/** rocksdb's iterator family, trimmed: the public Iterator, a nested namesake, and an implementation a guess reaches. */ +const ROCKSDB = { + 'include/rocksdb/iterator_base.h': lines( + 'namespace rocksdb {', + 'class IteratorBase {', + ' public:', + ' virtual ~IteratorBase() {}', + ' virtual bool Valid() const = 0;', + ' virtual void Next() = 0;', + '};', + '} // namespace rocksdb', + ), + 'include/rocksdb/iterator.h': lines( + '#include "include/rocksdb/iterator_base.h"', + 'namespace rocksdb {', + 'class Iterator : public IteratorBase {', + ' public:', + ' virtual int value() const = 0;', + '};', + '} // namespace rocksdb', + ), + 'include/rocksdb/memtablerep.h': lines( + 'namespace rocksdb {', + 'class MemTableRep {', + ' public:', + ' class Iterator {', + ' public:', + ' virtual bool Valid() const = 0;', + ' virtual void Next() = 0;', + ' };', + '};', + '} // namespace rocksdb', + ), + 'db/arena_wrapped_db_iter.h': lines( + '#include "include/rocksdb/iterator.h"', + 'namespace rocksdb {', + 'class ArenaWrappedDBIter : public Iterator {', + ' public:', + ' bool Valid() const override { return true; }', + ' void Next() override {}', + ' int value() const override { return 0; }', + '};', + '} // namespace rocksdb', + ), + 'include/rocksdb/db.h': lines( + '#include "include/rocksdb/iterator.h"', + 'namespace rocksdb {', + 'class DB {', + ' public:', + ' virtual int Get(int key) = 0;', + ' virtual Iterator* NewIterator() = 0;', + '};', + 'class DBGetBatch {', + ' public:', + ' int Get(int key) { return key; }', + '};', + '} // namespace rocksdb', + ), +}; + +describe('a `->` call on a standard smart pointer or optional reaches the type it holds', () => { + it('reproduction: `std::unique_ptr iter; iter->Valid()` is the Iterator the scope sees, not a namesake', async () => { + const cg = await indexed({ + ...ROCKSDB, + 'db/db_impl.cc': lines( + '#include ', + '#include "include/rocksdb/db.h"', + '#include "db/arena_wrapped_db_iter.h"', + 'namespace rocksdb {', + 'int CountKeys(DB* db) {', + ' std::unique_ptr iter(db->NewIterator());', + ' int n = 0;', + ' for (; iter->Valid(); iter->Next()) n += iter->value();', + ' return n;', + '}', + '} // namespace rocksdb', + ), + }); + try { + // `Valid` and `Next` are IteratorBase's, which Iterator derives from. + expect(calls(cg, 'rocksdb::CountKeys')).toEqual([ + 'rocksdb::DB::NewIterator', + 'rocksdb::Iterator::value', + 'rocksdb::IteratorBase::Next', + 'rocksdb::IteratorBase::Valid', + ]); + } finally { + cg.close(); + } + }); + + it('`std::shared_ptr`, `std::optional`, a parameter, and a member the class declares in its header', async () => { + const cg = await indexed({ + ...ROCKSDB, + // Classes the receivers' names point a guess at. + 'table/table_db.h': lines( + 'namespace rocksdb {', + 'class TableDB {', + ' public:', + ' int Get(int key) { return key; }', + '};', + 'class CuckooTableDB {', + ' public:', + ' int Get(int key) { return key; }', + '};', + '} // namespace rocksdb', + ), + 'db/db_impl.h': lines( + '#include ', + '#include "include/rocksdb/db.h"', + 'namespace rocksdb {', + 'class DBImpl {', + ' public:', + ' int Lookup(int key);', + ' private:', + ' std::shared_ptr table_db_;', + '};', + '} // namespace rocksdb', + ), + 'db/db_impl.cc': lines( + '#include ', + '#include "db/db_impl.h"', + '#include "table/table_db.h"', + 'namespace rocksdb {', + 'int DBImpl::Lookup(int key) {', + ' return table_db_->Get(key);', + '}', + 'int Probe(const std::shared_ptr& cuckoo_db) {', + ' std::optional batch;', + ' return cuckoo_db->Get(1) + batch->Get(2);', + '}', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::DBImpl::Lookup')).toEqual(['rocksdb::DB::Get']); + expect(calls(cg, 'rocksdb::Probe')).toEqual(['rocksdb::DB::Get', 'rocksdb::DBGetBatch::Get']); + } finally { + cg.close(); + } + }); + + it('a nested class is found first from inside the class that declares it', async () => { + const cg = await indexed({ + ...ROCKSDB, + 'memtable/skiplistrep.cc': lines( + '#include ', + '#include "include/rocksdb/iterator.h"', + '#include "include/rocksdb/memtablerep.h"', + 'namespace rocksdb {', + 'class SkipListRep : public MemTableRep {', + ' public:', + ' bool Check();', + ' private:', + ' std::unique_ptr iter_;', + '};', + 'bool SkipListRep::Check() { return iter_->Valid(); }', + '} // namespace rocksdb', + ), + }); + try { + // MemTableRep's own `Iterator`, which SkipListRep inherits, hides rocksdb::Iterator. + expect(calls(cg, 'rocksdb::SkipListRep::Check')).toEqual(['rocksdb::MemTableRep::Iterator::Valid']); + } finally { + cg.close(); + } + }); + + it('a held type named through an alias, a `using namespace`, and a class local to the function', async () => { + const cg = await indexed({ + ...ROCKSDB, + 'table/internal_iterator.h': lines( + 'namespace rocksdb {', + 'template ', + 'class InternalIteratorBase {', + ' public:', + ' virtual void SeekToFirst() = 0;', + '};', + 'using InternalIterator = InternalIteratorBase;', + 'class MergingIterator : public InternalIterator {', + ' public:', + ' void SeekToFirst() override {}', + '};', + '} // namespace rocksdb', + ), + 'table/scan.cc': lines( + '#include ', + '#include "table/internal_iterator.h"', + 'namespace rocksdb {', + 'void ScanTable(InternalIterator* raw) {', + ' std::unique_ptr it(raw);', + ' it->SeekToFirst();', + '}', + 'class EventListener {', + ' public:', + ' void OnFlushCompleted() {}', + '};', + 'void Notify() {', + ' class FlushListener {', + ' public:', + ' void OnFlushCompleted() {}', + ' };', + ' std::shared_ptr listener;', + ' listener->OnFlushCompleted();', + '}', + '} // namespace rocksdb', + ), + 'tools/dump.cc': lines( + '#include ', + '#include "include/rocksdb/db.h"', + '#include "include/rocksdb/memtablerep.h"', + 'using namespace rocksdb;', + 'int Dump(DB* db) {', + ' std::unique_ptr it(db->NewIterator());', + ' return it->Valid() ? 1 : 0;', + '}', + ), + }); + try { + expect(calls(cg, 'rocksdb::ScanTable')).toEqual(['rocksdb::InternalIteratorBase::SeekToFirst']); + expect(calls(cg, 'rocksdb::Notify')).toEqual(['rocksdb::Notify::FlushListener::OnFlushCompleted']); + expect(calls(cg, 'Dump')).toEqual(['rocksdb::DB::NewIterator', 'rocksdb::IteratorBase::Valid']); + } finally { + cg.close(); + } + }); +}); + +describe('a held type that is no class of the project\'s gets no guess', () => { + it('a template parameter, a library type, and an alias of one', async () => { + const cg = await indexed({ + ...ROCKSDB, + 'table/block.h': lines( + 'namespace rocksdb {', + 'class Block {', + ' public:', + ' unsigned size() const { return 0; }', + ' int front() const { return 0; }', + '};', + '} // namespace rocksdb', + ), + 'db_stress_tool/stress.cc': lines( + '#include ', + '#include ', + '#include ', + '#include "db/arena_wrapped_db_iter.h"', + '#include "table/block.h"', + 'namespace rocksdb {', + 'template ', + 'void TestIterate(IterType* raw) {', + ' std::unique_ptr iter(raw);', + ' iter->Next();', + '}', + 'using BlockList = std::vector;', + 'class BlockBasedTableIterator {', + ' public:', + ' int First();', + ' private:', + ' std::unique_ptr> block_handles_;', + ' std::shared_ptr block_list_;', + '};', + 'int BlockBasedTableIterator::First() {', + ' return block_handles_->front() + static_cast(block_list_->size());', + '}', + '} // namespace rocksdb', + ), + }); + try { + // Not ArenaWrappedDBIter::Next, Block::front or Block::size. + expect(calls(cg, 'rocksdb::TestIterate')).toEqual([]); + expect(calls(cg, 'rocksdb::BlockBasedTableIterator::First')).toEqual([]); + } finally { + cg.close(); + } + }); + + it('a class that has no such method, nor does any class it derives from', async () => { + const cg = await indexed({ + ...ROCKSDB, + 'db/compaction.cc': lines( + '#include ', + '#include "include/rocksdb/db.h"', + '#include "db/arena_wrapped_db_iter.h"', + 'namespace rocksdb {', + 'class CompactionIterator {', + ' public:', + ' void Release() {}', + '};', + 'void Compact(DB* raw) {', + ' std::unique_ptr db(raw);', + ' db->Release();', + '}', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::Compact')).toEqual([]); + } finally { + cg.close(); + } + }); +}); + +describe('what keeps the call as it was', () => { + it('a type the project does not declare, such as a generated message, and a class template\'s specialization', async () => { + // protocolbuffers/protobuf's tests hold generated messages, which derive + // from Message; facebook/rocksdb's order-maintenance tree has members only + // a specialization declares. + const cg = await indexed({ + 'src/google/protobuf/message.h': lines( + 'namespace google {', + 'namespace protobuf {', + 'class Message {', + ' public:', + ' unsigned SpaceUsedLong() const { return 0; }', + '};', + '} // namespace protobuf', + '} // namespace google', + ), + 'src/google/protobuf/extension_set_unittest.cc': lines( + '#include ', + '#include "src/google/protobuf/message.h"', + 'namespace google {', + 'namespace protobuf {', + 'unsigned SpaceUsed() {', + ' std::unique_ptr message;', + ' return message->SpaceUsedLong();', + '}', + '} // namespace protobuf', + '} // namespace google', + ), + 'util/omt.h': lines( + '#include ', + 'namespace toku {', + 'template ', + 'class omt_node {', + ' public:', + ' void clear_stolen_bits() {}', + '};', + 'template ', + 'class omt_node {', + ' public:', + ' bool get_marked() const { return false; }', + '};', + 'bool HasMarks() {', + ' std::unique_ptr> node;', + ' return node->get_marked();', + '}', + '} // namespace toku', + ), + }); + try { + expect(calls(cg, 'google::protobuf::SpaceUsed')).toEqual(['google::protobuf::Message::SpaceUsedLong']); + expect(cg.getCallees(callable(cg, 'toku::HasMarks').id).filter((r) => r.edge.kind === 'calls').map((r) => r.node.name)).toEqual(['get_marked']); + } finally { + cg.close(); + } + }); + + it('`->` through a raw pointer to a smart pointer is the smart pointer\'s own member', async () => { + const cg = await indexed({ + ...ROCKSDB, + 'cache/cache.h': lines( + 'namespace rocksdb {', + 'class Cache {', + ' public:', + ' void reset() {}', + '};', + '} // namespace rocksdb', + ), + 'db/version_set.cc': lines( + '#include ', + '#include "include/rocksdb/iterator.h"', + '#include "cache/cache.h"', + 'namespace rocksdb {', + 'void Drop(std::unique_ptr* iter_cache) {', + ' iter_cache->reset();', + '}', + '} // namespace rocksdb', + ), + }); + try { + expect(calls(cg, 'rocksdb::Drop')).toEqual([]); + } finally { + cg.close(); + } + }); +}); diff --git a/src/resolution/cpp-namespaces.ts b/src/resolution/cpp-namespaces.ts new file mode 100644 index 0000000000..79c4d9ca68 --- /dev/null +++ b/src/resolution/cpp-namespaces.ts @@ -0,0 +1,113 @@ +/** + * C and C++ namespaces the source spells in ways the index cannot see: a + * namespace alias (`namespace py = pybind11;`) and a macro that opens a + * namespace (`FMT_BEGIN_NAMESPACE`). Shared by name matching and the C++ + * class lookup (cpp-supertypes.ts). + */ +import type { ResolutionContext } from './types'; +import { stripCommentsForRegex } from './strip-comments'; + +const CPP_NS_MACROS = new WeakMap; openerFns: Set; closers: Map; aliases: Map }>(); +const CPP_NS_FRAMES = new WeakMap>>(); +/** A closing macro's body: `}` / `} }`, maybe beside a pragma macro (`PYBIND11_WARNING_POP }`). */ +const CPP_CLOSER_BODY = /^(?:[A-Za-z_]\w*\s+)*\}(?:\s*\})*\s*;?$/; +const CPP_NS_ALIASES = new WeakMap>(); + +/** The project's namespace aliases: `namespace py = pybind11;`. */ +export function cppNamespaceAliases(context: ResolutionContext): Map { + const hit = CPP_NS_ALIASES.get(context); + if (hit) return hit; + const aliases = new Map(); + for (const file of context.getAllFiles()) { + if (!/\.(?:h|hh|hpp|hxx|inl|c|cc|cpp|cxx)$/i.test(file)) continue; + const source = context.readFile(file); + if (!source || !source.includes('namespace')) continue; + for (const m of source.matchAll(/^[ \t]*namespace[ \t]+([A-Za-z_]\w*)[ \t]*=[ \t]*(?:::)?([A-Za-z_][\w:]*)[ \t]*;/gm)) { + if (!aliases.has(m[1]!)) aliases.set(m[1]!, m[2]!); + } + } + CPP_NS_ALIASES.set(context, aliases); + return aliases; +} + +/** + * The project's namespace-opening macros — `#define FMT_BEGIN_NAMESPACE + * namespace fmt { inline namespace v12 {`, `#define RAPIDJSON_NAMESPACE_BEGIN + * namespace RAPIDJSON_NAMESPACE {` (through `#define RAPIDJSON_NAMESPACE + * rapidjson`) — as the namespace path each opens (inline namespaces are + * transparent), and the closing macros as how many scopes each closes. + */ +function cppNamespaceMacros(context: ResolutionContext): { openers: Map; openerFns: Set; closers: Map; aliases: Map } { + const hit = CPP_NS_MACROS.get(context); + if (hit) return hit; + const openers = new Map(); + // `#define PYBIND11_NAMESPACE_BEGIN(name) namespace name {`, used as `PYBIND11_NAMESPACE_BEGIN(detail)`. + const openerFns = new Set(); + const closers = new Map(); + const aliases = new Map(); + const bodies: Array<[string, string]> = []; + for (const file of context.getAllFiles()) { + if (!/\.(?:h|hh|hpp|hxx|h\+\+|inl|ipp|tcc)$/i.test(file)) continue; + const raw = context.readFile(file); + if (!raw || !raw.includes('#') || !raw.includes('define')) continue; + const source = stripCommentsForRegex(raw.replace(/\\\r?\n/g, ' '), 'cpp'); + for (const m of source.matchAll(/^[ \t]*#[ \t]*define[ \t]+([A-Za-z_]\w*)(\(\s*([A-Za-z_]\w*)?\s*\))?[ \t]+([^\n]*)$/gm)) { + const body = m[4]!.trim(); + if (m[2] !== undefined) { + // (a trailing pragma macro — `PYBIND11_WARNING_PUSH` — rides along) + if (m[3] && new RegExp(`^namespace\\s+${m[3]}\\s*\\{[\\w\\s]*$`).test(body)) openerFns.add(m[1]!); + else if (CPP_CLOSER_BODY.test(body)) closers.set(m[1]!, (body.match(/\}/g) ?? []).length); + continue; + } + if (/^[A-Za-z_]\w*$/.test(body)) aliases.set(m[1]!, body); + else if (CPP_CLOSER_BODY.test(body)) closers.set(m[1]!, (body.match(/\}/g) ?? []).length); + // An inline namespace (transparent, and often named by a macro call) is skipped. + else if (/^(?:inline\s+namespace\s+[^{}]*\{\s*|namespace\s+[A-Za-z_]\w*\s*\{\s*)+[\w\s]*$/.test(body)) bodies.push([m[1]!, body]); + } + } + for (const [name, body] of bodies) { + if (openers.has(name)) continue; + const path = [...body.replace(/inline\s+namespace\s+[^{}]*\{/g, '').matchAll(/namespace\s+([A-Za-z_]\w*)/g)] + .map((m) => aliases.get(m[1]!) ?? m[1]!); + if (path.length > 0) openers.set(name, path); + } + const macros = { openers, openerFns, closers, aliases }; + CPP_NS_MACROS.set(context, macros); + return macros; +} + +/** The line ranges of a C / C++ file each namespace macro opens, with the namespace path it opens. */ +export function cppMacroNamespaceFrames(file: string, context: ResolutionContext): Array<{ start: number; end: number; path: string[] }> { + let memo = CPP_NS_FRAMES.get(context); + if (!memo) { + memo = new Map(); + CPP_NS_FRAMES.set(context, memo); + } + const hit = memo.get(file); + if (hit) return hit; + const frames: Array<{ start: number; end: number; path: string[] }> = []; + const { openers, openerFns, closers, aliases } = cppNamespaceMacros(context); + if (openers.size > 0 || openerFns.size > 0) { + const lines = context.getFileLines?.(file) ?? context.readFile(file)?.split(/\r?\n/) ?? []; + const open: Array<{ start: number; path: string[] }> = []; + lines.forEach((text, i) => { + const m = /^[ \t]*([A-Z_][A-Z0-9_]*)(?:\(\s*([A-Za-z_]\w*)?\s*\))?[ \t]*;?[ \t]*(?:\/\/.*|\/\*.*\*\/[ \t]*)?\r?$/.exec(text); + const token = m?.[1]; + if (!token) return; + const arg = m[2]; + const path = arg !== undefined && openerFns.has(token) ? [aliases.get(arg) ?? arg] : arg === undefined ? openers.get(token) : undefined; + if (path) open.push({ start: i + 1, path }); + else if (closers.has(token) && open.length > 0) frames.push({ ...open.pop()!, end: i + 1 }); + }); + for (const frame of open) frames.push({ ...frame, end: lines.length }); + } + memo.set(file, frames); + return frames; +} + +/** Drop the per-context namespace memos (see ReferenceResolver.clearCaches). */ +export function clearCppNamespaceMemos(context: ResolutionContext): void { + CPP_NS_MACROS.delete(context); + CPP_NS_FRAMES.delete(context); + CPP_NS_ALIASES.delete(context); +} diff --git a/src/resolution/cpp-supertypes.ts b/src/resolution/cpp-supertypes.ts index ff47ebad21..f473e5e3bb 100644 --- a/src/resolution/cpp-supertypes.ts +++ b/src/resolution/cpp-supertypes.ts @@ -36,6 +36,10 @@ * * Only C and C++ declarations are candidates, and no other strategy (a * framework's, the import resolver, name matching) sees these references. + * + * The same lookup (cppClassWritten) finds the class a `std::unique_ptr`, + * `std::shared_ptr` or `std::optional` holds, for a `->` call on it, looked + * up from the calling function (name-matcher's matchCppHeldTypeCall). */ import type { Node } from '../types'; import type { ResolvedRef, ResolutionContext, UnresolvedRef } from './types'; @@ -47,7 +51,7 @@ import { cppTypeSegments, type CppNamedClass, } from './cpp-type-aliases'; -import { cppMacroNamespaceFrames, cppNamespaceAliases } from './name-matcher'; +import { cppMacroNamespaceFrames, cppNamespaceAliases } from './cpp-namespaces'; import { stripCommentsForRegex } from './strip-comments'; const CLASS_KINDS: ReadonlySet = new Set(['class', 'struct', 'union']); @@ -73,30 +77,73 @@ function visibleFrom(file: string): (n: Node) => boolean { export function matchCppSupertype(ref: UnresolvedRef, context: ResolutionContext): ResolvedRef | null { const derived = context.getNodeById?.(ref.fromNodeId); if (!derived || !isCFamily(derived) || !CLASS_KINDS.has(derived.kind)) return null; - const written = writtenBase(ref, context); + const found = cppClassWritten(writtenBase(ref, context), { + scope: cppParentScope(derived.qualifiedName), + file: derived.filePath, + line: ref.line, + parameters: cppTemplateParameters(derived, context), + self: derived, + }, ref, context); + if (!found) return null; + return found.exact + ? { original: ref, targetNodeId: found.node.id, confidence: 0.9, resolvedBy: 'qualified-name' } + : { original: ref, targetNodeId: found.node.id, confidence: 0.7, resolvedBy: 'exact-match' }; +} + +/** Where a C or C++ type name is written, which is where its lookup starts. */ +export interface CppNameSite { + /** + * The scope it is looked up from: a class's enclosing scope for its base + * clause, the class of a method for code in the method. + */ + scope: string; + /** The file that writes it: its translation unit decides which classes it sees, and its `using`s before `line` apply. */ + file: string; + line: number; + /** The template parameters in scope there; a name that starts with one names no particular class. */ + parameters: ReadonlySet; + /** A class the name cannot be: the deriving class, in its own base clause. */ + self?: Node; +} + +/** The class a C or C++ type name names. */ +export interface CppWrittenClass { + /** The class, or an alias of a type that is none of the project's classes. */ + node: Node; + /** Found by C++ name lookup, rather than as the only class of that name the file can see. */ + exact: boolean; +} + +/** + * The class a C or C++ type name written at `site` names, looked up as the + * comment at the top of this file says. Null when the name names no + * particular class (a template parameter, an enum, a member of a type outside + * the project); undefined when nothing the file can see is named so, or + * several classes could be. + */ +export function cppClassWritten(written: string, site: CppNameSite, ref: UnresolvedRef, context: ResolutionContext): CppWrittenClass | null | undefined { const segments = cppTypeSegments(written); if (!segments) return null; const global = written.startsWith('::'); - if (!global && cppTemplateParameters(derived, context).has(segments[0]!)) return null; + if (!global && site.parameters.has(segments[0]!)) return null; - const visible = visibleFrom(derived.filePath); - const scopes = cppScopesWithin(cppParentScope(derived.qualifiedName)); - const resolved = (target: Node | null | undefined): ResolvedRef | null => - target ? { original: ref, targetNodeId: target.id, confidence: 0.9, resolvedBy: 'qualified-name' } : null; + const visible = visibleFrom(site.file); + const scopes = cppScopesWithin(site.scope); + const resolved = (node: Node): CppWrittenClass => ({ node, exact: true }); - // The scopes around the class, or only the global one for `::Base`. - const lexical = classNode(cppClassNamed(written, global ? [''] : scopes, ref, context, visible), derived, context); - if (lexical !== undefined) return resolved(lexical); + // The scopes around the name, or only the global one for `::Base`. + const lexical = classNode(cppClassNamed(written, global ? [''] : scopes, ref, context, visible), site, context); + if (lexical !== undefined) return lexical && resolved(lexical); - // What the file writes before the class, the nearest first: each stands for + // What the file writes before the name, the nearest first: each stands for // the name's first segment, and the rest stays as written. const path = segments.join('::'); const first = /^\s*(?:::\s*)?[A-Za-z_]\w*/.exec(written)?.[0] ?? ''; const after = written.slice(first.length); - const usings = usingsIn(derived.filePath, context).filter((u) => u.line < ref.line).reverse(); + const usings = usingsIn(site.file, context).filter((u) => u.line < site.line).reverse(); const lookUp = (spelled: string): Node | undefined => { const absolute = spelled.startsWith('::'); - return classNode(cppClassNamed(spelled, absolute ? [''] : scopes, ref, context, visible), derived, context) ?? undefined; + return classNode(cppClassNamed(spelled, absolute ? [''] : scopes, ref, context, visible), site, context) ?? undefined; }; // `::_pbi::MessageGlobalsBase` under `namespace _pbi = ::google::protobuf::internal;`. if (segments.length > 1) { @@ -121,16 +168,16 @@ export function matchCppSupertype(ref: UnresolvedRef, context: ResolutionContext } // `fmt::detail::buffer`, where a macro opens `namespace fmt` around `detail::buffer`. if (segments.length > 1) { - const hit = pick(namedClasses(segments, derived, context, visible) - .filter((n) => macroQualifiedName(n, context) === path), derived); + const hit = pick(namedClasses(segments, site, context, visible) + .filter((n) => macroQualifiedName(n, context) === path), site.file); if (hit) return resolved(hit); } // The only class of that name the file can see, by its qualified tail. - const tail = namedClasses(segments, derived, context, visible) + const tail = namedClasses(segments, site, context, visible) .filter((n) => n.qualifiedName === path || n.qualifiedName.endsWith(`::${path}`)); - const own = tail.filter((n) => n.filePath === derived.filePath); + const own = tail.filter((n) => n.filePath === site.file); const sole = own.length > 0 ? own : tail; - return sole.length === 1 ? { original: ref, targetNodeId: sole[0]!.id, confidence: 0.7, resolvedBy: 'exact-match' } : null; + return sole.length === 1 ? { node: sole[0]!, exact: false } : undefined; } /** @@ -179,36 +226,36 @@ function stripTemplateArguments(text: string): string { } /** - * The node of the class a lookup found, as the deriving class's file sees it - * (null: the name names no particular class; undefined: keep looking). + * The node of the class a lookup found, as the file that writes the name sees + * it (null: the name names no particular class; undefined: keep looking). */ -function classNode(named: CppNamedClass, derived: Node, context: ResolutionContext): Node | null | undefined { +function classNode(named: CppNamedClass, site: CppNameSite, context: ResolutionContext): Node | null | undefined { if (named === null || named === undefined) return named; if ('alias' in named) return named.alias; const classes = context .getNodesByQualifiedName(named.cls) - .filter((n) => isCFamily(n) && CLASS_KINDS.has(n.kind) && n.id !== derived.id); + .filter((n) => isCFamily(n) && CLASS_KINDS.has(n.kind) && n.id !== site.self?.id); // A lookup that ended on the class itself (its namespace lost) found nothing. if (classes.length === 0) return undefined; - const visible = classes.filter(visibleFrom(derived.filePath)); - return pick(visible.length > 0 ? visible : classes, derived); + const visible = classes.filter(visibleFrom(site.file)); + return pick(visible.length > 0 ? visible : classes, site.file); } /** * One of several declarations of a class (separate translation units, - * vendored copies): the deriving class's own file's, then its header's, then - * the nearest by path. + * vendored copies): the one in `file` itself, then in its header, then the + * nearest by path. */ -function pick(classes: Node[], derived: Node): Node | undefined { +function pick(classes: Node[], file: string): Node | undefined { if (classes.length <= 1) return classes[0]; - const own = classes.find((n) => n.filePath === derived.filePath); + const own = classes.find((n) => n.filePath === file); if (own) return own; const stem = (p: string): string => p.replace(/\.[^./\\]+$/, ''); - const header = classes.find((n) => stem(n.filePath) === stem(derived.filePath)); + const header = classes.find((n) => stem(n.filePath) === stem(file)); if (header) return header; const shared = (p: string): number => { const a = p.split('/'); - const b = derived.filePath.split('/'); + const b = file.split('/'); let i = 0; while (i < a.length - 1 && i < b.length - 1 && a[i] === b[i]) i++; return i; @@ -216,11 +263,11 @@ function pick(classes: Node[], derived: Node): Node | undefined { return classes.reduce((best, n) => (shared(n.filePath) > shared(best.filePath) ? n : best)); } -/** C and C++ classes named like the last segment of `segments` that `derived`'s file can see. */ -function namedClasses(segments: readonly string[], derived: Node, context: ResolutionContext, visible: (n: Node) => boolean): Node[] { +/** C and C++ classes named like the last segment of `segments` that the file at `site` can see. */ +function namedClasses(segments: readonly string[], site: CppNameSite, context: ResolutionContext, visible: (n: Node) => boolean): Node[] { return context .getNodesByName(segments[segments.length - 1]!) - .filter((n) => isCFamily(n) && CLASS_KINDS.has(n.kind) && n.id !== derived.id && visible(n)); + .filter((n) => isCFamily(n) && CLASS_KINDS.has(n.kind) && n.id !== site.self?.id && visible(n)); } /** A declaration's qualified name with the namespaces macros open around it: `fmt::detail::buffer` for `detail::buffer`. */ diff --git a/src/resolution/cpp-type-aliases.ts b/src/resolution/cpp-type-aliases.ts index 374d579190..924aa3ecd3 100644 --- a/src/resolution/cpp-type-aliases.ts +++ b/src/resolution/cpp-type-aliases.ts @@ -124,6 +124,23 @@ export function cppTypeSegments(raw: string): string[] | null { return cppTypeName(raw)?.names ?? null; } +/** + * The top-level arguments of the first template argument list in a written + * type: `['Iterator']` for `const std::unique_ptr&`, `['Foo', + * 'Deleter']` for `std::unique_ptr>`. Null when it has + * none, or the list is not closed. + */ +export function cppTemplateArguments(raw: string): string[] | null { + const open = raw.indexOf('<'); + if (open < 0) return null; + let depth = 0; + for (let i = open; i < raw.length; i++) { + if (raw[i] === '<') depth++; + else if (raw[i] === '>' && --depth === 0) return splitTopLevel(raw.slice(open + 1, i)).map((a) => a.trim()); + } + return null; +} + /** C and C++ declare the names a C++ file can see (a header may be either). */ function isCFamily(n: Node): boolean { return n.language === 'cpp' || n.language === 'c'; diff --git a/src/resolution/index.ts b/src/resolution/index.ts index f71e8c0865..eb67236a50 100644 --- a/src/resolution/index.ts +++ b/src/resolution/index.ts @@ -288,6 +288,8 @@ export class ReferenceResolver { // 565s of combined worker time (~581µs each, recursion-multiplied). private supertypeGen = 0; private supertypeMemo = new Map(); + // getSupertypeNodes' memo, by node id, under the same generation tag. + private supertypeNodeMemo = new Map(); /** Invalidate the getSupertypes memo — call when resolved edges may have advanced. */ private advanceSupertypeGeneration(): void { @@ -295,6 +297,7 @@ export class ReferenceResolver { // Lazy invalidation via the gen tag; bound the map so a long run over many // batches doesn't accrete dead entries. if (this.supertypeMemo.size > 50_000) this.supertypeMemo.clear(); + if (this.supertypeNodeMemo.size > 50_000) this.supertypeNodeMemo.clear(); } // Node kinds are a small fixed set (~24), so this is a plain Map, not an LRU. // getNodesByKind returns the FULL node list for a kind; it was previously @@ -511,6 +514,7 @@ export class ReferenceResolver { this.methodMatchCache.clear(); this.methodOwnerIndexCache.clear(); this.supertypeMemo.clear(); + this.supertypeNodeMemo.clear(); this.supertypeGen++; this.nodesByKindCache.clear(); this.fileExistsMemo.clear(); @@ -874,6 +878,18 @@ export class ReferenceResolver { return supers; }, + getSupertypeNodes: (id: string) => { + const hit = this.supertypeNodeMemo.get(id); + if (hit && hit.gen === this.supertypeGen) return hit.nodes; + const nodes: Node[] = []; + for (const edge of this.queries.getOutgoingEdges(id, RESOLUTION_READ_EDGE_KINDS)) { + const target = this.nodeById(edge.target); + if (target) nodes.push(target); + } + this.supertypeNodeMemo.set(id, { gen: this.supertypeGen, nodes }); + return nodes; + }, + getImportMappings: (filePath: string, language) => { const cacheKey = filePath; const cached = this.importMappingCache.get(cacheKey); diff --git a/src/resolution/name-matcher.ts b/src/resolution/name-matcher.ts index 308e363dce..281c98b1db 100644 --- a/src/resolution/name-matcher.ts +++ b/src/resolution/name-matcher.ts @@ -14,7 +14,9 @@ import { SWIFT_TYPE_PATH_CALL, resolveSwiftTypePathCall } from './swift-type-vis import { dartImportPrefixes, dartLibrarySees, dartPrefixSees, inSameDartLibrary } from './dart-libraries'; import { isDartLocallyBound } from './dart-local-scope'; import { breakVbTie, isVbMemberInScope, isVbNestedTypeInScope, isVbTypeQualifiedBy, matchVbTypedCall, preferVbProject, sameVbProject } from './vbnet-receivers'; -import { cppAliasedTypeName, cppTypeSegments, isCppPointerType, resolveCppAliasedType } from './cpp-type-aliases'; +import { cppAliasedTypeName, cppTemplateArguments, cppTemplateParameters, cppTypeSegments, isCppPointerType, resolveCppAliasedType } from './cpp-type-aliases'; +import { clearCppNamespaceMemos, cppMacroNamespaceFrames, cppNamespaceAliases } from './cpp-namespaces'; +import { cppClassWritten } from './cpp-supertypes'; import { isTestPath } from '../search/query-utils'; import { isMinifiedContent } from '../extraction/generated-detection'; import { getCargoWorkspaceCrateMap } from './frameworks/cargo-workspace'; @@ -8444,6 +8446,84 @@ function isCppLibraryType(raw: string, context: ResolutionContext): boolean { return !declared.some((n) => n.qualifiedName === spelled || n.qualifiedName.endsWith(`::${spelled}`)); } +/** Standard-library types whose `->` reaches the value they hold. */ +const CPP_STD_HOLDERS: ReadonlySet = new Set(['unique_ptr', 'shared_ptr', 'optional']); + +/** + * A `->` call on a C++ receiver declared as a standard smart pointer or + * optional (`std::unique_ptr iter; iter->Valid()`) calls a member of + * the type it holds. That type is looked up as C++ looks it up where the call + * is written (cpp-supertypes.ts): the scopes around the call, the file's + * `using`s, and aliases followed. Looking it up by its last name instead + * finds namesakes: 21 other classes in rocksdb are named `Iterator`, nested + * ones like `MemTableRep::Iterator`, while the `Iterator` the call means + * inherits `Valid` from `IteratorBase`. + * - The member of the class it names, or of a class that class derives from. + * - null when no project method is called: the type is a template + * parameter's, a library type (`std::unique_ptr>`) or an + * alias of one, or a class that lacks the method (but a class template's + * specialization may declare it, so not there). + * - undefined when `raw` declares no such holder, or the type it holds is + * none the project declares (a generated message, a class the index + * doesn't name): the call goes on as before. + */ +function matchCppHeldTypeCall(raw: string, methodName: string, ref: UnresolvedRef, context: ResolutionContext): ResolvedRef | null | undefined { + const holder = cppTypeSegments(raw); + if (holder?.length !== 2 || holder[0] !== 'std' || !CPP_STD_HOLDERS.has(holder[1]!) || !isCppLibraryType(raw, context)) return undefined; + const held = cppTemplateArguments(raw)?.[0]; + const caller = context.getNodeById?.(ref.fromNodeId); + if (!held || !caller || isCppPointerType(held)) return undefined; + if (isCppLibraryType(held, context)) return null; + // A template parameter's type (`std::unique_ptr`, `typename T::Iter`) names no particular class: null. + const found = cppClassWritten(held, { + // A class local to the calling function is found first. + scope: caller.kind === 'file' ? '' : caller.qualifiedName, + file: ref.filePath, + line: ref.line, + parameters: cppTemplateParameters(caller, context), + }, ref, context); + if (found === undefined) return undefined; + if (found === null) return null; + if (!CPP_CLASS_KINDS.has(found.node.kind)) { + // An alias that leads to none of the project's classes. When it names a + // library type (`using VectorWalPtr = std::vector<…>;`) or a template + // parameter's, nothing of the project's is called. Otherwise it may be one + // each `#if` arm declares differently (rocksdb's `port::Thread`, a + // `std::thread` or a `WindowsThread`) or name a class the index keeps + // under another namespace (protobuf's `MapField`): no telling. + const aliased = resolveCppAliasedType(held, ref, context); + return aliased === null || (aliased && isCppLibraryType(aliased.target.join('::'), context)) ? null : undefined; + } + const method = cppMethodOf(found.node, methodName, ref, context); + if (method) return { original: ref, targetNodeId: method.id, confidence: found.exact ? 0.9 : 0.85, resolvedBy: 'instance-method' }; + return cppTemplateParameters(found.node, context).size > 0 ? undefined : null; +} + +const CPP_CLASS_KINDS: ReadonlySet = new Set(['class', 'struct', 'union']); + +/** + * C++ class `cls`'s method `name`: its own, else the nearest one of a class + * it derives from, through the base edges every declaration of `cls` has. + */ +function cppMethodOf(cls: Node, name: string, ref: UnresolvedRef, context: ResolutionContext, depth = 0, seen = new Set()): Node | null { + const want = `${cls.qualifiedName}::${name}`; + const named = context.getMethodMatches?.(cls.name, name, ref.language) ?? + context.getNodesByName(name).filter((n) => n.kind === 'method' && sameLanguageFamily(n.language, ref.language)); + const own = named.filter((n) => n.qualifiedName === want); + if (own.length > 0) return preferCallSiteFile(own, ref.filePath)[0]!; + seen.add(cls.qualifiedName); + if (depth >= 4 || !context.getSupertypeNodes) return null; + for (const decl of context.getNodesByQualifiedName(cls.qualifiedName)) { + if ((decl.language !== 'cpp' && decl.language !== 'c') || !CPP_CLASS_KINDS.has(decl.kind)) continue; + for (const base of context.getSupertypeNodes(decl.id)) { + if (!CPP_CLASS_KINDS.has(base.kind) || seen.has(base.qualifiedName)) continue; + const inherited = cppMethodOf(base, name, ref, context, depth + 1, seen); + if (inherited) return inherited; + } + } + return null; +} + /** * A C++ source line as code, for reading declarations: null for a line of a * comment (`// …`, ` * …`), else the line with its comments blanked, columns @@ -9069,9 +9149,7 @@ export function clearNameMatcherMemos(context: ResolutionContext): void { KOTLIN_RECEIVER_TYPES.delete(context); KOTLIN_HIERARCHIES.delete(context); KOTLIN_FRAMES.delete(context); - CPP_NS_MACROS.delete(context); - CPP_NS_FRAMES.delete(context); - CPP_NS_ALIASES.delete(context); + clearCppNamespaceMemos(context); CPP_CLASSES_IN_FILE.delete(context); SOLIDITY_SUPERS.delete(context); DECLARED_SUPERS.delete(context); @@ -10322,6 +10400,15 @@ export function matchMethodCall( isCppLibraryType(cppDecl.raw!, context)) { return null; } + // A `->` call on a standard smart pointer or optional the calling + // function or its class declares is a call on the type it holds: + // rocksdb's `std::unique_ptr iter; iter->Valid()` reached + // `ArenaWrappedDBIter::Valid` by the method's name. + if (ref.language === 'cpp' && cppDecl.callers && !cppDecl.aliased && !cppDecl.shadowed && !cppDecl.pointer && + cppMemberOperator(objectOrClass!, ref, context) === '->') { + const held = nmTimedT('mc-cppheld', ref, () => matchCppHeldTypeCall(cppDecl.raw!, methodName!, ref, context)); + if (held !== undefined) return held; + } } // Go 2-hop field chain `base.field.Method` (#1276): the base's type comes @@ -12824,104 +12911,6 @@ function erlangImportedModule(name: string, arity: string, ref: UnresolvedRef, c return undefined; } -const CPP_NS_MACROS = new WeakMap; openerFns: Set; closers: Map; aliases: Map }>(); -const CPP_NS_FRAMES = new WeakMap>>(); -/** A closing macro's body: `}` / `} }`, maybe beside a pragma macro (`PYBIND11_WARNING_POP }`). */ -const CPP_CLOSER_BODY = /^(?:[A-Za-z_]\w*\s+)*\}(?:\s*\})*\s*;?$/; -const CPP_NS_ALIASES = new WeakMap>(); - -/** The project's namespace aliases: `namespace py = pybind11;`. */ -export function cppNamespaceAliases(context: ResolutionContext): Map { - const hit = CPP_NS_ALIASES.get(context); - if (hit) return hit; - const aliases = new Map(); - for (const file of context.getAllFiles()) { - if (!/\.(?:h|hh|hpp|hxx|inl|c|cc|cpp|cxx)$/i.test(file)) continue; - const source = context.readFile(file); - if (!source || !source.includes('namespace')) continue; - for (const m of source.matchAll(/^[ \t]*namespace[ \t]+([A-Za-z_]\w*)[ \t]*=[ \t]*(?:::)?([A-Za-z_][\w:]*)[ \t]*;/gm)) { - if (!aliases.has(m[1]!)) aliases.set(m[1]!, m[2]!); - } - } - CPP_NS_ALIASES.set(context, aliases); - return aliases; -} - -/** - * The project's namespace-opening macros — `#define FMT_BEGIN_NAMESPACE - * namespace fmt { inline namespace v12 {`, `#define RAPIDJSON_NAMESPACE_BEGIN - * namespace RAPIDJSON_NAMESPACE {` (through `#define RAPIDJSON_NAMESPACE - * rapidjson`) — as the namespace path each opens (inline namespaces are - * transparent), and the closing macros as how many scopes each closes. - */ -function cppNamespaceMacros(context: ResolutionContext): { openers: Map; openerFns: Set; closers: Map; aliases: Map } { - const hit = CPP_NS_MACROS.get(context); - if (hit) return hit; - const openers = new Map(); - // `#define PYBIND11_NAMESPACE_BEGIN(name) namespace name {`, used as `PYBIND11_NAMESPACE_BEGIN(detail)`. - const openerFns = new Set(); - const closers = new Map(); - const aliases = new Map(); - const bodies: Array<[string, string]> = []; - for (const file of context.getAllFiles()) { - if (!/\.(?:h|hh|hpp|hxx|h\+\+|inl|ipp|tcc)$/i.test(file)) continue; - const raw = context.readFile(file); - if (!raw || !raw.includes('#') || !raw.includes('define')) continue; - const source = stripCommentsForRegex(raw.replace(/\\\r?\n/g, ' '), 'cpp'); - for (const m of source.matchAll(/^[ \t]*#[ \t]*define[ \t]+([A-Za-z_]\w*)(\(\s*([A-Za-z_]\w*)?\s*\))?[ \t]+([^\n]*)$/gm)) { - const body = m[4]!.trim(); - if (m[2] !== undefined) { - // (a trailing pragma macro — `PYBIND11_WARNING_PUSH` — rides along) - if (m[3] && new RegExp(`^namespace\\s+${m[3]}\\s*\\{[\\w\\s]*$`).test(body)) openerFns.add(m[1]!); - else if (CPP_CLOSER_BODY.test(body)) closers.set(m[1]!, (body.match(/\}/g) ?? []).length); - continue; - } - if (/^[A-Za-z_]\w*$/.test(body)) aliases.set(m[1]!, body); - else if (CPP_CLOSER_BODY.test(body)) closers.set(m[1]!, (body.match(/\}/g) ?? []).length); - // An inline namespace (transparent, and often named by a macro call) is skipped. - else if (/^(?:inline\s+namespace\s+[^{}]*\{\s*|namespace\s+[A-Za-z_]\w*\s*\{\s*)+[\w\s]*$/.test(body)) bodies.push([m[1]!, body]); - } - } - for (const [name, body] of bodies) { - if (openers.has(name)) continue; - const path = [...body.replace(/inline\s+namespace\s+[^{}]*\{/g, '').matchAll(/namespace\s+([A-Za-z_]\w*)/g)] - .map((m) => aliases.get(m[1]!) ?? m[1]!); - if (path.length > 0) openers.set(name, path); - } - const macros = { openers, openerFns, closers, aliases }; - CPP_NS_MACROS.set(context, macros); - return macros; -} - -/** The line ranges of a C / C++ file each namespace macro opens, with the namespace path it opens. */ -export function cppMacroNamespaceFrames(file: string, context: ResolutionContext): Array<{ start: number; end: number; path: string[] }> { - let memo = CPP_NS_FRAMES.get(context); - if (!memo) { - memo = new Map(); - CPP_NS_FRAMES.set(context, memo); - } - const hit = memo.get(file); - if (hit) return hit; - const frames: Array<{ start: number; end: number; path: string[] }> = []; - const { openers, openerFns, closers, aliases } = cppNamespaceMacros(context); - if (openers.size > 0 || openerFns.size > 0) { - const lines = context.getFileLines?.(file) ?? context.readFile(file)?.split(/\r?\n/) ?? []; - const open: Array<{ start: number; path: string[] }> = []; - lines.forEach((text, i) => { - const m = /^[ \t]*([A-Z_][A-Z0-9_]*)(?:\(\s*([A-Za-z_]\w*)?\s*\))?[ \t]*;?[ \t]*(?:\/\/.*|\/\*.*\*\/[ \t]*)?\r?$/.exec(text); - const token = m?.[1]; - if (!token) return; - const arg = m[2]; - const path = arg !== undefined && openerFns.has(token) ? [aliases.get(arg) ?? arg] : arg === undefined ? openers.get(token) : undefined; - if (path) open.push({ start: i + 1, path }); - else if (closers.has(token) && open.length > 0) frames.push({ ...open.pop()!, end: i + 1 }); - }); - for (const frame of open) frames.push({ ...frame, end: lines.length }); - } - memo.set(file, frames); - return frames; -} - /** * Resolve `fmt::format` / `fmt::detail::to_unsigned` to the declaration a * namespace macro puts there: a node named `format` (qualified `format` or diff --git a/src/resolution/types.ts b/src/resolution/types.ts index 7eb6dead9e..b5090530e9 100644 --- a/src/resolution/types.ts +++ b/src/resolution/types.ts @@ -153,6 +153,14 @@ export interface ResolutionContext { * method). Optional so external/test contexts compile without it. */ getSupertypes?(typeName: string, language: Language): string[]; + /** + * The nodes one type declaration extends or implements, through its own + * resolved `implements`/`extends` edges: where getSupertypes unions those of + * every type sharing a simple name, this is the supertypes of the node + * `id`. Empty during the first resolution pass, as getSupertypes is. + * Optional so external/test contexts compile without it. + */ + getSupertypeNodes?(id: string): Node[]; /** * Look up a node by its id. Lets matchers derive the FROM-symbol's * enclosing-class scope (Swift implicit-self method scoping, `this.X`