Repository navigation
Conversation
bnusunny
force-pushed
the
fix/nodejs-closure-from-lockfile
branch
from
October 9, 2026 22:25
8abc789 to
1b5b92b
Compare
…pm ls The artifacts link asks npm which packages a workspace function resolves, and `npm ls --all --parseable --omit=dev` costs 0.33s per function - the largest remaining per-function cost once the install is no longer the bottleneck. The lockfile the build already locates describes the same resolved tree, so read it instead: walk the lockfileVersion 2/3 `packages` map from the member's own entry, resolve each name the way node does (`<key>/node_modules/<name>`, walking up), follow `link: true` entries to their source directory, and skip anything marked `dev`. A version 1 lockfile has no path map, so that falls back to `npm ls`, as does a lockfile that cannot be read. An entry can be in the lockfile without being on disk: `fsevents` is darwin-only and optional, so webpack's and jest's closures list it while Linux never installs it. `npm ls` reports the installed tree, so the resolver keeps only directories that exist; a required dependency missing there means the install itself failed, which the install step already surfaces. Behaviour is unchanged, which is the point - the existing integration tests that assert each function's artifacts hold its own dependencies and not its sibling's are the regression proof, and they pass unchanged. The new unit tests cover the resolver itself: nested copies, scoped names, workspace links, dev entries, version 1 fallback, an unreadable lockfile, and an entry that is not on disk.
bnusunny
force-pushed
the
fix/nodejs-closure-from-lockfile
branch
from
October 9, 2026 23:09
1b5b92b to
fb8fcca
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available: N/A — a performance change on the path #933 added. No linked issue.
Description of changes
Goal: stop spawning one
npm lsper function when building a workspaces monorepo in source, by reading the closure out of the lockfile npm already wrote.#935 narrows a function's artifacts to its own dependencies by asking
npm ls --all --parseable --omit=devwhich packages that function resolves. That answer is correct, and it costs an npm process per function — on a 20-function monorepo, twenty sequential node startups whose only job is to re-derive whatpackage-lock.jsonalready records.lockfile_closure.production_closurederives the same set from the lockfile. The lockfile'spackagesmap is keyed by install path, so resolving a dependency is the same walk node performs: look for<prefix>/node_modules/<name>, strip a path segment, repeat. A workspace link resolves to its real source directory, which is what npm's--parseableoutput also prints.npm lsnpm lsMeasured on two generated fixture sets, npm 11.19.0, Linux; medians of repeated runs.
npm lsstays as the fallback and nothing about the artifacts changes.lockfileVersion1 has no per-path map, so the resolver returnsNonefor it, as it does for an unreadable lockfile or an install directory the lockfile does not describe — and the caller then asks npm exactly as before. The experimental flag gating is unchanged: this runs only where #935's link step already runs.Description of how you validated changes
The lockfile's answer is checked against npm's, not against a fixture.
test_the_lockfile_closure_agrees_with_npm_ls_on_a_real_installruns a realnpm installon theworkspaces-monorepofixture and asserts both endpoints' closures are set-equal betweenproduction_closureandnpm ls --all --parseable --omit=dev. A unit test cannot establish this — it would compare this code against a fixture written from this code.Mutation-checked: removing the walk-up from
_resolve_keymakes that test fail, naming the two directories it then loses (packages/sharedand the hoistedminimal-request-promise).Unit tests cover the resolver directly — a member's own dependencies plus its shared workspace package, a conflicting version resolving to the nested copy rather than the hoisted one, a member that wants the hoisted version getting both copies, dev dependencies excluded, an entry not on disk excluded — and each way it declines to answer: version 1, an undescribed install directory, an unreadable file.
Two tests pin the wiring, which is where a silent regression would hide:
test_a_usable_lockfile_answers_the_closure_without_running_npmassertsresolve_dependency_closureis never called, andtest_a_lockfile_that_cannot_answer_falls_back_to_npmasserts it is called once. Without the first, the lockfile path could stop being used and every test would still pass.ruff checkandblack --checkclean.#933monorepo integration test now exercises the lockfile path end to end, since the workflow passes the lockfile in — so the artifact assertions it already makes cover this change too.Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.