Repository navigation
Type the stubs of imported JavaScript modules from TypeScript and JSDoc - #7951
FarhanAliRaza wants to merge 21 commits into
Conversation
The stub written next to an imported JavaScript module typed every function as `(*args: Any)`. It now reads each export's call signatures with the TypeScript checker, from annotations in a .ts file or JSDoc in a .js file, through script-signatures.js, which ships to .web beside the other build scripts an app runs: /** @PARAM {number} count @PARAM {string} elementId @PARAM {boolean} [loud] */ export function celebrate(count, elementId, loud = false) {} def celebrate(count: float | Var[float], elementId: str | Var[str] | EventCallback[*tuple[Any, ...]], loud: bool | Var[bool] = ..., /) -> EventSpec: ... Arguments are positional-only, as JavaScript passes them. JSON types map to Python (number to float, T[] to Sequence, unions, literals, `| null`); types with no JSON form become Any. A string parameter also takes an event handler, which the call sends as its name. Overloads become @overload, rest parameters *args, and exports that are not functions are left out. typescript (pinned to the version reflex-frontend checks itself with, which a test keeps equal) is installed only with apps that import a module, and runs after the install, once for every stale module. The stub's header records a digest of the module and the scripts beside it, so typescript does not run while they are unchanged. When the types cannot be read (no node or bun, or the script fails) the stub stays untyped, with a warning, and is retried on the next compile. Part of ENG-9127. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
[Medium impact] The changes affect development builds for apps that import JavaScript modules from Python. The complete-output testing requirement must be satisfied before merging; recursive types also need a safe fallback. FindingsSummaryAdds typed Python stubs for imported JavaScript modules using TypeScript annotations and JSDoc.
Reviews (8) · Last reviewed commit: "Cover re-exported names in the signature..." · Reviewed by Greptile |
Merging this PR will not alter performance1 warning
Comparing Footnotes
|
…cused-darwin-dx76ek-js-stubs # Conflicts: # packages/reflex-base/src/reflex_base/js/modules.py # reflex/compiler/utils.py # tests/units/compiler/test_compiler_utils.py # tests/units/reflex_base/js/test_modules.py
…s, cache key - A parameter renamed from a non-identifier (`$value` -> `arg0`) no longer takes the name of another parameter, which made the stub invalid Python. - Only trailing optional parameters get defaults: in `f(prefix = "", count)` Python cannot omit `prefix`, so both are required. - A tuple rest parameter (`...args: [number, string]`) has no single element type and becomes `*args: Any` instead of `*args: Sequence[Any]`, which rejected the valid call. - An export of unknown type (`any`, e.g. re-exported from a package without declarations) keeps an untyped def instead of disappearing from the stub. - The stub digest covers the local files the module imports (as copied by the compiler) and the app's package.json, so a package upgrade re-reads the types; paths are hashed relative to the module, so the header is the same on every machine. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
…cused-darwin-dx76ek-js-stubs
…cused-darwin-dx76ek-js-stubs # Conflicts: # reflex/compiler/utils.py # tests/units/compiler/test_compiler_utils.py
…cused-darwin-dx76ek-js-stubs # Conflicts: # tests/units/reflex_base/js/test_modules.py
…cused-darwin-dx76ek-js-stubs
Review fixes:
- The stub imports the types it uses under private aliases (`_str`, `_Var`,
`_Sequence`, ...), so an export named `str`, `Var` or `Sequence` no longer
replaces the type its annotations name; parameters keep their own names. An
export named like one of the aliases is left out of the stub. Checked with
pyright: a stub exporting `str`, `Var` and `any` type-checks, and calls are
checked against it.
- The stub digest, through module_files, also covers the files a module takes
types from: `.d.ts` files an import resolves to, JSDoc `import("./types")`
and `/// <reference path>`. Editing them rewrites the stub.
Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
An earlier merge left conflict markers, then indented by the formatter, in compile_script_modules's Returns section; it now describes the three returned values once. The stub tests that compared only the last line or two of the rendered stub compare the complete output. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
…de it Trying `.d.ts` before `.js` when resolving `./helpers` picked helpers.d.ts whenever both existed, so helpers.js was not copied and a clean frontend build could not resolve the import. Runtime files resolve as the bundler does again, and the declaration file that types an import (helpers.d.ts for `./helpers` or `./helpers.js`, or alone for a types-only import) is tracked beside it, for the stub digest. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
Dropping .d.ts from runtime resolution also dropped the lookup of types/index.d.ts for `./types`, so editing it left the stub digest unchanged. The declaration lookup now tries it after types.d.ts. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
|
The CodSpeed failure on 7c653f1 is not from this change. All three regressions are
So this is runner variance. Nothing to fix in the diff; it can be acknowledged on CodSpeed. Generated by Claude Code |
test_js_module_stub failed when another test module had already run an app named `callscript` in the same process: the harness reloads that package, which keeps its first directory, so page_script.js and its stub live there, not under this harness's app_path. The test now takes the module's path from the imported module, and normalizes the digest in the header, which depends on the app's package.json (its value is covered by the unit tests), before comparing the whole stub. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
Adapt the stubs to the reworked module API. A stub declares only the names
ScriptSource.functions lists, and names TypeScript reports as types only are
dropped too. module_files is the copy walk again: the files a module reads
types from (declarations beside its imports, type-only imports, JSDoc
import("...") and triple-slash references) are found by type_files, only for
the digest, so a type reference outside the package no longer fails the
compile and type-only files are not copied into .web. The digest hashes the
scripts the copy walk already read.
script-signatures.js now classifies each parameter's type with the checker
into the JSON value Python can send (a small type tree with sorted unions), and
stubs.py only renders it, importing just the aliases a stub uses. Stubs are
written in development only, a stub whose module the app stopped importing is
removed, and failures to read types say what happened and how to fix it.
Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
Read the kind of every import, type-only ones included, through the base's new walk of template and JSX expressions. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
Granian watched .pyi files, so each stub Reflex wrote next to a JavaScript module restarted the backend once more. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
The Type Stubs section shows what a JSDoc signature becomes, maps each TypeScript type to its Python type (Any included), says the stubs are written in development only and recommends committing them, and lists the warnings a compile can print with their fixes. The news fragments say what changed for users instead of naming internal files. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
`boolean` is always the union `true | false`, so classify never saw a lone BooleanLike type. A function re-exported under another name is in the stub, and a re-exported type is not. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
| const index = checker.getIndexInfoOfType(type, ts.IndexKind.String); | ||
| if (index) return { kind: "mapping", value: classify(index.type) }; |
There was a problem hiding this comment.
Recursive types disable argument checks
A recursive type such as interface Tree { [key: string]: Tree } makes classifyObject call classify on the same type without stopping. The script fails before printing its results, so every stale module gets an untyped stub. Later compiles repeat the failure.
Track types already being visited and use any when a type repeats.
| json.dumps([str(handwritten), str(stub_app / "gone.pyi")]) | ||
| ) | ||
| _compile_stubs() | ||
| assert party_stub.read_text().startswith("# Generated by Reflex from party.js (") |
There was a problem hiding this comment.
Test misses unexpected stub output
The new cleanup test checks generated output with startswith, so unexpected declarations after the header still pass. The repository guide requires tests to compare generated output in full.
Before merging, compare the whole stub, deriving or normalizing the digest first.
Context Used: CLAUDE.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Type of change
Description
Third of the ENG-9127 stack, on top of #7950.
Problem. Python can now import a JavaScript module's functions, but nothing checks the calls. A wrong argument type, or a missing argument, shows up only in the browser.
Change. In dev mode, Reflex writes a type stub (
party.pyi) next to each imported JS/TS module, so pyright and mypy check every call:becomes
Argument types come from TypeScript annotations or, in
.jsfiles, from JSDoc. The TypeScript checker classifies each type. Nothing is parsed out of printed type strings.stringstr(also accepts an event handler, which is sent as its name)number/booleanfloat/boolT[], tuplesSequence[...]Record<string, T>Mapping[str, T]Literal[...]Set,Map,bigint,Promise, functions, classes, DOM types, untypedAny, because arguments arrive as JSONRules.
.pyiextension is ignored by hot reload, so writing a stub doesn't restart the backend..web/package.json;.pyiis never touched.import("…"),/// <reference>,import type,.d.ts) count toward the stub but are never copied into.web. A type file outside the package is no longer a compile error.Warnings. Each states the problem, its cause and the fix. When the types can't be read, the stubs accept any arguments and the app still runs. For example:
Tests
script-signatures.jsfrom a fixture that vitest also checks. The failure paths run a real subprocess: no runtime, non-zero exit, bad JSON, timeout. The digest is compared exactly, orphaned stubs are removed, and a production compile writes no stub.script-signatures.jsis fully covered.test_call_script.py: the stub generated for a module is checked against its exact text.npm run checkare clean.typecheck_bad.pyand none ontypecheck_good.pyor the app.https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf