Repository navigation
Import JavaScript modules from Python and call their exports by name - #7950
FarhanAliRaza wants to merge 19 commits into
Conversation
`from my_app.scripts.party import celebrate` now finds my_app/scripts/party.js (or .ts) when no Python module of that name exists. Each named export becomes a JsFunction; calling it returns a `_call_js` event naming the module and the function, with the arguments as JSON. It works in event triggers, returned or yielded from handlers, and uncalled as a trigger (it takes the trigger's arguments, like a handler). An event handler argument arrives as its full name, for `env.addEvents([env.event(name, payload)])`. The compiler copies each imported module, with the .js/.ts files beside it so relative imports resolve, under utils/script_modules, and utils/scripts.js re-exports it under a key `$` makes unique. The frontend imports that index on the first call, so nothing is evaluated and an app that never calls a module does not fetch the chunk: it works under a CSP without unsafe-eval. npm packages the modules import are installed with the app. A stub (party.pyi) with untyped arguments is written next to the module, so type checkers resolve the import; typed stubs come next. The finder is stdlib-only and reflex_base.js is lazy, so `import reflex`, which installs it, still does not load the event system. The function's return value is not sent back: results will come from awaiting the event, a separate design. Part of ENG-9127. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
…arwin-dx76ek-js-modules
[Medium impact] The PR appears safe to merge, with no new blocking issues found. SummaryAdds Python imports for JavaScript module functions and calls them in the browser without evaluating strings.
Reviews (9) · Last reviewed commit: "Hold an event back behind every module c..." · Reviewed by Greptile |
Merging this PR will not alter performance1 warning
Comparing Footnotes
|
…t reflex cheap
Review fixes for the JavaScript module import:
- Exports and imports are read from tokens (reflex_base/js/source.py) instead
of regular expressions over the source. Text inside strings, template
literals, regular expressions and comments is never read as code, so a
string such as "/api/*" no longer hides the exports after it. Every name of
`export const a = ..., b = ...` is found, as are destructured exports, TS
annotations with commas, and `"string" as name` export lists. Imports are
found wherever line breaks fall, including dynamic `import("x")`.
- The compiler copies the module and the local files it imports, followed
transitively and resolved as the bundler does (extensionless, index files,
`.js` naming a `.ts`), at their paths from the directory of the module's
package, instead of the script files beside it. Nested helpers and imported
CSS now resolve, and unused siblings are no longer bundled or installed. A
file outside the package's directory is a clear error.
- RegistrationContext.fork() copies script_modules, so an app built in a fork
still compiles the modules imported before it.
- The finder no longer imports pathlib, importlib.abc or typing, which
`import reflex` does not load otherwise (CodSpeed: test_import_reflex
+1.6ms); a test pins it.
- Test docstrings.
Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
…arwin-dx76ek-js-modules
Review fixes: - Files a module imports that are not scripts (stylesheets, images, fonts) were read as UTF-8 into the compile output, so `import logo from "./logo.png"` stopped the compile with UnicodeDecodeError. compile_script_modules now returns them as (source, target) copies, which compile_app copies byte for byte after the dry-run return, skipping targets that are up to date. - A copied stylesheet's `@import` and `url(...)` references are followed too, so `./theme.css` and `./bg.png` used by an imported `party.css` are bundled. Absolute paths, URLs, data URIs and fragments are left alone. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
CodSpeed still showed test_import_reflex about 15% slower on c0ebd9c. The benchmark clears the reflex modules and imports `reflex` again in the same process, so it measures module execution: importing the lazy reflex_base.js package and its finder module on every `import reflex` cost about 200us of about 1300us. It also showed a bug: install() matched the finder with isinstance, and a re-imported finder module is a new class, so every fresh import appended another finder to sys.meta_path (205 after 200 rounds). reflex_base/__init__.py, which every Reflex import loads first, now registers a small finder class that loads reflex_base.js.finder only on the first import no other finder found, matched by name so a fresh import never adds a second. `import reflex` loads no module for it (about 12us of execution), and reflex/__init__.py is back to its previous form, so the generated reflex/__init__.pyi stays free of runtime code. Tests pin one finder after a re-import and that `import reflex` does not load reflex_base.js. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
Review fixes: - call_event_fn counted `*args` as one required parameter and passed only as many arguments as the callable declared, so an uncalled JS module function failed to compile on a trigger with no arguments (`on_mount=initialize`) and got only the first argument of one with several. The same broke any `lambda *args: ...`. Variadic parameters are no longer required, and a `*args` callable gets every argument. - A relative import with a query or fragment (Vite's `./message.txt?raw`, `./logo.png?url`) resolves to its file again, so the file is copied with the module, which keeps the import as written. Package specifiers drop the query before install. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
…arwin-dx76ek-js-modules Keep reflex_base.js a package: the base's js.py becomes js/api.py, whose ref, add_events and library the package re-exports lazily, since the module finder `import reflex` registers lives in the package too. The obsolete environment.py goes, test_js.py moves to js/test_api.py, and the JavaScript module tests, docs and demo use `reflex` from $/utils/reflex. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
A callable taking `**kwargs` or a keyword-only parameter was called with one argument per parameter, so `lambda key, **kw: ...` on on_key_down got two positional arguments and raised TypeError. Count only positional parameters, both to call the function and to check that the trigger provides enough arguments; `*args` still takes all of them. The varargs test moves next to the new one in the event module's test file. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
Scope: the finder only claims modules of the app's package, the package rxconfig's app_name names, so a third-party package's .js file is never imported or registered. The config loads only once a file is found, and the finder is now a function the stub in reflex_base calls. Parsing: one scanner serves tokenize, template expressions and JSX, and each file is read and tokenized once per compile (module_files returns each script's text, exports and imports). `=>`, `++` and `--` are single tokens, so annotations like `() => Map<K, V>` and `i++ / 2` read right; `.jsx`/`.tsx` files skip JSX text; type items of export lists, generic arrow functions, `import type` and `export type` are handled. A `const` bound to a literal is a value, not a function. Every new function stays within a complexity of 8. Compile: files must be inside the app package's directory, not merely the project's. utils/scripts.js now maps each module's Python name, as a JSON string, to a loader, so any module name or path is safe in the generated code. Specifiers with a scheme, `#` or `@/` are not installed, and Node.js built-ins fail the compile with what to do instead. Errors: importing a missing function names the file and its functions; importing a value says only functions can be called; the browser error for a missing function gives both causes and both fixes. Frontend: state.js loads the app's modules with the page and caches them, so a `_call_js` runs its function before awaiting anything and the next event, even one processEvent starts concurrently, sees its effect. An app without modules loads nothing. Errors of event code are reported through one helper. Public API: the module helpers are private (_CALL_JS, _JsFunction, _load_module, _render_stub, _package_imports), module_key is gone, and RegistrationContext.script_modules is _script_modules. Tests follow the module layout: test_finder.py, test_modules.py, test_source.py, and reflex_base/test_init.py. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
A handler yields a module function and then a script that reads what it wrote; each arrives in a message of its own. With the module loaded on demand, the script ran first and read the old text. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
The section leads with a runnable app, then the rules (the app package is the scope, files must be inside it, what is installed, arguments as JSON, how to get a value back, JSX only in .jsx/.tsx) and each error with its fix. News fragments say the same for downstream users. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
The helper that reports errors of event code now serves four blocks, so a test pins what each logs and reports, with and without window.onerror. A string after `from` always ends an import or export clause in valid code, so finding the clause's keyword needs no bail-out branches. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
Loading the modules at page load does not guarantee they finish before events arrive. While a _call_js waits for its module, processEvent now holds the events after it, in the order they arrived, until the call has run, so they see what its function did. Nothing waits once the module has loaded. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
The tokenizer skipped the expressions of templates and JSX without keeping
their tokens, so an import("./x.js") in ${...} or {...} was never copied.
A literal's token now holds the tokens of its expressions, and the import
search walks them; the export search still sees one token per literal.
Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
An event that waited for one pending module call could run ahead of the next one: the event before it may be a module call that only starts waiting for its module once the first has run. processEvent now waits until no call is pending. Claude-Session: https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf
|
CodSpeed reports
So the code under that benchmark is byte-for-byte the same in both runs. Generated by Claude Code |
Type of change
Description
Second of the ENG-9127 stack, on top of #7949.
Problem. More than a line of JavaScript ends up as strings passed to
rx.call_script. Those strings get no editor support or lint, the browser evals them, and they break under a Content Security Policy withoutunsafe-eval.Change. Write the JavaScript in a
.jsor.tsfile in your app package, and import its exported functions from Python:Each named export becomes a Python function that returns an event. Use it in an event trigger, or return or yield it from a handler. Passed to a trigger without being called, it receives the trigger's arguments. The browser receives the module, the function name and the arguments as JSON, and evaluates nothing.
Rules.
.jsfiles in other installed packages are never claimed../helpers.js, CSS with its@import/url()references, images) are bundled with it. They must be inside the app package's directory.virtual:,#…,@/…and type-only imports. A Node built-in with no browser package (fs,node:*) stops the compile with an error.reflex.addEvents.Errors. Each one states the problem, its cause and the fix, and is pinned by an exact test:
from app.scripts.party import nopenames the file, lists its functions, and shows theexport function nope() { ... }to add.export const notAFunction = 5says it is a value, not a function, which Python cannot call.Design.
import reflexdoesn't load the event system. It claims a module only if a matching script exists under the app package, and reads the config only then.export { type X }, and JSX elements in.jsxand.tsxfiles. Each file is read and tokenized once per compile.utils/scripts.jsmaps module names to() => import(...)with JSON-escaped strings, so each module is its own lazy chunk in the production build. Loading starts at page load and is cached, so a module function runs before anything awaited after it.lambda key, **kw: ...and keyword-only lambdas no longer fail.Tests
test_finder.py,test_modules.py,test_source.py,test_init.py): the import scope, the folder limit, generated names, TypeScript exports, npm detection, the error texts, and the*args/**kwargsregression._call_js, the sharedreportEventErrorhelper, and the ordering of a module function before the next event.test_call_script.py: module functions from a trigger and from a handler, an async function, an npm package, CSS, an uncalledon_change, and the ordering.npm run checkare clean. The productionreflex exportbuilds each module as its own chunk.https://claude.ai/code/session_01LwJUCVxPh3ihrnaFn6CWnf