Skip to content

Compiler: JS: initial tooling for ecmascript modules#2163

Draft
hhugo wants to merge 14 commits into
masterfrom
esm4
Draft

Compiler: JS: initial tooling for ecmascript modules#2163
hhugo wants to merge 14 commits into
masterfrom
esm4

Conversation

@hhugo

@hhugo hhugo commented Feb 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

This PR adds initial compiler tooling for working with ECMAScript modules.

New compiler support:

  • Esm module (compiler/lib/esm.ml): parses ES modules, extracts imports/exports, builds a module dependency graph (with cycle support), resolves star re-exports, and topologically sorts modules.
  • Esm_bundle module (compiler/lib/esm_bundle.ml): bundles a module graph into a single program — rewrites imports to direct references, generates namespace bindings, optimizes ns.field accesses into named imports, and supports tree shaking from entry-point exports.
  • Script-to-module conversion.

Supporting changes:

  • js_assign.ml: the program top level is now recorded as a Params block instead of Var_scope, so that under the Preserve naming strategy top-level var declarations (which ESM output places directly at the top level, e.g. vars hoisted by tree shaking) get names allocated.
  • New test suite compiler/tests-esm/ (38 tests) using the vendored ppx_expect_light.

Preliminary work for: #1532, #1161, #551

@hhugo hhugo marked this pull request as draft February 4, 2026 01:22
hhugo and others added 5 commits July 11, 2026 10:08
@hhugo @claude
Compiler: esm: address review findings
d1ce20b 
- Esm.resolve_reexport: error out on missing export names and cyclic
  re-export chains instead of looping forever (a typo in
  [export { y } from ...] used to hang the compiler)
- Support [export * as ns from]: desugar into a synthetic namespace
  import plus a plain export so the namespace object is materialized
- Rewrite exports of imported bindings into proper re-exports; the
  bundled output used to reference the erased import binding
  ([import { x } ...; export { x }] and chains through middle modules)
- Esm_bundle: never optimize a namespace import that is itself exported
- Esm_bundle.merge_modules: substitute import bindings in export
  statements and deduplicate conflicting export names (duplicate
  exports are a JavaScript syntax error)
- Esm_tree_shake: count [var]s hoisted out of compound statements as
  definitions ([if (c) var x = 1] used to be dropped while live code
  referenced [x])
- Esm_tree_shake: class expressions with side-effecting extends
  clauses, computed names or static blocks are no longer dropped
- Esm_tree_shake: mark modules traversed while resolving imports and
  re-export chains as reached so their side effects are preserved
  (re-exporting "barrel" modules), keep intermediate re-export entries
  and pure re-export modules needed for bundle-time resolution
- Linker: align exported types of Fragment.attach_annot and
  Fragment.script_to_module with what Parse_js.parse' produces
- Esm: drop the unused has_default_export field and the duplicated
  topological sort

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hhugo @claude
Compiler: tests-esm: use vendored ppx_expect_light
55f44e4 
ppx_expect is not a dependency of this repo; CI fails with
'Library "ppx_expect" not found' on every Tests job.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hhugo @claude
Compiler: tests-esm: fix Windows CI failures
3782dc3 
- Lexer.of_file keeps the input channel open, which prevents the test
  cleanup from deleting the file on Windows (Permission denied); read
  the file and use Lexer.of_string instead
- Normalize path separators when printing bundler error messages so
  the expected output is stable across platforms

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hhugo @claude
Tests: fix esm_bundle path normalization on Windows
c9ff431 
normalize_path only split on '/', but Filename.concat joins with '\\' on
Windows, so '.' components stayed glued to the previous segment. In the
cyclic-dependency tests each lap around the cycle then produced a new
module id with an extra '\.' segment, growing the path until it exceeded
MAX_PATH and resolution failed. Unify separators before splitting.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hhugo @claude
Tests: use normalized paths for esm entry points
f81513b 
Module ids are raw path strings. After the previous fix, resolved module
paths use '/' on Windows but entry points still used the '\\'-joined path
from Filename.concat, so entry ids sorted differently relative to their
dependencies than on Unix ('/' < '\\'). The SCC-based topological sort
then ordered independent sibling modules differently, breaking the
'import alias collision' expect test on Windows. Return normalized paths
from the test [write] helper so all module ids agree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hhugo hhugo mentioned this pull request Jul 12, 2026
Draft
hhugo and others added 4 commits July 12, 2026 15:55
@hhugo @claude
Compiler: esm: emit bundled modules in ES evaluation order
d073001 
The bundler ordered modules with an SCC topological sort whose tie-break
is the module id (a path): independent sibling modules came out in
alphabetical order instead of the evaluation order mandated by the spec
(post-order depth-first traversal following import declarations). Add
[Esm.evaluation_order] and use it in [Esm_bundle.bundle]; the affected
expect tests change to the ES-correct order.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hhugo @claude
Compiler: esm: support external imports
9c58e12 
Import specifiers that [resolve] cannot resolve (e.g. "node:module")
used to fail graph construction. They are now external: the import takes
no part in the module graph and the statement is kept verbatim in the
module body, so it survives bundling. Re-exporting from an external
module remains an error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hhugo @claude
Compiler: esm: keep entry-module side effects when tree shaking
c6da015 
Tree shaking seeded liveness only from entry exports: an entry module
that exports nothing was shaken away entirely, side effects included.
Entry modules are evaluated by definition; mark them as reached so their
side-effecting statements (and, transitively, the modules they import)
survive.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hhugo @claude
Compiler: esm: optimize namespace imports before tree shaking
bb78bb5 
A namespace import marks every export of the imported module live, so
tree shaking kept modules whole even when only a few fields were used.
Run the ns.field -> named-import optimization on the graph before
shaking; the bundler's own pass remains for the no-shaking path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hhugo

hhugo commented Jul 12, 2026

Copy link
Copy Markdown
Member Author

Pushed four Compiler: esm: commits fixing bugs found while building the ESM-output experiment on top of this branch (#2402):

  • ES evaluation order in the bundler — modules were emitted in an SCC topological order whose tie-break is the module id, so independent sibling modules came out in alphabetical order instead of import-declaration order (also the root cause of the platform-dependent ordering previously seen on Windows CI). New Esm.evaluation_order, used by Esm_bundle.bundle; two expect tests updated to the ES-correct order.
  • External imports — unresolvable specifiers (e.g. node:module) failed build_graph; resolve returning None now marks the import as external and keeps it verbatim.
  • Tree shaking: entry side effects — an entry module exporting nothing was shaken to an empty file; entry modules are now always marked reached.
  • Tree shaking vs namespace imports — import * as ns kept every export of the source alive; the ns.field → named-import optimization now runs before shaking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

no changelog

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@hhugo