Try out the new math-expressions - #1622
Open
siefkenj wants to merge 84 commits into
Open
Conversation
Member
|
Right now, merges to main auto-publish to the I'll have to learn the best ways to maintain multiple lines. If you have suggestions, let me know! |
Member
|
Or maybe, we release one more |
# Conflicts: # packages/doenetml-worker-javascript/package.json
**`nthroot` could not be evaluated at all.** `<function>nthroot(x,3)</function>`
is ordinary authored DoenetML, and on this branch it drew nothing — at any
input, not only negative ones. The engine's `f()` — the plotting and
root-finding entry point — compiles through math.js, which spells the function
`nthRoot`; an unknown head is not a compile error there, it becomes a symbol
that throws `Undefined function nthroot` on the first sample. Legacy plotted it
fine. Fixed upstream in `functionConversions`, pinned there by mutation and
here by a `functionTag.test.ts` case that reads `numericalfs` — the path that
was wrong. The path that was *right* is why no test caught it:
`<number>$$f(-8)</number>` substitutes and evaluates symbolically, and answered
correctly throughout.
Mapping it also puts an odd root of a negative on the real branch, matching
`cbrt` and the eleventh pass's odd-root work.
**The changeset described a plotting change that did not happen.** It claimed
`x^(1/3)` now draws for negative inputs "matching its `\sqrt[3]{x}` spelling".
Measured on both engines: `x^(1/3)` returns the principal complex value from
`numericalf` on legacy *and* here, so nothing changed and nothing draws; what
did change is `cbrt(x)`, which used to return complex and now returns the real
root. Rewritten to say that, and to disclose that a `<function>` and an
`<answer>` therefore disagree about `x^(1/3)` at a negative input.
Also documented: the extrema rewrite removes issue Doenet#940's spurious minimum,
which is user-visible and was in no changeset.
Four plumbing holes, from a review pass over the build and CI surface the
earlier cycles covered least:
- `publish-doenetml-to-pretext-python.yml` checks out without the submodule.
Its wheel build reaches `../math:build`, which throws without
`vendor/math-expressions` and needs `wasm-bindgen` on PATH. It is the one
checkout in the repo that needed updating and did not get it, so cutting a
release would have failed at the build step.
- `setup-math-wasm` wrote its wasm-bindgen version twice and guarded the
install on the binary *existing*. A bump that missed the cache key would
restore the old binary, skip the install, and fail with a schema mismatch
naming a version the file no longer mentions; a damaged entry under that key
would never heal, since `main`'s cache is what every branch reads. One input
now feeds both, and the guard compares versions.
- `.devcontainer/postCreateCommand.sh` runs `npm run build` under `set -eu`
with no submodule init, so container creation aborted for anyone who cloned
without `--recurse-submodules`, and for Codespaces, which never initialize
them. CI could not see this: both devcontainer jobs check out recursively
first.
- `doenetml-iframe`'s wireit restructure left the two iife builds as siblings
of `../doenetml:build` rather than downstream of it. Siblings run in
parallel, and both run `dts()` over a tsconfig that reaches
`@doenet/doenetml` — so they can resolve against a dist that
`../doenetml:build` (`clean: true`) is mid-clean. That is the same race the
file's own note describes, made *more* reachable than the chained
`npm run` it replaced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
A measurement of a leak is only as good as its control. The 7.0 GB was the whole delta across the file, my own browser's drift included; the same measurement with `dispose()` in place is 1.9 GB and plateaus, which is what makes the attribution safe. And CI now has the after-number the diagnosis predicted: the same file went from 166,595 ms with two failures to ~52,000 ms with none, so the extra two minutes were the runner thrashing rather than work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
**Merging this PR would have published a broken package.** `publish.yml`'s `dev-release` job fires on `workflow_run` for every successful CI run on `main` and runs `npm run publish -- --tag dev`, so merging *is* releasing. Verified against the built tree rather than restated: `packages/doenetml/dist/` contains `import ... from "math-expressions"` in 14 chunks, and its generated `package.json` declared `"private": false` with `react`/`react-dom` as its only peers — because `vite.config.ts` deliberately filtered the seam out of the published peers, a `file:../math` range being meaningless to a consumer. So the tarball's `math-expressions` import resolves to nothing, or to npm's unrelated `math-expressions@2.x`. (`standalone`, `doenetml-iframe` and `v06-to-v07` bundle the seam and carry no such import; only `@doenet/doenetml` is affected.) The guard is one invariant rather than a special case: **a package whose bundle imports something by name that an npm consumer cannot install is not publishable.** `scripts/transform-package-json.ts` now checks every externalized dependency's range, and if one is missing or local (`file:`/`link:`/`portal:`/ `workspace:`) it leaves `"private": true` in the built manifest and says why in the build log. `npm publish` refuses a private package, and the publish wrapper refuses first so the error names the reason instead of being npm's generic EPRIVATE. `packages/doenetml/vite.config.ts` therefore stops filtering, and `math-expressions` becomes a real `peerDependencies` entry that reads `file:../math` today and `^3.x` after Step 6 — at which point the block clears itself, with no flag to remember to flip. Verified end to end: `npm run build -w packages/doenetml` prints the warning and emits `"private": true`; the other three published manifests are unaffected (their externals all have registry ranges). New unit tests in `packages/standalone/scripts/` pin all of it — 5 of the 8 fail with the guard reverted. **`erf` was the mirror image of the twelfth pass's `nthroot`, and it reaches grading.** Fixed upstream (pin bumped); on this side, a new `src/test/math/appliedFunctionSymbols.test.ts` probes all 69 spellings an author can type through *both* numeric paths — `f()`, which compiles to math.js, and the engine's own `evaluate_to_constant` — because a head can be missing from either one and neither absence is louder than a `NaN`. Its first assertion fails if a spelling is added to `appliedFunctionSymbolsDefault` without a probe. **A second worker leak, same shape as the one the twelfth pass fixed.** `doenetml-prototype`'s `wasm-tests.test.browser.ts` boots a core worker per test — thirty-odd in one page — and terminated none. Now tracked and terminated in `afterEach`; 30/30 still pass. Documentation reconciled against the tree, all measured: - `PUBLISHED_PEER_DEPS` was the last dangling reference in the repo; the notes still said deleting it is what unblocks publication. It is the dependency range that does. - "Six `get_component` shadow reads" is five, and there are three *other* unguarded reads (`parallelCoords` in `LineSegment.js` twice and `Line.js`, `unnormalizedDirection` in `DirectionComponent.js`) that no enumeration mentioned — eight in total. `DirectionComponent.js:274`, listed as unguarded, is guarded by a shape test the route list did not credit. - 14 × `submodules: recursive` and 12 × `setup-math-wasm`, not 13 and 11; the file the twelfth pass added them to was missing from both lists. - Five wrong-answer-on-grading defects is six: on the unfixed engine `equals(erf(0.5), 0.5204998778130465)` is `false`, measured. - Sizes re-measured (1.69 MiB wasm, 792 kB gzipped), `~8,100` added lines is 8,583, `~160` `.simplify()` sites is 147, and §2.1's call-count tables are labelled audit-era with three spot-checks, since they have drifted. - `packages/math/README.md`'s type-surface delta covered three ODE types; it is those plus `setWasmModule` and the default export. - `substituteMathIntoExpression` does not exist; the call site is `invertMath`. - `playground/` is `packages/playground/` in the pinned submodule (4 paths). - Issue Doenet#1375 is a VS Code diagnostics bug, not the blob-URL `fetch` block. `CoreWorker.ts` has cited it wrongly since before this branch; three new docs had copied it, and now say what is true without a number. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…coincidence
**The publish guard had a hole in it, in new code.** `transform-package-json.ts`
merged the dependency fields `{...dependencies, ...peerDependencies,
...devDependencies}` — devDependencies *last*, so highest precedence. Adding a
`"math-expressions": "^3.0.0"` devDependency to `packages/doenetml` (for a test,
say) would have silently cleared the block the thirteenth pass built, while the
published peer range stayed `file:../math`. Measured through the real
transformer: `private: false`, peer `^3.0.0`. The merge now runs lowest
precedence first. No manifest in the repo declares the same dep twice today, so
no emitted manifest changes. `catalog:` joins the local-range set.
**And the exact regression it exists to prevent was not pinned.** Re-adding the
`PUBLISHED_PEER_DEPS` filter to `packages/doenetml/vite.config.ts` left all eight
of the thirteenth pass's tests green. Three new tests read the real config and
manifest: `math-expressions` is externalized, the *unfiltered* list reaches
`createPackageJsonTransformer`, and every externalized dep has a declared range.
Verified by reintroducing the filter — the middle one fails.
Two limits of the guard are now stated rather than overstated, in the vite
config's comment and in the notes: it is a shape test on the range with no
network, so `^3.0.0` before `math-expressions@3.x` exists reads as publishable;
and `npm run publish` continues past a workspace that exits non-zero, so a
premature merge refuses `@doenet/doenetml` and still ships its three siblings.
**A changeset claim that was a grid coincidence.** "A cell straddling a pole no
longer brackets a sign change" is not what happens. `evaluate_many` reports
`NaN` only for a sample landing *exactly on* the pole; issue Doenet#940's function
works because `[-100, 100]` over 1000 intervals puts a sample bit-exactly on
`x = 5`. Move the pole to 5.1 and the spurious minimum returns at
`5.100000624836199` (measured). Changeset and test comment corrected; the general
fix — `exactCriticalPointsOf` already returns the complete root set, so a
bracketed sign change matching no exact root is provably a discontinuity — is
written up as a follow-up.
**A seventh grading-reaching divergence, upstream (pin bumped).** `equals` reads
`det`/`trace` of a literal matrix as an *opaque variable*, so
`\det[[1,2],[3,4]]` and `-2` compare unequal although both simplify and evaluate
to `-2`. That is a third numeric path the thirteenth pass's sweep did not cover,
and it fails in a different shape — a `false` equality rather than a `NaN`.
`appliedFunctionSymbols.test.ts` excused `det` on a premise that measurement
contradicts (`det` of a real matrix *does* evaluate); the comment is corrected
and the matrix form of both reducers is now probed on both paths it does guard.
**Two latent copies of the worker leak the twelfth and thirteenth passes fixed.**
`getFlatDast` in both `test/utils/dast-to-flat-dast.ts` files boots a core worker
per call and is installed as a page global driven from a page that outlives every
call — the same shape, one directory over from the file that was fixed. Both now
terminate in a `finally`. `doenetml-to-pretext/src/index.ts`'s exported
`createWrappedCoreWorker` was a third copy that returned only the Comlink proxy,
so its worker was unreachable and could never be terminated; it has no caller and
is deleted rather than duplicated. A sweep of all 22 `new Worker(` sites across
both repos found no third *live* leak.
Smaller, each measured:
- `MatchesPattern.js`'s comment on the declared match kinds was wrong. `pi`, `e`
and `i` all still bind under `"variable"` (they are strings in the AST); the
one real narrowing is `i` under `"number"`. Pinned in `matchespattern.test.ts`,
verified to fail on the wrong expectation.
- `packages/math/package.json` listed `"import"` before `"types"` under `"."`
and had no `"types"` at all on `"./engine-rust"` — conditions match in order,
so both resolved only by file adjacency.
- `wasm-loader.ts`'s uninitialized-access error told every realm to `await` on
the browser main thread, including when the real cause was a build with no
inlined bytes.
- `engine-smoke.test.ts`'s `differentiates` opened with an assertion that cannot
fail — that an `Expression` has an `evaluate_to_constant` method. It now
asserts the derivative's tree.
- `check-bundle-size.mjs`'s `startsWith(prefix, start - prefix.length)` passes a
negative position for a blob near the top of a file, which clamps to 0 and
silently becomes "does the file start with the prefix". Guarded; the arrow-
valued `const finish` is a function declaration, per AGENTS.md.
- `domain.ts`'s two entry points disagreed about `undefined` (`domain[0]` versus
`domain?.[0]`); `Curve.js`'s comment claimed a live source that this PR closed.
Documentation reconciled against the tree: the notes said "six shadow reads"
where the follow-up beside it says eight unguarded reads of which five are
shadow reads; the migration plan still said five grading defects; the changesets
skill said the transformer emits `private: false`, which is now conditional.
New follow-ups written up: the general pole fix and two cheaper extrema items,
`static-assets`' generated `math-assets.json` being stale (missing `cbrt` and
`nthroot`, and `schema-freshness` does not cover the generator that produces it —
pre-existing on main), CI installing no `binaryen` so the shipped core is
unoptimized and a local build differs from CI's, the six undisposed LSP-test
workers, and `Function.js`'s four look-alike `inputMathFs` blocks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…th a second door **The general pole fix, implemented.** The fourteenth pass showed that the changeset's "a cell straddling a pole no longer brackets a sign change" was a grid coincidence — `[-100,100]/1000` puts a sample bit-exactly on issue Doenet#940's pole at `x = 5`, and moving it to 5.1 brought the spurious minimum back at `5.100000624836199`. Its write-up said the fix was to decline the *derivative* branch. Measured, that is not enough: with the derivative branch declined the minimum is still reported, because `fminbr` descends into the pole and the four strict-minimum comparisons all pass (`result.tol` steps far enough either side of the point it converged on that both neighbours read less deep than it does). `critical_points` gives the **complete** real root set of `f'` when the derivative is rational — reduced over a common denominator, so a pole cannot masquerade as a root. Where such an `f'` exists, `f` is differentiable wherever `f'` is finite, so an interior extremum is in that list; and where `f'` flips sign at a pole instead, `f` diverges there (the antiderivative of `c/(x-a)^k` is unbounded at `a` for every k ≥ 1), so that point is an infimum, not a minimum. A cell holding none of the roots therefore holds no extremum, and `extrema.js` now declines **both** estimators there. Independent of where the grid falls. **And a second seam, found while testing it.** `find_local_global_maxima` is the minimum hunt on a negated formula, and it negated with `fromAst(["-", formula.tree])`. Reading `.tree` serializes to a JSON AST, where every non-integer literal becomes an f64 — the engine holds `5.1` exactly, as `51/10` — and `critical_points` declines outright once a coefficient is inexact. So every *maximum* of a function with a decimal coefficient silently lost both its exact location and the pole rejection that needs the complete root set, while its minima kept both. `-(x+8)(x-8)/((x-2)(x+4)(x-5.1)^2)` reported a spurious maximum at `5.100000370673212`. Now `formula.multiply(-1)`, which stays inside the engine. Written up as a convention in the notes: rebuild an expression with an engine method, never from its `.tree`. `functionTag.test.ts` pins the off-grid pole and its negation, and asserts the four locations to 1e-12 — which also closes the "no refinement round-off" half of the claim, unpinned until now because the file's other assertions use a 1e-3 tolerance that cannot tell an exact root from a refined one. All three parts verified by revert-fail-restore, separately. **The publish guard had a second door.** The thirteenth pass built it and the fourteenth closed a hole in it; it still read npm's *bare-path* spelling of a local dependency as publishable. `"math-expressions": "../math"` installs the sibling directory exactly as `file:../math` does — npm accepts the path with or without the protocol — and the guard tested protocols only, so that spelling would have shipped a tarball whose import resolves to nothing, which is the one thing the guard exists to stop. `git+file:` joins the list for the same reason. Nothing legitimate is caught: no semver range and no dist-tag begins with `.` or `/` (`^1.2` and `~1.2` do not), which the new registry-range cases assert. Also, the fourteenth pass's justification for its own fix is wrong where it landed in the code: the comment says a devDependency could clear the guard "while the published peer range was still `file:../math`". It could not — the tested range and the emitted peer range are one lookup, so they cannot come apart. What actually happened, and what its commit message measured, is that both jobs went to the field a consumer's installer never sees. Comment corrected to the reason that is true. **Upstream (pin bumped to `a2c468a`, green):** `equals` said a determinant differed from its own value. Established as a regression against legacy, which answers `-2`/`5`/`true` to every case, and wider than reported — `evaluate_to_constant` was `None`, not `-2`. Fixed there; `appliedFunctionSymbols .test.ts`'s `NO_SCALAR_KERNEL` exemption for `det` was where the omission had been codified on this side, exactly as the deny list was upstream, and it is now empty and annotated. The matrix-reducer block also probes `equals`, the path that was actually broken and that neither of its two assertions covered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
… contradicts **`domain.ts`'s two entry points still disagreed, and the comment named a function that does not exist.** The fourteenth pass changed `find_effective_domain` to read `domain?.[0]` and said the resulting behaviour now matched "`find_domain_endpoints` below" — there is no such function anywhere in the repo; it means `find_effective_domains_piecewise_children`. And the agreement it claimed was partial: the sibling tests `if (domain1)`, this one still tested `if (domain1 !== undefined)`, so a `domain[0]` of `null` returned the real line there and threw `Cannot read properties of null` here. That is the same crash shape the piecewise-domain changeset is about, one value over. Unreachable today — `Function.js` stores an interval or leaves a hole, never a `null` — but the two now read and test the value identically, which is the property the comment was trying to state. `packages/utils/test/domain.test.ts` pins all five spellings of "no domain" against both entry points, and fails on the `null` case without the fix. **`check-bundle-size.mjs`'s negative-position guard is right, but it cannot fire, and the comment says the opposite.** The comment asserts "the length test is not redundant". It is: `WASM_URI_PREFIX` ends in `,`, which is not a base64 character, so a run beginning before the prefix ends is cut off there and never reaches `BIG_BLOB_MIN` — `finish` returns before the classification. Checked both ways: the set of run starts below the prefix length that could survive to the classification is empty, and a differential run of the shipped scanner against one with the guard removed produced no difference over nine adversarial inputs (blob at index 0/1/22, prefix-then-blob, prefix-minus-comma-then-blob, prefix-then-wasm-magic). Kept, because its deadness is a property of the prefix constant rather than of the loop and no test can pin it, but the comment now says which of those two things it is. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
**`extrema.js` had two `.tree` negations, not one.** `flip_function_children`
negates each *piece* of a piecewise function the same way
`find_local_global_maxima` negated the whole formula, and so lost exactness the
same way: the AST round trip turns the engine's exact `51/10` into an f64 and
`critical_points` declines a formula with one inexact coefficient. Measured on a
`<piecewiseFunction>` over `(x+8)(x-8)/((x-2)(x+4)(x-5.1)^2)`: the piece's
minimum came back exact and its maxima came back bracketed, `9.731226447456773`
against the exact `9.731224303522758`. Both are `multiply(-1)` now.
`piecewisefunction.test.ts` pins the piece's maxima to 1e-12, verified to fail
against the round trip.
**Written up, not fixed:** `flip_function_children`'s interpolated branch writes
`flippedStateValues.coeffsFlip` while its consumer reads `coeffs`, which
`{...stateValues}` already copied un-negated — so a `<piecewiseFunction>` with a
through-points piece computes that piece's maxima as the minima of the
*un-negated* spline. Identical on `upstream/main` and predating the branch
(`Global extrema (#2205)`), and no engine call is involved, so it is not this
PR's to fix; recorded in the notes because this pass was in that function and
would otherwise have looked straight past it.
**The grading-defect narrative said six in three places and seven in two.** All
seven are now fixed, so the notes, the migration plan and the PR body agree on
the count and on the reading that matters: four of the seven were invisible
because a *different* numeric path answered correctly — `nthroot` graded while
it could not be plotted, `erf` plotted while it could not be graded, and
`det`/`trace` did both while they could not be compared — and two of the seven
had been written into a test's own exemption list as deliberate before anyone
measured them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…hird door **The pole rejection was not grid-independent after all.** The fifteenth pass declined both estimators in a cell holding none of `critical_points`' roots and called that "independent of where the grid falls". Verified against 43 rational functions with poles at 5.1, 3.14159, −2.71828 and 0.7071 and their negations, that fix is sound — it removes 36 pole artifacts (|f| from 2e5 to 3.6e17, all within 1e-5 of a pole) and loses no genuine extremum, including the ones that sit close to a pole. But a cell can hold a root **and** a pole, and then the cell test is satisfied and `fminbr` still descends into the pole: `(x-5)^2/(x-5.1)^2` reported a maximum of 3.01e10 at `5.100000576002722`, since the only root of f' is 5 and it lies on the left edge of the cell holding 5.1. Asking the question of the converged *point* instead closes it: where the complete root set is known, a reported minimum has to be **at** one of those roots, within `fminbr`'s own convergence width. That subsumes the cell test for that estimator, and it is the same argument the fifteenth pass made — an interior extremum of a function with a rational derivative has f' = 0 — applied where the estimator actually lands rather than where its bracket started. `functionTag.test.ts` pins the shape and its negation; every extremum in the 43-function corpus is unchanged. **The derivative and the exact roots are computed once, not once per recursion level.** They are pure functions of `formula` and `variables`, and the recursion re-enters with the same ones on a narrower domain. The follow-up write-up said they were recomputed "on each of up to ~1000 cells × 100 recursion levels"; measured, the per-cell half was wrong — the loop only reads the finished list. Per recursion level it was real, and `critical_points` is the expensive call: for a degree-12 polynomial over the default domain it is 98ms of the 131ms a hunt takes. 392ms → 316ms over minima and maxima together, −19%; shallow cases are inside the noise. Corrected in the notes rather than restated. **The publish guard had a third door, so it now mirrors npm instead of listing spellings.** The thirteenth pass wrote the protocols, the fifteenth added the bare path after finding `../math` slipped through — and `~/src/math-expressions` still did, along with a bare tarball path. Checked against `npm-package-arg` directly: its `isFileSpec` accepts `.`, `~/`, `/`, `\` and a drive letter, and a spec ending in `.tgz`/`.tar(.gz)` with no protocol is a local file too. The guard is now that definition — "a range npm resolves to a `directory` or a local `file`" — split into three named tests. A first draft caught `https://…/math.tgz`, which a consumer *can* fetch; the protocol test is what keeps it out, and it is asserted. The four new local spellings fail against the old guard, and the registry ranges (including `~3.0.0`, one character from `~/`) still publish. **Upstream (pin bumped to `3b27de9`, green):** an eighth grading defect, the same folder-versus-sampler split as `det`, reached by an extra pair of parentheses — `simplify(mod((7,3)))` was `1` while `equals(mod((7,3)), 1)` was `false`. Both layers now go through one helper. Two corrections went with it: the open ledger had blamed the *fold*, and narrowing the fold is a regression against legacy rather than a fix; and the fifteenth pass's claim that `rootof` is unreachable is false — the canonicalizer's rewrite needs an already-expanded polynomial, so `rootof((x-1)(x-2), 0)` stays opaque. Filed, not fixed, with the reason. The grading-defect narrative moves to eight in all five places that count it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…ibling upstream Three follow-ups to the pass, none of them behavioral: **The `-19%` was measured across two different code states.** The before-number came from a build without the point-test pole guard and the after-number from one with it, which is not a controlled comparison — and the point test replaced a `find` with a `filter`, so it has an allocation cost of its own that the number was hiding rather than including. Re-measured by toggling *only* the memo on otherwise identical code: `(x-1)(x-2)…(x-12)` over the default domain goes 412ms → 314ms, `-24%`, and `sin(20x)+x^2/100` — sixteen recursion levels over a much cheaper derivative — goes 72ms → 63ms. The three non-recursing cases are inside the run-to-run noise, as before. **`derivativeInfoFor` returned the same four-field literal from two branches.** One `NO_DERIVATIVE` constant now, which is what the two branches always meant: a formula with a hole in it and one the engine cannot compile are the same answer to the caller. **`MATH_EXPRESSIONS_UPSTREAM_REQUESTS.md` had the `det` fix and not its sibling.** The `f((a,b))` fix is the same two layers and the same failure shape, so it belongs in the same closed section, together with the two things that had to be measured before it could be the right fix — that the ledger had blamed the fold, and that narrowing the fold is a regression against legacy. The `rootof` claim that went with the `det` sweep is recorded there as **open**, since it is the one part of that sweep that measurement contradicted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…llers it hid
A pass over the files this review has touched least — the ones in the diff but in
no `Review cycle N:` commit — found one thing worth fixing and two worth stating
accurately.
**`evaluate_to_constant()` can return a math.js `Complex`, and the vendored
declaration said `number | null`.** Legacy returned a plain number for a real
value and a complex one for a non-real value, and the engine keeps that contract
— `fromText("i").evaluate_to_constant()` is `{re: 0, im: 1}`, which the compat
layer documents at the implementation. The narrow declaration was harmless until
the seventh pass fixed `copyDtsFiles` and these declarations began reaching
consumers; from then on it type-checked code that cannot hold what it is handed,
and it contradicted a comment in `utils/math.ts` that says in so many words "a
complex result passes through".
Widening it to `number | Complex | null` — `Complex` is already declared in that
file, and `evaluate()` beside it already returns it — turned up two callers, and
they are the same function copied into two packages: `numberFromSerializedAst` in
`doenetml-to-pretext` and in `doenetml-prototype` promises `number` and returns
`evaluate_to_constant() ?? NaN`, so `??` caught the `null` and let a `{re, im}`
object out of a function declared to return a number. `graph-point.tsx` feeds
that straight into a plotted coordinate. Both now call `toNumberOrNaN`, the
helper that exists for this and covers both cases. Both packages type-check clean
afterwards; they were clean before only because the declaration was wrong.
**`UNSPECIFIED_COMPONENT` had no test that could fail.** Its defence against
colliding with an author's own variable of that name is a U+E000 prefix, and its
own docstring warns the prefix is invisible in editors and in `git diff`. The one
other test that mentions the marker imports the constant and compares it against
itself, so it stays green whether or not the prefix survives a paste. One test
now asserts the code point — written with an explicit escape, since a test for an
invisible character must not itself be invisible, and ordered so the readable
assertion fails first (`expected 117 to be 57344`, rather than
`'unspecifiedComponent'` is not `'unspecifiedComponent'`). Verified by dropping
the prefix.
**`doenetml-worker`'s `dedupe` comment stated a mechanism the tree contradicts.**
It said that without the entry the seam "resolve[s] to distinct module instances"
and the bundle "ends up with two copies of the engine". There is one install —
a single root-hoisted `node_modules/math-expressions` symlink, no nested copy,
because all seven declaring packages use the identical `file:../math` range — and
vite resolves symlinks to one realpath. The entry is defensive, not a fix for
anything observed, and the comment now says which of the two it is, matching the
framing `virtual-keyboard`'s config already uses for the same specifier.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…use it **A ninth wrong-answer-on-grading defect, and it is the other recurring shape.** `<isBetween lowerLimit="-1" upperLimit="1">x</isBetween>` answered **true** for a free variable. `evaluate_to_constant` returns `null` for an expression that is not a constant, `null` coerces to `0` in a `<`/`>` comparison, and zero is inside that interval. The legacy library returned `NaN` here — its `nan_for_non_numeric` default, measured on `2.0.0-alpha95` — and `NaN` fails both comparisons, so this read as correct until the engine switch. `<isBetween>` is a boolean and reaches `<answer>` through `<when>`. Its two siblings in the same file, `<isInteger>` and `<isNumber>`, both guard with `Number.isFinite`; this one did not. That is the whole lesson: the helper already existed (`evaluateToNumber`), and a helper is worth nothing at the one call site that does not call it. Pinned in `booleanoperatorsonmath.test.ts` with limits deliberately asymmetric about zero — so a wrong `0` cannot be right by accident — and a numeric control case, verified by reverting the fix (`expected true to equal false`). The count moves to nine everywhere it is stated. **The sixteenth pass's type widening was right and left 23 type errors behind.** It reported that "both packages type-check clean afterwards" — true of the two packages it edited. Nothing runs `tsc` in CI (`Lint Typescript Code` is Prettier plus a filename-collision check), and `packages/doenetml`'s `vite build` *logs* every `vite-plugin-dts` diagnostic and exits `0`, so 22 fresh `TS2322`s in `packages/doenetml` and one in `doenetml-worker-rust` turned nothing red. All 23 are the same `evaluate_to_constant() ?? NaN` the widening was about, and take the same fix: `toNumberOrNaN`, or `typeof scale === "number"` where the value is compared rather than assigned. One of them was a real contract violation, not just a type: `evaluateToNumber` in `doenetml-worker-rust` is documented — and doctested — as "returning NaN if the expression cannot be evaluated", and had been returning `null`, which the Rust side turns into an `anyhow` error instead. The 18 errors that remain in `packages/doenetml` predate this branch (JSXGraph label typings, one `RoundType`), so the missing CI job is written up as follow-up 9 rather than added on top of them. **The publish guard's fourth door, and the first one found by diffing against npm rather than by thinking of a spelling.** Checked every spec form against `npm-package-arg@14` directly: the guard mirrored `isFileSpec` but never mirrored npa's *last* branch, where any protocol-less spec containing a separator is a local path — no leading dot, no `~/`, no drive letter, no `.tgz` required. So `"math-expressions": "vendor/math-expressions/packages/…"`, which is how a path in *this* monorepo is actually spelled, read as publishable and would have shipped an unresolvable peer range. Confirmed against npm 11.12.1, which installs `sub/dir/pkg` as a symlink to that directory. `user/repo` GitHub shorthand still publishes, and the three places the guard is deliberately stricter than npa (`link:`/`portal:`/`workspace:`/`catalog:`, which npa *throws* on; UNC paths, local only on Windows; a one-slash `.tar.gz`, a repository name to npa) are now named as such instead of being claimed as npa's own answer. Five new spellings fail against the old guard; the registry ranges, including `~3.0.0` one character from `~/`, still pass. **Upstream (pin bumped to `10fdf8a`, green at `e2b3bfd`):** `rootof` is fixed rather than filed. The sixteenth pass asked whether a cross-spelling inequality on a grading path is shippable; the premise does not hold — `rootof` is in neither of DoenetML's applied-function lists, and legacy has no `rootof` at all — but it is a self-inconsistency in the surface Doenet#84 publishes to npm, so `expr_to_upoly` now multiplies polynomials instead of reading only a sum of monomials. Two of the filing's details were wrong: a *scaled* polynomial was never the problem, and the obvious degree guard narrows `rootof(x^70 - x^69, 0)`. The published `types/math-expressions.d.ts` also gets the sixteenth pass's `Complex` widening, which had reached only DoenetML's vendored copy of it. **Three claims of the sixteenth pass re-measured and upheld**: the `f((a,b))` shared helper is consistent across all nineteen heads that can spread and regresses nothing (every changed cell moves `false`→`true` or `None`→`Some`); the extrema point test loses no genuine extremum over an independent 66-function sweep, including minima 1e-4 from a pole, and removes every pole artifact; and the widening's two `numberFromSerializedAst` fixes are correct. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…re it lands The sixteenth pass's `spread_list_argument` closes the heads whose spread arity is evaluable — `mod` and the six aggregates — which is the subset that reached grading. The other fifty-odd heads are still two distinct opaque atoms, and asking why turned up a layer below equality: `to_js` is not injective. `f((x, y))` and `f(x, y)` are different Rust trees that serialize to the same JS AST, and that AST is the contract with DoenetML. Both halves of the severity question were measured rather than argued. **Grading is unaffected.** `checkEquality.js` hands raw `.tree` values to `check_equality`, which rebuilds both operands with `me.fromAst` one line before `.equals()` — so the distinction is erased on the way in. An `<answer>` awards full credit for `sin((7,3))` against `sin(7,3)`, and `<boolean>$m1 = $m2</boolean>` is `true`, on every head tried, while the underlying objects compare `false`. That is why this is *not* a tenth grading defect and the count stays at nine. **Rendering is affected, and it is a regression.** A `<mathInput>` holding `\sin\left(\left(x,y\right)\right)` renders with its inner parentheses; after a save/restore through `serializedComponentsReviver` the *same saved JSON* renders without them, and `floor((x,y))` changes notation outright. Legacy is stable across the same round trip. LaTeX output has stopped being a function of the AST. Filed upstream with the fix identified — flatten a lone `Tuple` argument in the parsers, since the printers read the raw tree and a canonical-form fix would repair `equals` and leave the display — rather than attempted at the end of a pass, because it reaches the round-trip suites. Pin bumped to `6a39cc3`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…cated helper It asked whether `numberFromSerializedAst`, which it had to fix in two places, should be shared. Yes, and the move is free — the facts, so the follow-up can be opened from the description alone: the two copies are byte-identical (`diff` exits 0) and were identical in pre-switch DoenetML too, so the duplication predates this branch; `toNumberOrNaN` and `serializedComponentsReviver`, the helper's only imports, already live in `@doenet/utils`, which already depends on `math-expressions`; neither consumer is in `@doenet/utils`' dependency set, so there is no cycle; and both files already import from `@doenet/utils`, so no call site gains an import. Recorded rather than done because the duplication is pre-existing, and because the move exposes something that should be fixed with it: neither package *declares* `@doenet/utils`, in `dependencies` or in its wireit `build.dependencies`. Both build today only because `doenetml-worker:build` happens to pull `../utils:build` in first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…h's shape, and a tsc gate **The `f((a, b))` serialization defect is fixed rather than filed again** (upstream pin bumped to `ea3e378`, green at that head). `to_js` was not injective: `f((x, y))` and `f(x, y)` were different Rust trees serializing to byte-identical JSON, so an expression was not equal to itself after a round trip through its own `.tree`, and the same saved `<math>` rendered `\sin\left(\left( x, y \right)\right)` before a save/restore and `\sin\left( x, y \right)` after. The fix is in the parsers, not canonicalization — the printers read the raw tree — and it exposed an older printer defect underneath: the LaTeX bracket notations were guarded on `args.len() == 1` and otherwise emitted `\abs` and a braceless `\sqrt`, neither of which MathJax can render. Both restore legacy's output byte for byte, which is why no changeset entry follows: relative to what users have today, nothing changes. **A tenth and eleventh grading defect, from reading the population the ninth came out of instead of waiting for the follow-up.** Both are the `null`-coerces-to-`0` shape, and both sit in a file that already imports the guard and uses it a few lines away. - `<lineSegment>`'s public `slope` read its four endpoint coordinates raw, so `endpoints="($blank,1) (3,4)"` answered **`1`** — the slope of `(0,1)–(3,4)` — for a segment that has none. `slope` is `createComponentOfType: "number"` and reaches `<answer>` through `<when>`, so a student who submitted nothing scored full credit. The same commit range had already fixed `numericalEndpoints` 180 lines above and the inverse definition 35 lines below, and walked past the forward definition between them. - `periodicSetEquality` mapped a student's offsets through the raw call before `mod(offset₀ − offset, period)`, so an answer made entirely of free variables looked like it started on the set and collected a third of the credit under `matchPartial`. Fixed by mapping to `NaN`, **not** by refusing the list: the match loop counts a consecutive run, so a partly symbolic `0, y, 6` must keep the 0.3 it earned, exactly as it did before the switch. The first draft refused the whole list and was a regression against legacy; measured before shipping. - `<piecewiseFunction>`'s `numericalfs` is character-for-character the code this branch already fixed in `Function.js`, and is fixed for the same contract. No document was found that reaches it, so it is a contract fix, not a measured wrong answer. - `utils/lineSegment.js`'s two reads, on the drag path, where `null - null` took the *identical endpoints* branch instead of the no-slope one. Both defects verified by revert-fail-restore. The sweep re-measured follow-up 3's population: **254 live call sites, 44 unguarded** (the older 277/204 figures were counted differently and are superseded); all 44 were read, and the other 42 are rendering-only, latent with no consumer, or pointer-coordinate paths. **The `tsc` gate is added rather than written up**, because the type surface that went unchecked is one this PR introduced. `npm run typecheck` (`scripts/typecheck.mjs`) runs `tsc --noEmit` over every package with a `tsconfig.json`, as a step in the Build job — there rather than in the lint job because the root `paths` mapping resolves `@doenet/…` to a package's `dist`. It discovers packages instead of listing them and fails on a stale exclusion. **22 packages pass in ~30s**, including `packages/doenetml`, where the 22 unchecked errors were. Unblocking it cost three edits, not eighteen: 16 of that package's 18 pre-existing errors were one `LabelLikeJXG.update` declaration. Five packages are still excluded with their counts named (35/18/2/2/2); none is in a file this branch touches. The seventeenth pass's "18 pre-existing errors" was `packages/doenetml` alone — the repo-wide figure is 75. **A fifth door in the publish guard, opened by the fourteenth pass's own fix.** The guard `.trim()`ed before classifying, on the stated grounds that "npm trims the range first". npm does not: `hosted-git-info` refuses any spec containing whitespace, so `"vendor/math "` never becomes a repository to npm — but trimming turned it back into a clean `user/repo` shape and called it publishable. Measured against npm 11.12.1: that range installs `node_modules/math-expressions` as a *dangling* symlink and exits 0, so the consumer's import throws at run time. The prefix tests still run on the trimmed range (a leading space must not slip `" file:../math"` past an anchored pattern); only the GitHub-shorthand test sees the raw one. The same sweep — a diff against the real `npm-package-arg@14` over 173 spellings — found one realistic *over*-strictness and fixed it too: `git@github.com:user/repo.git` read as a local path and would have blocked a release over a clonable dependency. Eight new spellings fail against the old guard. **Cycle 17's items verified.** `rootof`: `expr_to_upoly` really does multiply now, `rootof((x-1)(x-2), 0)` equals `1` and `rootof(x^2-3x+2, 0)`, and the claim that the obvious degree guard would have *narrowed* `rootof(x^70 - x^69, 0)` is reproduced — `max_rootof_degree` is 64, degree 70 reaches the leaf only through the uncapped monomial arm and `make_rootof`'s squarefree radical. `rootof` really does appear nowhere in DoenetML's sources. `<isBetween>`'s siblings really do guard, and its test really does fail without the fix — but its comment and commit message gave the repro as `lowerLimit`/`upperLimit`, attributes that do not exist; with them `limits` is empty and the answer is `false` for an unrelated reason. The comment now names `limits`, which is what the test uses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
The PR description and the guard's own doc comment had it; the durable notes file still said "the fourth form the test has taken" and listed three deliberate divergences from `npm-package-arg`. It now names the fifth door — the `.trim()` an earlier pass added on the false premise that npm trims, which turned `"vendor/math "` back into a `user/repo` shape — the scp-style git URL the same sweep found the guard over-strict on, and the fourth deliberate divergence. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
… wrong A pass over what seventeen cycles had covered least — `packages/math`, the seam the whole PR rests on — and over every checkable number in the four documents. **A broken `.d.ts` was being shipped, and it was the exact leak two long comments claim to prevent.** `vite.config.ts` passes `copyDtsFiles: true`, which copies `src/vendor-shims.d.ts` into `dist/` — and `vite-plugin-dts` rewrites its module specifiers on the way, turning `declare module "math-expressions-js-compat"` into `declare module '../../../vendor/math-expressions/.../lib/math-expressions.ts'`. That is invalid TypeScript (`TS2436: Ambient module declaration cannot specify relative module name`, three times, plus a `TS2439` and two `TS2307`s) and it put a submodule source path inside `files: ["/dist"]`. `engine-rust.ts` and `wasm-loader.ts` both explain at length why no submodule path may reach a consumer's type program; both were true of their own files and both were defeated by the copy. The file is input to our type-check, not part of our published surface, so it is excluded. It matters at Step 6, when `@doenet/math` is meant to be publishable. **Three of the four gap fills the seam deleted had no test.** `engine-rust.ts` says in so many words that "deleting them is how we verify each upstream fix actually covers our usage" — and only `.f()` was pinned. Added: the non-finite `fromAst`/`.tree` fixpoint (`NaN`, `±Infinity`, and one nested), `fromAst` accepting an `Expression` where a tree is expected, and the context-level operation family. Also `dopri`, which is the one export whose regression would be silent in *both* the type system and the suite: `engine-rust.ts` reads it off the context through an `as unknown as` cast, so an upstream rename yields `undefined` with no type error and the failure surfaces at run time in `ODESystem.js` and `@doenet/utils`. All four verified to produce real values, not vacuous passes. **Three smaller ones in the same files.** `initMathWasmSync` tested `initialized` and never `initPromise`, so it could instantiate a second time on top of an in-flight `initMathWasm()` — the exact hazard its own header says the two entry points are written to avoid. Unreachable today; it now refuses rather than returning a lie about being ready. `dropDefaultWasmPath`'s `hits` counter is never reset, so under `vite build --watch` a rebuild inherits the first build's count and the "glue changed shape" check can never fire again — putting the ~2.25 MiB duplicate silently back. And `getComponent`'s bare `catch` swallowed the `wasm-loader` Proxy's "core is not initialized" throw as though it meant "no such component", turning a setup error into a silently dropped value; it now re-throws anything the seam itself raised. Its `?? undefined` was dead — upstream throws rather than returning `undefined` — and the comment right above it says so. **Numbers.** Every count in the four documents was re-derived. Wrong, and fixed: the import-count table (136/142/147 → 139/145/150, with the command that reproduces them and the merge-base column beside it — only the *first* row is identical at the base, which is the row "no call site changed" actually means; the other two moved because this branch adds five importing files); `../math:build` is a dependency of seven packages, not four; `evaluate_to_constant` is 206 in `worker-javascript/src` and 21 in `utils/src`, not 194 and 17, and `.tree` is 39 in `utils/src`, not 42; the vendored declarations are 504 lines against upstream's 525 with comments stripped, not "1,152 either side", and the delta is exactly the 21-line v3 block already enumerated there. The exact byte counts for the WASM and the bundle are removed rather than corrected — the file itself says the wasm is not byte-reproducible, so those digits could only ever be wrong; the MiB figures they accompanied are right and stay. `147` was repeated in ten places. It is now stated once, in the table, and the other nine say "the files that import the library" — a number repeated in ten files goes stale in ten files. **One follow-up was misdiagnosed and would not have worked.** Follow-up 6 said the committed `math-assets.json` had "drifted" from its generator and proposed running the generator in CI. Measured: `generate-math-assets.ts` holds its own hard-coded 65-entry list, byte-for-byte what the JSON already contains, so re-running it changes nothing. The divergence is between that generator and the worker's 67-entry list, which it duplicates instead of importing. Rewritten with the real cause and the fix that follows from it. Two more corrections: the `get_component` audit said "eight reads … the other three" and then listed four locations (nine and four, verified); and follow-up 10's undeclared-`@doenet/utils` defect has a third instance, `doenetml-worker-rust`, whose import list this branch adds to. CI green end to end on `74e0df251`, including the new Type check step, which type-checked 22 packages in 60 seconds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…ass opened Verified the eighteenth pass head by head against `math-expressions@2.0.0-alpha95` before looking for anything new. Both of its central claims hold: the `f((a, b))` parser flatten is byte-identical to legacy on all seventeen adversarial parses tried — nested applications, a two-argument head receiving a tuple, `f((x,y),z)`, an integral with a differential, both parsers — and the LaTeX printer fix reproduces legacy exactly on every notation it touched. What it did *not* reach is upstream, in Doenet#84. **A twelfth, thirteenth and fourteenth grading defect, out of the population the eighteenth pass declared safe.** It sorted the 44 unguarded `evaluate_to_constant` sites into three groups; two of the three claims were wrong, and one was hiding live defects. "Pointer-coordinate paths where the input is a number by construction (the six `nearestPoint` definitions)" is not true: `<constrainTo>` hands `nearestPoint` the constrained point's coordinates as *math expressions*, so a point with a blank or symbolic coordinate arrives un-evaluated, and `null` is `0` to the distance arithmetic where `NaN` had made every candidate comparison fail and left the point alone. `<polygon>`, `<polyline>` and `<regionBetweenCurveXAxis>` therefore snapped such a point onto themselves: <mathInput name="mi" /> <graph> <polygon name="poly" vertices="(0,0) (4,0) (4,4) (0,4)" /> <point name="P" x="$mi" y="9"> <constraints><constrainTo>$poly</constrainTo></constraints> </point> </graph> <answer><award><when>$P = (0,4)</when></award></answer> With the input empty, `P` sits at exactly `(0, 4)` and a student who entered nothing scores full credit. `<circle>`, `<line>`, `<parabola>` and `<curve>` already opened with the identical four-line `Number.isFinite` idiom, which is what made the group look uniform from a distance — the ninth/tenth/eleventh lesson a fourth time: the guard existed and the call sites did not use it. Fixed in all five that lacked it; `<discreteSimulationResultPolyline>` and `<regionBetweenCurves>` are the same code on the same path and are fixed for symmetry rather than from a reproducer. Three tests, each verified to fail against its own fix reverted and no other. The other two group claims are corrected rather than fixed: `<angle>`'s renderer guards `numericalRadius` and *not* its point coordinates, so `numericalPoints` really does carry a `null` into JSXGraph — but it is `forRenderer` and not `public`, so no grading path reaches it and it belongs to the follow-up; and `<line>`'s `numericalCoeff*` are consumed after all, by a caller that guards. **The publish guard's sixth door, opened by the eighteenth pass's own fix.** That pass corrected an over-strictness — `git@github.com:user/repo.git` would have blocked a legitimate release — by whitelisting any `user@host:rest`. npm has no general scp-style parser: `npa` calls such a range `git` only when `hosted-git-info` recognises *both* the host and a `user/project` tail, and otherwise falls through to `fromFile` and calls it a **`directory`**. So `git@gitlab.example.com:group/repo.git#v3` and `git@git.company.internal:team/math.git` — a self-hosted GitLab or Gitea remote, the realistic spelling — went from blocked to publishable, and `user@host:../math` was measured against npm 11.12.1 installing `node_modules/math-expressions` as a *dangling* symlink at exit 0. The repo's own test table had the GitLab case pinned in the **publishable** column, so the hole was asserted rather than merely missed. Closed three ways, all measured against the real `npm-package-arg`: the scp test is restricted to the hosts `hosted-git-info` knows; a protocol-less range carrying an `@` or `:` that it does not recognise is unpublishable (no version range, dist-tag or shorthand contains either character, and `npa` *throws* on these rather than classifying); and a `%` disqualifies the GitHub shorthand, because `hosted-git-info` `decodeURIComponent`s and swallows the `URIError`, so `vendor/math%po` is a `directory` to npm and installs as a *working* local symlink — the silent kind. A 300-spelling differential sweep now shows **0 leaks**; the 28 remaining divergences are all over-strict, which is the safe direction. Nine cases added to the table, all nine verified to fail without the fix. `user/repo#semver:^3` keeps publishing — a committish may carry a colon. **The `tsc` gate could not notice an exclusion that had been fixed.** Its comment says a stale entry "silently un-gates a package that someone has already cleaned up", but the check only caught a package that had *left the tree*. A package whose errors somebody fixed stayed on the list and was never gated again — the likelier way an exclusion rots. Excluded packages are now type-checked like any other and the run fails when one comes back clean; verified by putting a clean package on the list. 22 pass and 5 are confirmed still unclean, in 36 seconds. Two of the five exclusions were described wrongly when recorded, and the descriptions are what a follow-up would budget from: `doenetml-worker-javascript`'s 35 are not "all implicit `any`" (23 are; 12 are real mismatches, ten of them `TS2339 Property 'returnDependencies' does not exist`), and `vscode-extension`'s 18 are not React code needing work — every one is a missing-`dom` diagnostic that `tsc` names the fix for, because that package's `lib` is `["es2020", "WebWorker"]`. The eighteenth pass's repo-wide "75" is also wrong: the five entries sum to **59** (77 with `packages/doenetml`'s own 18, which that pass fixed). The submodule pin moves to `3335f38`, which fixes the three printer arms the eighteenth pass's `args.len() == 1` sweep left behind, puts back a compat expectation that had been rewritten away from the JS output, and replaces a divergence ledger that was stale in four places with one a test can check. The changeset is unchanged and still accurate: every fix here restores parity with the released engine rather than altering behaviour a user has seen. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
Bumps `vendor/math-expressions` to `ca5168d`, where `evaluate_to_constant` answers `NaN` rather than `null` for an expression with no numeric value, and carries the DoenetML half of that landing. The Rust core answers `Option<f64>` — right there, where nothing coerces — and that crossed to JS as `null`. `null` is the one "no value" marker that does not behave like one: `Number(null)` is `0`, `null + 5` is `5`, `null <= 1` is `true`, `Number.isNaN(null)` is `false`. So an expression with no value did not fail loudly in a consumer that had not been individually taught about it, it quietly behaved like zero. That single inversion is the root of the class of grading defects these review passes have been finding one at a time — a rectangle 0 wide, a polygon centroid pulled to the origin, a vertical line with slope 1, `<isBetween>` true for a free variable, a `<constrainTo>` point landing on the shape so a blank answer scored full credit. Legacy `math-expressions@2.x` answered `NaN` for every one of these inputs (measured against `2.0.0-alpha95`), so this restores the contract rather than inventing one. - `packages/math/src/vendored/math-expressions.d.ts` is re-synced with upstream's declarations: `evaluate_to_constant` is `number | Complex`, not `number | Complex | null`, and `match` is `MatchResult | false` taking a `MatchOptions` object rather than `MatchResult | null` taking a bare boolean (it has always returned `false`, so a `!== null` test against the old declaration was unconditionally true). Declaration lines are byte-identical to upstream's again — 509 now, and still differing only by the trailing ODE block and default export the header documents. - The two helper contracts are restated. `isNumericConstant` and `toNumberOrNaN` keep their jobs — the `Complex` arm is still there, and a numeric state variable still has exactly one way to say "not a number" — but they are no longer the only thing standing between a `null` and a wrong number, so a site that forgets one now degrades loudly. - `engine-smoke.test.ts` pins the contract at the seam, which is where ~890 DoenetML call sites read it: seven no-value spellings answer `NaN`, the marker survives `+ * / Number() <= >= isNaN`, `i` is still a `Complex`, and `1/0` is still `Infinity`. Revert-fail-restore verified against the previous pin — four of them fail. The changeset's account of this is corrected: it claimed the engine reports "cannot evaluate" separately from "evaluates to NaN". It did; it does not now, and the user-visible half it was explaining (`NaN` rather than zero) is unchanged and now uniform.
The maintainer's instruction, and he is right: he is the one who will publish `math-expressions@3.x` and merge these two PRs in the right order, so code protecting him from doing it in the wrong order is superfluous. It also has a record. The guard was a build-time *shape* test on a dependency range, and it was holed six times across review passes 13 through 19 — twice by the very pass that had just "fixed" it. A test that has to be rewritten every other pass to stay correct about `npm-package-arg`'s classification is not protection, it is a second thing to get wrong. Removed: - `scripts/transform-package-json.ts`: the seven range-classifying regexes, `isUnpublishableRange`, the `unpublishable` accumulator, `pkg.private = …` and its warning. 388 lines to 159. What is left is what the file is *for*: promoting externalized dependencies to `peerDependencies` with the range the source manifest declares, stripping the workspace-only fields, and rewriting `files`/`exports` paths relative to the output directory. The `devDependencies`-lowest precedence in `allDeps` stays and is still tested — it decides which range ships, which is true independently of any guard. A dep with no declared range now warns and is skipped rather than blocking. - `.github/scripts/npm-publish-with-retry.mjs`: the `pkg.private` preflight, which existed only to turn the guard's `EPRIVATE` into a sentence. Nothing else in this repo writes `private` into a built manifest. Retries, transient patterns, idempotency and dist-tag handling are untouched. - `packages/standalone/scripts/transform-package-json.test.mjs`: the 73-entry and 21-entry range tables and the four `private`-asserting cases. 348 lines to 212, ten tests. The kept ones pin what survives, including one new pair that the guard's presence had made unstateable: the range is copied *verbatim* whatever its shape, and the source manifest's `"private": true` does not reach the tarball. Kept, because it is correct independently and is what lets the range become `^3.x` at the swap: `packages/doenetml/vite.config.ts` still hands the *unfiltered* `EXTERNAL_DEPS` — `math-expressions` included — to `createPackageJsonTransformer`, and the test that pins that stays, with its reason restated. A filtered list would drop the seam out of the published manifest's `peerDependencies` entirely, which is a real defect and has nothing to do with publish ordering. Verified by building: `@doenet/doenetml`, `@doenet/standalone`, `@doenet/doenetml-iframe` and `@doenet/v06-to-v07` all emit a `dist/package.json` with no `private` key at all (the pre-guard behavior — the guard is what had been writing an explicit `false`), and `@doenet/doenetml`'s `peerDependencies.math-expressions` is `file:../math`, ready to become the published range. Documentation: the *order* stays, stated once, under a renamed "Release order" heading in `MATH_EXPRESSIONS_RUST_MIGRATION_PLAN.md`, with a short note that nothing enforces it and why. The five paragraphs in `MATH_EXPRESSIONS_ENGINE_NOTES.md` describing the enforcement mechanism, its two limits, its fifth door, and the four deliberate divergences from `npa` are gone; so is the follow-up item about the guard not being atomic across the four publish targets, which was a limitation of a thing that no longer exists. `.github/skills/changesets/SKILL.md` no longer describes a transformer that keeps `private: true`, and now says what the built manifest actually contains. Step 6 of the plan is rewritten as a checklist someone can execute in one sitting, because the maintainer will run it before merging rather than after. Twelve numbered steps with the line numbers to edit, re-verified against this tree, and four corrections to what it said before: - **`^3.x` will not install `3.0.0-alpha1`.** Upstream's version is a prerelease and npm semver excludes prereleases from a caret range; measured both directions. Every range in the old table said `^3.x`. Nothing checks this now that the guard is gone — and the guard never resolved ranges against the registry anyway, so it would not have caught it either. - **The Rust toolchain does not disappear.** `packages/doenetml-worker-rust` runs `wasm-pack build --target web`, which needs `rustup target add wasm32-unknown-unknown`, and that build is on `packages/doenetml:build`'s critical path. Only the *version-locked* `wasm-bindgen-cli` leaves with the submodule. So `ci.yml`'s Rust lint job and the devcontainer's `rust` feature must not be touched, and the `wasm-toolchain` feature can only be deleted after confirming wasm-pack still resolves the target in the rebuilt image. - **`packages/math` survives**, which the old text hedged on. Inlining the wasm as base64 rather than fetching it is DoenetML's decision, not upstream's, and `wasm-loader.ts` is where it lives. So six of the seven consumer manifests keep `file:../math` and it is `packages/math/package.json` that gains the npm dependency — with a name collision to check, since `@doenet/math` is itself installed *as* `math-expressions`. - `src/vendor-shims.d.ts` is deletable too and was never mentioned; `Dopri` is a type alias here against upstream's function declaration, so that one re-export is not literal; the vendored declaration count is 509, not 504. Also recorded: the submodule URL is `siefkenj/math-expressions`, not `Doenet/math-expressions` as the prose says, and `math-expressions@3.x` is not on the registry yet (`latest` is `2.0.0-alpha95`).
Second half of the sentinel landing, separate from the semantics change as asked. Three kinds of thing. **Seven call sites that were still leaking the old sentinel into arithmetic.** A sweep of all 269 non-test `evaluate_to_constant()` calls found these reading the raw result and computing with it, so each was a live defect until the pin bump and is fixed by it. They are guarded here anyway, because the `Complex` arm is still there and each of these feeds a renderer, a grade or a state variable that cannot hold one: - `Angle.js` `numericalPoints` — `forRenderer`, handed to JSXGraph, which coerces. `<angle through="(a,b) (0,0) (1,0)" />` drew a real angle with the undefined vertex at the origin, in the graph and in the PreTeXt export. - `Curve.js` `controlPoints` inverse definition — `$curve.controlPoints` on a bezier curve with a symbolic through point reported the bare control vector, as if the point sat at the origin. - `Curve.js` `calculateControlVectorFromSpline` (four reads) — the *default* path for `<curve through="…">`, so `<curve through="(0,0) (a,b) (2,3)" />` produced finite, plausible control vectors computed as if the symbolic point were `(0,0)`, and its own `numericEntries` flag agreed they were numbers. - `FunctionIterates.js` multi-dimensional branch — the one-dimensional branch directly above it already asked `isNumericConstant` and filled `＿`; this one did not, so `initialValue="(a,b)"` reported the orbit of the **origin** while the same thing in one dimension correctly reported blanks. That asymmetry now has a test, revert-fail-restore verified. - `Polyline.js` `referenceVertices` in the rotate/dilate inverse — dragging a `rigid` polygon with one symbolic vertex and `rotateAround="vertex"` swung that vertex about the origin and wrote a concrete number back over the symbol. - `CobwebPolyline.js` vertex reads — the distance to the attractor decides whether a student's vertex is **marked correct**, and it was being measured from the origin, so a vertex with no value could land inside `attractThreshold` and score. `utils/sequence.js`'s three `Math.floor(… evaluate_to_constant() …)` reads were in the same class — `<sequence type="math" to="n" />` computed `null + 1` and emitted a one-item sequence — and need no guard: `Math.floor` of either `NaN` or a `Complex` is `NaN`, which `Number.isInteger` already rejects with the existing "Invalid length of sequence" warning. **Dead `null` handling, simplified.** `Integer.js` (`=== null` and a `?? NaN`, both now `evaluateToNumber`/`toNumberOrNaN`, which also stops a complex value reaching `Math.round`), `Number.js` (three `number === null ||` tests, and `valueIgnoringUnits` returning `null` from its catch, now `NaN`, which makes the `?? number` fallbacks provably dead and removable), `SolveEquations.js` (`res !== null`), `utils/components/function.ts` (`?? NaN` → `toNumberOrNaN`, which the complex arm needed anyway). Left in place with their reasons restated: `Math.js` and `utils/math.ts`'s `?? NaN`, which are belt and braces around `plainComplex`, and `jsxgraph.ts`'s `typeof` test, which is load-bearing against `Complex` and always was. **Prose.** Twenty-odd comments and four test docblocks said `evaluate_to_constant` answers `null`. They now describe the sentinel in the past tense, keeping the defect each one records, because that history is why the guards are there. One of them was not merely stale but backwards. `Number.js`'s `valueIgnoringUnits` docblock claimed, "measured", that `2$`, `-5$` and a bare `$` all answer `null` and "none of them `NaN`", and concluded that the callers' `number === null` test was the half that reached the function. All three answer `NaN`, and did so even under the old sentinel — `$` is one of the unit names `evaluate_to_constant` excluded from "free variables", so they fell through to its `NaN` arm. It was `Number.isNaN` that reached it all along. Re-measured both against the previous pin and against this one, and the comment now says so. `packages/utils`'s `isNumericConstant` docblock had the same fact right, so the two contradicted each other. Targeted suites run: angle, curve, curve.bezier, integer, number, math, text, functioniterates, sequence, solveequations, polyline, polygon, stickygroup, linesegment, periodicSetEquality, booleanoperatorsonmath, piecewisefunction, rectangle, regularPolygon, point. All green.
Two errors in the previous commit, both mine, both the kind these passes exist to catch. **The guard I added to `FunctionIterates`'s multi-dimensional branch was wrong, and CI caught it.** The one-dimensional branch fills `＿` when the initial value has no numeric value; I made the multi-dimensional one match. It should not: `change dimensions, numerical` already pins `(NaN,NaN)` there, and that is a correct "no value" marker for a `<mathList>` of vectors. Making the two agree is a behavior change with nothing to do with the sentinel and no business riding along with it. Reverted; `evaluateToNumber` stays, which is the part that belongs here (the complex arm), and both spellings are now documented at the site as deliberate. **I claimed "all green" on a suite that was red.** The run was `npx vitest run … 2>&1 | tail -25`, and a pipeline exits with the status of its *last* command, so `tail` reported 0 no matter what vitest did. The failure was in the scrollback the whole time. Every test run in this cycle has been re-run redirecting to a file and checking `$?` directly; 13 files / 390 tests in the touched set, plus cobweb, angle and both curve suites, plus `@doenet/utils`, all actually green. **And the previous commit's "seven live defects" is an overclaim I am withdrawing.** Seven *sites* were reading a raw `evaluate_to_constant()` result; that much is measured. Whether each produced a wrong number is a separate question, and I have now checked two end to end against the previous pin: - `<angle>`'s `numericalPoints` **reproduces**. With the old sentinel and no guard, the state variable holds a literal `[null, null]` — measured — and it is `forRenderer`, so JSXGraph coerces it and the undefined vertex is drawn at the origin. A real defect, fixed twice over now. - `<functionIterates>` **does not**. The numerical function guards its own input, so the orbit came out `(NaN,NaN)` even with the old sentinel and the bare call. The site was worth hardening; it was not producing a wrong number. The other five are unverified end to end. The honest count is therefore *seven sites hardened, one confirmed user-visible defect among them*, and the sweep-agent classification that produced the list sorts by local code shape, which is a hypothesis and not a measurement. The rewritten test pins the behavior that matters — a symbolic initial value never yields the orbit of the origin — in both branches and in both spellings.
**The twentieth pass hardened seven call sites and verified two.** This one verified the other five, the only way that means anything: restore the previous engine pin (`ca5168d^`, where `evaluate_to_constant` still answered `null`), revert the site's guard, and read what the document reports. **Four reproduce.** - `<curve through="(1,2) (a,b) (5,6)">` with `<bezierControls>` — `controlPoints` for the symbolic through point was `[[1,1],[-1,-1]]`, the bare control vector, because `null + vectorX` is `vectorX`. (The twentieth pass called this the "inverse definition"; it is the forward `arrayDefinitionByKey`.) - `<curve through="(0,0) (a,b) (2,3)" />` — the *default* path, with no `<bezierControls>` at all. `calculateControlVectorFromSpline` interpolates three consecutive through points, so one symbolic point read as `0` gave *every* control vector a finite, plausible value (`[-0.267, -0.4]` at index 1) describing a spline through the origin that nothing asked for. - `<polygon vertices="(1,2) (a,b) (3,4)" rigid rotateAround="vertex" />` — dragging it rotated the symbolic vertex about the origin and wrote `(0.156, -0.071)` back over the symbol, pulling the third vertex to `(2.84, 4.15)` with it. The only one of the class that *destroys* authored content rather than mis-reporting it. - `<cobwebPolyline>` with a vertex the student never placed — `null - null` is `0`, so the squared distance from the attractor was exactly `0`, inside any `attractThreshold`, and `correctVertices` read `[false, true]`. A grade. `<functionIterates>` **does not** reproduce, confirming the twentieth pass's own correction independently: the numerical function guards its own input, so the orbit was `(NaN,NaN)` even unguarded. Three new tests pin the four, in `curve.bezier.test.ts`, `polygon.test.ts` and `cobwebpolyline.test.ts`, each with an all-numeric control leg so it cannot pass vacuously, and each verified to fail against the previous pin with its guard reverted. The running total is **eighteen** wrong-answer-on-grading defects, ten of them this one `null`-coerces-to-`0` shape — which is the argument for having fixed it at the source rather than at ~890 call sites. **The sentinel's uniformity, checked rather than assumed.** Every path out of `evaluate_to_constant` was probed at the seam — free variables, bare and subscripted blanks, placeholder holes, indeterminate forms, matrices and their `det`/`trace` reductions, units under all three option combinations, the complex arm, `±Infinity`. There is no `null` left on any of them, and nothing depended on the deleted `UNIT_NAMES`/`treeHasBlank`/`treeHasBareBlank` for a second reason. `evaluate_to_complex` still answers `null`, correctly: no legacy counterpart, not in the published declarations, and `Complex(NaN, NaN)` is a real value in its range. **Twenty-three comments and docblocks still described the sentinel in the present tense**, which after the twentieth pass makes them false rather than merely stale — a reader would have concluded the engine answers `null` today. Two were worse: `<polygon>`'s and its four copies said "the engine returns `null` where the old one returned `NaN`", which is now backwards in both halves. All of them now say what the engine does *and* why the guard stays anyway (the `Complex` arm), which is a reason with a future rather than an archaeology of superseded workarounds. **The publish range now states both cases wherever it is stated.** `MATH_EXPRESSIONS_ENGINE_NOTES.md` and `packages/doenetml/vite.config.ts` both said "the published `^3.x` range" flatly; `^3.x` is right for a released `3.0.0` and wrong for a prerelease, which is what upstream's `package.json` currently says. Both now name the two cases and point at the plan. Re-verified alongside: `packages/doenetml-worker-rust:build:rust` really is on `packages/doenetml:build`'s critical path (through `doenetml-worker`), so `wasm32-unknown-unknown` does not leave with the submodule; and all seven `"math-expressions": "file:../math"` manifests are at the line numbers the plan gives, six of which stay. **Counts the review had asserted and not re-derived.** `MATH_EXPRESSIONS_RUST_ MIGRATION_PLAN.md` said "eleven" grading defects while citing a document that said "fourteen"; the notes' "Two came from CI ... the other three" read as an enumeration of all of them rather than of the first five. Both corrected. Upstream's `PR84_REVIEW_KNOWN_ISSUES.md` suite state was seven passes stale, and is fixed in the submodule bump. The import-count table reproduces exactly with its own documented command (139/145/150); the PR body's leftover "147" — a number the notes had already retracted — is fixed there. **Nine more declaration-vs-implementation lies**, and the vendored declarations re-synced. Detail is in the submodule commit; the DoenetML-visible part is that `packages/math/src/vendored/math-expressions.d.ts` tracks them, that its header no longer claims a byte-identity that a Prettier-gated copy of a non-Prettier-gated file cannot have, and that the header now lists the three hunks its own `diff` check should show so the check can be run and believed. Every test run here was redirected to a file with the vitest process's own exit status checked. 18 worker-javascript files / 571 tests, `@doenet/utils` 790, `packages/math` 31, and `npm run typecheck` (22 clean, 5 not gated, 59 errors — unchanged) all green.
**The twenty-first pass's four new guard tests hold, but not the way the pass
described them.** Reverting a guard no longer falsifies its test, because the
`NaN` sentinel makes `evaluateToNumber` and the bare call agree — so they were
re-falsified by simulating the *old* sentinel at the boundary
(`evaluateToNumber` returning `0` for a non-number). All four fail under it,
every numeric control leg stays green, and all four pass at the head. The
guards reject nothing legitimate: the only value they turn away is one that has
no numeric reading.
**The string-taking wasm sweep that pass reported was re-run rather than
re-read, and it had missed the two biggest entry points.** `me.fromText` and
`me.fromLatex` handed a non-string straight to a wasm entry typed `&str`, so
`fromText(5)`, `fromText(anExpression)` and `fromText({})` were all
`RuntimeError: memory access out of bounds`. Fixed in the submodule with a
`TypeError` naming the argument type; nothing that used to succeed changed.
**The two decisions.** The **48 declared-but-absent `Expression` members** are
**narrowed away**, not documented: TypeScript accepted `expr.sin()` and it
threw, and a declaration whose job is to describe a drop-in earns nothing by
promising members that are not there. `packages/math/src/vendored/
math-expressions.d.ts` loses 96 declarations, plus `Context`'s own `ZmodN` and
`parser_parameters` — Context-only properties, so outside the `Expression`
audit that measured the 48 — in the same shape as upstream's copy, so the two
still differ only by the trailing v3 block and one Prettier line wrap. Every
member either interface still declares (66 / 87) is present at runtime.
`npm run typecheck` is unchanged by the removal — 22 clean, the same 5 not
gated, the same 59 errors — which is the measurement that nothing called them.
The gap is unchanged and stays the open ask in
`MATH_EXPRESSIONS_UPSTREAM_REQUESTS.md`.
The **six unresolved declaration divergences** got one verdict each: five
narrowed to what the code does, `solve_linear`'s `ABSENT_EXPRESSION` accepted
with its reason, and `Expression.match`'s dropped `allow_extended_match` fixed
upstream. None reaches DoenetML, which uses `me.utils.match` and never
`Expression.match`.
**Four documentation defects of exactly the shape the twenty-first pass was
fixing.** `MATH_EXPRESSIONS_ENGINE_NOTES.md`'s "conventions" bullet and
`MATH_EXPRESSIONS_UPSTREAM_REQUESTS.md`'s ledger entry both still described the
`null` sentinel in the **present tense**, two passes after it was removed —
which is worse than stale, because a reader would have concluded the engine
answers `null` today. Follow-up 3 was still titled "sweep the remaining raw
null-coercions", a task the twentieth pass closed at the source; it is retitled,
its population re-derived, and what the residue now risks (a `Complex` reaching
a real-valued consumer, not a `0`) stated instead. Its `<angle>` entry said the
display defect was "left for the follow-up" when the twentieth pass had fixed
it. And follow-up item 7 had been swallowed into item 6's last line by a missing
newline, so the list read 1-6, 8, 9.
**Counts re-derived rather than carried.** The import table (139/145/150) and
the applied-function lists (65/65/67) reproduce exactly. `evaluate_to_constant`
is now **259 occurrences across 67 files, 230 of them live**, down from the
eighteenth pass's 254 live. Step 6's vendored-declaration line count said 1,223
against a real 1,333 before this pass touched the file, and the 21-line
difference from upstream is 23 once `evaluate_to_constant`'s wrap is counted;
both corrected, both now 1,283 / 422. The vendored header's "~147 call sites"
and the published file's identical claim are replaced by a pointer to the one
table that owns that number.
**One new follow-up, written up rather than fixed because the engine switch did
not cause it.** Dragging `<polygon vertices="(1,2) (a,b) (3,4)" rigid
rotateAround="vertex" />` writes `NaN` over all three vertices, losing two
authored numeric ones with the symbol. That is what the twentieth pass's guard
*improved* — before it, the symbol was overwritten with a plausible number —
and it is loud rather than silent, so it is pinned as current behaviour. But the
right answer is to decline the update, which the same function already does for
a non-finite *moved* coordinate. Legacy answered `NaN` for `(a,b)` too, so the
outcome predates this branch; follow-up 11 in the notes has the fix.
No changeset change: nothing here alters DoenetML-visible behaviour. Every test
run was redirected to a file with the vitest process's own exit status checked —
`packages/math` 31, `@doenet/utils` 790, 8 worker-javascript files / 453 tests,
`npm run typecheck` — all green.
…rose sweep
**The twenty-second pass's discovery, applied to the whole population.** It found
that the `NaN` sentinel makes reverting a guard a no-op for its test, and
re-falsified only its own four. **All 23 guard tests the seventh through
twenty-second passes added for the `null`-coercion class were audited this pass,
and none is vacuous.** 15 fail under one patch — `toNumberOrNaN` returning `0`
for a value with no numeric reading, which is what the unguarded code did under
the old sentinel — with `@doenet/utils` rebuilt so the worker tests see it.
5 more survive that patch only because their guard is not routed through the
helper, and fail under the same simulation applied at their own boundary:
`<math>`'s `plainComplex(...) ?? NaN`, the three `nearestPoint`
`Number.isFinite` guards in `<polygon>`/`<polyline>`/`<regionBetweenCurveXAxis>`,
and `domain?.[0]`. The remaining 2 are not of the class in the first place —
`<line>`'s `UNSPECIFIED_COMPONENT` marker test, and `periodicSetEquality`'s
deliberate positive control, which is falsified by the opposite mistake
(refusing a partly symbolic list whole). The method is now a convention bullet in
`MATH_EXPRESSIONS_ENGINE_NOTES.md`, with the patch spelled out, so the next pass
does not check one of these by reverting it.
**Two of them are re-anchored on the arm that is still live.** A test that only
observes the engine's `NaN` is a behaviour lock, not a guard lock. Where a
`Complex` reaches the same guard, a plain revert does falsify — so `<math>`'s
`.number` now also asserts that `sqrt(-4)` arrives as a plain `{re, im}`
(`plainComplex`'s own job: without it the object crosses the structured clone
prototype-stripped), and `<vector>`'s `numericalEndpoints` asserts that a
`sqrt(-4)` head reads `NaN`. Both legs verified to fail with their guard
reverted and to pass with it. Two more were tried and **measured not to work**,
and say so in place rather than being left to look like guard tests:
`<circle>`'s `numericalRadius` is not reached by a `sqrt(-4)` radius, and
`<polygon>`'s centroid answers `NaN` for a `Complex` either way, because `+=` on
one produces a string that divides back to `NaN`.
**The twenty-second pass verified.** Every claim re-derived rather than read.
`me.fromText`/`me.fromLatex` reject a number, an `Expression`, an AST array, a
plain object, `null` and `undefined` with a `TypeError`, and still parse a
`String` object; no DoenetML call site passes a non-string, and every one that
could is inside a `try`/`catch` — the throw's `name` is `TypeError`, not
`ParseError`, so `<mathInput showPreview>`, which renders only the latter, shows
exactly what it showed before. All 48 removed `Expression` members are absent at
runtime, as are `Context`'s `ZmodN` and `parser_parameters`; nothing in the repo
consumes any of them (swept including Cypress and script trees, not just what
`tsc` sees). The 66/87 that remain are all present at runtime, counted with the
TypeScript compiler rather than by eye. `Expression.match` and `me.utils.match`
now answer identically for `allow_extended_match`. The five narrowed
declarations and `solve_linear`'s accepted `ABSENT_EXPRESSION` each match what
the built package does. The two `.d.ts` files, comments stripped, differ by
exactly the two hunks their header says. The import table (139/145/150), the
applied-function lists (65/65/67), `evaluate_to_constant`'s
259/67 files/230 live, and the compat suite's 6,383/6,372/11 all reproduce.
**Eleven live code comments still described the `null` sentinel in the present
tense** — the same defect the twenty-second pass fixed in two documents, and the
twenty-first in the tests, neither of which reached the components.
`periodicSetEquality.js` (twice, one of them naming the wrong helper),
`Line.js`, `LineSegment.js`, `BooleanOperatorsOfMath.js`, `Function.js` (twice),
`Polyline.js`, `PeriodicSet.js`, `MathBaseOperator.js`, `eval-math.ts` and
`mathFormatParse.ts` all asserted that the engine answers `null` today, and
`math.test.ts`, `polygon.test.ts` and `answer.test.ts` did the same. All now say
what the engine did, what it does, and what the guard is still for. Comment-only:
`git diff` over the component and util sources has no non-comment line in it.
**Follow-up 11 confirmed rather than fixed.** `<polygon vertices="(1,2) (a,b)
(3,4)" rigid rotateAround="vertex" />` answering `NaN` for all three vertices
predates the branch: the only change this diff makes to `calculateNumericalCentroid`
and to the rotate/dilate inverse's reference-vertex read is
`evaluate_to_constant()` → `evaluateToNumber(...)`, the identity for a number and
for `NaN`, so the branch and its merge base compute it the same way. Recorded in
the follow-up.
Nothing changed in `math-expressions` this pass; Doenet#84 is unchanged at `46f4c49`,
and its compat suite, `verify:package` and `typecheck` were re-run here and are
green. No changeset change: every DoenetML-source edit is a comment.
Runs redirected to a file with the process's own exit status checked: the 23
guard tests, `npm run typecheck` (22 clean, the same 5 not gated, the same 59
errors), Prettier over every changed file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
The PR body said installing a Rust toolchain with `wasm32-unknown-unknown` and a matching `wasm-bindgen-cli` was "the one new hard prerequisite for building the repo at all". Verified against the build graph, and it is false in its main clause. `packages/doenetml:build` depends on `../doenetml-worker:build`, which depends on `../doenetml-worker-rust:build`, whose `build:rust` is `npx wasm-pack build lib-js-wasm-binding --target web --dev …`; `npm run build` is `packages/doenetml:build`; and every one of those edges is identical at `upstream/main`. DoenetML has compiled its own Rust core on the default build path all along. What this diff actually adds is narrower, and is now what the documents say: - a **`wasm-bindgen-cli` on `PATH`** whose version matches the submodule's `wasm-bindgen = "=0.2.126"`. Nothing before this branch invoked that CLI — `git grep` for it at the merge base finds only Cargo manifests and comments, because `wasm-pack` carries its own copy. The engine's `build-wasm.sh` calls `cargo build` and then bare `wasm-bindgen`, so the binary has to be there and has to match. - an **explicit `rustup target add wasm32-unknown-unknown`**, for the same reason: `wasm-pack` adds the target for itself, a direct `cargo build --target wasm32-unknown-unknown` does not. - the **submodule checkout**. All three leave at Step 6; the toolchain does not. The migration plan's item 10 already said that correctly — it was the PR body, the changeset, the notes' "Building" section and `packages/math/README.md` that carried the old claim, and all four are corrected. The changeset's line is the user-facing one, so it now says the toolchain was already required and names only what is added. Also answered there, because the maintainer asked what becomes of it: the `../math:build` dependency this branch adds to `doenetml-worker-rust`'s wireit `build` **stays** after the swap. Step 6's own step 2 leaves the six non-publishing manifests — `doenetml-worker-rust`'s among them — on `"file:../math"`, so `@doenet/math` survives as the resolution target of the bare specifier and still has to be built first. What changes under the edge is that `../math:build` stops compiling Rust and starts unpacking a prebuilt `.wasm` out of `node_modules`. Documentation only; no code changed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
**The lead the twenty-third pass named, worked through.** Roughly twenty guard
sites had no complex-arm coverage; this pass probed each with a `sqrt(-4)` where
the symbolic value had been, first as they stand and then with the boundary
helpers made pass-throughs (`toNumberOrNaN` and `plainComplex` the identity,
`isNumericConstant`'s complex-rejecting half removed) — the same method as the
old-sentinel patch, aimed at the other arm. Instrumenting the helpers to log a
`Complex` argument shows one **reaches** the guard at **19 of the 20**;
`<circle>`'s `numericalRadius` is the exception, as the twenty-third pass
reported. But at only **five** does removing the guard change what the document
reports.
**The reason is the measurement worth having**, and it is now the conventions
bullet: `-`, `*`, `/`, `Number()` and `Math.abs()` all take a math.js `Complex`
to `NaN`, so a site that merely subtracts already degrades loudly on its own.
Three operations do not. Every comparison answers `false` in *both* directions
(`c >= -3` and `c < -3`), so a range test silently excludes and a comparator
sort orders arbitrarily; `+` returns a **string** (`2i` + `1` is `"2i1"`, the
structured-clone form `"[object Object]1"`); and `me.fromAst` throws outright.
The sites where a guard is observably load-bearing are exactly the ones that
store the value or rebuild with it.
**Five tests anchored on the `Complex` arm**, each verified to fail with its own
guard plainly reverted, for the right reason, and to pass with it restored:
`<angle>`'s `numericalPoints`, `<cell>`'s `.number`, `<ray>`'s
`numericalEndpoint`, `<curve>`'s bezier `controlPoints` and
`periodicSetEquality`'s per-tuple offset guard. The `<curve>` one is the
`+`-returns-a-string case — unguarded, a control point's x coordinate comes out
as the math variable named `2i1`. The `periodicSetEquality` one is the
`fromAst`-throws case, and a throw there is a dead document rather than a wrong
grade. That makes seven such legs on the branch with the twenty-third pass's
two, and the guard-test population 26.
**The other fourteen are reachable and not falsifiable**, and are recorded in
the notes rather than in fourteen comments: `<cobwebPolyline>`, `<isBetween>`,
`<curve>`'s default spline control vectors, `<functionIterates>`, `<line>`'s
`parallelTo`, `<lineSegment>`'s `slope`, `<rectangle>`'s width and height,
`<stickyGroup>`'s drag, `<polygon>`'s rigid rotate, `<polygon>`'s centroid,
`find_effective_domain`'s endpoint reads, and the three `nearestPoint` guards.
In each the `Complex` meets a `-`, a `/` or a comparison before it can be
observed. They keep their guards; their tests are behaviour locks and are
falsified by the sentinel patch, not by a revert.
**Four comments claimed a `Complex` justification that measurement contradicts,
and all four are corrected in place.** `<lineSegment>`'s slope said a `Complex`
"subtracts without complaint" — it subtracts to `NaN`. `<periodicSet>`'s said
`%` does not reject one — `%` yields `NaN`. `periodicSetEquality`'s said the
helper stays because `mod` would carry one — the subtraction on the previous
line gets there first, and reverting that one reading changes no result.
`<polygon>`'s `nearestPoint` said a `Complex` "orders against numbers in ways
JavaScript will happily invent"; it was traced arriving at that guard's own
`x1` as `{re: 0, im: 2}`, and reverting the guard still leaves the point alone.
`isNumericConstant`'s own doc comment had the same shape — `{re, im} >= -3`
implying a `true` where the answer is `false` — and now states what was
measured.
**The twenty-third pass verified.** The vacuous-test audit reproduces: seven
guard tests spot-checked across `<lineSegment>`, `<rectangle>`, `<tabular>`,
`<isBetween>`, `<cobwebPolyline>`, `<stickyGroup>` and `<functionIterates>` all
fail under the one patch, and its correction to the twenty-second's recipe is
right — `typeof value === "number" ? value : 0` leaves every one of them green,
because the sentinel is a number. Its comment sweep is comment-only, confirmed
by filtering the diff, but the count is **twelve**, not eleven: the sentence's
own list enumerates twelve hunks in ten files, each a genuine present-tense
`null` claim. Its toolchain correction holds in full — `build:rust` is
byte-identical at the merge base, `doenetml:build → doenetml-worker:build →
doenetml-worker-rust:build` are the same edges there, `git grep wasm-bindgen` at
the base finds only manifests and comments, and Step 6's step 2 leaves all six
non-publishing manifests on `file:../math` at the line numbers it names, so the
`../math:build` edge does stay. The PR body, changeset, notes and
`packages/math/README.md` agree.
**Coherence sweep**: this description said "nine unguarded `get_component`
reads" where the notes have said eight since the ninth was withdrawn, and gave
the branch's added lines as 8,583 against a measured 11,898 (the notes said
~10,900); both corrected, along with the review-history count. The
applied-function figures reconcile — the 69 probes are exactly the union of the
67- and 65-entry lists, checked both directions.
`math-expressions` is untouched; Doenet#84 is still at `46f4c49` and still ready to
publish from — its compat suite (6,372 passed / 11 skipped of 6,383),
`verify:package` and `typecheck` were re-run here and are green. No changeset
change: every DoenetML source edit in this diff is a comment.
Runs redirected to a file with the process's own exit status checked: the five
new legs (pass, then fail one at a time under their own reverts, then pass),
the eight affected test files, `npm run typecheck` (22 clean, the same 5 not
gated, the same 59 errors), Prettier over every changed file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…ot reproduce
**The twenty-fourth pass verified, end to end.** Its five new legs were each
re-falsified by plainly reverting that leg's own guard, with the run redirected
to a file and the vitest process's own exit status read: `<angle>`'s
`numericalPoints` reports `{re: 0, im: 2}` into a `forRenderer` array;
`<cell>`'s `.number` holds a `Complex` in a `number`-typed public state
variable; `<ray>`'s `numericalEndpoint` the same; `<curve>`'s bezier
`controlPoints` comes out `"2i1"`/`"2i-1"` — the `+`-returns-a-string case, the
coordinate becoming a math variable of that name; and `periodicSetEquality`
throws out of `fromAst` at `periodicSetEquality.js:340`, a dead document rather
than a wrong grade. All five restored and green.
**Its central measurement reproduces exactly**, on both a math.js `Complex` of
`2i` and the prototype-stripped `{re, im}`: `-`, `*`, `/`, `%`, `Number()` and
`Math.abs()` all give `NaN`; `+` gives the string `"2i1"` / `"[object Object]1"`;
`>=`, `>`, `<=`, `<` are all `false` in both directions; `me.fromAst` throws.
That is what the fourteen "reachable but not falsifiable" verdicts rest on, and
it holds. Its four corrected comments hold too — including the claim that
reverting `periodicSetEquality`'s `number_list` reading changes no result, which
was checked by reverting it (all three tests stay green).
**A merge-readiness pass re-derived every count in the four documents and both
PR bodies. Four do not reproduce**, and all four are corrected here:
- The conventions bullet said the `null` inversion is where "roughly fourteen"
grading defects came from. The ledger 300 lines below it enumerates the ninth
through the eighteenth — **ten** — and both PR bodies say ten. Written at the
twenty-second pass and wrong when written.
- `packages/math/README.md` said the vendored declarations are "504 lines here
against 525 upstream, and the whole 21-line delta is upstream's v3 additions
block", and called the shared contract "byte-identical". Measured: **422
against 445**, a **23**-line delta of which 21 are the v3 block and 2 are
`evaluate_to_constant`'s signature wrapped across three lines upstream and one
here — so byte-identity is not achievable in both directions, which is
exactly what Step 6 of the plan already says. The counts moved at the
twenty-second pass, when both files were narrowed; the plan was updated and
the README was not.
- The `evaluate_to_constant()` population read 259 occurrences / 29 comments.
Measured 260 / 30 — the twenty-third pass's comment sweep netted one. The
live count, which is the figure the follow-up uses, is unchanged at 230.
- The branch's added lines read 11,898, which was measured at the twenty-fourth
pass *before* it committed. Real figure 12,082, GitHub agreeing. Now stated
as ~12,000 with the reason: this denominator has gone stale at three separate
passes and the numerator it qualifies has not moved.
Everything else re-derived and reproducing: the import table (139/145/150), the
applied-function lists (67 default, 65 latex, 65 in the generated JSON, union
69 = the probe table), 22 raw `get_component` reads in the worker outside
comments and tests with 36/4 elsewhere, `npm run typecheck` (22 clean, the same
5 not gated, 35+2+2+2+18 = 59 errors), 66 `Expression` / 87 `Context` members,
1,283 vendored declaration lines, 14 submodule checkouts and 12 uses of
`setup-math-wasm`, all seven `file:../math` manifest line numbers Step 6 names,
and Step 6 being twelve steps.
**The publish path confirmed one more time**, at the pinned `46f4c49`:
`cargo fmt --all -- --check` and `clippy --workspace --all-targets -D warnings`
clean, `cargo test --workspace` **876 passed / 0 failed**, `npm run typecheck`
clean, the compat suite **6,372 passed / 11 skipped of 6,383**, and
`verify:package` green — it packs `math-expressions-3.0.0-alpha1.tgz`, installs
it into a throwaway project outside the workspace and runs both consumer
scripts, with `vendor/wasm-web/math_expressions_wasm_bg.wasm` (1.8 MB,
`--target web`) in the tarball and reached through the `./wasm-web/*` export.
The version is a prerelease, so the `^3.0.0-alpha1`-not-`^3.0.0` case is the
live one; both cases are stated in Step 6, the notes and both PR bodies.
No source, test or changeset change: every edit here is documentation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
…mmary The same sweep, one paragraph further on. The "read them together" summary of the wrong-answer-on-grading defects still said "read the **fourteen** together" and, twice, that `nthroot` and the `f((a, b))` serialization defect are "not among the **eleven**" — while the section head six paragraphs above says **eighteen** and enumerates eighteen. "The eleven" was stale by the nineteenth pass, which wrote "the fourteen" beside it without touching it, and neither moved when the twentieth and twenty-first passes took the total to eighteen; a reader looking for "the eleven" finds no antecedent anywhere in the file. All four references are now the eighteen the ledger actually holds, including the enumeration "the ninth, tenth, eleventh and twelfth-through-eighteenth", which is what makes the guard point about all ten of the `null` shape rather than the six of them that had been found when the sentence was written. Documentation only; no count in the ledger itself changed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F9QoEYrYzcLeJKxWoeheK8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Switch to the Rust/WASM math-expressions engine
Replaces the legacy
math-expressions@2.xJavaScript library with the Rust core compiled to WASM, via themath-expressions-js-compatdrop-in from Doenet/math-expressions#84, pinned as thevendor/math-expressionssubmodule. Review #84 first — this PR is its downstream half. The switch is permanent: Rust is the only supported configuration, and to A/B against the old engine you check out a commit before this branch. The mechanism is an alias, not a codemod —@doenet/math(packages/math) backs the bare specifier, every consumingpackage.jsonsays"math-expressions": "file:../math", and no file that importsmath-expressionschanges.Building requires a Rust toolchain, as it already did:
npm run buildispackages/doenetml:build, which reachesdoenetml-worker-rust'swasm-pack buildon the default path at the merge base too. What this PR adds on top is awasm-bindgen-clionPATHmatching the submodule's pinnedwasm-bindgen, an explicitrustup target add wasm32-unknown-unknown, andgit submodule update --init --recursive— all three of which leave at Step 6, while the toolchain requirement stays (see the notes' "Building" section).Status: CI is green at the head
484100b5— 23 passing, 1 skipped, 0 failed. #84 is green at46f4c49, which the submodule pin names, and it is ready to publish from: itspackage publishabilityjob packs the tarball, installs it into a throwaway project outside the workspace and drives both consumer paths.User-visible behavior changes are in
.changeset/rust-math-expressions-engine.md— read it rather than a summary here..changeset/piecewise-domain-without-intervals.mdcovers an unrelated pre-existing crash fixed in passing.Where to start reviewing:
packages/math/(the seam —engine-rust.tsis a straight re-export,wasm-loader.tscarries the one ordering constraint), thenMATH_EXPRESSIONS_ENGINE_NOTES.md, then the changesets. The worker-javascript component and test churn is mostly the NaN-boundary convention and unpadded-delimiter expectations the notes describe.In this diff but not the engine switch, kept and called out: a
<video>source-change fix; the memory-bench instruments and two Cypress specs from the memory-reduction workstream; a bash-3.2 rewrite of.husky/format-staged.sh; the piecewise-domain crash fix. The case for splitting each into its own PR is inMATH_EXPRESSIONS_ENGINE_NOTES.md, "What is still riding along, and should not be".The residual risk, plainly: no differential grading harness exists, which is the primary risk of an engine swap — the suites are green, and the review still found eighteen wrong-answer-on-grading defects that no existing test named. All eighteen are fixed. The confidence argument is structural rather than exhaustive: most of them shared one cause —
evaluate_to_constantansweringnull, which coerces to0in JavaScript, where the legacy library answeredNaN— and that sentinel was fixed at its source rather than one call site at a time, which is what makes the unswept reads safe by default. The eighteen, the sweeps and the guard-audit method are inMATH_EXPRESSIONS_ENGINE_NOTES.md, "Known risks and open product decisions".Four documents carry the detail:
MATH_EXPRESSIONS_ENGINE_NOTES.md(architecture, the conventions the switch established, known risks, follow-up PRs, what is riding along),MATH_EXPRESSIONS_RUST_MIGRATION_PLAN.md(the two-stage plan and the Step 6 checklist),MATH_EXPRESSIONS_UPSTREAM_REQUESTS.md(engine-level divergences still open upstream), and, in #84,active-plans/PR84_REVIEW_KNOWN_ISSUES.md. The pass-by-pass review history is in the git log (Review cycle N:commits) and in this description's edit history (GitHub web UI only).🤖 This description was written by an agent during review; the findings ledger lives in the documents above.