Skip to content

feat: add second order to SScalar - #88

Merged
oberbichler merged 1 commit into
mainfrom
feat/sscalar-second-order
Jul 27, 2026
Merged

oberbichler merged 1 commit into
mainfrom
feat/sscalar-second-order

Conversation

@oberbichler

Copy link
Copy Markdown
Owner

Completes the three-step plan for SScalar: characterize (#84), change the storage (#85), add second order (this).

SScalar carried a value and a gradient keyed by name. This adds the Hessian, so named derivatives reach the same order the indexed DDScalar already offered. The Python types are SScalar (order 1) and SSScalar (order 2), following the one-letter-per-order naming of DScalar/DDScalar; dd(a, b) is absent from the first-order class rather than raising.

x = hj.SSScalar.variable("x", 0.5)
y = hj.SSScalar.variable("y", 0.4)

f = (x * y).sqrt()
f.d("x")          # 0.4472135954999579
f.dd("x", "y")    # 0.5590169943749475
f.dd("x", "z")    # 0.0 -- an unknown name, as with d

Why dense over the names

The Hessian is dense over the names a value carries, laid out as the upper triangle in the same packing DDScalar uses. That was measured rather than assumed: any nonlinear function contributes the full outer product of its own gradient, so a simulation of the fill rate reaches 100% after two operations. What stays sparse is the set of names — which is the point of the type, since it does not have to be known in advance. A pair-keyed map would allocate per entry to store what is dense anyway (25 allocations against 0 for k=6).

Cost

Order 1 pays nothing. The Hessian storage sits behind [[no_unique_address]] and collapses to an empty member:

sizeof runtime, k=6 allocations/iter
SScalar<1> 32 B 1.93 µs 60
SScalar<2> 56 B 2.52 µs 73

Second order costs 1.31× the first — for six names and 21 Hessian entries.

Tests

34 new C++ cases (each checking the value, three gradient entries and all six Hessian entries, plus symmetry) and 8 new Python cases. The expected values were generated symbolically with sympy, so they share no formulas with the header.

Writing them first paid off. Of 413 assertions, 174 failed at the start; the last 8 to fall pointed at four operators — -u, s - u, u * s, u / s — that copied the storage and rewrote value and gradient in place without ever touching the Hessian. Routing them through the existing apply() helper fixes it and is shorter than what it replaced.

Verification

  • C++ 118 cases / 1886 assertions, Debug and Release with -Werror
  • clang ASan + UBSan clean
  • Python 259 tests
  • clang-format, ruff clean; just tidy 35 findings against 41 on main (all pre-existing, none in the new code)

SScalar carried a value and a gradient keyed by name. This adds the
Hessian, so named derivatives reach the same order the indexed DDScalar
already offered. The Python types are SScalar (order 1) and SSScalar
(order 2), following the one-letter-per-order naming of DScalar and
DDScalar.

The Hessian is dense over the names a value carries, laid out as the
upper triangle in the packing DDScalar uses. That is deliberate: any
nonlinear function contributes the full outer product of its own
gradient, so the triangle fills up after a couple of operations. What
stays sparse is the set of names, which is what makes the type useful --
it does not have to be known in advance. A pair-keyed map would allocate
per entry to store what is dense anyway.

Order 1 pays nothing for the addition. The Hessian storage sits behind
[[no_unique_address]] and collapses to an empty member, so sizeof stays
at 32 bytes against 56 for order 2. Second order costs 1.31x the runtime
of first order over six names.

The expected values in the new tests were generated symbolically, so
they share no formulas with the header. Writing them first paid off: it
caught four operators (-u, s - u, u * s, u / s) that copied the storage
and rewrote the gradient in place without touching the Hessian. Routing
them through the existing apply() helper fixes that and is shorter than
what it replaces.
@oberbichler
oberbichler merged commit 8736f1d into main Jul 27, 2026
17 checks passed
@oberbichler
oberbichler deleted the feat/sscalar-second-order branch July 27, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant