Skip to content

Single-pass expression analysis groundwork - answer type questions from ExpressionResults - #5857

Open
ondrejmirtes wants to merge 424 commits into
2.2.xfrom
resolve-type-rewrite-2
Open

Single-pass expression analysis groundwork - answer type questions from ExpressionResults#5857
ondrejmirtes wants to merge 424 commits into
2.2.xfrom
resolve-type-rewrite-2

Conversation

@ondrejmirtes

@ondrejmirtes ondrejmirtes commented Jun 12, 2026

Copy link
Copy Markdown
Member

Groundwork for the "new world" where an expression is traversed once: after processExpr, its ExpressionResult knows the before/after scopes, the type (typeCallback) and the narrowing (specifyTypesCallback), composed from child results instead of re-walking subtrees. Handlers then stop implementing TypeResolvingExprHandler; the old entry points (MutatingScope::resolveType, the TypeSpecifier dispatcher) are guarded behind NewWorld::disableOldWorld() and get mass-deleted in PHPStan 3.0.

What's on the branch, bottom up:

  • Guards + ExpressionResultFactory: old-world type resolution entry points throw when NewWorld::disableOldWorld() is flipped (the migration meter); all ExpressionResult construction goes through a generated factory.
  • ExpressionResult carries beforeScope, expr, typeCallback, specifyTypesCallback and is stored per node in ExpressionResultStorage (layered O(1) duplicate()), replacing the stored before-Scope.
  • ExprHandler / TypeResolvingExprHandler split: resolveType/specifyTypes move to the sub-interface so handlers can shed them one by one.
  • ExpressionResultStorageStack: old-world consumers (TypeSpecifier dispatcher, extensions, rules below PHP 8.1, unconverted handlers' resolveType) keep working for converted handlers' nodes. Every scope shares the stack created by its internal scope factory; NodeScopeResolver pushes the storage of the analysis in progress through MutatingScope::pushExpressionResultStorage() (always popped in finally, throwing on imbalance), and MutatingScope answers from the stored result - or processes a synthetic node on demand. Scopes never reference a storage directly, so nothing pins the result graph with the cycle collector disabled in bin/phpstan. Also adds MutatingScope::applySpecifiedTypes - filterBySpecifiedTypes without Scope::getType().
  • First two migrations: ScalarHandler and ArrayHandler no longer implement TypeResolvingExprHandler. The array migration is a precision win the old world cannot reach: each item type is captured at its own evaluation point, so [$b = 1, $b + 1, $c = $b, $c + 2, $c++, $c] infers array{1, 2, 1, 3, 1, 2}.

Verified: full test suite green, make phpstan clean, and analysis memory back at baseline (no leak from the result graph despite gc_disable()).

Closes phpstan/phpstan#13944
Closes phpstan/phpstan#12207
Closes phpstan/phpstan#7155
Closes phpstan/phpstan#14396
Closes phpstan/phpstan#11953
Closes phpstan/phpstan#13802
Closes phpstan/phpstan#13789
Closes phpstan/phpstan#12780
Closes phpstan/phpstan#14914
Closes phpstan/phpstan#14908

🤖 Generated with Claude Code

return $this->withFlavor(false);
}

private function withFlavor(bool $fiber): self

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this read withFiber?

@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch 2 times, most recently from eb31077 to 59cbf22 Compare June 19, 2026 11:44
@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch from 59cbf22 to 125cf22 Compare June 20, 2026 11:56
@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch 4 times, most recently from f98892f to 4455baa Compare July 6, 2026 22:20
@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch from 61fe06e to e38aadd Compare July 16, 2026 14:56
ondrejmirtes referenced this pull request Jul 23, 2026
Every property fetch / method call resolves its type by walking down to
the chain root to detect a nullsafe operator (NullsafeShortCircuitingHelper),
costing O(N²) walk steps per chain of depth N — with or without an actual
nullsafe operator in the chain. Deep loop-wrapped plain chains make that
walk dominate: 3.71s -> 3.14s wall (-15%), -18% user CPU from the
recursion-to-loop rewrite. The real-world counterpart is Symfony
TreeBuilder fluent chains (300+ calls in one statement) in Sylius bundle
Configuration classes, which dropped up to 23% per file.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016szvNF5RXhACdfMQNc6DVL
@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch 12 times, most recently from fb22d34 to 84b1614 Compare July 28, 2026 17:31
ondrejmirtes and others added 29 commits July 30, 2026 23:54
…ng it

applyWrite() read the value to assign back out of the expression-result
storage, so every caller had to pre-store a result for the assigned
expression first: AssignOpHandler and PreInc/PreDecHandler through the
storeProvisionalExpressionResult() crutch (a store invisible to outside
askers, existing only for this one synchronous read), processVirtualAssign()
through a plain store of the passed or fabricated result.

The caller now hands the assigned expression's result to applyWrite()
directly; the reads (value type in both flavours, truthy/falsey narrowing,
the sentinel comparisons, and the conditional-holder refinement in
currentTypeForConditionalHolder - whose storage miss would re-enter the
still-in-flight assignment and recurse) consume the threaded result, falling
back to stored results or on-demand pricing only where no result exists
(virtual assigns of non-type-carrying expressions, nested assign chains).
The provisional-store machinery is deleted; AssignOpHandler builds its
callbacks after the value evaluation, so the by-ref capture goes away too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
The isset-semantics read for `$lvalue ??= ...` was a full extra walk of the
target chain before the write walk - the chain's sub-expressions were
processed twice.

Each target branch now produces the read after its own children are walked,
via processExprNodeConsumingStored, so the read is composed from the stored
child results instead of re-walking them. Ordering keeps the storage
observable behavior: for an ArrayDimFetch target the write-flavoured
per-dimension results are deferred until after the composition (parked rule
asks resume with the read-flavoured results, as they did off the pre-read's
stores, and the write-flavoured results replace them in storage as before);
for property targets the existing parked-asker composition consumes and
re-anchors the read's store exactly as it consumed the pre-read's. The
fallback branch composes too - synthetic ??= nodes (e.g.
InvalidBinaryOperationRule's TypeExpr-operand clones) reach it when priced
on demand.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
… results

Pricing a read for an already-walked node meant pushing it through the walk
machinery with a NoopNodeCallback and reading the product back out of the
expression-result storage. The four chain-link handlers now expose the pure
tail of their processExpr as composeResult() - building the read's
ExpressionResult from the already-walked child results, with processExpr
routing through the same method so the two paths cannot drift.

prepareTarget() composes the target reads directly from the child results it
holds: the reads are built without Noop walks, stored explicitly where parked
rule asks need them, and the chain results for the coalesce composition are
assembled from the same objects instead of re-read from storage. The
read-modify-write modes now produce the target read for every shape, so
AssignOpHandler's typeCallback resolves both operands from the threaded
results (the target read and the value expr's result) and the Div/Mod throw
check uses the value result directly - no storage round-trips. The fabricated
per-dimension results likewise capture their neighbouring link results
instead of lazily re-reading them by node.

The remaining NoopNodeCallback uses are a dynamic-name variable target (its
name expression has no earlier pricing), the ExistingArrayDimFetch clone
walks, the ternary convergence-pass fallback, and synthetic method-call
simulations; the remaining storage reads are guaranteed-present stores of the
same walk or fallbacks for callers that cannot pass a result.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
…e paths

More storage round-trips replaced by results the code already holds:

- createSpecifyTypesCallback() takes the assigned expression's result as
  non-null (its lazy closure read it back out of the scope's result bridge
  before) and captures the walked call's argument results at creation, so the
  array_key_first/array_search/array_rand/count-minus-one narrowing reads the
  captured results instead of maybe-stored types per ask.
- The AssignRef intertwined-variable types are plain scope-state reads, spelled
  as such.
- applyWrite() prices the offsetSet receiver from a threaded
  PreparedAssignTarget result (the second-outermost chain link) and the List_
  item dimensions from the walked key results - a literal index constructs its
  ConstantIntegerType directly instead of pricing a synthetic node.
- produceArrayDimFetchAssignValueToWrite() reads tracked chain links via
  Scope::getStateType() - the hasExpressionType() gate's actual contract - and
  the additional-expression dimensions from the walk-captured offset types,
  which also fixes their flavour on the native pass (both flavour invocations
  read the same plain-scope type before).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
The upstream SwitchConditionRule (6e24090) listens on a node emitted from
the Switch_ statement processing; the branch's version of that block needs
the same emission.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
`$$name OP= ...` evaluates the name before reading the old value, so
prepareTarget() walks it once with the real node callback at that point,
composes the read from the result, and threads it on the
PreparedAssignTarget - the write flow consumes it instead of walking the
name again after the value. Mirrors the same change on 2.2.x.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
… walks

The Unset_ handling deep-cloned the already-processed offset chain and
wrapped the clone's levels in ExistingArrayDimFetch nodes, so the virtual
assign had to re-walk the clones with a noop callback to have results for
them. The wrappers now reference the original sub-expressions directly - the
unset statement's own walk already processed and stored them - and the
assign target preparation reads those stored results instead of walking.
DeepNodeCloner loses its last consumer and is removed.

The wrappers deliberately carry no ExpressionResults themselves: they end up
inside scope-held synthetic expressions, and a carried result would pin its
whole scope graph (worker memory exhaustion in self-analysis proved it).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
…tic calls

The ArrayAccess offsetGet/offsetExists/offsetSet/offsetUnset and magic __set
simulations fabricated a MethodCall node and pushed it through the whole
expression walk with a noop callback, only to read the throw points off the
result. MethodThrowPointHelper::getThrowPointsForCallOnType() now resolves
the method on the already-priced receiver type and derives the throw point
directly - same reflection resolution, dynamic throw-type extensions and
implicit-throws semantics as the walk's, without processing anything. The
fabricated node remains only as the throw-point anchor and the payload the
extensions receive.

This also stops the __set simulation from re-walking the real receiver
expression and double-counting its throw points.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
The last two synthetic-call pricings walked a fabricated MethodCall through
the on-demand machinery just to obtain the call's return type: the
ArrayAccess offsetGet read in ArrayDimFetchHandler's composed result and the
__toString() type feeding the string-cast throw point.

Both now resolve through MethodCallReturnTypeHelper::methodCallReturnType()
on the already-priced receiver type - the same reflection resolution,
argument-driven acceptor selection and dynamic return type extension
dispatch the walk performed, without processing anything. The native flavour
mirrors the walk's native arm (the combined acceptor's native return type),
and the fabricated node remains only as the payload extensions receive.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
The static/function call handlers' lazy callbacks already read their
operands from captured results; the maybe-stored fallback arms remained for
cases that cannot occur - the class/name results are always captured when
the class or name is an expression. Those arms are now guards, so a
violation of the single-pass invariant surfaces loudly instead of silently
re-pricing.

The array_key_first/find_key null-narrowing in FuncCallHandler reads the
array argument from the call's ArgsResult (falling back to the stored read
only for rewritten calls keyed by normalized nodes); processArgs' by-ref
writeback pass reads argument types from its own argResults map; and the
@var tag's native variable type is a plain scope-state read, spelled as
such.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
…backs

The dynamic-name/class fallbacks in PropertyFetchHandler,
StaticPropertyFetchHandler, MethodCallHandler and NewHandler were
structurally dead: every caller walks the non-Identifier name (and
non-Name class) and passes its ExpressionResult. Guard them with
ShouldNotHappenException instead of silently re-pricing through
readTypeOfMaybeStored().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
The $getType bridges handed to InitializerExprTypeResolver answer the
held operand results first; everything else they are asked about is a
composed synthetic node (e.g. Mod($left, $right)) - price it through
processSyntheticOnDemand() explicitly instead of the undecided
readTypeOfMaybeStored().

The comparison specify reader's residue asks are operand subexpressions
(count() arguments, subtraction operands) processed by the same walk -
read their guaranteed-stored results. The chain-link reader and the
assign-narrowing arg reader only ever see captured results - guard the
impossible misses with ShouldNotHappenException. Implicit __toString
callers all hold the operand result now, so the maybe-stored arm is gone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
…paths into findScopeStateType

findStoredResult()'s only caller was readTypeOfMaybeStored()'s first
line - fold it in. The new findScopeStateType() answers what the scope
itself knows (variable state, tracked expression holder) without any
node processing and returns null otherwise, letting each caller decide
whether a miss means a synthetic walk or an invariant violation.

AssignHandler's assigned-value reads route through readAssignedValueType()
whose no-result arm is now explicitly virtual-assign pricing (scope state
or synthetic walk); the unwalked-ternary arms use the same form on the
narrowed cond scopes. The chain-root receiver, by-ref array keys and
keep-list dimension reads consume their guaranteed-stored results from
the walk's storage instead of the undecided maybe-stored lookup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
The disjunction/conditional holder-projection sites guard every target
with hasExpressionType()->yes() on the very ask scope, the foreach
key-loop dim fetch is tracked by enterForeach(), and the virtual
wrappers (IssetExpr, PossiblyImpureCallExpr) read remembered scope
state - none of them want a storage lookup. The new
readScopeStateOrSyntheticType() spells that out: scope state answers
without a walk, anything else is priced as a synthetic node.

currentTypeForConditionalHolder() now probes the walk's own storage
explicitly (threaded through the sure/sure-not helpers) instead of the
scope-visible stack, which loop-convergence passes miss.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
The specify callback lives on the ExpressionResult and the storage holds
the results - capturing the storage in the callback creates a cycle the
disabled GC never collects (self-analysis workers hit the 450M ceiling).
Capture the two shape-determined subexpression results (count()/strlen()
argument, subtraction operands) into a plain map when the callback is
built.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
The REBASE_HEAD-restore resolution of NullCoalesceRuleTest kept the
branch's historical file state and lost the two test methods 2.2.x
gained with the falsey-isset ??= narrowing; their data files came
through. Re-add them unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
Mirrors 2.2.x's 56c357f in the branch's composed form: the value
expression of $x ??= y only evaluates when the left side is null OR
unset, so the value scope applies the falsey narrowing of isset($x)
(composed from the left read via createIssetSingleSubjectNonTrueTypes -
a certainty reduction for surely-set non-nullable subjects) instead of
a bare null pin that collapsed non-nullable subjects to NEVER. The
coalesce VALUE-falsey narrowing (getFalseySpecifiedTypes) is unchanged -
a falsey value does not imply an unset left side.

The ??= node's specify callback now also composes its target's
truthiness narrowing directly - the raw sure-not term on the assign
node itself cannot be unpacked into the target at the application
point (what TypeSpecifier::create()'s AssignOp\Coalesce arm recovered;
the specify-side mirror of the existing createTypesCallback).

Covers the upstream bug-15021 fixture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
…osure type extensions

Adopts 2.2.x's 1d345a6 at the branch's two closure-argument sites:
after a non-null phpdoc-side override, the same extension resolves the
native flavour on the natively-promoted scope. This restores the native
precision the branch had downgraded in bug-11014 (expectations reverted
to the 2.2.x fixture) and satisfies the precise native match shape
expectation in preg_replace_callback_shapes.php.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
The invalidation pre-filters treated every '__phpstan' occurrence as
non-compositional: any virtual-node key on either side of a check - the
stored holder's key or the invalidated expression's own key - forced the
full per-holder containment work, and an invalidation of a virtual-keyed
expression swept the scope's whole holder population. On a
src/Analyser+src/Rules run that was 2.41M shouldInvalidateExpression()
calls, 3.2x what 2.2.x pays for the same 35k invalidation events over
the same holder population.

Three changes, mirrored in the native twin:
- invalidateExpressionEntries() gets the same caller-level compositional
  shortcut its siblings already had (skip holders and conditional-holder
  groups whose keys cannot contain the invalidated key; a conditional
  holder's map key embeds every condition's key verbatim).
- The invalidated-expression side of every gate uses
  keyMayHideSubExpressions() instead of the coarse '__phpstan' substring,
  so invalidating a compositionally-printed virtual expression keeps the
  shortcut usable.
- ForeachValueByRef, IntertwinedVariableByReference and
  PropertyInitialization join COMPOSITIONAL_VIRTUAL_KEY_PREFIXES - their
  printers emit every walked sub-node (or walk none). The foreach and
  parameter original-value markers stay out deliberately: they hide a
  synthesized Variable child that containment-based invalidation must
  keep finding when the variable is reassigned.

Same run afterwards: 420k calls, 308k containment checks (from 2.41M and
1.16M).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
With the compositional-key shortcut clearing virtual-node keys, only
308k invalidation checks reach the containment step on a
src/Analyser+src/Rules run (down from 1.16M) - the population where the
index was measured worthless on 2.2.x (phpstan-src PR #6152, 12
interleaved pairs, dead neutral). ExpressionTypeHolder, the ScopeOps
containment (containsExpressionToInvalidate) and their native twins
return to the 2.2.x shapes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
WeakMap is PHP 8.0+ and the downgraded phar must run on PHP 7.4.

The per-file capture maps (TernaryHandler, MatchHandler,
ClosureTypeResolver) become spl_object_id-keyed arrays: their keys are
AST nodes the parser cache keeps alive for the whole file's analysis, so
ids of live entries cannot collide, and the existing per-file reset
empties the maps before another file could reuse them.

FiberNodeScopeResolver's flush memo is keyed by usually-synthetic nodes
that rules rebuild per ask - a dead node's id can be reused - so the
entry stores the node itself and hits are identity-checked; the resolver
now takes part in the per-file reset to release the entries.

PhpFunctionFromParserNodeReflection's static generator cache becomes an
attribute on the FunctionLike node itself: it lives and dies with the
parser-cached AST, with no id-reuse hazard and no pinning.

Same-tree interleaved benchmark: user CPU -0.7% (neutral within noise),
make-phpstan Used memory 2.31 GB -> 2.25 GB.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
The same hazard f935366 closed for the isset/empty/coalesce nodes:
these five are emitted to node callbacks for their rules, and
third-party collectors call $scope->getType() on every expression they
receive - with no printer method that crashes the type-cache key
printer. Rule-facing virtual nodes extend NodeAbstract so they can
never reach getType().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DaBZjgksga4c5s6Q9FniY7
@ondrejmirtes
ondrejmirtes force-pushed the resolve-type-rewrite-2 branch from d644970 to ad9303a Compare July 30, 2026 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment