Improve path finding

[Giszmo]

Draft. Replaces the chamfer gradient descent with weighted distance fields, in place: the existing gradients become Uint16 and are kept per swim class instead of per canSwim bool.

What changes

• Gradients. resourcesGradient, forbiddenGradient, guardAreasGradient, clearAreasGradient and Building::globalGradient are Uint16 fields indexed [team][swimClass]. A cell holds GRADIENT_AT_GOAL - cost, cost in tenths of a land step (10 cardinal, 14 diagonal, water at the unit's swim-class rate), so "higher is closer", 0 obstacle and 1 unreachable keep their meaning (MapInternal.h). Map::propagateGradient (Dial's bucket-queue Dijkstra, MapGradientField.cpp) replaces the chamfer for pathfinding; the Uint8 chamfer stays only for the Castor and Warrush helper maps.
• Swim classes. Unit::swimClass() buckets walk/swim speed into 7 classes (0 = cannot swim); a gradient is allocated the first time a unit of that class asks for it, so unused classes cost nothing. The round-robin scheduler updates only the gradients in use.
• Steering. Map::directionByGradient picks the neighbour with the highest value minus step cost; when blocked it may sidestep to an equal cell (deterministic syncRand). This replaces the 5x5 minigrad and the 32x32 local building grids: a building has one full-map gradient per class, rebuilt when the map changes nearby (dirtyBuildingGradients), when a unit is stuck, and every 125 ticks for clearing flags (Map::buildingGradient). clearingFlagStep, localResources and updateLocalResources go away with it.
• Timing. A diagonal step advances delta at speed/sqrt(2), so diagonals are no longer 41 % faster than cardinal moves (second commit; the octile costs assume it).
• A*. pathfindPointToPoint uses the same step costs and a total order on equal keys.

Net: 43 files, +912 / -1401. MapGradientLocal.cpp and MapMinigrad.cpp are deleted, MapGradientField.cpp and test/GradientTest.cpp added.

Testing

Headless (-test-games-nox), release build, Nicowar on every team. "old" is master's pathfinder and timing, "old + sqrt2" is master's pathfinder with only the diagonal timing fix, "new" is this branch. 10 game minutes (15000 ticks), 5 seeds each, mean units per team at the end:

map	old	old + sqrt2	new
Mazury 128x128, 2 Nicowars	72.1	61.4	59.5
balanced_for_2 64x64, 2 Nicowars	88.8	86.5	85.9
Mazury 4x4 512x512, 8 Nicowars x 4 swarms (seed 1, all teams together)	266	400	389

The diagonal timing fix alone costs master's units about 15 % of their economy on Mazury (they were moving 41 % faster on diagonals); the new pathfinder is within noise of "old + sqrt2" on both small maps and about even on the big one.

Ticks per second at matched unit counts (512-tick windows, same seeds, runs one after the other on an idle machine):

map	units	old	old + sqrt2	new	new/old
Mazury 128x128, 2 teams	50	3770	4075	5332	1.41
	80	3413	2935	3785	1.11
	100	2534	2554	3110	1.23
	130	2273	2498	2938	1.29
balanced_for_2 64x64, 2 teams	30	20585	21609	17608	0.86
	50	17432	17778	16408	0.94
	80	13869	13864	14212	1.02
Mazury 4x4 512x512, 8 teams	300	104	114	207	1.98
	400	105	114	216	2.05
	500	139	129	196	1.40
	600	113	117	203	1.79

The 10-minute 512x512 game takes 128 s wall on master and 75 s here. One Dijkstra costs about what one chamfer pass costs, but the chamfer needs several passes on a big map and there is no 32x32 grid to rebuild any more; on the tiny map the per-class gradients cost a little at low unit counts.

Peak RSS on Mazury 4x4 (512x512, 8 Nicowars, 10 game minutes): old 140 MB, new 179 MB (one Uint16 field per building and swim class in use, instead of one Uint8 field per building and swim variant plus the local grids).

Determinism: same seed twice gives identical per-team timelines and end-of-game lines on 128x128 and 512x512; replays differ only in the known pointer-looking bytes of the Echo AI save in the header. 166 unit tests pass (8 new in GradientTest); the CI harnesses (building footprint, selection, terrain, savegame safety) pass; server=1 builds. A mixed Castor/Warrush/Nicowar/Numbi game (war flags, clearing flags, guard areas) runs clean.

Known limits

• Uint16 costs cap a path at 6553 land tiles; hunger ends a trip after ~400 steps, so it never binds.
• Building gradients are kept per swim class until the building dies; the memory of a very long game on a huge map could be bounded with an LRU.
• Round-trip aware hiring (walk to the resource tile that is cheapest for the whole trip to the building) was measured on an earlier version of this branch (+8 % deliveries on Mazury) and is left for a follow-up PR to keep this one small.
• Save files and the network protocol are unchanged; replays of games recorded before this change do not play back identically.

[stephanemagnenat]

I do not understand at all why we have these extra cost fields. Why not simply switching the existing gradient to u16 and multiply their number per "swim class" (compared to boolean CAN_SWIM)?

So I'll ask the AI why we have duplication rather than updating our existing gradients.

[stephanemagnenat]

In general, while it is totally fine to do AI-based development for Glob2, we should really be a bit mindful for duplication and slop. Even frontier models (gpt 6 astra and fable 5.1) still have a very tunnel vision and would create a huge mess in the code if not kept in check and nudge towards simplicity/DRY.

[Giszmo]

Restructured along those lines. The existing gradients are now the Uint16 fields themselves, indexed by swim class instead of the canSwim bool, and Map::propagateGradient (a bucket-queue Dijkstra) replaces the chamfer for them in place. There are no parallel cost fields, no per-team switch and no stats any more. The 5x5 minigrad and the 32x32 local building grids went with it, since a weighted full-map gradient makes them redundant: net -490 lines. The Uint8 chamfer stays only because Castor and Warrush run it on their own helper maps.

The round-trip aware hiring is out of this PR; it can come separately if wanted. Description rewritten with the measurements against master (economy, ticks per second at matched unit counts, memory).

[genixpro]

I support the proposed gameplay direction, including swimming-aware weighted routing and the diagonal timing correction. I am happy for this to merge once we have tightened up the computational performance.

We are currently benchmarking and profiling this against the existing pathfinder with my recent gradient optimizations, including a comparison with diagonal timing matched. That should help identify which costs remain and where further optimization is worthwhile.

This is conditional support: the performance work and verification are still in progress, so please keep that as an outstanding condition before merging.

[genixpro]

Added a focused performance optimization to the weighted propagation loop. It reuses wrapped neighboring coordinates, the two terrain costs, and their queue buckets for each popped cell. Neighbor visitation remains NW, N, NE, E, SE, S, SW, W, preserving queue insertion order. No changes to weights, swimming classes, scheduling or diagonal timing.

Measured against this PR's original head cddad361, on an Apple M3 with Apple Clang -O3: median core thread CPU seconds for 15,000 ticks, three paired serial repetitions with alternating order:

Fixture	Before	After	Less core CPU
G2, 128x128	2.870 s	2.176 s	24.2%
Playground, 8 Numbi, 128x128	3.327 s	2.580 s	22.5%
Archipelago, 128x128	2.989 s	2.204 s	26.3%
BigArena, 256x256	15.269 s	11.139 s	27.0%

Whole-process elapsed time fell by about 17–20%. Separate function-timing runs showed about 35–39% less time in Dijkstra propagation, with identical call counts. Core timing excludes AI order generation, order execution, loading and replay output; function profiles include all calls, so their totals are not interchangeable. No new persistent memory or production profiling code.

Validation: 167 unit tests pass. The new test compares 700 randomized fields against an independent heap-based shortest-path solver, covering all seven classes, water, obstacles, mixed seed costs, absent sources, toroidal wrapping and thin/rectangular maps. ASan+UBSan checks pass with the changed propagation and test translation units instrumented (remaining linked objects were not instrumented). Clean release binaries produce byte-identical replays and complete per-tick checksum sidecars across all four fixtures, 60,000 ticks per binary.

This also compares favorably with my optimized existing gradients on these fixtures, but the weighted/old routing versions produce different populations; the before/after table above is the stronger comparison because the simulation results match exactly. These measurements do not cover 512x512 or late-game worst cases.

For #193's rebase: its direction-dependent lane penalties must remain per edge; the two shared terrain costs alone do not include those penalties.

[genixpro]

Pushed a second behavior-preserving optimization, b99734c8e, carrying over the useful 8-bit AI helper gradient work from the now-closed #195 and #196. It reuses row pointers, takes one neighbor maximum, skips maximal seeds, and returns early for inert fields. The weighted 16-bit pathfinder is unchanged by this follow-up.

On the AI-heavy 256×256 BigArena fixture, three alternating paired 15,000-tick release runs against 28ddbc0b6 gave these medians:

Metric	Before	After
Whole-process CPU	20.65 s	20.11 s (2.6% less)
Whole-process elapsed	20.93 s	20.80 s (0.6% less)
Retired instructions	298.79 billion	264.96 billion (11.3% less)
Core-only simulation CPU	11.370 s	11.655 s (2.5% higher)

The core timer excludes most AI order generation; the core regression is reported explicitly, so this is not a universal speedup. Separate diagnostic profiling showed helper propagation falling from 3.337 s to 2.496 s with the same 3,642 calls. Numbi-only workloads do not use these helpers.

Validation: clean before/after releases produced identical replays and every per-tick checksum on four fixtures, totaling 60,000 ticks per binary. Both passed the independent helper oracle covering 3,000 random fields plus thin/toroidal maps, obstacles, cutoff, inert inputs and mixed seeds. The harness is added to Linux CI. The targeted ASan/UBSan harness aborted in the local Homebrew SDL2 initializer before the tests, so that check is incomplete.

Also tested queue-ring sizing, direct candidate-gradient comparisons, early downhill rejection and compact terrain-cost buffers. None showed enough repeatable benefit to retain. This follows the earlier 22–27% core-CPU reduction; further straightforward changes are showing diminishing returns. Measurements are Apple M3 / Apple Clang release builds, with final pairs isolated from a competing build that disturbed screening.

[genixpro]

Final code-quality follow-up is pushed as 097d8c4e0:

• Fix the disabled BURST_UNIT_MODE build: its action speed is now in scope, retaining its original unscaled behavior.
• Give normal diagonal action timing a named helper and three focused regression tests covering travel directions/modes, non-travel actions, arrival timing boundaries and replay-visible integer rounding. The normal arithmetic is unchanged.
• Document shared scratch storage's serial/non-reentrant requirement, bounded seed costs and reseeding contract.
• Clarify buffer ownership, refresh throttling, the two passability variants versus seven weighted swim classes, and weighted land-step equivalents versus literal tile counts.

Validation: 170 unit tests passed; actual Unit.cpp compiled with BURST_UNIT_MODE; the release game and 3,000-field helper harness passed. All four final-release replay files and every per-tick checksum match the previously validated build across 60,000 ticks. The timing unit tests exercise the extracted calculation, not the entire Unit lifecycle. The earlier SDL initialization limitation on sanitizer execution remains disclosed.

Final Linux/Windows CI is running; merge will wait for those checks.