Round-trip gradients: send workers to the resource nearest for the whole trip - #244
Conversation
A worker fetching for a building walked to the resource nearest to itself, which is often on the far side of the base from the building it delivers to. Buildings now own a round-trip gradient per resource type and swim class: every tile of that resource is seeded with the cost of carrying from beside it to the building, so descending the gradient minimises the trip to the resource plus the trip back. Hiring and the choice of the next resource after a delivery score by the same distance, and a unit heading for a resource for a building follows it, falling back to the plain resource gradient when blocked or stale. Map::propagateGradient accepts seeds at any cost for this; seeds beyond its bucket window enter the sweep when it reaches their cost. Round-trip gradients are built on demand, refreshed at most every 40 ticks and freed after 500 ticks without use. (cherry picked from commit 02185c5) Fable 5.1 helped authoring this commit.
They were only dropped when the same building rebuilt one of its gradients, so a building that stopped fetching kept every field it had ever built. On a 512x512 map with 1400 units that reached 820 MB. Each building now sweeps its unused fields every 256 ticks. (cherry picked from commit 53627ae) Fable 5.1 helped authoring this commit.
…trip ones Every building that ever hired kept its full-map gradient per swim class for good, although at any moment only the buildings with fetchers need one. In a two-hour game on a 512x512 map with 250 buildings alive that was most of 895 MB. The sweep that already drops idle round-trip fields now drops idle building gradients too; the next unit that asks rebuilds. (cherry picked from commit 687b0c6) Fable 5.1 helped authoring this commit.
The scorer's two arms were on different scales. With a field, roundTrip is a real round trip; without one the fallback was distBuilding + distResource, the walk out plus however far the unit happens to be from the building, which for a unit standing at the building is half a round trip. Which arm a resource gets turns on whether anything has fetched it for this building yet, so a resource already being fetched was ranked against a fresh one on two scales. Estimate the carry leg instead: a unit at the building carries about as far as it walks out, and a unit at the resource carries about the building distance, so distResource + max(distBuilding, distResource) is exact at both ends and no worse than the old sum in between. No field is built for this; the numbers are already in hand. RoundTripHungerGateHarness covers the hunger gate (the walk to the resource, not the whole trip, decides whether a unit is hireable) and the fallback score; the second case fails without this fix, reporting a one-way 14 instead of the round trip of 24. Fable 5.1 helped authoring this commit.
Since the continuation state (format 91) a saved game carries every cached routing field and its timestamps, so a loaded game continues bit for bit. The round-trip fields, their refresh and last-use stamps and the building gradient's last-use stamp were not saved, so after a load hiring scored by the estimate where the uninterrupted game read a field, and the two games drifted apart within twenty steps (SavegameSafetyHarness, continuation check). They join the building part of the map runtime section under format 95. Older saves load as before with the fields empty. The replay floor moves to 95 as well: routing and hiring by round trip change the simulation. Fable 5.1 helped authoring this commit.
Petri benchmark: round trip vs master on a human-paced colonyMap: Leo's Binaries:
The delivery, walking, clearing, school and starvation counters and the perturbation seed are local measurement tooling, not part of this PR. The islands are all taken from FourSqares1 with swapped resources. Team 0: 108 workers, 4 corner inns with 1 fetcher each, wheat far (the starvation race) 
Team 3: 8 workers, 12 inns asking 13 to 19 workers each, 3 schools (every freed worker is hired within the tick by whichever building asks first) 
Team 2: 108 workers, 2 inns far apart is inn-capacity bound: 72 starve on master and this PR, 70 with #197. 
Team 1: 8 workers, 8 inns with 1 fetcher each plus a papyrus clearing flag delivers 104 on every binary; the 25 papyrus tiles are gone after 3482 ticks on master and this PR, 3290 with #197. 
80 game minutes, each colony cut where it settles120000 ticks, 10 seeds. Red = master, blue = this PR, green = #197. Left column is each colony's own KPI (workers alive, workers fully trained, papyrus tiles left), right column deliveries per game minute over a 1-minute window. Lines are the mean over seeds, the band the min to max. The x axis stops where the colony reached its steady state: 30, 20, 22 and 5 minutes.
Why the earlier AI-vs-AI games showed little: Nicowar re-plans its job requests every 100 ticks and keeps slack labour everywhere, so the first-asker hire already picks a nearby worker. A human colony with more job slots than workers is where the walk-length choice decides who lives. |
|
I hope we can merge this and #197 as it's just so much more fun to play with less micro management. |
|
Cross-referencing a finding from independent benchmarking on #197 (full methodology there): across a 48-scenario sweep (8 maps, 3 AI mixes, 2 seeds, same tick-0 saves replayed through master vs. this branch alone vs. this branch+#197), round-trip gradients on their own show a flat 0% median change in population/building growth at every checkpoint (early/mid/late) versus master — the measured +13–18% throughput cost from my earlier comment doesn't show a matching growth benefit in isolation. The benefit only shows up once #197's job/inn swaps are layered on top (+6.7%/+7.6% units, +9.1%/+6.7% buildings by mid/late game). That doesn't mean this branch is wrong — #197 depends on it, and round-trip fields are presumably what make swap scoring cheap and correct in the first place — but on its own, as measured here, it's a cost without a demonstrated standalone benefit. Full data: #197 (comment) |
genixpro
left a comment
genixpro
left a comment
There was a problem hiding this comment.
Approving and merging with Bradley's explicit sign-off, per this repo's policy that substantive engine/gameplay changes need a human maintainer's approval beyond any validation an agent can attach.
Summary of what's on record for this change by the time it merges:
- Save/replay compatibility checked directly:
VERSION_MINOR94→95,MINIMUM_VERSION_MINORcorrectly left at 58 (old saves still load),REPLAY_MINIMUM_VERSION_MINORcorrectly bumped to reject pre-95 replays (this is a real simulation change, so that's the right call), and the newroundTripsave section is properly gated behindversionMinor >= FILE_FORMAT_VERSION_ROUND_TRIP_FIELDSon load. - Cross-platform determinism confirmed independently: built this branch on macOS/Apple Clang and a clean Ubuntu 22.04/GCC 11 environment, ran identical seeded games on two save fixtures with
GLOB2_CHECKSUM_SIDECAR=1— replays and per-tick checksums came back byte-for-byte identical on both. (Both environments were arm64; x86_64 coverage — what CI's Linux runners actually use — is the one gap I couldn't close from here.) - Throughput cost measured: +13–18% simulation time on its own, attributable to round-trip gradient computation (see #197 for the fuller breakdown separating this PR's cost from #197's).
- Standalone benefit: across a 48-scenario sweep (8 maps, 3 AI mixes, 2 seeds), round-trip gradients alone showed a flat 0% median population/building-growth change vs. master at every checkpoint — the benefit shows up once #197's swaps are layered on top. Noted here for the record; not a blocker, since #197 depends on this branch either way.
Merging.
2 participants
Successor to #193, which GitHub closed when its head branch was renamed from
feat/pathfinding-lanestofeat/pathfinding-round-trip(the lanes commit had already left it; the traffic-lane work is #213). genixpro's review and Leo's measurements are on #193. Rebased onto current master (e3a01b0, after #223 fetch apportionment and #186); five commits, linear, no merge commits. #197 (task swaps) and #213 (traffic lanes) are stacked on this branch.Round-trip gradients. A worker fetching for a building walked to the resource nearest to itself, often on the far side of the base from where it delivers. A building now owns a round-trip gradient per resource type and swim class: every tile of that resource is seeded with the cost of carrying from beside it to the building, and
Map::propagateGradientruns from those seeds (it now takes seeds at any cost, and an optional cost bound). Descending it minimises trip-to-resource plus trip-back, sopathfindResourcewith the target building follows it, falling back to the plain resource gradient when blocked or stale. Hiring (considerUnitForResource, per wanted resource since #223) and the next-resource choice after a delivery score by the same distance when a fetcher has already built the field, and by an estimate otherwise:distResource + max(distBuilding, distResource), exact at both ends of the walk (hiring looks at every needed resource of every building, far more than ever get fetched, so no field is built for it). Refreshed at most every 120 ticks, propagated 128 tiles beyond the cheapest fetch, freed after 500 ticks unused; the building's own gradient is freed on the same timer.Commits
RoundTripHungerGateHarness(now also run in CI). On master the hunger gate already measures the walk to the resource, so this commit is the scale fix plus the harness.SavegameSafetyHarnesscaught the drift within twenty steps. Older saves load with the fields empty.Verified on this rebase (Linux, release build): 187 CppUnit tests;
RoundTripHungerGateHarness,ResourceFetchTargetHarness(7 swim classes),FetchApportionmentHarness,TrappedUnitLifecycleTest,BuildingExpelHarness,ImmobileUnitGradientHarnessandSavegameSafetyHarness(save, load, 300-step continuation) all pass.Measured on a human-paced test bed (Leo's petri maps: four AINone island colonies, more job slots than workers, no forbidden areas; details, screenshots and the 80-minute chart in this comment): on the 108-worker island with four far-off inns, deliveries go from 194 to 217 (+12%) and survivors from 72 to 79 of 108 in 10 game minutes, 66 to 74 over 80 minutes; on the island where two inns are the bottleneck the round trip alone changes nothing and #197's inn swap is what saves workers; on the 8-worker island with 215 open job slots the gain is +3% here and +23% with #197's job swaps. The earlier Nicowar-vs-Nicowar numbers on #193 (+6%) undersell it because the AI re-plans its job requests every 100 ticks and never saturates its labour market.
Feel: workers pick the wheat or wood patch on the building's side of the base rather than the one nearest to where they were freed; visible as fewer long diagonal hauls across a base. Save format 95: the round-trip fields travel with the map runtime state, like the other cached routing fields.