diff --git a/README.md b/README.md index fb73f7b..4ee6d4a 100644 --- a/README.md +++ b/README.md @@ -94,6 +94,10 @@ The last path is remembered. A green download arrow appears beside the gear when out; the same progress bar shows the download, and the toolkit restarts to install it. *(A "Resource Editor" tile exists but is a disabled stub.)* +The library browser shows a picture of each character, wardrobe piece and car, drawn from its +archive: people head and shoulders, a car on its wheels. Pictures are kept on disk and only the +tiles on screen are read. + ### Viewport and streaming - Area catalog of open-world districts and interiors, from `cityareas.bin`. @@ -200,6 +204,9 @@ with it: *Blended* becomes a translucent surface (glass), anything else a cut-ou grille). The material is written to the library on Save and its textures are packed into the archive on Build. +A skinned mesh - a car body - can gain vertices: its remap pools are rebuilt from the pushed skin. +Two levels of detail of one object can go in one push. A person opens standing on their rig. + Limits worth knowing: untouched geometry round-trips bit-exactly (that is how a real reshape is told apart from an untouched one); a topology rebuild leaves lower LODs and collision with the old shape; collision placements refuse scale and mirror pushes (resize the hull with the toolkit's own @@ -209,9 +216,15 @@ split it into several objects. ### Saving, building, backups -**Ctrl+S** writes edited FrameResources back to the extracted mirror. **Build** repacks every -edited archive into its `.sds`, creating a timestamped versioned backup first; archives are packed -independently, so one failure (the game holding a file open, say) does not block the rest. +**Ctrl+S** writes edited FrameResources back to the extracted mirror. **Build** opens a window +first: every archive edited in the session, ticked, and under each the files of its working copy that +differ from the archive the game has now. A Build packs a whole working copy, so that list is +everything that is about to change in the game - including a file changed in that folder in an +earlier session and forgotten, which is marked and counted out loud. Untick an archive to leave it +for a later Build; **Add archive…** packs one whose working copy was changed by hand (a script, a +table). Each ticked archive is then repacked into its `.sds` with a timestamped versioned backup +first; archives are packed independently, so one failure (the game holding a file open, say) does +not block the rest. **Restore Backup** rolls a single archive back to an earlier version - replacing both the live `.sds` and its extracted mirror - from the File menu, the tree's context menu or the viewport's. *(Material-library edits are not covered by the backup flow.)* @@ -233,6 +246,50 @@ beside the working copy in `illusion_memory.json`, and stated again on every Bui resource changed size. A repacked stock car states exactly what the original did (`--probe-car-clone`). +### Tools menu + +What the MCP server can do to a scene, the menus can too: each item under **Tools** opens a window +over the same job the matching tool runs, so a result does not depend on who asked for it. + +- **Hide triangles…** (map and resource editor) - cuts an opening in the selected mesh without + rebuilding it: a doorway in a stock wall, a pane of glass. With the window open the triangles are + picked by clicking them on the mesh in the viewport (a click on a marked one takes it back), or - + for many at once - by a box that holds them. What would be hidden is marked on the mesh, the count + says what goes on each level of detail, and Hide is one undo step. +- **Mirror to winter…** (map editor) - carries the loaded district's edits into its winter archive. + The item is greyed out, and says why, when the district has no winter variant or the winter one is + what is loaded. +- **Loading zones…** (map editor) - lists the zones of `city_univers` that hold a point (the + camera's, or one typed in), says which districts a player who appears there gets, and moves one + face of a zone. A district streams in where a zone named with two words after its number + (`AREA341_GREENFIELD_KINGSTONE`) holds the player; one named with a single word + (`AREA0019_GREENFIELD`) does not load it by itself. A moved face is written at once, redrawn in the Loading + zones layer, queued for Build, and is one step of the editor's Undo. + The zones are also edited in the viewport, with no window: with the **Loading zones** layer on, a + click picks the zone that spot is in (the same spot again takes the next one there) and the tool + shelf works on it - **Move** puts three arrows at the zone's centre and moves it whole, **Scale** + puts an arrow at the centre of each face and pulls that face alone. A side sliced off by a slanted + plane has no arrow. Letting go writes `city_univers` and queues it for Build; the right button or + Esc drops a drag, and Undo / Redo take back exactly what the drag changed. With **Whole map** on the + editor holds `city_univers` itself; a zone write is carried into that scene too, so the editor's own + save does not put the zone back. + The layer draws each zone as the edges of its box with a faint fill, hidden by the scene in front of + it and fading with distance (not in a parallel view); a zone with one word in its name - which does + not load its district by itself - is drawn fainter, the picked zone carries its name and + districts over it, and a zone that was added to the game's own has a second, near-white frame + round it (and "added" in its label and in the Loading zones list). + **New loading zone** (the button at the foot of the tool shelf, with the layer on) makes a zone + where the view looks: a name and the one or two districts it keeps loaded are asked, the box appears + picked, and Move and Scale put it in place. The name wants two words after its number + (`AREA900_DOCK_SOUTH`): that is what makes the game load the districts for a player who appears + inside, and the flyout says so when a name has one. It is written at once - the scene, its name + table and `cityareas.bin` - and Undo takes it out again. Not with Whole map on. +- **Stream map…** - find and replace across the text of a `StreamMapa.bin` (archive paths, instance, + line and group names), with every string it would change listed before anything is written and a + backup kept beside the file. +- **Clone car…**, **Replace a car…**, **Export car for multiplayer…** (resource editor) - the three + car jobs described under the MCP server below, starting from the car on the stage. + ### MCP server An MCP endpoint runs for the lifetime of the application at `http://127.0.0.1:2010/mcp` - loopback @@ -240,7 +297,7 @@ only, no authorization - with its live status in the launcher's status bar. Poin with `claude mcp add --transport http illusion http://127.0.0.1:2010/mcp`; change the port with `McpPort` in settings. -It serves 76 tools. The file tools all read through the same format layer the editor uses, so what +It serves 86 tools. The file tools all read through the same format layer the editor uses, so what a model is told about a file is what the toolkit itself sees; the editor tools drive the running map editor itself. @@ -254,11 +311,12 @@ map editor itself. | **Tables** | `list_tables`, `dump_rows`, `lookup_by_row` | | **Stream map** | `parse_stream_map`, `edit_stream_map` | | **Effects** | `parse_effects_file`, `parse_effects_from_bytes` | -| **Utility** | `hash_fnv32`, `hash_fnv64`, `hash_batch`, `convert_number`, `detect_file_format`, `detect_format_from_bytes`, `list_game_files`, `get_configured_games` | +| **Utility** | `hash_fnv32`, `hash_fnv64`, `hash_batch`, `convert_number`, `detect_file_format`, `detect_format_from_bytes`, `list_game_files`, `get_configured_games`, `ping` | | **Editor** | `editor_status`, `editor_list_areas`, `editor_open_area`, `editor_save`, `editor_build`, `editor_mirror_winter`, `editor_undo`, `editor_redo`, `editor_notices` | -| **Scene** | `scene_find`, `scene_select`, `scene_delete_selected`, `scene_duplicate_selected`, `object_move`, `object_properties`, `object_set_property`, `actor_import`, `object_import` | +| **Scene** | `scene_find`, `scene_select`, `scene_delete_selected`, `scene_duplicate_selected`, `object_move`, `object_properties`, `object_set_property`, `actor_import`, `object_import`, `mesh_hide_triangles`, `mesh_materials`, `collision_unused_hulls`, `crash_placements` | | **Resource editor** | `editor_target`, `resource_list`, `resource_open`, `resource_status`, `car_tuning`, `car_tuning_set` | | **Cars** | `car_clone`, `car_substitute`, `car_export_m2o`, `archive_build` | +| **Loading zones** | `zones_at`, `zones_map`, `zone_move_face`, `zone_create`, `zone_delete` | | **Blender session** | `blender_open`, `blender_push`, `blender_end` | | **Viewport** | `camera_get`, `camera_set`, `camera_look_at`, `camera_frame_selection`, `view_set`, `viewport_screenshot` | @@ -271,6 +329,27 @@ and nothing reaches the game before `editor_build` (which keeps the usual timest a car, opened with `resource_open`); `car_tuning` and `car_tuning_set` read and edit that car's entity-data tables the way the Tuning tab does. +`mesh_hide_triangles` cuts an opening into a stock mesh without rebuilding it: the triangles inside +a world-space box are hidden on every level of detail and no vertex is touched, so a stock facade +keeps the channels Blender never sees. It reports by default and hides with `apply`. +`mesh_materials` lists a mesh's material slots and re-points one at another material - no geometry +and no UV is touched. `collision_unused_hulls` counts, and with `apply` removes, the hulls no placement references. +`crash_placements` lists the crash-layer props (trees, lamps, bins) standing in a box and can +delete them, the twin season included. + +`zones_at` lists the load zones of `city_univers` that hold a point and the districts they name, +`zones_map` draws where a district is asked for as a text plan, and `zone_move_face` moves one face +of a zone (its plane and its box together; reports by default, writes with `apply`). Measured in +the game: a district streams in where a zone named with two words after its number holds the player +- one named with a single word does not load it by itself - and free ride reads the base game's +`city_univers`, not the copy a DLC ships. +`zone_create` adds a new zone: a box between two corners, made like an existing zone, with a line in +`cityareas.bin` for the districts it keeps loaded. Measured in the game, with new zones over one +spot: a player who appears inside a zone named `AREA901_FOO_BAR` gets the district - with one +district in the table or two - and inside one named `AREA902_FOOXBAR` he does not. With the map +editor open the new zone is a step of its history; `zone_delete` takes an added zone out again (a +zone the game ships with is refused). Zones are made in the base game's `city_univers` only. + **Cars.** `car_clone` makes a new car out of an existing one for single player: a copy of its archive (and the winter `_z` twin) with the root frame, name table, prefab entry, entity data and geometry buffers filed under the new model name, registered in the vehicle, paint, cover-point and diff --git a/src/Illusion.Assets/Adapters/FrameNodeAdapter.cs b/src/Illusion.Assets/Adapters/FrameNodeAdapter.cs index 2897399..0380fbe 100644 --- a/src/Illusion.Assets/Adapters/FrameNodeAdapter.cs +++ b/src/Illusion.Assets/Adapters/FrameNodeAdapter.cs @@ -150,6 +150,29 @@ public bool SetSlotMaterial(int slotIndex, ulong hash) return true; } + public IReadOnlyList GetSlotTable() + { + if (Lod0Structs() == null || _frame is not FrameObjectSingleMesh mesh) return []; + return [.. mesh.Material.Materials.Select(level => (level ?? []).Select(slot => slot.MaterialHash).ToArray())]; + } + + public bool SetSlotTable(IReadOnlyList table) + { + ArgumentNullException.ThrowIfNull(table); + if (Lod0Structs() == null || _frame is not FrameObjectSingleMesh mesh) return false; + List levels = mesh.Material.Materials; + if (levels.Count != table.Count) return false; + for (int lod = 0; lod < levels.Count; lod++) + { + if ((levels[lod]?.Length ?? 0) != table[lod].Length) return false; + } + for (int lod = 0; lod < levels.Count; lod++) + { + for (int slot = 0; slot < table[lod].Length; slot++) levels[lod][slot].MaterialHash = table[lod][slot]; + } + return true; + } + // The LOD0 material table, with the same construct-as-side-effect guard GetMaterials uses. private MaterialStruct[]? Lod0Structs() { diff --git a/src/Illusion.Assets/Adapters/SceneDocumentAdapter.cs b/src/Illusion.Assets/Adapters/SceneDocumentAdapter.cs index 8de4132..ab9aca6 100644 --- a/src/Illusion.Assets/Adapters/SceneDocumentAdapter.cs +++ b/src/Illusion.Assets/Adapters/SceneDocumentAdapter.cs @@ -398,6 +398,48 @@ public IReadOnlyList GeometrySharers(FrameObjectSingleMes return sharers; } + /// + /// The other frame objects that draw from one of the given index buffers through a geometry block of their + /// OWN - which does not see. No shipped district has one (measured: 0 of 1310 + /// buffers in uppertown); an object imported twice with a level's draw distance changed in between does. + /// + public IReadOnlyList IndexBufferSharers(FrameObjectSingleMesh mesh, IReadOnlySet indexBuffers) + { + ArgumentNullException.ThrowIfNull(mesh); + ArgumentNullException.ThrowIfNull(indexBuffers); + var sharers = new List(); + foreach (object? value in _frame.FrameObjects.Values) + { + if (value is not FrameObjectSingleMesh other || ReferenceEquals(other, mesh) + || other.Geometry is not { LOD: { } levels } || ReferenceEquals(other.Geometry, mesh.Geometry)) + { + continue; + } + for (int lod = 0; lod < levels.Length; lod++) + { + if (other.GetIndexBuffer(lod) is { } buffer && indexBuffers.Contains(buffer.Hash)) + { + sharers.Add(other); + break; + } + } + } + return sharers; + } + + /// The other frame objects that draw with the same material block: re-pointing one of its slots + /// re-points it for all of them. Rare in the shipped districts (8 of 3099 mesh frames in uppertown). + public IReadOnlyList MaterialSharers(FrameObjectSingleMesh mesh) + { + ArgumentNullException.ThrowIfNull(mesh); + // Asked through the reference table first: the Material property makes an empty block for a mesh that + // has none, and a question must not leave every such mesh of the scene with one. + if (!mesh.Refs.ContainsKey(FrameEntryRefTypes.Material) || mesh.Material is not { } material) return []; + return [.. _frame.FrameObjects.Values.OfType() + .Where(other => !ReferenceEquals(other, mesh) && other.Refs.ContainsKey(FrameEntryRefTypes.Material) + && ReferenceEquals(other.Material, material))]; + } + /// The canonical adapter for a frame object of this document. public FrameNodeAdapter Node(FrameObjectBase frame) { diff --git a/src/Illusion.Assets/Bridge/BridgeMeshApplier.cs b/src/Illusion.Assets/Bridge/BridgeMeshApplier.cs index d6109c8..b5e2134 100644 --- a/src/Illusion.Assets/Bridge/BridgeMeshApplier.cs +++ b/src/Illusion.Assets/Bridge/BridgeMeshApplier.cs @@ -68,6 +68,14 @@ public sealed class ApplyResult internal Vector3 OldDecompressionOffset; internal float OldDecompressionFactor; + /// A skinned model's pools, face ranges, hit boxes and skeleton tables before the push and + /// as the push leaves them. They go in and out with the buffers: old vertex bytes read through new + /// pools name other bones. + internal SkinState? SkinBefore; + + /// + internal SkinState? SkinAfter; + /// Pre-push packed vertex bytes (diagnostics/probes; also the undo payload). public byte[] OldVertexData { get; internal set; } = null!; @@ -125,6 +133,7 @@ public sealed class ApplyResult public void ApplyNew() { ApplyBuffers(); + if (Frame is FrameObjectModel skinned) SkinAfter?.Apply(skinned); // …and the per-BONE boxes, which are derived from the vertices that just moved. Measured on 88 // cars: a box bounds every vertex with any weight on its bone, in that bone's own space, and @@ -194,6 +203,32 @@ public void RestoreOriginal() Rebuild.IndexBuffer.SetData(Rebuild.OldIndexData); Document?.MarkIndexBufferDirty(Rebuild.IndexBuffer.Hash); } + // …and the skin as it was, boxes included: ApplyNew rebuilt those from the new geometry, and the + // old geometry is entitled to the ones it had. + if (Frame is FrameObjectModel skinned) SkinBefore?.Apply(skinned); + } + } + + /// + /// for an object whose FRAME this push has already produced results for — the + /// other level of the same mesh. Both levels are one frame: they are packed against one quantization + /// lattice, share its bounds and, on a model, its pools. A result is computed against the frame as it + /// stands and carries "the other level as it was", so two results computed side by side each undo the + /// other when applied. This one is computed with the ones in place (and the + /// frame put back afterwards): the caller applies them in the same order and undoes them in reverse. + /// + public static ApplyResult? TryApplyAfter(IReadOnlyList earlier, IFrameNode node, + MeshObjectPayload payload, out string? skipReason, int lod = 0) + { + ArgumentNullException.ThrowIfNull(earlier); + foreach (ApplyResult applied in earlier) applied.ApplyNew(); + try + { + return TryApply(node, payload, out skipReason, lod); + } + finally + { + for (int i = earlier.Count - 1; i >= 0; i--) earlier[i].RestoreOriginal(); } } @@ -218,7 +253,13 @@ public void RestoreOriginal() // remap must not have every later push refused on account of it — the question below is whether THIS // push breaks it, not whether it was whole to begin with. FrameObjectModel? skinned = node is FrameNodeAdapter { Frame: FrameObjectModel m } ? m : null; - bool resolvedBefore = skinned != null && SdsMeshLoader.GlobalBoneIds(skinned) != null; + bool resolvedBefore = skinned != null && SdsMeshLoader.GlobalBoneIds(skinned, lod) != null; + + // The paths below work a skinned model's pools, face ranges and skeleton tables out by writing them. + // That is taken as the push's result and then taken BACK: a push that is refused must leave nothing + // behind, and one that is accepted is the caller's to commit — and to undo. What was written rides in + // the result and goes in with ApplyNew, out with RestoreOriginal. + SkinState? skinBefore = skinned != null ? SkinState.Take(skinned) : null; // A level left with a 32-bit index buffer — something an earlier version of this code wrote for a mesh // over 65535 vertices, and the game does not draw — is rebuilt whatever the push changed: the rebuild @@ -230,6 +271,16 @@ public void RestoreOriginal() { result = TryApplyRebuild(node, payload, out skipReason, lod); } + if (skinned != null && skinBefore != null) + { + if (result is { Unchanged: false }) + { + result.SkinBefore = skinBefore; + result.SkinAfter = SkinState.Take(skinned); + } + skinBefore.Apply(skinned); + } + // A push that changed nothing has nothing to break, and its result carries no buffers to apply — // ApplyNew on one of those is a null reference, not a check. if (result == null || skinned == null || !resolvedBefore || result.Unchanged) return result; @@ -243,15 +294,15 @@ public void RestoreOriginal() // Applied, checked and put back: the caller is the one that commits, and a push that would break the // skin has to be refused while the modeller is still in Blender and can split the vertex groups. result.ApplyNew(); - bool resolvesAfter = SdsMeshLoader.GlobalBoneIds(skinned) != null; + bool resolvesAfter = SdsMeshLoader.GlobalBoneIds(skinned, result.Lod) != null; string broke = resolvesAfter ? "" : SdsMeshLoader.DescribeBoneRemap(skinned); result.RestoreOriginal(); if (resolvesAfter) return result; skipReason = "this push would leave a skin the game cannot read, though the editor would still draw " - + "it: " + broke + ". A bone id has to fit the remap pool of the face group that draws it, and " - + "the pools are fixed at 64 entries in all. Give the affected faces one vertex group instead of " - + "two, or move them onto the material their bones already belong to, and push again."; + + "it: " + broke + ". A bone id has to fit the remap pool of the face group that draws it. " + + "Give the affected faces one vertex group instead of two, or move them onto the material " + + "their bones already belong to, and push again."; return null; } @@ -307,7 +358,10 @@ private static int UngroupedVertices(IFrameNode node, MeshObjectPayload payload) private static bool NeedsRebuild(string reason) => reason.StartsWith("topology changed", StringComparison.Ordinal) - || reason.Contains("source vertex index out of range", StringComparison.Ordinal); + || reason.Contains("source vertex index out of range", StringComparison.Ordinal) + // A re-weight that puts one vertex under two pools keeps its vertex count only on paper: the + // vertex has to become two, and making vertices is the rebuild's job. + || reason.StartsWith(SharedAcrossPoolsReason, StringComparison.Ordinal); /// Computes the count-preserving application of to /// 's mesh. Null with a reason when it cannot apply (topology changed, @@ -588,8 +642,12 @@ private static bool NeedsRebuild(string reason) => // whose split table and OPCODE partition the native core builds (one split + one burst per // material — see FrameLOD.CreateRebuilt). Lower LODs and the separate collision resource keep // their old shape — the caller warns the user once. + // + // poolOfSlot is the second pass of a skinned rebuild: the pool each surviving material slot was + // planned to draw from. A corner then belongs to its pool as much as to its normal and UV, so a + // vertex two pools would have shared comes out as one vertex per pool. private static ApplyResult? TryApplyRebuild(IFrameNode node, MeshObjectPayload payload, - out string? skipReason, int lod = 0) + out string? skipReason, int lod = 0, int[]? poolOfSlot = null) { skipReason = null; // A skinned model may be re-topologised: below, its remap pools AND its per-bone face ranges @@ -676,7 +734,7 @@ private static bool NeedsRebuild(string reason) => // agree on pos/normal/uv0 can still differ in channels Blender never saw (colors, extra UV // sets, damage groups), and merging them would corrupt those. Only Blender-born corners // (orig −1) deduplicate purely by attributes. - var keyToSplit = new Dictionary<(int Orig, uint Welded, int NormalKey, uint UvKey), int>(loops); + var keyToSplit = new Dictionary<(int Orig, uint Welded, int NormalKey, uint UvKey, int Pool), int>(loops); var positions = new List(); var normals = new List(); var uvs = new List(); @@ -696,7 +754,9 @@ private static bool NeedsRebuild(string reason) => ? payload.LoopOrigIndex[i] : -1; Vector3 normal = payload.LoopNormals[i]; Vector2 uv = new(payload.LoopUvs[i].X, 1f - payload.LoopUvs[i].Y); - var key = (orig, welded, PackNormalKey(normal), PackUvKey(uv)); + int keptSlot = slotRemap[payload.FaceMaterials[i / 3]]; + int pool = poolOfSlot != null && keptSlot >= 0 && keptSlot < poolOfSlot.Length ? poolOfSlot[keptSlot] : 0; + var key = (orig, welded, PackNormalKey(normal), PackUvKey(uv), pool); if (!keyToSplit.TryGetValue(key, out int split)) { split = positions.Count; @@ -931,6 +991,30 @@ private static bool NeedsRebuild(string reason) => SkeletonData? renderRig = null; if (newGlobal != null && frame is FrameObjectModel rebuiltModel) { + // The pools FIRST, and only as a plan: everything below writes into the model, and a push the + // pools cannot answer for has to be refused before any of it has. + PoolPlan? pools = PlanPools(rebuiltModel, outVerts, newGlobal, newIndexData, newMats, existingMats, + decoded.Lod, out string? blendReason); + if (pools == null) + { + skipReason = blendReason; + return null; + } + if (pools.SharedAcrossPools) + { + // A vertex on the seam between two face groups that read different pools. It used to be the + // modeller's job to cut the mesh there; the cut is mechanical, so it is made here — the same + // push again, with each corner keyed by the pool its face draws from. + if (poolOfSlot != null) + { + skipReason = SharedAcrossPoolsReason + ", and giving each its own copy did not separate " + + "them — split the mesh along that material boundary and push again"; + return null; + } + return TryApplyRebuild(node, payload, out skipReason, lod, + [.. pools.Groups.Select(g => (int)g.AssignedPoolIndex)]); + } + // The per-bone face ranges are ONE table on the model — there is no copy per level, and the // ranges it holds address LOD0's index buffer. Rebuilding it from a coarser level's indices // would point the physics splits at triangles of the wrong mesh, so a push into any other @@ -952,12 +1036,9 @@ private static bool NeedsRebuild(string reason) => for (int k = 0; k < 4; k++) renderWeights[(v * 4) + k] = outVerts[v].BoneWeights[k]; renderRig = SdsMeshLoader.RigOf(rebuiltModel); - if (!RemapBlendInfo(rebuiltModel, outVerts, newGlobal, newIndexData, newMats, existingMats, - decoded.Lod, out string? blendReason)) - { - skipReason = blendReason; - return null; - } + // Last, once nothing can refuse any more: the face ranges above were matched against the table + // as it shipped, and this is what moves the table. + CommitPools(rebuiltModel, pools); } byte[] newData = VertexCompressor.CompressBuffer( @@ -1491,62 +1572,144 @@ private static bool RebuildMeshSplits( } /// - /// Re-points a re-topologised skinned model's vertices at the remap pools it already ships with, and says - /// which pool each new face group draws from. Returns false with a reason when the pools cannot answer for - /// the geometry that came back. + /// What one level's remap pools have to be for the geometry that came back: the pools, which of them each + /// face group draws from, and where every entry of the old table has moved to. Worked out without + /// touching the model, so a push that turns out not to fit leaves it exactly as it was. + /// + private sealed class PoolPlan + { + internal int Level; + internal byte[] Sizes = []; + internal byte[] Remap = []; + internal FrameBlendInfo.SkinnedMaterialInfo[] Groups = []; + + /// The table before and after, pool by pool — an old entry keeps its pool and its place in + /// it, so its new index is the pool's new start plus the same offset. + internal int[] OldStart = []; + internal byte[] OldSizes = []; + internal int[] NewStart = []; + + /// A pool was made longer, or a new one opened: the table moved and everything that + /// describes it has to follow. + internal bool Grew; + + /// Some vertex is drawn by two face groups that read their bones from different pools. It + /// carries one set of ids, so it has to become two vertices before the plan can be carried out. + internal bool SharedAcrossPools; + + internal int Reindex(int oldIndex) + { + for (int p = 0; p < OldSizes.Length; p++) + { + if (oldIndex >= OldStart[p] && oldIndex < OldStart[p] + OldSizes[p]) + return NewStart[p] + (oldIndex - OldStart[p]); + } + return oldIndex; + } + } + + private const string SharedAcrossPoolsReason = + "a vertex is shared by two materials that read their bones from different pools"; + + /// + /// Re-points a skinned model's vertices at its remap pools and writes the pools back — the count-preserving + /// path's whole answer, where no vertex can be added. False with a reason when the pools cannot be made to + /// answer for the weights that came back; a vertex that would have to be split says so in a form the + /// caller turns into a rebuild. + /// + private static bool RemapBlendInfo( + FrameObjectModel model, Vertex[] vertices, byte[] globalOf, uint[] indices, + MaterialStruct[] newMats, MaterialStruct[] oldMats, int lod, out string? reason) + { + PoolPlan? plan = PlanPools(model, vertices, globalOf, indices, newMats, oldMats, lod, out reason); + if (plan == null) return false; + if (plan.SharedAcrossPools) + { + reason = SharedAcrossPoolsReason + " — the mesh has to be rebuilt to give each its own copy"; + return false; + } + CommitPools(model, plan); + return true; + } + + /// + /// Works out the remap pools a re-topologised (or re-weighted) level needs, and localizes the vertices' + /// bone ids against them. Null with a reason when no arrangement of pools the game is known to read can + /// answer for the geometry. + /// + /// The shipped pools are the starting point and are never reshuffled: a car does not put its whole rig in + /// one pool — shubert_38 ships 59 bones in pool 0 and 33 in pool 1 for 83 bones total — and replacing that + /// with one pool over every bone is what tore a repacked car apart. A face group that fits a shipped pool + /// keeps it, which leaves the skin channel byte-identical wherever nothing moved. + /// /// - /// The pools themselves are NOT rebuilt. A car does not put its whole rig in one pool — shubert_38 ships - /// 59 bones in pool 0 and 33 in pool 1 for 83 bones total (--probe-skinning) — and replacing that - /// with a single pool over every bone is what tore a repacked car apart: a draw can only reach so far into - /// the palette, so ids past the end came back as garbage transforms. Keeping the shipped pools also keeps - /// the skin channel byte-identical for every vertex that kept its material, which is the common case. + /// A face group whose bones no shipped pool holds together GROWS the pool that is missing fewest of them, + /// by appending — so every id already written against that pool still means the same bone — or, when no + /// pool has the room, opens a new one. What that asks of the rest of the file is measured + /// (--probe-remap-pools, 353 models, 650 levels, no exception): the skeleton's count per level, its + /// usage and reference arrays and its level masks are all derived from the pools and are derived again by + /// , and a split's BlendIndex is a position in level 0's + /// table and moves with the entry it names. The ceilings are the shipped ones: no pool anywhere holds more + /// than bones, no table more than entries. /// /// - private static bool RemapBlendInfo( + private static PoolPlan? PlanPools( FrameObjectModel model, Vertex[] vertices, byte[] globalOf, uint[] indices, MaterialStruct[] newMats, MaterialStruct[] oldMats, int lod, out string? reason) { reason = null; FrameBlendInfo blend; try { blend = model.GetBlendInfoObject(); } - catch (Exception) { reason = "the model's blend info cannot be read"; return false; } + catch (Exception) { reason = "the model's blend info cannot be read"; return null; } FrameBlendInfo.BoneIndexInfo[] lods = blend.BoneIndexInfos ?? []; - if (lods.Length == 0) { reason = "the model carries no remap pools"; return false; } + if (lods.Length == 0) { reason = "the model carries no remap pools"; return null; } // The edited level's own pools — each level has its own palette and its own face groups. int level = Math.Clamp(lod, 0, lods.Length - 1); FrameBlendInfo.BoneIndexInfo edited = lods[level]; - byte[] sizes = edited.BonesPerRemapPool ?? []; - byte[] remap = edited.BoneRemapIDs ?? []; + byte[] oldSizes = edited.BonesPerRemapPool ?? []; + byte[] oldRemap = edited.BoneRemapIDs ?? []; FrameBlendInfo.SkinnedMaterialInfo[] oldGroups = edited.SkinnedMaterialInfo ?? []; // Pool p is a run of sizes[p] ids in the flat remap table. - var poolStart = new int[sizes.Length]; + var oldStart = new int[oldSizes.Length]; int poolCount = 0, at = 0; - for (int p = 0; p < sizes.Length; p++) + for (int p = 0; p < oldSizes.Length; p++) { - poolStart[p] = at; - at += sizes[p]; - if (sizes[p] > 0) poolCount = p + 1; + oldStart[p] = at; + at += oldSizes[p]; + if (oldSizes[p] > 0) poolCount = p + 1; } - if (at > remap.Length) { reason = "the model's remap pools overrun its remap table"; return false; } - if (poolCount == 0) { reason = "the model carries no remap pools"; return false; } + if (at > oldRemap.Length) { reason = "the model's remap pools overrun its remap table"; return null; } + if (poolCount == 0) { reason = "the model carries no remap pools"; return null; } - var localOf = new Dictionary[poolCount]; - sizes = (byte[])sizes.Clone(); - for (int p = 0; p < poolCount; p++) + var pools = new List[oldSizes.Length]; + var localOf = new Dictionary[oldSizes.Length]; + for (int p = 0; p < oldSizes.Length; p++) { - var map = new Dictionary(sizes[p]); - for (int i = 0; i < sizes[p]; i++) map.TryAdd(remap[poolStart[p] + i], (byte)i); + pools[p] = [.. oldRemap.AsSpan(oldStart[p], oldSizes[p])]; + var map = new Dictionary(oldSizes[p]); + for (int i = 0; i < pools[p].Count; i++) map.TryAdd(pools[p][i], (byte)i); localOf[p] = map; } // A slot that kept its material keeps that material's pool — draw order is what ties a face group to - // a pool, so the shipped answer is the right one wherever it still applies. + // a pool, so the shipped answer is the right one wherever it still applies. A material drawn by two + // slots (279 shipped levels do that) is matched occurrence for occurrence. var groups = new FrameBlendInfo.SkinnedMaterialInfo[newMats.Length]; + var seen = new Dictionary(); for (int slot = 0; slot < newMats.Length; slot++) { - int was = Array.FindIndex(oldMats, m => m.MaterialHash == newMats[slot].MaterialHash); + ulong hash = newMats[slot].MaterialHash; + int nth = seen.GetValueOrDefault(hash); + seen[hash] = nth + 1; + int was = -1; + for (int old = 0, hit = 0; old < oldMats.Length; old++) + { + if (oldMats[old].MaterialHash != hash) continue; + was = old; + if (hit++ == nth) break; + } FrameBlendInfo.SkinnedMaterialInfo donor = was >= 0 && was < oldGroups.Length ? oldGroups[was] : new FrameBlendInfo.SkinnedMaterialInfo { AssignedPoolIndex = 0, NumWeightsPerVertex = 1 }; @@ -1558,10 +1721,10 @@ private static bool RemapBlendInfo( } // Which bones each slot actually draws, and how many influences its heaviest vertex carries. - var bonesOfSlot = new HashSet[newMats.Length]; + bool grew = false; for (int slot = 0; slot < newMats.Length; slot++) { - HashSet set = bonesOfSlot[slot] = []; + var set = new HashSet(); int influences = 1; int from = newMats[slot].StartIndex; int to = Math.Min(from + (newMats[slot].NumFaces * 3), indices.Length); @@ -1583,53 +1746,87 @@ private static bool RemapBlendInfo( // A slot whose bones its inherited pool cannot name (Blender moved faces between materials, or // the slot is new) takes any pool that can. - if (set.All(b => localOf[groups[slot].AssignedPoolIndex].ContainsKey(b))) continue; + int inherited = groups[slot].AssignedPoolIndex; + if (set.All(b => localOf[inherited].ContainsKey(b))) continue; int fit = -1; for (int p = 0; p < poolCount && fit < 0; p++) if (set.All(b => localOf[p].ContainsKey(b))) fit = p; if (fit >= 0) { groups[slot].AssignedPoolIndex = (byte)fit; continue; } - // No pool has them all — so GROW one. A pool is just "the bones this face group may name", and a - // vertex addresses it with a byte; the shipped data says the engine's real limit is per pool and - // not on the total, since cars run their totals to 108 entries while no single pool anywhere - // exceeds 60 (--probe-bullets). So the bones that are missing get appended to the pool that is - // missing fewest, and only a pool that would pass 60 is refused. - // - // This is what a cube weighted to a bonnet AND its deform bone needs: the two are not in one - // pool on any car, and until this existed the push either refused or — worse, before the guard — - // wrote an id past the end of the pool, which the editor drew correctly and the game placed - // somewhere else entirely. - // NOT YET. Growing a pool is more than lengthening the remap table: the SKELETON carries the same - // total — measured on 173 of 173 LODs, its blend-id count equals the sum of the pool sizes and - // its usage array is exactly that long — and what belongs in the new usage entries has not been - // measured. Writing the longer table alone leaves the two halves disagreeing, and that is worse - // than refusing: the editor reads the pools directly and shows the part in its right place while - // the game reads through the skeleton's mapping and puts it somewhere else. That was reported - // twice, and the second time it was this code that caused it. - // A model has dozens of groups and the refusal is about ONE material, so saying only that a pool - // cannot cover it leaves the modeller hunting blind. Name the material, its bones, and the pool - // each bone does sit in — that is the whole fix: drop the bones that sit apart, or split the - // faces onto a material whose bones share a pool. + // No pool has them all — so one is made to. A pool is just "the bones this face group may name"; + // the missing ones are appended to the pool that lacks fewest (its own first, on a tie), and a + // pool that would pass the shipped ceiling is left alone in favour of a new one. string material = MafiaMaterials.GetMaterialName(newMats[slot].MaterialHash) ?? $"0x{newMats[slot].MaterialHash:X16}"; - string[] boneNames = BoneNamesOf(model); - IEnumerable told = set.Order().Select(b => + if (set.Count > MaxBonesPerPool) { - string name = b < boneNames.Length ? boneNames[b] : $"bone{b}"; - string pools = string.Join("/", Enumerable.Range(0, poolCount) - .Where(p => localOf[p].ContainsKey(b))); - return pools.Length == 0 ? $"{name} (in no pool)" : $"{name} (pool {pools})"; - }); - reason = $"material '{material}' is weighted to bones that no single remap pool of the model " - + "covers: " + string.Join(", ", told) + ". " - + "A pool CAN be made longer — no shipped pool exceeds " + MaxBonesPerPool + " and the " - + "totals run past a hundred — but the rig stores that same total in its own blend-id count " - + "and usage array, and what goes in the new entries has not been measured yet. Growing one " - + "half alone gives a car that looks right in the editor and lands the part somewhere else " - + "in game, so it is refused instead. For now: give those faces ONE vertex group, or move " - + "them onto a material whose bones already sit in one pool."; - return false; + reason = $"material '{material}' is weighted to {set.Count} different bones, and one face group " + + $"can name at most {MaxBonesPerPool} (no shipped model goes past it). Split those faces " + + "over two material slots — the same material may be used by both — so that each half " + + "stays under the limit."; + return null; + } + + int best = -1, bestMissing = int.MaxValue; + for (int step = 0; step <= poolCount; step++) + { + int p = step == 0 ? inherited : step - 1; + if (step > 0 && p == inherited) continue; + int missing = set.Count(b => !localOf[p].ContainsKey(b)); + if (pools[p].Count + missing > MaxBonesPerPool) continue; + if (missing < bestMissing) { best = p; bestMissing = missing; } + } + if (best < 0) + { + if (poolCount >= pools.Length) + { + string[] boneNames = BoneNamesOf(model); + reason = $"material '{material}' needs bones that no remap pool of the model has room " + + $"for (" + string.Join(", ", set.Order().Select(b => b < boneNames.Length + ? boneNames[b] : $"bone{b}")) + $"): every pool would pass {MaxBonesPerPool} " + + $"bones and all {pools.Length} pools are in use. Weight those faces to fewer bones."; + return null; + } + best = poolCount++; + } + foreach (byte bone in set.Order()) + { + if (localOf[best].ContainsKey(bone)) continue; + localOf[best][bone] = (byte)pools[best].Count; + pools[best].Add(bone); + } + groups[slot].AssignedPoolIndex = (byte)best; + grew = true; + } + + var sizes = new byte[oldSizes.Length]; + var newStart = new int[oldSizes.Length]; + var remap = new List(oldRemap.Length + 16); + for (int p = 0; p < pools.Length; p++) + { + newStart[p] = remap.Count; + sizes[p] = (byte)pools[p].Count; + remap.AddRange(pools[p]); + } + if (remap.Count > MaxBlendIds) + { + reason = $"this push needs a remap table of {remap.Count} entries, and the longest any shipped " + + $"model carries is {MaxBlendIds} — past that nothing is known about what the game does. " + + "Weight the new geometry to bones its material already uses."; + return null; } + var plan = new PoolPlan + { + Level = level, + Sizes = sizes, + Remap = [.. remap], + Groups = groups, + OldStart = oldStart, + OldSizes = oldSizes, + NewStart = newStart, + Grew = grew, + }; + // A vertex carries one set of ids, so every group drawing it must read them against the same pool. var poolOfVertex = new int[vertices.Length]; Array.Fill(poolOfVertex, -1); @@ -1644,27 +1841,34 @@ private static bool RemapBlendInfo( if (v < 0 || v >= vertices.Length) continue; if (poolOfVertex[v] >= 0 && poolOfVertex[v] != pool) { - reason = "a vertex is shared by two materials that read their bones from different " - + "pools — split the mesh along that material boundary and push again"; - return false; + plan.SharedAcrossPools = true; + return plan; } poolOfVertex[v] = pool; } } // Ids back to pool-local. A zero-weight slot keeps the donor's byte: it names nothing, and rewriting - // it would move bytes for no reason. + // it would move bytes for no reason — UNLESS that byte points past the end of this vertex's pool. It + // does when the vertex took its unused ids from a neighbour drawn through a bigger pool (a new corner + // copies the nearest old vertex before its own weights land), and although the weight is zero the + // id is still read: a skin with any id past the table does not resolve at all, here or on the next + // pull. Such a slot is pointed at the pool's first entry, which is always there. for (int v = 0; v < vertices.Length; v++) { int pool = poolOfVertex[v]; if (pool < 0) continue; // nothing draws this vertex for (int k = 0; k < 4; k++) { - if (vertices[v].BoneWeights[k] <= 0f) continue; + if (vertices[v].BoneWeights[k] <= 0f) + { + if (vertices[v].BoneIDs[k] >= sizes[pool]) vertices[v].BoneIDs[k] = 0; + continue; + } if (!localOf[pool].TryGetValue(globalOf[(v * 4) + k], out byte local)) { reason = "a vertex came back weighted to a bone its material's remap pool does not name"; - return false; + return null; } vertices[v].BoneIDs[k] = local; } @@ -1686,24 +1890,54 @@ private static bool RemapBlendInfo( for (int k = 0; k < 4; k++) { if (vertices[v].BoneWeights[k] <= 0f) continue; - int slot = poolStart[pool] + vertices[v].BoneIDs[k]; - if (slot < remap.Length && remap[slot] == globalOf[(v * 4) + k]) continue; + int slot = newStart[pool] + vertices[v].BoneIDs[k]; + if (vertices[v].BoneIDs[k] < sizes[pool] && slot < plan.Remap.Length + && plan.Remap[slot] == globalOf[(v * 4) + k]) + { + continue; + } reason = "the rebuilt skin does not read back — a vertex's bone id resolves to the wrong " + "bone through its own remap pool. Nothing was written; the mesh is as it was."; - return false; + return null; } } + return plan; + } + + /// + /// Writes a plan into the model: the edited level's pools and face groups, and — when the table moved — + /// everything else in the file that describes it. + /// + private static void CommitPools(FrameObjectModel model, PoolPlan plan) + { + FrameBlendInfo blend = model.GetBlendInfoObject(); + FrameBlendInfo.BoneIndexInfo[] lods = blend.BoneIndexInfos ?? []; + // The edited level only: the other levels keep their own vertex buffers and the pools that go with // them. - lods[level] = new FrameBlendInfo.BoneIndexInfo + lods[plan.Level] = new FrameBlendInfo.BoneIndexInfo { - BonesPerRemapPool = sizes, - BoneRemapIDs = remap, - SkinnedMaterialInfo = groups, + BonesPerRemapPool = plan.Sizes, + BoneRemapIDs = plan.Remap, + SkinnedMaterialInfo = plan.Groups, }; blend.BoneIndexInfos = lods; - return true; + if (!plan.Grew) return; + + // A split names its bone by POSITION in level 0's table. A pool that grew pushed every later pool + // along, so the positions are moved with it — same pool, same place in the pool, same bone. + if (plan.Level == 0) + { + foreach (FrameObjectModel.WeightedByMeshSplit split in model.BlendMeshSplits ?? []) + { + split.BlendIndex = (ushort)plan.Reindex(split.BlendIndex); + } + } + + // …and the skeleton's own account of the table: the count per level, the usage and reference arrays, + // the level masks. All derived from the pools, so derived again. + Frames.BlendPoolTables.Sync(model); } /// @@ -1933,6 +2167,13 @@ private static void DropEmptyPieces(FrameObjectModel model) /// private const int MaxBonesPerPool = 60; + /// + /// The most entries one level's remap table may hold — all its pools together. The longest shipped table + /// is 115 entries (--probe-remap-pools, 353 skinned models); the format would hold 255, since the + /// skeleton files table positions in a byte, but nothing is known about the game past what it ships. + /// + private const int MaxBlendIds = 115; + /// How many bones the model's rig has, or 0 when it cannot be read. private static int BoneCountOf(FrameObjectModel model) { diff --git a/src/Illusion.Assets/Bridge/BridgeMeshExporter.cs b/src/Illusion.Assets/Bridge/BridgeMeshExporter.cs index 6b563ac..99fdfb1 100644 --- a/src/Illusion.Assets/Bridge/BridgeMeshExporter.cs +++ b/src/Illusion.Assets/Bridge/BridgeMeshExporter.cs @@ -117,7 +117,12 @@ public static class BridgeMeshExporter return new MeshObjectPayload { - Id = MakeId(frame, document), + // The LEVEL is part of the identity. Both levels of a car are one frame, so sent together under + // the frame's id alone they were one object to the session: the push of the near level came back + // into the far one (whose pools do not hold its bones) and the other arrived as a brand-new mesh. + Id = decoded.Lod > 0 + ? MakeId(frame, document) + "|lod" + decoded.Lod.ToString(System.Globalization.CultureInfo.InvariantCulture) + : MakeId(frame, document), BoneIndices = boneIndices, BoneWeights = boneWeights, SkeletonId = skeletonId, diff --git a/src/Illusion.Assets/Bridge/SkinState.cs b/src/Illusion.Assets/Bridge/SkinState.cs new file mode 100644 index 0000000..6aa128c --- /dev/null +++ b/src/Illusion.Assets/Bridge/SkinState.cs @@ -0,0 +1,125 @@ +using Illusion.Formats.Frames.ObjectTypes; +using Illusion.Formats.Frames.Resources; +using Illusion.Formats.Mathematics; + +namespace Illusion.Assets.Bridge; + +/// +/// Everything about a skinned model that a geometry push changes BESIDE its buffers: the remap pools and the +/// face groups that draw from them, the per-bone face ranges with their pieces and hit boxes, and the +/// skeleton's own account of the pools with its per-bone boxes. +/// +/// A push rewrites these, and they are not derived from the buffers alone — which pool a face group draws +/// from is a choice, and the vertex bytes only mean anything read through that choice. Undoing the buffers +/// while leaving them is how an undone push kept the old vertices and read them through the new pools: on a +/// car whose body had moved to another pool, one bone id in six came out as another bone. So a push carries +/// the state before and after, and puts back whichever belongs with the buffers it puts back. +/// +/// A snapshot is never handed to the model itself: applying it installs copies, because the next push edits +/// the split table in place and a redo must still find this one as it was taken. +/// +internal sealed class SkinState +{ + private readonly FrameBlendInfo.BoneIndexInfo[]? _levels; + private readonly FrameObjectModel.WeightedByMeshSplit[] _splits; + private readonly FrameObjectModel.HitBoxInfo[] _hitBoxes; + private readonly (int PhysSplitSize, int HitBoxSize, short NPhysSplits) _counters; + + private readonly bool _hasSkeleton; + private readonly int[] _levelCounts = []; + private readonly int _blendIds; + private readonly byte[] _levelMasks = []; + private readonly FrameSkeleton.MappingForBlendingInfo[] _mappings = []; + + private SkinState(FrameObjectModel model) + { + try { _levels = Copy(model.GetBlendInfoObject().BoneIndexInfos ?? []); } + catch (Exception) { _levels = null; } // no blend info to read: nothing of it to put back either + + _splits = Copy(model.BlendMeshSplits ?? []); + _hitBoxes = [.. (model.HitBoxes ?? []).Select(box => new FrameObjectModel.HitBoxInfo(box))]; + _counters = model.StoredSplitCounters; + + try + { + FrameSkeleton skeleton = model.GetSkeletonObject(); + _levelCounts = [.. skeleton.LodRemapIDCount ?? []]; + _blendIds = skeleton.NumBlendIDs; + _levelMasks = [.. skeleton.BoneLODUsage ?? []]; + _mappings = Copy(skeleton.MappingForBlendingInfos ?? []); + _hasSkeleton = true; + } + catch (Exception) + { + _hasSkeleton = false; + } + } + + /// The model's skin state as it stands. + public static SkinState Take(FrameObjectModel model) + { + ArgumentNullException.ThrowIfNull(model); + return new SkinState(model); + } + + /// Puts the model's skin state back to what it was when this was taken. + public void Apply(FrameObjectModel model) + { + ArgumentNullException.ThrowIfNull(model); + if (_levels != null) + { + try { model.GetBlendInfoObject().BoneIndexInfos = Copy(_levels); } + catch (Exception) { /* the blend info it was read from is gone — nothing to write into */ } + } + + model.BlendMeshSplits = Copy(_splits); + model.HitBoxes = [.. _hitBoxes.Select(box => new FrameObjectModel.HitBoxInfo(box))]; + // The counters as they were stored, not recomputed: this is "as it was", and a shipped model is + // entitled to whatever it shipped with. + model.StoredSplitCounters = _counters; + + if (!_hasSkeleton) return; + try + { + FrameSkeleton skeleton = model.GetSkeletonObject(); + skeleton.LodRemapIDCount = [.. _levelCounts]; + skeleton.NumBlendIDs = _blendIds; + skeleton.BoneLODUsage = [.. _levelMasks]; + skeleton.MappingForBlendingInfos = Copy(_mappings); + } + catch (Exception) + { + // as above + } + } + + private static FrameBlendInfo.BoneIndexInfo[] Copy(FrameBlendInfo.BoneIndexInfo[] levels) => + [.. levels.Select(level => new FrameBlendInfo.BoneIndexInfo + { + BonesPerRemapPool = [.. level.BonesPerRemapPool ?? []], + BoneRemapIDs = [.. level.BoneRemapIDs ?? []], + SkinnedMaterialInfo = [.. level.SkinnedMaterialInfo ?? []], + })]; + + // The types' own copy constructors copy one level down and share everything below it; the face-range + // rebuild replaces a piece's bursts in place, so the copy has to go all the way. + private static FrameObjectModel.WeightedByMeshSplit[] Copy(FrameObjectModel.WeightedByMeshSplit[] splits) => + [.. splits.Select(split => new FrameObjectModel.WeightedByMeshSplit(split) + { + Data = [.. (split.Data ?? []).Select(piece => new FrameObjectModel.BlendMeshSplitInfo(piece) + { + Data = [.. (piece.Data ?? []).Select(burst => new FrameObjectModel.MiniMaterialBurst(burst) + { + Data = [.. (burst.Data ?? []).Select(range => new FrameObjectModel.FacesBurst(range))], + })], + })], + })]; + + private static FrameSkeleton.MappingForBlendingInfo[] Copy(FrameSkeleton.MappingForBlendingInfo[] mappings) => + [.. mappings.Select(map => new FrameSkeleton.MappingForBlendingInfo + { + Bounds = [.. map.Bounds ?? Array.Empty()], + RefToUsageArray = [.. map.RefToUsageArray ?? []], + UsageArray = [.. map.UsageArray ?? []], + })]; +} diff --git a/src/Illusion.Assets/Frames/BlendPoolTables.cs b/src/Illusion.Assets/Frames/BlendPoolTables.cs new file mode 100644 index 0000000..17061e8 --- /dev/null +++ b/src/Illusion.Assets/Frames/BlendPoolTables.cs @@ -0,0 +1,135 @@ +using Illusion.Formats.Frames.ObjectTypes; +using Illusion.Formats.Frames.Resources; + +namespace Illusion.Assets.Frames; + +/// +/// The SKELETON'S account of a skinned model's remap pools, rebuilt from the pools themselves. +/// +/// +/// The pools live in the blend info: per level, a flat table of bone ids cut into slices, and a vertex names +/// its bones by position in the slice its face group draws from. The skeleton describes the same table a +/// second time, and a pool cannot be made longer without that description following — the editor reads the +/// pools directly and would draw the part in place, while the game reads through the skeleton and would not. +/// Measured over all 353 skinned models of the install, 650 levels, with no exception +/// (--probe-remap-pools): +/// +/// +/// LodRemapIDCount[level] is the length of that level's table. +/// NumBlendIDs is the length of the longest table (always level 0's). +/// UsageArray is the table's own INDICES, sorted by the bone each entry names — a bone that +/// sits in two pools has its two entries side by side. +/// RefToUsageArray[bone] is where that bone's run ENDS in the usage array (a running total), so +/// a bone the level never names repeats its predecessor's value. +/// BoneLODUsage[bone] is a bit per level whose table names the bone. +/// +/// Every one of them is derived data, so it is derived again rather than patched. +/// +public static class BlendPoolTables +{ + /// + /// The usage and reference arrays one level's table implies. is the flat table + /// (already cut to the pools' total), the rig's bone count. + /// + public static (byte[] Usage, byte[] Refs) Describe(ReadOnlySpan remap, int boneCount) + { + // A counting sort by bone, stable in table order — exactly the shape of the two arrays. + var perBone = new int[Math.Max(boneCount, 1)]; + foreach (byte bone in remap) + { + if (bone < perBone.Length) perBone[bone]++; + } + + var refs = new byte[boneCount]; + var next = new int[perBone.Length]; + int running = 0; + for (int bone = 0; bone < boneCount; bone++) + { + next[bone] = running; + running += perBone[bone]; + refs[bone] = (byte)Math.Min(running, byte.MaxValue); + } + + var usage = new byte[remap.Length]; + for (int i = 0; i < remap.Length; i++) + { + byte bone = remap[i]; + if (bone >= boneCount) continue; // a table naming a bone the rig lacks has no place to file it + usage[next[bone]++] = (byte)i; + } + return (usage, refs); + } + + /// The level mask BoneLODUsage holds for each bone: bit n set when level n's table names it. + public static byte[] LevelMasks(FrameBlendInfo.BoneIndexInfo[] levels, int boneCount) + { + var masks = new byte[boneCount]; + for (int level = 0; level < levels.Length && level < 8; level++) + { + foreach (byte bone in Table(levels[level])) + { + if (bone < boneCount) masks[bone] |= (byte)(1 << level); + } + } + return masks; + } + + /// A level's flat table, cut to what its pools add up to. + public static ReadOnlySpan Table(FrameBlendInfo.BoneIndexInfo level) + { + byte[] remap = level.BoneRemapIDs ?? []; + int total = 0; + foreach (byte size in level.BonesPerRemapPool ?? []) total += size; + return remap.AsSpan(0, Math.Min(total, remap.Length)); + } + + /// + /// Rewrites the skeleton's counts, usage arrays, references and level masks from the model's pools as + /// they stand. A no-op in bytes for a model nobody has edited. False when the rig cannot be read or its + /// shape does not line up with the pools (a level the skeleton has no mapping for) — nothing is written. + /// + public static bool Sync(FrameObjectModel model) + { + ArgumentNullException.ThrowIfNull(model); + + FrameSkeleton skeleton; + FrameBlendInfo blend; + try + { + skeleton = model.GetSkeletonObject(); + blend = model.GetBlendInfoObject(); + } + catch (Exception) + { + return false; + } + + FrameBlendInfo.BoneIndexInfo[] levels = blend.BoneIndexInfos ?? []; + int bones = skeleton.BoneNames?.Length ?? 0; + FrameSkeleton.MappingForBlendingInfo[] maps = skeleton.MappingForBlendingInfos ?? []; + if (bones == 0 || levels.Length == 0 || maps.Length < levels.Length) return false; + + var counts = new int[levels.Length]; + int longest = 0; + for (int level = 0; level < levels.Length; level++) + { + ReadOnlySpan table = Table(levels[level]); + // The usage array holds table INDICES in a byte and the references running totals in one. + if (table.Length > byte.MaxValue) return false; + counts[level] = table.Length; + longest = Math.Max(longest, table.Length); + + (byte[] usage, byte[] refs) = Describe(table, bones); + FrameSkeleton.MappingForBlendingInfo map = maps[level]; + map.UsageArray = usage; + map.RefToUsageArray = refs; + maps[level] = map; + } + + skeleton.MappingForBlendingInfos = maps; + skeleton.LodRemapIDCount = counts; + skeleton.NumBlendIDs = longest; + skeleton.BoneLODUsage = LevelMasks(levels, bones); + return true; + } +} diff --git a/src/Illusion.Assets/Frames/FrameDuplicator.cs b/src/Illusion.Assets/Frames/FrameDuplicator.cs index d15d67c..4cd684f 100644 --- a/src/Illusion.Assets/Frames/FrameDuplicator.cs +++ b/src/Illusion.Assets/Frames/FrameDuplicator.cs @@ -213,6 +213,125 @@ bool CloneBuffers() return added; } + /// A mesh frame given a geometry block and buffers of its own, with what undo needs to take them + /// away again. + public sealed class OwnedGeometry + { + internal FrameResource Resource = null!; + internal SceneDocumentAdapter Adapter = null!; + internal FrameObjectSingleMesh Mesh = null!; + internal FrameGeometry Before = null!; + internal FrameGeometry After = null!; + internal List VertexBuffers = new(); + internal List IndexBuffers = new(); + internal Dictionary IndexCopies = new(); + + /// The copy that stands, in the mesh's own block, for an index buffer of the shared one. + public IndexBuffer? CopyOf(IndexBuffer shared) => shared == null ? null : IndexCopies.GetValueOrDefault(shared.Hash); + + /// Puts the mesh back on the block it shared (undo). + public void Revert() + { + Mesh.Geometry = Before; + Mesh.ReplaceRef(FrameEntryRefTypes.Geometry, Before.RefID); + Resource.FrameGeometries.Remove(After.RefID); + foreach (VertexBuffer vb in VertexBuffers) Resource.VertexBuffers.Remove(vb.Hash); + foreach (IndexBuffer ib in IndexBuffers) Resource.IndexBuffers.Remove(ib.Hash); + } + + /// Puts the mesh on its own block again (redo). Re-registers what a save-time sanitize may + /// have pruned while it was away. + public void Apply() + { + if (!Resource.FrameGeometries.ContainsKey(After.RefID)) Resource.FrameGeometries.Add(After.RefID, After); + foreach (VertexBuffer vb in VertexBuffers) + { + Resource.VertexBuffers.TryAddToPool(vb); + Adapter.MarkVertexBufferDirty(vb.Hash); + } + foreach (IndexBuffer ib in IndexBuffers) + { + Resource.IndexBuffers.TryAddToPool(ib); + Adapter.MarkIndexBufferDirty(ib.Hash); + } + Mesh.Geometry = After; + Mesh.ReplaceRef(FrameEntryRefTypes.Geometry, After.RefID); + } + } + + /// + /// Gives a mesh frame that draws a geometry block other frames draw too a block of its OWN: a copy of the + /// block over copies of its vertex and index buffers - what a duplicate of the object gets, without the + /// duplicate. The frame looks exactly as before; a change to its geometry now changes it alone. Applied + /// when returned. Null with a reason when it cannot be done (missing buffers, full pools). + /// + public static OwnedGeometry? TryOwnGeometry(ISceneDocument document, IFrameNode node, out string? reason) + { + reason = null; + if (document is not SceneDocumentAdapter adapter || node is not FrameNodeAdapter { Frame: FrameObjectSingleMesh mesh }) + { + reason = "only a mesh has geometry to copy"; + return null; + } + FrameResource resource = adapter.Frame; + if (mesh.Geometry?.LOD is not { Length: > 0 }) + { + reason = "the mesh has no geometry"; + return null; + } + + string unique = Guid.NewGuid().ToString("N")[..8]; + string BufferName(ulong sourceHash, string kind) => $"{mesh.Name.String}_{kind}{sourceHash:x8}_{unique}"; + var owned = new OwnedGeometry { Resource = resource, Adapter = adapter, Mesh = mesh, Before = mesh.Geometry }; + var vertexCopies = new HashSet(); + bool room = true; + foreach (FrameLOD lod in mesh.Geometry.LOD) + { + VertexBuffer? vb = resource.VertexBuffers.GetBuffer(lod.VertexBufferRef.Hash); + IndexBuffer? ib = resource.IndexBuffers.GetBuffer(lod.IndexBufferRef.Hash); + if (vb == null || ib == null) + { + reason = "a level of the mesh has no buffers in this archive"; + room = false; + break; + } + if (vertexCopies.Add(vb.Hash)) + { + var copy = new VertexBuffer(Fnv64.Hash(BufferName(vb.Hash, "vb"))) { Data = (byte[])vb.Data.Clone() }; + if (!resource.VertexBuffers.TryAddToPool(copy)) { room = false; break; } + owned.VertexBuffers.Add(copy); + } + if (!owned.IndexCopies.ContainsKey(ib.Hash)) + { + var copy = new IndexBuffer(Fnv64.Hash(BufferName(ib.Hash, "ib"))); + copy.SetFormat(ib.IndexFormat); + copy.SetData((uint[])ib.GetData().Clone()); + if (!resource.IndexBuffers.TryAddToPool(copy)) { room = false; break; } + owned.IndexCopies[ib.Hash] = copy; + owned.IndexBuffers.Add(copy); + } + } + if (!room) + { + foreach (VertexBuffer vb in owned.VertexBuffers) resource.VertexBuffers.Remove(vb.Hash); + foreach (IndexBuffer ib in owned.IndexBuffers) resource.IndexBuffers.Remove(ib.Hash); + reason ??= "the archive has no buffer pool to copy the geometry into"; + return null; + } + + FrameGeometry geometry = resource.ConstructFrameAssetOfType(); + geometry.CopyFrom(mesh.Geometry); + foreach (FrameLOD lod in geometry.LOD) + { + ulong vertices = lod.VertexBufferRef.Hash, indices = lod.IndexBufferRef.Hash; + lod.VertexBufferRef = new HashName(BufferName(vertices, "vb")); + lod.IndexBufferRef = new HashName(BufferName(indices, "ib")); + } + owned.After = geometry; + owned.Apply(); + return owned; + } + // "_copy", then "_copy2", … — the frame name table is keyed by name, so a copy must not // collide with any existing object. private static string UniqueName(FrameResource resource, string sourceName) @@ -226,7 +345,7 @@ private static string UniqueName(FrameResource resource, string sourceName) } } - private static FrameEntry? ResolveRef(FrameResource resource, FrameObjectBase source, FrameEntryRefTypes slot) + internal static FrameEntry? ResolveRef(FrameResource resource, FrameObjectBase source, FrameEntryRefTypes slot) { if (!source.Refs.TryGetValue(slot, out int id)) return null; if (resource.FrameScenes.TryGetValue(id, out FrameHeaderScene? scene)) return scene; @@ -236,7 +355,7 @@ private static string UniqueName(FrameResource resource, string sourceName) // Writes both parent slots the way the loader/reparenter do: SetParent maintains the frame-side // runtime links; scene folders hold their members in a separate list the setter does not touch. - private static void LinkParents(FrameObjectSingleMesh clone, FrameEntry? parent1, FrameEntry? parent2) + internal static void LinkParents(FrameObjectBase clone, FrameEntry? parent1, FrameEntry? parent2) { clone.SetParent(ParentInfo.ParentType.ParentIndex1, parent1); clone.SetParent(ParentInfo.ParentType.ParentIndex2, parent2); diff --git a/src/Illusion.Assets/Illusion.Assets.csproj b/src/Illusion.Assets/Illusion.Assets.csproj index 7207fa8..fc60df6 100644 --- a/src/Illusion.Assets/Illusion.Assets.csproj +++ b/src/Illusion.Assets/Illusion.Assets.csproj @@ -12,6 +12,12 @@ + + + + + diff --git a/src/Illusion.Assets/Sds/TriangleHider.cs b/src/Illusion.Assets/Sds/TriangleHider.cs new file mode 100644 index 0000000..80bb0c4 --- /dev/null +++ b/src/Illusion.Assets/Sds/TriangleHider.cs @@ -0,0 +1,248 @@ +using System.Numerics; +using Illusion.Formats.Frames.ObjectTypes; +using Illusion.Formats.Frames.Resources; +using Illusion.Formats.Geometry; + +namespace Illusion.Assets.Sds; + +/// +/// Takes triangles out of a mesh without rebuilding it: a triangle is "hidden" by pointing its three indices +/// at one vertex, which the rasterizer draws as nothing. +/// +/// +/// This is how an opening is cut into a STOCK mesh — the painted door of a building one wants to walk +/// through. A rebuild through Blender cannot be trusted with that: a stock facade carries channels Blender +/// never sees (the shadow/occlusion-map UVs behind ShadowTexture), and vertices re-created from a +/// Blender mesh get wrong values for them — the whole facade comes back with black patches. Here no vertex +/// is touched and none is added or removed: the vertex buffer, the material ranges and every index outside +/// the hidden triangles stay exactly as shipped, on every level of detail. +/// +/// +public static class TriangleHider +{ + /// One triangle found: its level of detail, the material of its slot and its corners in world space. + public sealed record Triangle(int Lod, string Material, Vector3 A, Vector3 B, Vector3 C); + + /// One index buffer as it is and as it would be with the triangles hidden. Nothing is applied. + public sealed record Change(IndexBuffer Buffer, uint[] Before, uint[] After); + + /// What a box takes out of a mesh. + public sealed record Plan(IReadOnlyList Triangles, IReadOnlyList Changes); + + /// + /// The triangles of whose three corners all lie inside the world-space box, on every + /// level of detail, and the index buffers with them hidden. A triangle already hidden is not found again. + /// + /// The matrix the mesh is drawn at. + /// Only triangles of slots whose material name contains this, or null for any. + public static Plan Find(FrameObjectSingleMesh mesh, Matrix4x4 world, Vector3 min, Vector3 max, string? material = null) + { + ArgumentNullException.ThrowIfNull(mesh); + const float Slack = 1e-3f; + Vector3 lo = Vector3.Min(min, max) - new Vector3(Slack); + Vector3 hi = Vector3.Max(min, max) + new Vector3(Slack); + bool Inside(Vector3 p) => p.X >= lo.X && p.X <= hi.X && p.Y >= lo.Y && p.Y <= hi.Y && p.Z >= lo.Z && p.Z <= hi.Z; + + var triangles = new List(); + // By buffer, so two levels drawing from one buffer are edited as one. + var working = new Dictionary(); + int levels = mesh.Geometry?.LOD?.Length ?? 0; + List? slotsByLod = mesh.Material?.Materials; + for (int lod = 0; lod < levels; lod++) + { + DecodedMesh? decoded = SdsMeshLoader.DecodeLod(mesh, lod); + IndexBuffer? buffer = mesh.GetIndexBuffer(lod); + if (decoded == null || buffer == null || slotsByLod == null || lod >= slotsByLod.Count) continue; + if (!working.TryGetValue(buffer.Hash, out (IndexBuffer Buffer, uint[] Before, uint[] After) entry)) + { + uint[] before = buffer.GetData(); + working[buffer.Hash] = entry = (buffer, before, (uint[])before.Clone()); + } + + uint[] indices = entry.After; + Vector3[] positions = decoded.Positions; + foreach (MaterialStruct slot in slotsByLod[lod]) + { + string name = slot.MaterialName ?? ""; + if (material != null && !name.Contains(material, StringComparison.OrdinalIgnoreCase)) continue; + int end = Math.Min(slot.StartIndex + (slot.NumFaces * 3), indices.Length); + for (int i = slot.StartIndex; i + 2 < end; i += 3) + { + uint a = indices[i], b = indices[i + 1], c = indices[i + 2]; + if (a == b && b == c) continue; + if (a >= positions.Length || b >= positions.Length || c >= positions.Length) continue; + Vector3 pa = Vector3.Transform(positions[a], world); + Vector3 pb = Vector3.Transform(positions[b], world); + Vector3 pc = Vector3.Transform(positions[c], world); + if (!Inside(pa) || !Inside(pb) || !Inside(pc)) continue; + triangles.Add(new Triangle(lod, name, pa, pb, pc)); + indices[i + 1] = a; + indices[i + 2] = a; + } + } + } + + List changes = [.. working.Values + .Where(w => !w.Before.AsSpan().SequenceEqual(w.After)) + .Select(w => new Change(w.Buffer, w.Before, w.After))]; + return new Plan(triangles, changes); + } + + /// A triangle of the mesh's FIRST level of detail - the one drawn up close, and so the one a click + /// lands on - named by where its three indices stand in that level's index buffer. + public sealed record Picked(int Index, float Distance, Triangle Triangle); + + /// + /// The triangle of the first level of detail a world-space ray meets first, from either side, or null when + /// it meets none. A triangle already hidden is not met. + /// + public static Picked? Pick(FrameObjectSingleMesh mesh, Matrix4x4 world, Vector3 origin, Vector3 direction) + { + ArgumentNullException.ThrowIfNull(mesh); + DecodedMesh? decoded = (mesh.Geometry?.LOD?.Length ?? 0) > 0 ? SdsMeshLoader.DecodeLod(mesh, 0) : null; + IndexBuffer? buffer = decoded == null ? null : mesh.GetIndexBuffer(0); + if (decoded == null || buffer == null || mesh.Material?.Materials is not { Count: > 0 } slotsByLod) return null; + + uint[] indices = buffer.GetData(); + Vector3[] positions = decoded.Positions; + Picked? best = null; + foreach (MaterialStruct slot in slotsByLod[0]) + { + int end = Math.Min(slot.StartIndex + (slot.NumFaces * 3), indices.Length); + for (int i = slot.StartIndex; i + 2 < end; i += 3) + { + uint a = indices[i], b = indices[i + 1], c = indices[i + 2]; + if ((a == b && b == c) || a >= positions.Length || b >= positions.Length || c >= positions.Length) continue; + Vector3 pa = Vector3.Transform(positions[a], world); + Vector3 pb = Vector3.Transform(positions[b], world); + Vector3 pc = Vector3.Transform(positions[c], world); + if (RayMeets(origin, direction, pa, pb, pc, out float t) && (best == null || t < best.Distance)) + { + best = new Picked(i, t, new Triangle(0, slot.MaterialName ?? "", pa, pb, pc)); + } + } + } + return best; + } + + /// + /// What hiding triangles picked one by one takes out of a mesh: those triangles of the first level of detail + /// (), and on every other level the triangles that LIE ON them - their corners, the + /// middles of their edges and their own middle within metres of a picked triangle. A level cut coarser than the first has + /// triangles that reach past what was picked; those stay, and the opening closes at the distance that level + /// is drawn from - the count per level says so before anything is hidden. + /// + public static Plan FindPicked(FrameObjectSingleMesh mesh, Matrix4x4 world, IReadOnlyCollection picked, float near = 0.05f) + { + ArgumentNullException.ThrowIfNull(mesh); + ArgumentNullException.ThrowIfNull(picked); + var triangles = new List(); + var working = new Dictionary(); + int levels = mesh.Geometry?.LOD?.Length ?? 0; + List? slotsByLod = mesh.Material?.Materials; + var chosen = new List(); + Vector3 lo = new(float.MaxValue), hi = new(float.MinValue); + for (int lod = 0; lod < levels && picked.Count > 0; lod++) + { + DecodedMesh? decoded = SdsMeshLoader.DecodeLod(mesh, lod); + IndexBuffer? buffer = mesh.GetIndexBuffer(lod); + if (decoded == null || buffer == null || slotsByLod == null || lod >= slotsByLod.Count) continue; + if (!working.TryGetValue(buffer.Hash, out (IndexBuffer Buffer, uint[] Before, uint[] After) entry)) + { + uint[] before = buffer.GetData(); + working[buffer.Hash] = entry = (buffer, before, (uint[])before.Clone()); + } + + uint[] indices = entry.After; + Vector3[] positions = decoded.Positions; + foreach (MaterialStruct slot in slotsByLod[lod]) + { + int end = Math.Min(slot.StartIndex + (slot.NumFaces * 3), indices.Length); + for (int i = slot.StartIndex; i + 2 < end; i += 3) + { + if (lod == 0 && !picked.Contains(i)) continue; + uint a = indices[i], b = indices[i + 1], c = indices[i + 2]; + if ((a == b && b == c) || a >= positions.Length || b >= positions.Length || c >= positions.Length) continue; + Vector3 pa = Vector3.Transform(positions[a], world); + Vector3 pb = Vector3.Transform(positions[b], world); + Vector3 pc = Vector3.Transform(positions[c], world); + var triangle = new Triangle(lod, slot.MaterialName ?? "", pa, pb, pc); + if (lod == 0) + { + chosen.Add(triangle); + lo = Vector3.Min(lo, Vector3.Min(pa, Vector3.Min(pb, pc))); + hi = Vector3.Max(hi, Vector3.Max(pa, Vector3.Max(pb, pc))); + } + else if (!Covered(pa, pb, pc)) + { + continue; + } + triangles.Add(triangle); + indices[i + 1] = a; + indices[i + 2] = a; + } + } + } + + // Corners on the picks do not make a triangle lie on them: one spanning a door from frame to frame has + // all three on the frame. Its middle and the middles of its edges are asked too. + bool Covered(Vector3 a, Vector3 b, Vector3 c) => + LiesOn(a) && LiesOn(b) && LiesOn(c) && LiesOn((a + b + c) / 3f) && LiesOn((a + b) * 0.5f) && LiesOn((b + c) * 0.5f) && LiesOn((c + a) * 0.5f); + + bool LiesOn(Vector3 p) + { + if (p.X < lo.X - near || p.Y < lo.Y - near || p.Z < lo.Z - near || p.X > hi.X + near || p.Y > hi.Y + near || p.Z > hi.Z + near) return false; + foreach (Triangle t in chosen) + { + if (Vector3.DistanceSquared(p, Closest(p, t.A, t.B, t.C)) <= near * near) return true; + } + return false; + } + + List changes = [.. working.Values + .Where(w => !w.Before.AsSpan().SequenceEqual(w.After)) + .Select(w => new Change(w.Buffer, w.Before, w.After))]; + return new Plan(triangles, changes); + } + + // Moller-Trumbore, from either side: the editor draws a mesh without culling, so what is seen is what is met. + private static bool RayMeets(Vector3 origin, Vector3 direction, Vector3 a, Vector3 b, Vector3 c, out float t) + { + t = 0f; + Vector3 ab = b - a, ac = c - a; + Vector3 p = Vector3.Cross(direction, ac); + float det = Vector3.Dot(ab, p); + if (MathF.Abs(det) < 1e-12f) return false; + float inv = 1f / det; + Vector3 s = origin - a; + float u = Vector3.Dot(s, p) * inv; + if (u < 0f || u > 1f) return false; + Vector3 q = Vector3.Cross(s, ab); + float v = Vector3.Dot(direction, q) * inv; + if (v < 0f || u + v > 1f) return false; + t = Vector3.Dot(ac, q) * inv; + return t > 1e-4f; + } + + // The point of triangle abc nearest to p (Ericson, Real-Time Collision Detection 5.1.5). + private static Vector3 Closest(Vector3 p, Vector3 a, Vector3 b, Vector3 c) + { + Vector3 ab = b - a, ac = c - a, ap = p - a; + float d1 = Vector3.Dot(ab, ap), d2 = Vector3.Dot(ac, ap); + if (d1 <= 0f && d2 <= 0f) return a; + Vector3 bp = p - b; + float d3 = Vector3.Dot(ab, bp), d4 = Vector3.Dot(ac, bp); + if (d3 >= 0f && d4 <= d3) return b; + float vc = (d1 * d4) - (d3 * d2); + if (vc <= 0f && d1 >= 0f && d3 <= 0f) return a + (ab * (d1 / (d1 - d3))); + Vector3 cp = p - c; + float d5 = Vector3.Dot(ab, cp), d6 = Vector3.Dot(ac, cp); + if (d6 >= 0f && d5 <= d6) return c; + float vb = (d5 * d2) - (d1 * d6); + if (vb <= 0f && d2 >= 0f && d6 <= 0f) return a + (ac * (d2 / (d2 - d6))); + float va = (d3 * d6) - (d5 * d4); + if (va <= 0f && d4 - d3 >= 0f && d5 - d6 >= 0f) return b + ((c - b) * ((d4 - d3) / ((d4 - d3) + (d5 - d6)))); + float denom = 1f / (va + vb + vc); + return a + (ab * (vb * denom)) + (ac * (vc * denom)); + } +} diff --git a/src/Illusion.Assets/Sds/WorkingCopyDiff.cs b/src/Illusion.Assets/Sds/WorkingCopyDiff.cs new file mode 100644 index 0000000..dd9ae67 --- /dev/null +++ b/src/Illusion.Assets/Sds/WorkingCopyDiff.cs @@ -0,0 +1,240 @@ +using System.Security.Cryptography; +using System.Text.RegularExpressions; +using Illusion.Formats.Archive; + +namespace Illusion.Assets.Sds; + +/// How a file of a working copy stands against the archive in the game. +public enum WorkingCopyChangeKind +{ + /// In both, with different contents. + Changed, + /// In the working copy only: a Build adds it to the archive. + Added, + /// In the archive only: a Build drops it from the archive. + Removed, +} + +/// One file a Build would make different in the game. +/// Relative to the working copy, with forward slashes. +/// The working copy's file; for a removed one, the archive's. +/// When the working copy's file was last written (for a removed one, the default). +public sealed record WorkingCopyChange(string Path, WorkingCopyChangeKind Kind, long Size, DateTime Modified); + +/// What packing a working copy would change in the game. +/// The archive in the game. +/// False when the game has no such archive yet - then every file is an addition. +/// The files that differ, by path. +/// Files the working copy's manifest names that are not there: a Build leaves them +/// out of the archive and strikes them from the manifest. +public sealed record WorkingCopyComparison( + FileInfo Archive, bool InGame, IReadOnlyList Changes, IReadOnlyList MissingEntries); + +/// +/// Compares an archive's working copy with the archive as it stands in the game, file by file. +/// +/// A Build packs the WHOLE working copy, not what was edited since it was last opened: a file changed months +/// ago and forgotten goes into the game with today's edit, and nothing said so (it happened with +/// city_univers - an old collision file and name table rode in beside an eight-byte change). This is what a +/// Build window shows before it packs. +/// +/// +/// The archive is unpacked afresh into a scratch folder and the two folders are compared by what the files +/// hold, not by what they are called (see ). Unpacking is deterministic, so a working copy +/// nobody touched compares equal; the game's archive is only read. +/// +/// +public static partial class WorkingCopyDiff +{ + /// Compares the working copy the toolkit keeps for . + /// The archive has no working copy. + public static WorkingCopyComparison Compare(FileInfo sds) + { + ArgumentNullException.ThrowIfNull(sds); + return Compare(sds, MafiaEnvironment.ExtractedDir(sds)); + } + + /// Compares the working copy in with . + /// The folder is not a working copy (it has no manifest). + public static WorkingCopyComparison Compare(FileInfo sds, string workingDir) + { + ArgumentNullException.ThrowIfNull(sds); + ArgumentNullException.ThrowIfNull(workingDir); + if (!File.Exists(Path.Combine(workingDir, "SDSContent.xml"))) + { + throw new FileNotFoundException($"{sds.Name} has no working copy - there is nothing to pack.", Path.Combine(workingDir, "SDSContent.xml")); + } + + IReadOnlyList missing = SdsWriter.MissingEntries(workingDir); + Dictionary mine = Files(workingDir); + // A file the manifest does not name is not packed, whatever lies in the folder (a note, a copy kept + // "just in case"): it is not something a Build puts into the game, and is not listed as one. + // unescaped: the list writes "&" as "&", and a file name is compared as it stands on disk + string manifest = System.Net.WebUtility.HtmlDecode(File.ReadAllText(Path.Combine(workingDir, "SDSContent.xml"))); + // A name that several files of the working copy carry, each in a folder of its own, is told apart by the + // folder: with "/a/foo" unsaid and "/b/foo" still listed, the name alone found it listed. + HashSet shared = [.. mine.Keys.GroupBy(Path.GetFileName, StringComparer.OrdinalIgnoreCase).Where(g => g.Count() > 1).Select(g => g.Key!)]; + // As a whole name - the text of an element, or the end of a path in one - and not as a piece of a longer + // one: with "a.dds" unsaid and "ba.dds" still listed, a search for the letters found it listed. + bool Says(string name) + { + for (int at = manifest.IndexOf(name, StringComparison.OrdinalIgnoreCase); at >= 0; + at = manifest.IndexOf(name, at + 1, StringComparison.OrdinalIgnoreCase)) + { + char before = at == 0 ? '>' : manifest[at - 1]; + int end = at + name.Length; + if (before is '>' or '/' or '\\' && (end >= manifest.Length || manifest[end] == '<')) return true; + } + return false; + } + bool Named(string path) + { + // By its place in the working copy when the name is not its alone, by the name otherwise. A texture's + // top level ("MIP_") is a resource with an entry of its own and is looked up as any other: it + // used to pass as named whenever its texture was, which hid the deletion of the Mipmap alone. + string name = shared.Contains(Path.GetFileName(path)) ? path : Path.GetFileName(path); + return path.Equals("SDSContent.xml", StringComparison.OrdinalIgnoreCase) + || Says(name) || Says(name.Replace('/', '\\')) + // an XML resource is named without the extension it is unpacked with + || (name.EndsWith(".xml", StringComparison.OrdinalIgnoreCase) && (Says(name[..^4]) || Says(name[..^4].Replace('/', '\\')))); + } + sds.Refresh(); + if (!sds.Exists) + { + return new WorkingCopyComparison(sds, false, + [.. mine.Where(f => Named(f.Key)).OrderBy(f => f.Key, StringComparer.OrdinalIgnoreCase) + .Select(f => new WorkingCopyChange(f.Key, WorkingCopyChangeKind.Added, f.Value.Length, f.Value.LastWriteTime))], + missing); + } + + string scratch = Path.Combine(Path.GetTempPath(), "illusion_compare_" + Guid.NewGuid().ToString("N")); + try + { + SdsArchive.Open(sds.FullName).Extract(scratch); + return new WorkingCopyComparison(sds, true, Match(mine, Files(scratch), Named), missing); + } + finally + { + try + { + if (Directory.Exists(scratch)) Directory.Delete(scratch, recursive: true); + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) + { + // a scratch folder left in %TEMP% is not worth failing a comparison over + } + } + } + + // "Actors_306.act", "AnimalTrafficPaths317.atp", "ItemDesc_096a92487af8f55f.ids": a file named by the toolkit, + // after its place in the archive or after a hash, because the archive gives the resource no name. + [GeneratedRegex(@"^[A-Za-z_]+?_?(\d+|[0-9a-f]{16})\.[^./]+$")] + private static partial Regex Unnamed(); + + // Pairs the working copy's files with the archive's. By name alone that goes wrong: a resource the archive + // does not name is unpacked as "_", and the index is its place in the archive - add one + // texture and every such file after it comes out of the rebuilt archive under a new number; and a resource + // the toolkit imported sits in the working copy as "_" and comes back as "_". A + // working copy that was just packed then "differs" from its own archive in dozens of files. So: + // 1. same name, same bytes - the same resource, nothing to say; + // 2. same bytes, same kind, another name - the same resource under another number: nothing to say; + // 3. same name, other bytes - changed; + // 4. unnamed files of one kind left over on both sides - the same resources changed AND renumbered: + // paired in order, as changed; + // 5. what is left: in the working copy only - added; in the archive only - removed. + // "Kind" is the extension, for a file at the top of the folder that is not a texture: a texture keeps the + // name its artist gave it, and so does anything in a folder of its own. + private static List Match(Dictionary mine, Dictionary theirs, Func named) + { + // the manifest is the list of the files, not one of them: an entry added or dropped shows as its file + mine.Remove("SDSContent.xml"); + theirs.Remove("SDSContent.xml"); + // A file the manifest does not name is not in the working copy as far as a Build goes, even when it lies + // there byte for byte as the archive has it: that is what deleting a resource leaves (the entry is + // unsaid, the payload stays). Left in, it was paired with the archive's copy below as "the same" and + // the deletion was not listed - with nothing else changed, the archive came up as "nothing differs". + foreach (string path in mine.Keys.Where(path => !named(path)).ToList()) mine.Remove(path); + var hashes = new Dictionary(StringComparer.OrdinalIgnoreCase); + string Hash(FileInfo file) + { + if (!hashes.TryGetValue(file.FullName, out string? hash)) + { + using FileStream stream = file.OpenRead(); + hashes[file.FullName] = hash = file.Length + ":" + Convert.ToHexString(SHA256.HashData(stream)); + } + return hash; + } + static string Kind(string path) => + !path.Contains('/') && !path.EndsWith(".dds", StringComparison.OrdinalIgnoreCase) ? "*" + Path.GetExtension(path).ToLowerInvariant() : path; + + foreach (string path in mine.Keys.Where(theirs.ContainsKey).ToList()) // 1 + { + if (mine[path].Length == theirs[path].Length && Hash(mine[path]) == Hash(theirs[path])) + { + mine.Remove(path); + theirs.Remove(path); + } + } + var byBytes = new Dictionary>(); // 2 + foreach ((string path, FileInfo file) in theirs.OrderBy(f => f.Key, StringComparer.OrdinalIgnoreCase)) + { + if (Kind(path) == path) continue; + string key = Kind(path) + "|" + Hash(file); + if (!byBytes.TryGetValue(key, out Queue? queue)) byBytes[key] = queue = new Queue(); + queue.Enqueue(path); + } + foreach (string path in mine.Keys.OrderBy(k => k, StringComparer.OrdinalIgnoreCase).ToList()) + { + if (Kind(path) == path) continue; + if (byBytes.TryGetValue(Kind(path) + "|" + Hash(mine[path]), out Queue? queue) && queue.Count > 0) + { + theirs.Remove(queue.Dequeue()); + mine.Remove(path); + } + } + + var changes = new List(); + void Changed(string path) => changes.Add(new WorkingCopyChange(path, WorkingCopyChangeKind.Changed, mine[path].Length, mine[path].LastWriteTime)); + foreach (string path in mine.Keys.Where(theirs.ContainsKey).ToList()) // 3 + { + Changed(path); + mine.Remove(path); + theirs.Remove(path); + } + foreach (IGrouping kind in mine.Keys.Where(k => Kind(k) != k && Unnamed().IsMatch(k)).GroupBy(Kind).ToList()) // 4 + { + var stock = new Queue(theirs.Keys.Where(k => Kind(k) == kind.Key && Unnamed().IsMatch(k)).OrderBy(k => k, StringComparer.OrdinalIgnoreCase)); + foreach (string path in kind.OrderBy(k => k, StringComparer.OrdinalIgnoreCase)) + { + if (stock.Count == 0) break; + Changed(path); + mine.Remove(path); + theirs.Remove(stock.Dequeue()); + } + } + foreach ((string path, FileInfo file) in mine) // 5 + { + changes.Add(new WorkingCopyChange(path, WorkingCopyChangeKind.Added, file.Length, file.LastWriteTime)); + } + foreach ((string path, FileInfo file) in theirs) + { + changes.Add(new WorkingCopyChange(path, WorkingCopyChangeKind.Removed, file.Length, default)); + } + changes.Sort((x, y) => StringComparer.OrdinalIgnoreCase.Compare(x.Path, y.Path)); + return changes; + } + + // Every file of a folder by its path inside it. The toolkit's own note of the archive's memory figures is + // not part of the archive's contents and is left out. + private static Dictionary Files(string root) + { + var files = new Dictionary(StringComparer.OrdinalIgnoreCase); + foreach (string path in Directory.EnumerateFiles(root, "*", SearchOption.AllDirectories)) + { + string relative = Path.GetRelativePath(root, path).Replace('\\', '/'); + if (string.Equals(relative, SdsMemoryRequirements.FileName, StringComparison.OrdinalIgnoreCase)) continue; + files[relative] = new FileInfo(path); + } + return files; + } +} diff --git a/src/Illusion.Assets/World/AreaZones.cs b/src/Illusion.Assets/World/AreaZones.cs index fd79f03..277f261 100644 --- a/src/Illusion.Assets/World/AreaZones.cs +++ b/src/Illusion.Assets/World/AreaZones.cs @@ -1,4 +1,5 @@ using System.Numerics; +using Illusion.Formats.Archive; using Illusion.Formats.CityAreas; using Illusion.Formats.Frames; using Illusion.Formats.Frames.ObjectTypes; @@ -40,10 +41,14 @@ public static List Load(Func ensureExtracted, if (areaDistricts.Count == 0) return zones; // FrameResource city_univers → AREA boxes (name + world AABB). - ExtractedSds scene = ExtractedSds.Load(extracted); - if (scene.FrameResource?.FrameObjects == null) return zones; + // The scene alone: a volume is a matrix and a box, and the archive's vertex and index pools - which + // ExtractedSds reads and decodes in full - say nothing about it. This runs again after every zone edit. + IReadOnlyList scenes = SdsManifest.Load(extracted).GetFiles("FrameResource"); + if (scenes.Count == 0) return zones; + var frame = new FrameResource(scenes[0]); + if (frame.FrameObjects == null) return zones; - foreach (var pair in scene.FrameResource.FrameObjects) + foreach (var pair in frame.FrameObjects) { if (pair.Value is not FrameObjectArea area) continue; string? name = area.Name?.ToString(); diff --git a/src/Illusion.Assets/World/LoadZones.cs b/src/Illusion.Assets/World/LoadZones.cs new file mode 100644 index 0000000..cfa0050 --- /dev/null +++ b/src/Illusion.Assets/World/LoadZones.cs @@ -0,0 +1,671 @@ +using System.Numerics; +using Illusion.Assets.Adapters; +using Illusion.Assets.Sds; +using Illusion.Formats.Archive; +using Illusion.Formats.CityAreas; +using Illusion.Formats.Frames; +using Illusion.Formats.Frames.ObjectTypes; +using Illusion.Formats.Frames.Resources; +using Illusion.Formats.Hashing; + +namespace Illusion.Assets.World; + +/// +/// The city's load zones as the game states them, open for editing. +/// +/// A zone is an AREA volume in the scene of city_univers.sds - a world matrix, a box in the volume's own +/// space and the planes that bound it - and missions\CITY\cityareas.bin names, for each volume, the one +/// or two districts the game keeps loaded while the camera is inside it. A place that lies in no volume asks +/// for nothing: put there by a teleport, the player stands in a district that never streams in, until he +/// walks into a zone that asks for it. +/// +/// +/// The planes are what decides: a point is inside when n.p + d >= 0 for every plane, in the volume's +/// space. The box is the planes' own extent and is kept in step with them here - a face is moved in both. +/// +/// +public sealed class LoadZones +{ + private readonly Dictionary _volumes = new(StringComparer.OrdinalIgnoreCase); + private readonly Dictionary> _districts = new(StringComparer.OrdinalIgnoreCase); + + // Asked about a point: the volumes in name order, and each one's way from the world into its own space. + // Worked out once - a plan of a district asks about tens of thousands of points - and dropped when a + // volume is moved. + private (string Name, FrameObjectArea Zone)[]? _ordered; + private readonly Dictionary _toLocal = []; + + private LoadZones(FileInfo archive, FrameResource frame, string sceneFile) + { + Archive = archive; + Frame = frame; + SceneFile = sceneFile; + } + + // For a zone that is ADDED: the working copy's folder, the scene read with its name table (a new volume + // has to be listed there), and the table of districts once it has been read to be changed. + private string _extracted = ""; + private HashSet _known = new(StringComparer.OrdinalIgnoreCase); + private bool _withNameTable; + private bool _volumeAdded; + private CityAreasTable? _table; + + private string TableFile => Path.Combine(_extracted, "missions", "CITY", "cityareas.bin"); + + /// The frame name table of the working copy - the file a new volume is listed in - or null when + /// the archive has none. + public string? NameTableFile => SdsManifest.Load(_extracted).GetFiles("FrameNameTable").FirstOrDefault(); + + /// The file of the working copy the scene was read from, and writes. + public string SceneFile { get; } + + /// The archive the zones live in: a copy of city_univers.sds (see ). + public FileInfo Archive { get; } + + /// + /// Every copy of city_univers.sds the game has: the base one first, then one for each DLC that ships + /// its own (pc\dlcs\<dlc>\sds\city_univers\ - Joe's Adventures does). A DLC's copy is a scene of + /// its own, with fewer zones. Measured in the game (free ride of a multiplayer client that mounts Joe's + /// Adventures): the zones in force were the BASE copy's - a face moved in the DLC's copy changed nothing. + /// + /// Measured the same way: what makes a district stream in at the point a player appears at is a zone of + /// the seam kind ("AREA341_GREENFIELD_KINGSTONE") - see . A district's own box + /// ("AREA0019_GREENFIELD") did not load it by itself, and a point in no seam zone keeps whatever was loaded + /// before - nothing, right after a login. + /// + /// + public static IReadOnlyList Copies() + { + var copies = new List { new(MafiaEnvironment.CityUniversSds) }; + string dlcs = Path.Combine(MafiaEnvironment.PcFolder, "dlcs"); + if (Directory.Exists(dlcs)) + { + foreach (string dlc in Directory.GetDirectories(dlcs).OrderBy(d => d, StringComparer.OrdinalIgnoreCase)) + { + var copy = new FileInfo(Path.Combine(dlc, "sds", "city_univers", "city_univers.sds")); + if (copy.Exists) copies.Add(copy); + } + } + return copies; + } + + /// + /// Whether a volume of this name loads its districts for a player who APPEARS inside it (a login, a + /// teleport), as opposed to keeping loaded what a neighbour brought in. Measured in the game, with new zones + /// over one test spot (free ride of a multiplayer client, 2026-10): the NAME decides. Two words after the + /// number - "AREA900_SANDISLAND_TUNEL", "AREA901_FOO_BAR", "AREA903_A_B" - and the district was loaded, + /// with one district in cityareas.bin or two, its byte 0 or 1. One word - "AREA900_TEST", + /// "AREA902_FOOXBAR" - and it was not, with two districts and the byte 1. The table says WHICH districts; + /// it does not say whether. The shipped names agree: a district's own box has one word and one district, a + /// seam two and two. The thirty shipped names written with hyphens ("AREA0223-DIPTON-KINGSTONE") were not + /// measured and are counted by their words too. + /// + public static bool LoadsOnArrival(string? name) => + (name ?? "").Split(['_', '-'], StringSplitOptions.RemoveEmptyEntries).Length >= 3; + + private static readonly Lazy> Shipped = new(() => + { + var names = new HashSet(StringComparer.OrdinalIgnoreCase); + System.Reflection.Assembly assembly = typeof(LoadZones).Assembly; + string? resource = assembly.GetManifestResourceNames().FirstOrDefault(n => n.EndsWith("ShippedLoadZones.txt", StringComparison.Ordinal)); + if (resource == null || assembly.GetManifestResourceStream(resource) is not { } stream) return names; + using var reader = new StreamReader(stream); + while (reader.ReadLine() is { } line) + { + if (line.Trim() is { Length: > 0 } name) names.Add(name); + } + return names; + }); + + /// + /// Whether a volume of this name came with the game: it is on the list of the AREA volumes the base game's + /// city_univers and the DLC copies ship with. One that is not was ADDED - by , here + /// or on the machine the archive came from - and is the kind that is marked in the layer and may be taken out. + /// + public static bool IsShipped(string? name) => name != null && Shipped.Value.Contains(name); + + /// How a copy is named to the user: "base", or the DLC's folder name. + public static string CopyName(FileInfo copy) + { + ArgumentNullException.ThrowIfNull(copy); + string full = copy.FullName; + int at = full.IndexOf(Path.DirectorySeparatorChar + "dlcs" + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase); + if (at < 0) return "base"; + string rest = full[(at + 6)..]; + return rest[..rest.IndexOf(Path.DirectorySeparatorChar)]; + } + + /// Its scene, as read from the working copy. + public FrameResource Frame { get; } + + /// Every volume, by name. + public IReadOnlyDictionary Volumes => _volumes; + + private readonly HashSet _ambiguous = new(StringComparer.OrdinalIgnoreCase); + + /// Names that more than one volume of the scene carries. Such a volume is listed (the first of + /// them) but not moved: a name is all a caller has to say which one is meant. + public IReadOnlyCollection AmbiguousNames => _ambiguous; + + /// The districts a volume keeps loaded, as resolved archive names; none for a volume the table does + /// not list (shop zones are bound elsewhere). + public IReadOnlyList DistrictsOf(string zone) => + _districts.TryGetValue(zone, out IReadOnlyList? districts) ? districts : []; + + /// Reads the zones from the working copy of a city_univers.sds. + /// Unpacks an archive when it has no working copy yet; returns its folder. + /// The district archives' base names ("midtown"), to resolve the table's targets against. + /// Which copy (one of ); the base game's when null. + /// Reads the scene together with its frame name table - what + /// needs, and what costs a full read of the archive's buffer pools; moving existing zones does not need it. + public static LoadZones Open(Func ensureExtracted, IReadOnlyCollection districts, FileInfo? copy = null, + bool forNewZones = false) + { + ArgumentNullException.ThrowIfNull(ensureExtracted); + ArgumentNullException.ThrowIfNull(districts); + FileInfo archive = copy ?? new FileInfo(MafiaEnvironment.CityUniversSds); + if (!archive.Exists) throw new FileNotFoundException("city_univers.sds is not in the game folder", archive.FullName); + + string extracted = ensureExtracted(archive); + // The scene alone. A zone is a matrix, a box and planes; the archive's vertex and index pools have + // nothing to say about it, and reading them all cost every click on a zone a full load of the archive. + IReadOnlyList scenes; + try + { + scenes = SdsManifest.Load(extracted).GetFiles("FrameResource"); + } + catch (System.Xml.XmlException ex) + { + throw new InvalidDataException("the working copy of city_univers.sds has a contents list that cannot be read: " + ex.Message, ex); + } + if (scenes.Count == 0) throw new InvalidDataException("city_univers.sds has no scene"); + // A volume that is added has to be listed in the frame name table, with the flags the other volumes + // carry there - and those are only on the frames when the scene is read together with the table. + FrameResource scene = forNewZones + ? ExtractedSds.Load(extracted).FrameResource ?? throw new InvalidDataException("city_univers.sds has no scene") + : new FrameResource(scenes[0]); + var zones = new LoadZones(archive, scene, scenes[0]) + { + _extracted = extracted, + _withNameTable = forNewZones, + _known = new HashSet(districts, StringComparer.OrdinalIgnoreCase), + }; + foreach (FrameObjectArea area in zones.Frame.FrameObjects.Values.OfType()) + { + if (area.Name?.ToString() is not { Length: > 0 } name) continue; + // Two volumes under one name could not be told apart by the table that names them, nor by a + // caller that names one to move: the first is kept, and the name is remembered as ambiguous. + if (!zones._volumes.TryAdd(name, area)) zones._ambiguous.Add(name); + } + + string table = Path.Combine(extracted, "missions", "CITY", "cityareas.bin"); + if (File.Exists(table)) + { + HashSet known = zones._known; + foreach (CityAreaEntry entry in CityAreasFile.Load(table).Areas) + { + var list = new List(2); + foreach (string? target in new[] { entry.Target1, entry.Target2 }) + { + if (DistrictNames.Resolve(target, known) is { } district && !list.Contains(district, StringComparer.OrdinalIgnoreCase)) + list.Add(district); + } + if (list.Count > 0) zones._districts[entry.Name] = list; + } + } + return zones; + } + + /// Whether a world point is inside a volume. is how far past the + /// nearest violated plane it lies (0 when inside). + public static bool Contains(FrameObjectArea zone, Vector3 world, out float outsideBy) + { + ArgumentNullException.ThrowIfNull(zone); + outsideBy = 0f; + return Matrix4x4.Invert(zone.WorldTransform, out Matrix4x4 toLocal) && Contains(zone, toLocal, world, out outsideBy); + } + + private static bool Contains(FrameObjectArea zone, Matrix4x4 toLocal, Vector3 world, out float outsideBy) + { + outsideBy = 0f; + Vector3 local = Vector3.Transform(world, toLocal); + if (zone.Planes is not { Length: > 0 } planes) + { + Vector3 lo = zone.Bounds.Min, hi = zone.Bounds.Max; + Vector3 beyond = Vector3.Max(Vector3.Max(lo - local, local - hi), Vector3.Zero); + outsideBy = MathF.Max(beyond.X, MathF.Max(beyond.Y, beyond.Z)); + return outsideBy <= 0f; + } + foreach (Vector4 plane in planes) + { + float depth = Vector3.Dot(new Vector3(plane.X, plane.Y, plane.Z), local) + plane.W; + if (-depth > outsideBy) outsideBy = -depth; + } + return outsideBy <= 0f; + } + + /// A volume's box in world space (the eight corners of its own box through its matrix). + public static (Vector3 Min, Vector3 Max) WorldBox(FrameObjectArea zone) + { + ArgumentNullException.ThrowIfNull(zone); + Vector3 lo = zone.Bounds.Min, hi = zone.Bounds.Max; + var min = new Vector3(float.MaxValue); + var max = new Vector3(float.MinValue); + for (int i = 0; i < 8; i++) + { + Vector3 corner = Vector3.Transform( + new Vector3((i & 1) == 0 ? lo.X : hi.X, (i & 2) == 0 ? lo.Y : hi.Y, (i & 4) == 0 ? lo.Z : hi.Z), + zone.WorldTransform); + min = Vector3.Min(min, corner); + max = Vector3.Max(max, corner); + } + return (min, max); + } + + /// The volumes that hold a point, or come within metres of holding it. + public IEnumerable<(string Name, FrameObjectArea Zone, bool Inside, float OutsideBy)> At(Vector3 world, float near = 0f) + { + _ordered ??= [.. _volumes.OrderBy(v => v.Key, StringComparer.Ordinal).Select(v => (v.Key, v.Value))]; + foreach ((string name, FrameObjectArea zone) in _ordered) + { + if (!_toLocal.TryGetValue(zone, out Matrix4x4? toLocal)) + { + toLocal = Matrix4x4.Invert(zone.WorldTransform, out Matrix4x4 inverse) ? inverse : null; + _toLocal[zone] = toLocal; + } + if (toLocal is not { } into) continue; + bool inside = Contains(zone, into, world, out float outsideBy); + if (inside || outsideBy <= near) yield return (name, zone, inside, outsideBy); + } + } + + /// The districts asked for at a point: those of every volume that holds it. + public IReadOnlyList DistrictsAt(Vector3 world) => + [.. At(world).SelectMany(z => DistrictsOf(z.Name)).Distinct(StringComparer.OrdinalIgnoreCase).OrderBy(d => d, StringComparer.Ordinal)]; + + /// + /// Moves one face of a volume to a world coordinate, IN MEMORY: the plane that bounds it on that side and + /// its box together. writes it. Only for a volume that is neither turned nor scaled and + /// a face that lies square to the axis - which is how the city's zones are cut, except where one corner is + /// sliced off by an extra plane, and that plane is left as it is. + /// + /// The volume's name. + /// "+x", "-x", "+y", "-y", "+z" or "-z": the face on that side. + /// Where the face is to stand, on that axis, in world coordinates. + /// Null on success, otherwise why not. + public string? MoveFace(string zone, string face, float to, out LoadZoneFaceMove? move) + { + move = null; + if (!_volumes.TryGetValue(zone ?? "", out FrameObjectArea? area)) return $"no load zone named '{zone}'"; + if (_ambiguous.Contains(zone!)) return $"more than one volume of the scene is named '{zone}' - which one is meant cannot be told"; + if (face is not { Length: 2 } || (face[0] != '+' && face[0] != '-') || "xyz".IndexOf(char.ToLowerInvariant(face[1])) < 0) + return "face is one of +x, -x, +y, -y, +z, -z"; + if (!float.IsFinite(to)) return "the coordinate must be a finite number"; + + Matrix4x4 world = area.WorldTransform; + if (MathF.Abs(world.M11 - 1f) > 1e-3f || MathF.Abs(world.M22 - 1f) > 1e-3f || MathF.Abs(world.M33 - 1f) > 1e-3f) + return $"{zone} is turned or scaled; only a volume standing square to the map can have a face moved"; + + int axis = "xyz".IndexOf(char.ToLowerInvariant(face[1])); + bool upper = face[0] == '+'; + float origin = axis == 0 ? world.M41 : axis == 1 ? world.M42 : world.M43; + float local = to - origin; + + // Inside is n.p + d >= 0: the face that limits the volume on its upper side has its normal pointing + // DOWN the axis and d equal to the limit; on the lower side the normal points up and d is minus it. + int found = -1; + for (int i = 0; i < area.Planes.Length; i++) + { + Vector4 plane = area.Planes[i]; + float[] n = [plane.X, plane.Y, plane.Z]; + bool square = true; + for (int k = 0; k < 3; k++) + { + if (k != axis && MathF.Abs(n[k]) > 1e-3f) square = false; + } + if (square && MathF.Abs(n[axis] - (upper ? -1f : 1f)) < 1e-3f) found = i; + } + if (found < 0) return $"{zone} has no face square to the axis on {face}"; + + Vector3 lo = area.Bounds.Min, hi = area.Bounds.Max; + float other = upper ? (axis == 0 ? lo.X : axis == 1 ? lo.Y : lo.Z) : (axis == 0 ? hi.X : axis == 1 ? hi.Y : hi.Z); + if (upper ? local <= other + 0.5f : local >= other - 0.5f) + return $"that would leave {zone} with no depth on {face[1]}: its opposite face stands at {other + origin:F2}"; + + Vector4 before = area.Planes[found]; + float from = (upper ? before.W : -before.W) + origin; + area.Planes[found] = new Vector4(before.X, before.Y, before.Z, upper ? local : -local); + if (upper) + { + if (axis == 0) hi.X = local; else if (axis == 1) hi.Y = local; else hi.Z = local; + } + else + { + if (axis == 0) lo.X = local; else if (axis == 1) lo.Y = local; else lo.Z = local; + } + var box = area.Bounds; + box.Min = lo; + box.Max = hi; + area.Bounds = box; + + (Vector3 min, Vector3 max) = WorldBox(area); + move = new LoadZoneFaceMove(area.Name.ToString(), face.ToLowerInvariant(), found, from, to, min, max); + return null; + } + + /// + /// The faces of a volume that can move: those with a plane square to their axis, on a + /// volume that stands square to the map. A corner sliced off by a slanted plane leaves its side without one. + /// + public IReadOnlySet SquareFaces(string zone) + { + var faces = new HashSet(StringComparer.Ordinal); + if (!_volumes.TryGetValue(zone ?? "", out FrameObjectArea? area)) return faces; + Matrix4x4 world = area.WorldTransform; + if (MathF.Abs(world.M11 - 1f) > 1e-3f || MathF.Abs(world.M22 - 1f) > 1e-3f || MathF.Abs(world.M33 - 1f) > 1e-3f) return faces; + foreach (Vector4 plane in area.Planes) + { + float[] n = [plane.X, plane.Y, plane.Z]; + for (int axis = 0; axis < 3; axis++) + { + bool square = MathF.Abs(MathF.Abs(n[axis]) - 1f) < 1e-3f && Enumerable.Range(0, 3).All(k => k == axis || MathF.Abs(n[k]) <= 1e-3f); + // the face on the upper side has its normal pointing DOWN the axis (see MoveFace) + if (square) faces.Add((n[axis] < 0 ? "+" : "-") + "xyz"[axis]); + } + } + return faces; + } + + /// + /// Moves a whole volume by a world offset, IN MEMORY: its place changes, its planes and its box - which are + /// in its own space - do not, so this works for a volume of any shape. writes it. + /// + /// Null on success, otherwise why not. + public string? Move(string zone, Vector3 by) + { + if (!_volumes.TryGetValue(zone ?? "", out FrameObjectArea? area)) return $"no load zone named '{zone}'"; + if (_ambiguous.Contains(zone!)) return $"more than one volume of the scene is named '{zone}' - which one is meant cannot be told"; + if (!float.IsFinite(by.X) || !float.IsFinite(by.Y) || !float.IsFinite(by.Z)) return "the offset must be finite numbers"; + // A volume's place in the world is its local place put through its parent (FrameObjectBase works the + // world transform out the same way), so the offset is taken into the parent's space before it is added. + // Only the parent's turn and scale are used: a frame matrix is not kept as a full 4 x 4, and inverting + // one as it stands fails. + Matrix4x4 parent = (area.Parent ?? area.Root)?.WorldTransform ?? Matrix4x4.Identity; + var turn = new Matrix4x4( + parent.M11, parent.M12, parent.M13, 0f, + parent.M21, parent.M22, parent.M23, 0f, + parent.M31, parent.M32, parent.M33, 0f, + 0f, 0f, 0f, 1f); + if (!Matrix4x4.Invert(turn, out Matrix4x4 unTurn)) return $"{zone} hangs on a frame whose transform cannot be inverted"; + Matrix4x4 local = area.LocalTransform; + local.Translation += Vector3.TransformNormal(by, unTurn); + area.LocalTransform = local; + _toLocal.Clear(); // its own way into its space, and that of anything hanging on it + return null; + } + + /// + /// Adds a NEW volume to the scene, IN MEMORY, and a line for it to the table of districts: a box standing + /// square to the map between two world corners, keeping the one or two districts named loaded while the + /// player is inside it. It is made as a copy of an existing volume () - its type, + /// flags, parent and place in the name table - given a name, a place and a shape of its own. + /// writes the scene, the name table and the districts table. + /// + /// Null on success, otherwise why not; nothing is changed on a refusal. + public string? Create(string name, string like, Vector3 min, Vector3 max, string district1, string? district2) + { + if (!_withNameTable) return "these zones were opened for moving, not for adding to"; + if (string.IsNullOrWhiteSpace(name)) return "a zone needs a name"; + if (Frame.FrameObjects.Values.OfType().Any(o => string.Equals(o.Name?.ToString(), name, StringComparison.OrdinalIgnoreCase))) + return $"the scene already has an object named '{name}'"; + if (!_volumes.TryGetValue(like ?? "", out FrameObjectArea? source)) return $"no load zone named '{like}' to make it like"; + foreach (float v in new[] { min.X, min.Y, min.Z, max.X, max.Y, max.Z }) + { + if (!float.IsFinite(v)) return "the corners must be finite numbers"; + } + Vector3 lo = Vector3.Min(min, max), hi = Vector3.Max(min, max), half = (hi - lo) * 0.5f; + if (half.X < 0.5f || half.Y < 0.5f || half.Z < 0.5f) return "the box must be at least a metre on every side"; + Matrix4x4 world = source.WorldTransform; + if (MathF.Abs(world.M11 - 1f) > 1e-3f || MathF.Abs(world.M22 - 1f) > 1e-3f || MathF.Abs(world.M33 - 1f) > 1e-3f) + return $"{like} is turned or scaled - take a volume that stands square to the map as the pattern"; + + // its own origin goes to the box's centre, said in the space of the frame the pattern hangs on + Matrix4x4 parent = (source.Parent ?? source.Root)?.WorldTransform ?? Matrix4x4.Identity; + var turn = new Matrix4x4( + parent.M11, parent.M12, parent.M13, 0f, + parent.M21, parent.M22, parent.M23, 0f, + parent.M31, parent.M32, parent.M33, 0f, + 0f, 0f, 0f, 1f); + if (!Matrix4x4.Invert(turn, out Matrix4x4 unTurn)) return $"{like} hangs on a frame whose transform cannot be inverted"; + + if (!File.Exists(TableFile)) return "the working copy of city_univers has no cityareas.bin"; + CityAreasTable table = _table ?? CityAreasTable.Parse(File.ReadAllBytes(TableFile)); + // The table names districts in its own words ("kingston" for the archive kingstone): a district is + // written as the table says it, and one it has no word for must at least be an archive of the city. + var resolved = new List(2); + var written = new List(2); + foreach (string? asked in new[] { district1, district2 }) + { + if (string.IsNullOrWhiteSpace(asked)) + { + if (resolved.Count == 0) return "a zone names at least one district"; + continue; + } + if (TableWord(table, asked.Trim()) is not var (word, district)) return $"'{asked.Trim()}' is not a district of the city"; + if (resolved.Contains(district, StringComparer.OrdinalIgnoreCase)) return "the two districts are the same one"; + written.Add(word); + resolved.Add(district); + } + if (table.Add(name, written[0], written.Count > 1 ? written[1] : null) is { } refused) return refused; + + // The copy constructor takes every serialized field of the pattern - flags, name-table membership, the + // references to its parents - and shares its planes array, which is replaced below. + var clone = new FrameObjectArea(source) { Name = new HashName(name) }; + var box = clone.Bounds; + box.Min = -half; + box.Max = half; + clone.Bounds = box; + // inside is n.p + d >= 0 in the volume's own space: an upper face looks down its axis, a lower one up + clone.Planes = + [ + new Vector4(-1, 0, 0, half.X), new Vector4(1, 0, 0, half.X), + new Vector4(0, -1, 0, half.Y), new Vector4(0, 1, 0, half.Y), + new Vector4(0, 0, -1, half.Z), new Vector4(0, 0, 1, half.Z), + ]; + clone.PlaneSize = clone.Planes.Length; + + Frame.FrameObjects.Add(clone.RefID, clone); + Frames.FrameDuplicator.LinkParents(clone, + Frames.FrameDuplicator.ResolveRef(Frame, source, FrameEntryRefTypes.Parent1), + Frames.FrameDuplicator.ResolveRef(Frame, source, FrameEntryRefTypes.Parent2)); + + // the pattern's place, carried by the difference between the centres + Matrix4x4 local = source.LocalTransform; + local.Translation += Vector3.TransformNormal(((lo + hi) * 0.5f) - world.Translation, unTurn); + clone.LocalTransform = local; + + _table = table; + _volumeAdded = true; + _volumes[name] = clone; + _districts[name] = resolved; + _ordered = null; + _toLocal.Clear(); + return null; + } + + // How the table says a district, and the archive that is: the table's own name for it when it has one + // (exactly, or one that resolves to the archive asked for), otherwise the archive's name as it stands. + private (string Word, string District)? TableWord(CityAreasTable table, string asked) + { + IReadOnlyList words = table.Districts; + string? word = words.FirstOrDefault(w => string.Equals(w, asked, StringComparison.OrdinalIgnoreCase)) + ?? words.FirstOrDefault(w => string.Equals(DistrictNames.Resolve(w, _known), asked, StringComparison.OrdinalIgnoreCase)); + if (word != null) return (word, DistrictNames.Resolve(word, _known) ?? word); + return _known.TryGetValue(asked, out string? archive) ? (archive, archive) : null; + } + + /// + /// Takes a volume out of the scene, IN MEMORY, and its line out of the table of districts - what undoing a + /// is. writes the scene, the name table and the districts table. + /// + /// Null on success, otherwise why not; nothing is changed on a refusal. + public string? Delete(string name) + { + if (!_withNameTable) return "these zones were opened for moving, not for taking out"; + if (!_volumes.TryGetValue(name ?? "", out FrameObjectArea? area)) return $"no load zone named '{name}'"; + if (_ambiguous.Contains(name!)) return $"more than one volume of the scene is named '{name}' - which one is meant cannot be told"; + if (Frame.FrameObjects.Values.OfType().Any(o => ReferenceEquals(o.Parent, area) || ReferenceEquals(o.Root, area))) + return $"other objects of the scene hang on {name}"; + if (!File.Exists(TableFile)) return "the working copy of city_univers has no cityareas.bin"; + CityAreasTable table = _table ?? CityAreasTable.Parse(File.ReadAllBytes(TableFile)); + table.Remove(name!); + + area.SetParent(ParentInfo.ParentType.ParentIndex1, null); + area.SetParent(ParentInfo.ParentType.ParentIndex2, null); + foreach (FrameHeaderScene scene in Frame.FrameScenes.Values) scene.Children.Remove(area); + Frame.FrameObjects.Remove(area.RefID); + + _table = table; + _volumeAdded = true; + _volumes.Remove(name!); + _districts.Remove(name!); + _ordered = null; + _toLocal.Remove(area); + return null; + } + + /// + /// Whether a scene an open editor holds has this volume exactly as this one has it - its place, its box and + /// its planes. It has not when the volume was changed in that editor and not saved: a write made from the + /// disk's copy would then be made from a state the editor is about to replace. A scene without the volume + /// has nothing to disagree with. + /// + public bool InStepWith(SceneDocumentAdapter document, string zone) + { + ArgumentNullException.ThrowIfNull(document); + if (!_volumes.TryGetValue(zone ?? "", out FrameObjectArea? mine) || ReferenceEquals(document.Frame, Frame)) return true; + FrameObjectArea? theirs = document.Frame.FrameObjects.Values.OfType() + .FirstOrDefault(a => string.Equals(a.Name?.ToString(), zone, StringComparison.OrdinalIgnoreCase)); + return theirs == null + || (SamePlace(theirs.LocalTransform, mine.LocalTransform) && theirs.Bounds.Min == mine.Bounds.Min && theirs.Bounds.Max == mine.Bounds.Max + && (theirs.Planes ?? []).SequenceEqual(mine.Planes ?? [])); + } + + // A frame's matrix is stored as three columns of four. The fourth is not in the file: read from disk it is + // all zeros, composed by an editor it ends in a one - so a volume moved in the editor and SAVED compared + // as "changed and not saved" for good. What counts is what a save writes. + private static bool SamePlace(Matrix4x4 a, Matrix4x4 b) => + a.M11 == b.M11 && a.M12 == b.M12 && a.M13 == b.M13 + && a.M21 == b.M21 && a.M22 == b.M22 && a.M23 == b.M23 + && a.M31 == b.M31 && a.M32 == b.M32 && a.M33 == b.M33 + && a.M41 == b.M41 && a.M42 == b.M42 && a.M43 == b.M43; + + /// + /// Brings the same volume in a scene an open editor holds in step with this one: its place, its box and its + /// planes. An editor that has the archive loaded (the map editor in Whole map mode does) writes its own copy + /// of the scene whole on its next save, and without this that save would put the volume back where it stood. + /// + /// False when that scene has no volume of this name. + public bool MirrorInto(SceneDocumentAdapter document, string zone) + { + ArgumentNullException.ThrowIfNull(document); + if (!_volumes.TryGetValue(zone ?? "", out FrameObjectArea? mine) || ReferenceEquals(document.Frame, Frame)) return false; + FrameObjectArea? theirs = document.Frame.FrameObjects.Values.OfType() + .FirstOrDefault(a => string.Equals(a.Name?.ToString(), zone, StringComparison.OrdinalIgnoreCase)); + if (theirs == null) return false; + theirs.Planes = [.. mine.Planes]; + var box = theirs.Bounds; + box.Min = mine.Bounds.Min; + box.Max = mine.Bounds.Max; + theirs.Bounds = box; + theirs.LocalTransform = mine.LocalTransform; + return true; + } + + /// Writes the scene into the working copy of city_univers.sds; a Build of that archive + /// takes it into the game. Returns the file written. + public string Save() + { + if (!_volumeAdded) return SdsWriter.SaveFrameResource(Frame, Archive); + + // A new volume is three files - the scene, the name table that lists it and the districts table that + // names it - and one without the others is a scene the game has no use for. What each held is kept + // until the last is written, and put back if one fails. + IReadOnlyDictionary before = StructureFiles(); + var replaced = new List(); + try + { + string written = SdsWriter.SaveFrameResource(Frame, Archive); + replaced.Add(written); + if (SdsWriter.SaveFrameNameTable(Frame, Archive) is { } names) replaced.Add(names); + AtomicFile.WriteAllBytes(TableFile, _table!.ToBytes()); + return written; + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or InvalidOperationException or InvalidDataException) + { + // a write is one file swapped in whole: the one that failed is as it was, the ones before it are not + string left = PutBack(replaced.ToDictionary(f => f, f => before.GetValueOrDefault(f))); + if (left.Length == 0) throw; + throw new IOException($"{ex.Message} - and the working copy could not be put back as it was ({left}): unpack city_univers again", ex); + } + } + + /// + /// The files a volume that is added or taken out is written into - the scene, the name table that lists it + /// and the districts table that names it - each with what it holds now, or null when it is not there. + /// + public IReadOnlyDictionary StructureFiles() + { + List files = [SceneFile, .. SdsManifest.Load(_extracted).GetFiles("FrameNameTable"), TableFile]; + return files.Distinct(StringComparer.OrdinalIgnoreCase).ToDictionary(f => f, f => File.Exists(f) ? File.ReadAllBytes(f) : null); + } + + /// + /// Writes those files as they are given: all of them or - when one cannot be written - none, the ones + /// already swapped in being put back. + /// + /// A file could not be written; the message says what was left changed, if anything. + public static void Restore(IReadOnlyDictionary files) + { + ArgumentNullException.ThrowIfNull(files); + var before = files.Keys.ToDictionary(f => f, f => File.Exists(f) ? File.ReadAllBytes(f) : null); + var replaced = new Dictionary(); + try + { + foreach ((string file, byte[]? bytes) in files) + { + if (bytes != null) AtomicFile.WriteAllBytes(file, bytes); + else File.Delete(file); + replaced[file] = before[file]; + } + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) + { + string left = PutBack(replaced); + throw new IOException(left.Length == 0 ? ex.Message : $"{ex.Message} - and the working copy was left half written ({left}): unpack city_univers again", ex); + } + } + + // Each file to the bytes given, or away when it had none. Returns what could not be put back ("" when all was). + private static string PutBack(IReadOnlyDictionary files) + { + var left = new List(); + foreach ((string file, byte[]? bytes) in files) + { + try + { + if (bytes != null) AtomicFile.WriteAllBytes(file, bytes); + else File.Delete(file); + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) + { + left.Add($"{Path.GetFileName(file)}: {ex.Message}"); + } + } + return string.Join("; ", left); + } +} + +/// One face of a load zone moved: which plane, from where to where on its axis (world), and the +/// volume's world box afterwards. +public sealed record LoadZoneFaceMove(string Zone, string Face, int Plane, float From, float To, Vector3 BoxMin, Vector3 BoxMax); diff --git a/src/Illusion.Assets/World/ShippedLoadZones.txt b/src/Illusion.Assets/World/ShippedLoadZones.txt new file mode 100644 index 0000000..a3f9f13 --- /dev/null +++ b/src/Illusion.Assets/World/ShippedLoadZones.txt @@ -0,0 +1,1009 @@ +AREA0001_SANDISLAND +AREA0002_SANDISLAND +AREA0003_SANDISLAND +AREA0004_SANDISLAND +AREA0005_SANDISLAND +AREA0006_SANDISLAND +AREA0007_SANDISLAND +AREA0008_SANDISLAND +AREA0009_HUNTERS +AREA000_1_Odevy +AREA0010_HUNTERS +AREA0011_HUNTERS +AREA0012_HUNTERS +AREA0013_HUNTERS +AREA0014_HUNTERS +AREA0015_HUNTERS +AREA0017_GREENFIELD +AREA0018_GREENFIELDF +AREA0019_GREENFIELD +AREA001_0_Odevy +AREA001_PORT_SOUTHPORT +AREA001_SICCESTA_SICMESTOA +AREA0020_KINGSTON +AREA0021_KINGSTON +AREA0022_KINGSTON +AREA0023_DIPTON +AREA0024_DIPTON +AREA0025_DIPTON +AREA0026_DIPTON +AREA0027_DIPTON +AREA0028_RIVERSIDE +AREA0029_RIVERSIDE +AREA002_SICCESTA_SICMESTOA +AREA002_SOUTHPORT_OYSTERBAY +AREA0030_RIVERSIDE +AREA0031_HIGH +AREA0032_HIGH +AREA0033_HILL +AREA0034_UPPERTOWN +AREA0035_UPPERTOWN +AREA0036_UPPERTOWN +AREA0037_UPPERTOWN +AREA0038_UPPERTOWN +AREA0039_UPPERTOWN +AREA003_SICMESTOA_sicradnice +AREA003_SOUTHPORT_port +AREA0040_UPPERTOWN +AREA0041_ITALY +AREA0042_MILLNEW +AREA0043_MILLNEW +AREA0044_MILLN +AREA0045_MILLN +AREA0046_FOUNDRY +AREA0048_CHINATOWN +AREA0049_EASTSIDE +AREA004_SICMESTOA_SICRADNICE +AREA004_SOUTHPORT_PORT +AREA0050_EASTSIDE +AREA0051_EASTSIDE +AREA0052_EASTSIDE +AREA0053_EASTSIDE +AREA0054_EASTSIDE +AREA0055_EASTSIDE +AREA0056_EASTSIDE +AREA0057_WESTSIDE +AREA0058_WESTSIDE +AREA0059_WESTSIDE +AREA005_SICMESTOA_SICRADNICE +AREA005_SOUTHPORT_PORT +AREA0060_WESTSIDE +AREA0061_WESTSIDE +AREA0062_WESTSIDE +AREA0063_WESTSIDE +AREA0064_WESTSIDE +AREA0065_WESTSIDE +AREA0066_MIDTOWN +AREA0067_MIDTOWN +AREA0068_MIDTOWN +AREA0069_MIDTOWN +AREA006_1_Respray +AREA006_SOUTHPORT_port +AREA006_SOUTHPORT_PORT +AREA0070_OYSTERBAY +AREA0072_OYSTERBAY +AREA0073_MILLS +AREA0074_MILLS +AREA0075_SOUTHPORT +AREA0076_SOUTHPORT +AREA0077_PORT +AREA0078_PORT +AREA0079_PORT +AREA007_0_Respray +AREA007_SOUTHPORT_OYSTERBAY +AREA0080_MILLN +AREA0081_MILLN +AREA0082_MILLN +AREA0083_MILLS +AREA0084_MILLN +AREA0085_MILLN +AREA0086_MILLN +AREA0087_MILLN +AREA0088_TUNEL +AREA0089_TUNEL +AREA008_1_Fast +AREA008_SOUTHPORT_port +AREA0090_GREENFIELD +AREA0091_MILLS +AREA0092_WESTSIDE +AREA0093_CHINATOWN +AREA0094_CHINATOWN +AREA0095_CHINATOWN +AREA0096_CHINATOWN +AREA0097_SEAGIFT +AREA0098_ITALY +AREA0099_SHEXTERIER +AREA009_0_Fast +AREA009_SOUTHPORT_port +AREA009_SOUTHPORT_PORT +AREA0100_OYSTERBAY +AREA010_1_Odevy +AREA010_SOUTHPORT_PORT +AREA010_SOUTHPORT_port +AREA011_0_Odevy +AREA011_SOUTHPORT_TUNEL +AREA012_1_Respray +AREA012_SOUTHPORT_WESTSIDE +AREA013_0_Respray +AREA013_WESTSIDE_SOUTHPORT +AREA014_1_Gas +AREA014_WESTSIDE_uppertown +AREA015_0_Gas +AREA015_UPPERTOWN_WESTSIDE +AREA016_1_Odevy +AREA016_SOUTHPORT_MIDTOWN +AREA017_0_Odevy +AREA017_SOUTHPORT_midtown +AREA018_1_Bruno +AREA018_SOUTHPORT_MIDTOWN +AREA019_0_Bruno +AREA019_MIDTOWN_EASTSIDE +AREA0200-WESTSIDE-MIDTOWN +AREA0201-WESTSIDE-UPPERTOWN +AREA0202-ITALY-UPPERTOWN +AREA0203-ITALY-MILLNEW +AREA0204-UPPERTOWN-KINGSTONE +AREA0205-MILLNEW-ITALY +AREA0206-OYSTERBAY-MIDTOWN +AREA0207-MIDTOWN-OYSTERBAY +AREA0208-MIDTOWN-EASTSIDE +AREA020_SOUTHPORT_oysterbay +AREA0210-HIGH-GVINTERIER +AREA0211-CHINATOWN-MILLNEW +AREA0212-CHINATOWN-EASTSIDE +AREA0213-EASTSIDE-MIDTOWN +AREA0214-OYSTERBAY-SOUTHPORT +AREA0215-OYSTERBAY-MILLS +AREA0216-EASTSIDE-MIDTOWN +AREA0217-UPPERTOWN-ITALY +AREA0218-UPPERTOWN-ITALY +AREA0219-HUNTERS-GREENFIELD +AREA021_SOUTHPORT_OYSTERBAY +AREA0220-WESTSIDE-EASTSIDE +AREA0221-SANDISLAND-TUNEL +AREA0222-SOUTHPORT-TUNEL +AREA0223-DIPTON-KINGSTONE +AREA0224-RIVERSIDE-SHEXTERIER +AREA0225-ITALY-RIVERSIDE +AREA0226-RIVERSIDE-ITALY +AREA0227-OYSTERBAY-MILLS +AREA0228-SOUTHPORT-MIDTOWN +AREA0229-MIDTOWN-WESTSIDE +AREA022_1_Illia +AREA022_SOUTHPORT_oysterbay +AREA0230-MILLNEW-ITALY +AREA0231-SOUTHPORT-OYSTERBAY +AREA023_0_Illia +AREA023_SOUTHPORT_MIDTOWN +AREA023_SOUTHPORT_oysterbay +AREA024_1_Mona +AREA024_SOUTHPORT_MIDTOWN +AREA025_0_Mona +AREA025_SOUTHPORT_MIDTOWN +AREA026_1_Odevy +AREA026_SOUTHPORT_MIDTOWN +AREA027_0_Odevy +AREA027_SOUTHPORT_MIDTOWN +AREA028_SOUTHPORT_MIDTOWN +AREA029_SOUTHPORT_westside +AREA030_1_Fast +AREA030_SOUTHPORT_MIDTOWN +AREA031_0_Fast +AREA031_MIDTOWN_SOUTHPORT +AREA032_1_Gas +AREA032_MIDTOWN_SOUTHPORT +AREA033_0_Gas +AREA033_MIDTOWN_SOUTHPORT +AREA034_1_Fast +AREA034_MIDTOWN_WESTSIDE +AREA035_0_Fast +AREA035_MIDTOWN_SOUTHPORT +AREA036_1_Odevy +AREA036_MIDTOWN_OYSTERBAY +AREA037_0_Odevy +AREA037_OYSTERBAY_SOUTHPORT +AREA038_1_Irish +AREA038_OYSTERBAY_MILLS +AREA039_0_Irish +AREA039_MILLS_OYSTERBAY +AREA040_1_Harry +AREA041_0_Harry +AREA041_MIDTOWN_EASTSIDE +AREA042_1_Odevy +AREA042_MIDTOWN_OYSTERBAY +AREA043_0_Odevy +AREA043_MIDTOWN_EASTSIDE +AREA044_1_Fast +AREA044_MIDTOWN_WESTSIDE +AREA045_0_Fast +AREA045_MIDTOWN_WESTSIDE +AREA046_1_Odevy +AREA046_MIDTOWN_SOUTHPORT +AREA047_0_Odevy +AREA047_MIDTOWN_EASTSIDE +AREA048_EASTSIDE_MIDTOWN +AREA049_EASTSIDE_UPPERTOWN +AREA050_1_Respray +AREA050_EASTSIDE_ITALY +AREA051_0_Respray +AREA051_EASTSIDE_chinatown +AREA052_1_Deli +AREA052_EASTSIDE_CHINATOWN +AREA053_0_Deli +AREA053_EASTSIDE_chinatown +AREA054_EASTSIDE_chinatown +AREA055_EASTSIDE_MIDTOWN +AREA056_1_Respray +AREA056_EASTSIDE_MIDTOWN +AREA057_0_Respray +AREA057_UPPERTOWN_EASTSIDE +AREA058_1_Respray +AREA058_UPPERTOWN_eastside +AREA058_UPPERTOWN_WESTSIDE +AREA059_0_Respray +AREA059_OYSTERBAY_SOUTHPORT +AREA060_1_Subway +AREA060_OYSTERBAY_midtown +AREA061_0_Subway +AREA061_OYSTERBAY_MIDTOWN +AREA062_1_04MArcade +AREA062_SANDISLAND_HUNTERS +AREA063_0_04MArcade +AREA063_SANDISLAND_HUNTERS +AREA064_1_Odevy +AREA064_HUNTERS_SANDISLAND +AREA065_0_Odevy +AREA065_HUNTERS_SANDISLAND +AREA066_1_Respray +AREA067_0_Respray +AREA067_HUNTERS_GREENFIELD +AREA068_1_OdevyL +AREA068_HUNTERS_GREENFIELD +AREA069_0_OdevyL +AREA069_GREENFIELD_KINGSTONE +AREA070_1_Lonestar +AREA070_GREENFIELD_HUNTERS +AREA071_0_Lonestar +AREA071_GREENFIELD_KINGSTONE +AREA072_1_05Byt +AREA072_GREENFIELD_hunters +AREA073_0_05Byt +AREA073_GREENFIELD_KINGSTONE +AREA074_1_Garage +AREA074_GREENFIELD_KINGSTONE +AREA075_0_Garage +AREA075_KINGSTONE_GREENFIELD +AREA076_1_Gunshop +AREA076_KINGSTONE_GREENFIELD +AREA077_0_Gunshop +AREA077_KINGSTONE_GREENFIELD +AREA078_1_Gunshop +AREA078_KINGSTONE_greenfield +AREA079_0_Gunshop +AREA079_KINGSTONE_DIPTON +AREA079_KINGSTONE_dipton +AREA07_ITALY +AREA080_1_Respray +AREA080_KINGSTONE_DIPTON +AREA081_0_Respray +AREA081_KINGSTONE_DIPTON +AREA083_DIPTON_KINGSTONE +AREA084_1_Gunshop +AREA084_DIPTON_RIVERSIDE +AREA085_0_Gunshop +AREA085_DIPTON_RIVERSIDE +AREA086_1_Gunshop +AREA086_RIVERSIDE_DIPTON +AREA087_0_Gunshop +AREA087_HUNTERS_greenfield +AREA088_1_Fredy +AREA088_HUNTERS_UPPERTOWN +AREA089_0_Fredy +AREA089_UPPERTOWN_HUNTERS +AREA090c_1_Subway +AREA090_UPPERTOWN_WESTSIDE +AREA091_UPPERTOWN_westside +AREA092_1_Giuseppe +AREA092_UPPERTOWN_hunters +AREA093_0_Giuseppe +AREA093_UPPERTOWN_westside +AREA094_1_Gunshop +AREA094_UPPERTOWN_WESTSIDE +AREA095_0_Gunshop +AREA095_UPPERTOWN_WESTSIDE +AREA096_1_Gunshop +AREA096_UPPERTOWN_westside +AREA097_0_Gunshop +AREA097_UPPERTOWN_HIGH +AREA098_1_Gunshop +AREA098_UPPERTOWN_ITALY +AREA099_0_Gunshop +AREA099_UPPERTOWN_ITALY +AREA100_UPPERTOWN_ITALY +AREA101_UPPERTOWN_WESTSIDE +AREA102_1_Respray +AREA102_WESTSIDE_MIDTOWN +AREA103_0_Respray +AREA103_WESTSIDE_EASTSIDE +AREA104_WESTSIDE_EASTSIDE +AREA105_UPPERTOWN_WESTSIDE +AREA106_1_Odevy +AREA106_HUNTERS_UPPERTOWN +AREA107_0_Odevy +AREA107_KINGSTONE_UPPERTOWN +AREA108_1_VitohouseB12 +AREA108_UPPERTOWN_KINGSTONE +AREA109_0_VitohouseB12 +AREA109_UPPERTOWN_KINGSTONE +AREA110_1_Joesflat_chodba +AREA110_OYSTERBAY_CHINATOWN +AREA111_0_Joesflat_chodba +AREA111_OYSTERBAY_MILLS +AREA112_1_Franhome +AREA112_OYSTERBAY_MILLS +AREA113_0_Franhome +AREA114_OYSTERBAY_MIDTOWN +AREA115_OYSTERBAY_MILLS +AREA116_1_Bruski +AREA116_OYSTERBAY_MILLS +AREA117_0_Bruski +AREA117_OYSTERBAY_CHINATOWN +AREA118_1_Erik_chodba +AREA118_ITALY_UPPERTOWN +AREA119_0_Erik_chodba +AREA119_ITALY_UPPERTOWN +AREA120_1_Gunshop +AREA120_ITALY_UPPERTOWN +AREA121_0_Gunshop +AREA121_ITALY_uppertown +AREA122_1_Odevy +AREA122_ITALY_EASTSIDE +AREA123_0_Odevy +AREA123_ITALY_eastside +AREA124_1_03Price +AREA124_ITALY_eastside +AREA125_0_03Price +AREA125_ITALY_eastside +AREA126_ITALY_MILLNEW +AREA127_ITALY_MILLNEW +AREA128_1_Respray +AREA129_0_Respray +AREA129_ITALY_MILLNEW +AREA130_1_Respray +AREA130_MILLNEW_ITALY +AREA131_0_Respray +AREA131_CHINATOWN_MILLNEW +AREA132_ITALY_MILLNEW +AREA133_ITALY_MILLN +AREA134_ITALY_MILLNEW +AREA135_ITALY_MILLNEW +AREA136_1_Gunshop +AREA136_ITALY_MILLNEW +AREA137_0_Gunshop +AREA137_ITALY_UPPERTOWN +AREA138_ITALY_MILLNEW +AREA139_ITALY_millnew +AREA140_1_ElGreco +AREA140_ITALY_uppertown +AREA141_0_ElGreco +AREA141_MILLNEW_ITALY +AREA142_1_Gvexterier +AREA143_0_Gvexterier +AREA143_SANDISLAND_TUNEL +AREA144_1_SHkanal +AREA144_SANDISLAND_HUNTERS +AREA145_0_SHkanal +AREA145_SANDISLAND_HUNTERS +AREA146_1_Falcone_satna +AREA147_0_Falcone_satna +AREA147_OYSTERBAY_chinatown +AREA148_1_Psycho +AREA148_MILLS_OYSTERBAY +AREA149_0_Psycho +AREA149_EASTSIDE_CHINATOWN +AREA150_1_Vitovila +AREA150_CHINATOWN_OYSTERBAY +AREA151_0_Vitovila +AREA151_CHINATOWN_SEAGIFT +AREA152_CHINATOWN_MILLN +AREA153_CHINATOWN_OYSTERBAY +AREA154_1_15Planetex +AREA154_CHINATOWN_OYSTERBAY +AREA155_0_15Planetex +AREA155_CHINATOWN_OYSTERBAY +AREA156_1_VitoA1 +AREA156_CHINATOWN_MILLNEW +AREA157_0_VitoA1 +AREA159_UPPERTOWN_HIGH +AREA160_HIGH_hill +AREA161_0_SGoffice +AREA161_HIGH_UPPERTOWN +AREA162_1_SGoffice +AREA162_HIGH_uppertown +AREA163_0_Argaraz +AREA163_HIGH_HILL +AREA164_1_Argaraz +AREA164_HIGH_GVINTERIER +AREA165_0_Derek +AREA165_HIGH_uppertown +AREA166_1_Derek +AREA166_HIGH_UPPERTOWN +AREA167_0_Odevy +AREA167_HIGH_GVINTERIER +AREA168_1_Odevy +AREA168_ITALY_RIVERSIDE +AREA169_0_Gas +AREA169_RIVERSIDE_italy +AREA170_1_Gas +AREA170_ITALY_MILLN +AREA171_0_Gas +AREA171_ITALY_MILLN +AREA172_1_Gas +AREA172_MILLNEW_MILLN +AREA173_0_Odevy +AREA174_1_Odevy +AREA175_0_Gunshop +AREA175_MILLN_ITALY +AREA176_1_Gunshop +AREA176_MILLN_MILLS +AREA177_0_Respray +AREA177_CHINATOWN_mills +AREA178_1_Respray +AREA178_CHINATOWN_MILLS +AREA179_0_Odevy +AREA179_MILLS_chinatown +AREA180_1_Odevy +AREA180_MILLN_MILLS +AREA181_0_Illia +AREA181_MILLS_MILLN +AREA182_1_Illia +AREA182_MILLS_MILLN +AREA183_0_Gunshop +AREA183_MILLS_MILLN +AREA184_1_Gunshop +AREA184_MILLS_MILLN +AREA185_MILLN_CHINATOWN +AREA186_MILLN_CHINATOWN +AREA187_CHINATOWN_MILLN +AREA188_MILLN_MILLS +AREA189_0_Respray +AREA189_CHINATOWN_milln +AREA190_1_Respray +AREA190_CHINATOWN_MILLN +AREA191_0_Gas +AREA191_MILLN_CHINATOWN +AREA192_1_Gas +AREA192_MILLN_MILLS +AREA193_MILLNEW_MILLN +AREA194_MILLN_foundry +AREA195_0_Illia +AREA195_MILLN_FOUNDRY +AREA196_1_Illia +AREA196_FOUNDRY_MILLN +AREA197_0_Maria_Agnelo +AREA197_MILLN_MILLNEW +AREA198_1_Maria_Agnelo +AREA198_MILLN_MILLS +AREA199_0_04Roof_MArcade +AREA199_MILLN_MILLS +AREA200_1_04Roof_MArcade +AREA200_MILLN_mills +AREA201_0_construction_prizemi +AREA201_EASTSIDE_ITALY +AREA202_1_construction_prizemi +AREA202_EASTSIDE_chinatown +AREA203_0_cathouse +AREA203_CHINATOWN_EASTSIDE +AREA204_1_cathouse +AREA204_WESTSIDE_uppertown +AREA204_WESTSIDE_UPPERTOWN +AREA205_0_triads0 +AREA205_HIGH_UPPERTOWN +AREA206_1_triads0 +AREA206_HIGH_UPPERTOWN +AREA207_SOUTHPORT_OYSTERBAY +AREA208_OYSTERBAY_SOUTHPORT +AREA209_0_Gas +AREA209_OYSTERBAY_SOUTHPORT +AREA210_1_Gas +AREA210_OYSTERBAY_MILLS +AREA211_0_Gas +AREA211_MILLS_OYSTERBAY +AREA212_1_Gas +AREA212_MILLS_oysterbay +AREA213_0_vitoa3shop +AREA213_MILLS_oysterbay +AREA214_1_vitoa3shop +AREA214_MILLS_MILLN +AREA215_0_Gas +AREA215_MILLS_MILLN +AREA216_1_Gas +AREA216_MILLS_MILLN +AREA217_MILLS_OYSTERBAY +AREA218_MILLS_OYSTERBAY +AREA219_0_Fast +AREA219_MILLS_OYSTERBAY +AREA220_1_Fast +AREA220_MILLS_OYSTERBAY +AREA221_1_04Roof_MArcade +AREA221_MILLS_OYSTERBAY +AREA222_1_crazy_horse +AREA222_MILLS_milln +AREA223_0_crazy_horse +AREA223_MILLS_MILLN +AREA224_1_crazy_horse2 +AREA224_MILLS_MILLN +AREA225_0_crazy_horse2 +AREA225_MILLS_OYSTERBAY +AREA226_1_Joesflat_chodba_b +AREA226_MILLS_OYSTERBAY +AREA227_0_Joesflat_chodba_b +AREA227_MILLS_OYSTERBAY +AREA228_0_l_garage +AREA228_MILLS_OYSTERBAY +AREA229_1_l_garage +AREA229_MILLS_MILLN +AREA230_0_mtrain +AREA230_MILLS_OYSTERBAY +AREA231_1_mtrain +AREA231_MILLS_oysterbay +AREA232_MILLS_oysterbay +AREA233_CHINATOWN_MILLNEW +AREA234_0_SHkanal_z +AREA234_CHINATOWN_eastside +AREA235_1_SHkanal_z +AREA235_MIDTOWN_WESTSIDE +AREA236_0_dlc_printery_storage +AREA236_HILL_HIGH +AREA237_1_dlc_printery_storage +AREA237_UPPERTOWN_PRICEOFFICE +AREA238_0_dlc_printery_roof +AREA238_UPPERTOWN_ITALY +AREA239_1_dlc_printery_roof +AREA239_UPPERTOWN_PRICEOFFICE +AREA240_0_dlc_ring +AREA240_UPPERTOWN_ITALY +AREA241_1_dlc_ring +AREA241_UPPERTOWN_ITALY +AREA242_1_dlc_cathouse_chodba +AREA242_UPPERTOWN_ITALY +AREA243_0_dlc_cathouse_chodba +AREA243_UPPERTOWN_ITALY +AREA244_1_dlc_lokace_supermarket +AREA244_UPPERTOWN_italy +AREA245_0_dlc_lokace_supermarket +AREA245_UPPERTOWN_WESTSIDE +AREA246_UPPERTOWN_WESTSIDE +AREA247_UPPERTOWN_ITALY +AREA248_1_dlc_port +AREA248_UPPERTOWN_ITALY +AREA249_0_dlc_port +AREA249_UPPERTOWN_ITALY +AREA250_1_dlc_hotel_entrance +AREA250_ITALY_CHINATOWN +AREA251_0_dlc_hotel_entrance +AREA251_ITALY_MILLNEW +AREA252_ITALY_EASTSIDE +AREA253_CHINATOWN_ITALY +AREA254_UPPERTOWN_ITALY +AREA255_UPPERTOWN_ITALY +AREA256_ITALY_UPPERTOWN +AREA257_ITALY_UPPERTOWN +AREA258_UPPERTOWN_ITALY +AREA259_DIPTON_UPPERTOWN +AREA260_CHINATOWN_ITALY +AREA261_CHINATOWN_eastside +AREA262_CHINATOWN_EASTSIDE +AREA263_UPPERTOWN_DIPTON +AREA264_FOUNDRY_MILLN +AREA265_FOUNDRY_MILLN +AREA266_FOUNDRY_milln +AREA267_FOUNDRY_milln +AREA269_FOUNDRY_MILLN +AREA270_FOUNDRY_MILLN +AREA271_FOUNDRY_FOUNDRYINT +AREA272_GVINTERIER_high +AREA273_HIGH_GVINTERIER +AREA274_RIVERSIDE_SHEXTERIER +AREA275_RIVERSIDE_SHEXTERIER +AREA276_RIVERSIDE_SHEXTERIER +AREA278_RIVERSIDE_DIPTON +AREA279_ITALY_riverside +AREA280_ITALY_MILLNEW +AREA281_ITALY_MILLNEW +AREA282_JOESFLAT_ITALY +AREA283_JOESFLAT_ITALY +AREA284_EASTSIDE_MIDTOWN +AREA285_EASTSIDE_MIDTOWN +AREA286_FALCONE_EASTSIDE +AREA287_KINGSTONE_DIPTON +AREA287_KINGSTONE_dipton +AREA288_KINGSTONE_dipton +AREA289_KINGSTONE_dipton +AREA289_KINGSTONE_DIPTON +AREA290_KINGSTONE_ERIK +AREA291_HILL_PLANETIN +AREA292_HILL_HIGH +AREA293_HILL_HIGH +AREA294_TRIADS1_prazdna +AREA295_TRIADS1_triads2 +AREA296_TRIADS1_TRIADS2 +AREA297_TRIADS1_TRIADS2 +AREA298_TRIADS1_TRIADS2 +AREA299_TRIADS2_TRIADS1 +AREA300_TRIADS2_triads1 +AREA301_TRIADS2_chinatown +AREA302_TRIADS2_chinatown +AREA303_TRIADS2_CHINATOWN +AREA304_TRIADS2_CHINATOWN +AREA305_CHINATOWN_TRIADS2 +AREA306_CHINATOWN_millnew +AREA307_TRIADS2_chinatown +AREA308_TRIADS2_chinatown +AREA309_TRIADS2_chinatown +AREA310_CHINATOWN_MILLNEW +AREA311_CHINATOWN_MILLNEW +AREA312_CHINATOWN_MILLNEW +AREA313_MIDTOWN_SOUTHPORT +AREA314_MIDTOWN_WESTSIDE +AREA315_MIDTOWN_WESTSIDE +AREA316_ARPRADELNA_MIDTOWN +AREA317_ARPRADELNA_arpatro +AREA318_ARPRADELNA_arpatro +AREA319_ARPRADELNA_arpatro +AREA320_arpatro_arpradelna +AREA321_ARPATRO_arstrecha +AREA322_ARPATRO_arstrecha +AREA323_ARSTRECHA_ARPATRO +AREA324_ARSTRECHA_ARPATRO +AREA325_ARSTRECHA_arpatro +AREA326_ARSTRECHA_ARPATRO +AREA327_ARPATRO_arstrecha +AREA331_MILLNEW_CHINATOWN +AREA332_MILLNEW_MILLN +AREA333_MILLNEW_ITALY +AREA334_MILLNEW_CHINATOWN +AREA335_MILLNEW_ITALY +AREA336_MILLNEW_CHINATOWN +AREA337_ITALY_eastside +AREA338_SANDISLAND_TUNEL +AREA339_SOUTHPORT_TUNEL +AREA340_GREENFIELD_KINGSTONE +AREA341_GREENFIELD_KINGSTONE +AREA342_KINGSTONE_DIPTON +AREA343_KINGSTONE_DIPTON +AREA344_KINGSTONE_DIPTON +AREA345_KINGSTONE_DIPTON +AREA346_KINGSTONE_GREENFIELD +AREA347_KINGSTONE_DIPTON +AREA348_KINGSTONE_DIPTON +AREA349_KINGSTONE_DIPTON +AREA350_KINGSTONE_GREENFIELD +AREA351_HIGH_DIPTON +AREA352_HIGH_DIPTON +AREA353_DIPTON_HIGH +AREA354_DIPTON_RIVERSIDE +AREA355_DIPTON_KINGSTONE +AREA356_DIPTON_RIVERSIDE +AREA357_MIDTOWN_OYSTERBAY +AREA358_MILLS_OYSTERBAY +AREA359_OYSTERBAY_MILLS +AREA360_EASTSIDE_MIDTOWN +AREA365_MILLN_ITALY +AREA366_WESTSIDE_MIDTOWN +AREA367_WESTSIDE_MIDTOWN +AREA368_WESTSIDE_MIDTOWN +AREA369_WESTSIDE_MIDTOWN +AREA371_WESTSIDE_MIDTOWN +AREA372_WESTSIDE_MARKETARCADE +AREA376_WESTSIDE_MIDTOWN +AREA377_WESTSIDE_MIDTOWN +AREA379_DISTILLERY_SANDISLAND +AREA380_UPPERTOWN_PRICEOFFICE +AREA381_EASTSIDE_FALCONE +AREA383_HUNTERS_GREENFIELD +AREA384_HUNTERS_GREENFIELD +AREA385_GREENFIELD_hunters +AREA386_FALCONE_EASTSIDE +AREA389_CHINATOWN_MILLNEW +AREA390_TRIADS2_chinatown +AREA391_TRIADS2_chinatown +AREA392_CONSTRUCTIONSITE_CONSTRSTRECHA +AREA393_EASTSIDE_WESTSIDE +AREA394_EASTSIDE_MIDTOWN +AREA395_WESTSIDE_MIDTOWN +AREA396_MIDTOWN_EASTSIDE +AREA397_EASTSIDE_UPPERTOWN +AREA398_OYSTERBAY_MIDTOWN +AREA399_UPPERTOWN_westside +AREA400_UPPERTOWN_westside +AREA401_UPPERTOWN_westside +AREA402_UPPERTOWN_VITOA3CITY +AREA403_UPPERTOWN_VITOA3CITY +AREA407_RIVERSIDE_SHEXTERIER +AREA408_RIVERSIDE_SHEXTERIER +AREA409_SHEXTERIER_SHINTERIER +AREA410_CHINATOWN_SEAGIFT +AREA411_OYSTERBAY_CHINATOWN +AREA412_SOUTHPORT_PORT +AREA413_ITALY_JOESFLAT +AREA414_ITALY_JOESFLAT +AREA415_JOESFLAT_ITALY +AREA417_CHINATOWN_MILLNEW +AREA418_CHINATOWN_MILLNEW +AREA419_MIDTOWN_SOUTHPORT +AREA420_SOUTHPORT_MIDTOWN +AREA421_SOUTHPORT_MIDTOWN +AREA422_CONSTRUCTIONSITE_MIDTOWN +AREA423_SOUTHPORT_MIDTOWN +AREA424_CONSTRUCTIONSITE_CONSTRSTRECHA +AREA425_MIDTOWN_SOUTHPORT +AREA426_MIDTOWN_SOUTHPORT +AREA427_MIDTOWN_SOUTHPORT +AREA428_HILL_HIGH +AREA429_EASTSIDE_CHINATOWN +AREA430_EASTSIDE_MIDTOWN +AREA431_OYSTERBAY_MILLS +AREA432_OYSTERBAY_chinatown +AREA433_ITALY_millnew +AREA434_ITALY_millnew +AREA435_ITALY_uppertown +AREA436_UPPERTOWN_DIPTON +AREA437_ITALY_milln +AREA438_WESTSIDE_southport +AREA439_HIGH_GVINTERIER +AREA440_HIGH_GVINTERIER +AREA441_HIGH_GVINTERIER +AREA442_HIGH_GVINTERIER +AREA443_GVINTERIER_high +AREA444_HIGH_GVINTERIER +AREA445_MIDTOWN_SOUTHPORT +AREA446_SOUTHPORT_MIDTOWN +AREA447_MIDTOWN_EASTSIDE +AREA448_ITALY_UPPERTOWN +AREA449_MIDTOWN_SOUTHPORT +AREA450_SOUTHPORT_WESTSIDE +AREA451_WESTSIDE_HUNTERS +AREA452_WESTSIDE_HUNTERS +AREA455_CONSTRUCTIONSITE_CONSTRSTRECHA +AREA458_KINGSTONE_DIPTON +AREA459_KINGSTONE_DIPTON +AREA459_KINGSTONE_dipton +AREA460_KINGSTONE_dipton +AREA464_PORT_SOUTHPORT +AREA465_MIDTOWN_WESTSIDE +AREA466_ITALY_JOESFLATVARA +AREA467_JOESFLATVARA_ITALY +AREA468_JOESFLATVARA_ITALY +AREA469_JOESFLATVARA_ITALY +AREA470_ITALY_JOESFLATVARA +AREA471_ITALY_JOESFLATVARB +AREA472_JOESFLATVARB_ITALY +AREA473_JOESFLATVARB_ITALY +AREA474_JOESFLATVARB_ITALY +AREA475_ITALY_JOESFLATVARB +AREA476_ERIK_KINGSTONE +AREA477_WESTSIDE_MIDTOWN +AREA478_PRISONCELBATH_PRISONI +AREA479_PRISONCELBATH_PRISONE +AREA480_PRISONE_PRISONCELBATH +AREA481_PRISONE_PRISONI +AREA482_PRISONI_PRISONCELBATH +AREA483_PRISONI_PRISONE +AREA484_MIDTOWN_SOUTHPORT +AREA485_SOUTHPORT_WESTSIDE +AREA486_CHINATOWN_millnew +AREA487_ITALY_eastside +AREA488_SEAGIFT_CHINATOWN +AREA489_CHINATOWN_SEAGIFT +AREA490_MILLN_MILLNEW +AREA491_EASTSIDE_MIDTOWN +AREA492_EASTSIDE_MIDTOWN +AREA493_PRAZDNA_PRAZDNA +AREA494_EASTSIDE_MIDTOWN +AREA495_MIDTOWN_WESTSIDE +AREA496_CHINATOWN_OYSTERBAY +AREA497_UPPERTOWN_ITALY +AREA498_EASTSIDE_UPPERTOWN +AREA499_PORT_SOUTHPORT +AREA500_PORT_SOUTHPORT +AREA501_SOUTHPORT_PORT +AREA502_SOUTHPORT_PORT +AREA503_SOUTHPORT_PORT +AREA505_HUNTERS_GREENFIELDF +AREA506_HUNTERS_GREENFIELDF +AREA507_GREENFIELDF_KINGSTONE +AREA509_GREENFIELDF_HUNTERS +AREA510_GREENFIELDF_KINGSTONE +AREA511_GREENFIELDF_hunters +AREA512_GREENFIELDF_KINGSTONE +AREA513_GREENFIELDF_KINGSTONE +AREA514_KINGSTONE_GREENFIELDF +AREA515_KINGSTONE_GREENFIELDF +AREA516_KINGSTONE_GREENFIELDF +AREA517_KINGSTONE_greenfieldf +AREA518_GREENFIELDF_KINGSTONE +AREA519_GREENFIELDF_KINGSTONE +AREA520_KINGSTONE_GREENFIELDF +AREA521_KINGSTONE_GREENFIELDF +AREA523_HUNTERS_GREENFIELDF +AREA524_HUNTERS_GREENFIELDF +AREA525_GREENFIELDF_hunters +AREA527_SANDISLAND_TUNEL +AREA528_SOUTHPORT_TUNEL +AREA529_PRISONE_PRISONPR +AREA530_PRISONI_PRISONPR +AREA531_PRISONPR_PRISONE +AREA532_CONSTRUCTIONSITE_CONSTRSTRECHA +AREA533_SOUTHPORT_MIDTOWN +AREA534_MILLN_MILLNEW +AREA535_UPPERTOWN_WESTSIDE +AREA544_CONSTRUCTIONSITE_CONSTRSTRECHA +AREA545_CONSTRSTRECHA_CONSTRUCTIONSITE +AREA546_GREENFIELDF_HUNTERS +AREA547_GREENFIELD_HUNTERS +AREA548_DLCPRINTERY_MILLS +AREA551_DLCPRINTERY_MILLS +AREA552_DLCPRINTERY_MILLS +AREA553_DLCPRINTERY_MILLS +AREA554_DLCPRINTERY_MILLS +AREA555_MILLS_OYSTERBAY +AREA557_MILLS_MILLN +AREA558_KINGSTONE_GREENFIELD +AREA559_EASTSIDE_WESTSIDE +AREA559_KINGSTONE_GREENFIELDF +AREA560_GREENFIELD_HUNTERS +AREA561_HUNTERS_GREENFIELD +AREA562_UPPERTOWN_ITALY +AREA563_EASTSIDE_WESTSIDE +AREA564_KINGSTONE_UPPERTOWN +AREA565_OYSTERBAY_DLCCATHOUSE +AREA566_DLCCATHOUSE_PRAZDNA +AREA567_DLCCATHOUSE_PRAZDNA +AREA567_WESTSIDE_UPPERTOWN +AREA568_MIDTOWN_SOUTHPORT +AREA569_SOUTHPORT_WESTSIDE +AREA570_PRAZDNA_PRAZDNA +AREA570_PRAZDNA_PRAZDNA01 +AREA570_PRAZDNA_PRAZDNA02 +AREA570_PRAZDNA_PRAZDNA03 +AREA570_PRAZDNA_PRAZDNA04 +AREA570_PRAZDNA_PRAZDNA05 +AREA571_PRAZDNA_PRAZDNA +AREA571_PRAZDNA_PRAZDNA01 +AREA571_PRAZDNA_PRAZDNA02 +AREA571_PRAZDNA_PRAZDNA03 +AREA571_PRAZDNA_PRAZDNA04 +AREA571_PRAZDNA_PRAZDNA05 +AREA572_PRAZDNA_PRAZDNA +AREA572_PRAZDNA_PRAZDNA01 +AREA572_PRAZDNA_PRAZDNA02 +AREA572_PRAZDNA_PRAZDNA03 +AREA572_PRAZDNA_PRAZDNA04 +AREA572_PRAZDNA_PRAZDNA05 +AREA573_PRAZDNA_PRAZDNA +AREA573_PRAZDNA_PRAZDNA01 +AREA573_PRAZDNA_PRAZDNA02 +AREA573_PRAZDNA_PRAZDNA03 +AREA573_PRAZDNA_PRAZDNA04 +AREA573_PRAZDNA_PRAZDNA05 +AREA574_PRAZDNA_PRAZDNA +AREA574_PRAZDNA_PRAZDNA01 +AREA574_PRAZDNA_PRAZDNA02 +AREA574_PRAZDNA_PRAZDNA03 +AREA574_PRAZDNA_PRAZDNA04 +AREA574_PRAZDNA_PRAZDNA05 +AREA575_PRAZDNA_PRAZDNA +AREA575_PRAZDNA_PRAZDNA01 +AREA575_PRAZDNA_PRAZDNA02 +AREA575_PRAZDNA_PRAZDNA03 +AREA575_PRAZDNA_PRAZDNA04 +AREA575_PRAZDNA_PRAZDNA05 +AREA576_PRAZDNA_PRAZDNA +AREA576_PRAZDNA_PRAZDNA01 +AREA576_PRAZDNA_PRAZDNA02 +AREA576_PRAZDNA_PRAZDNA03 +AREA576_PRAZDNA_PRAZDNA04 +AREA576_PRAZDNA_PRAZDNA05 +AREA577_PRAZDNA_PRAZDNA +AREA577_PRAZDNA_PRAZDNA01 +AREA577_PRAZDNA_PRAZDNA02 +AREA577_PRAZDNA_PRAZDNA03 +AREA577_PRAZDNA_PRAZDNA04 +AREA577_PRAZDNA_PRAZDNA05 +AREA577_PRAZDNA_PRAZDNA06 +AREA577_PRAZDNA_PRAZDNA07 +AREA577_PRAZDNA_PRAZDNA08 +AREA577_PRAZDNA_PRAZDNA09 +AREA577_PRAZDNA_PRAZDNA10 +AREA577_PRAZDNA_PRAZDNA11 +AREA577_PRAZDNA_PRAZDNA12 +AREA577_PRAZDNA_PRAZDNA13 +AREA577_PRAZDNA_PRAZDNA14 +AREA577_PRAZDNA_PRAZDNA15 +AREA577_PRAZDNA_PRAZDNA16 +AREA577_PRAZDNA_PRAZDNA17 +AREA578_PRAZDNA_PRAZDNA +AREA578_PRAZDNA_PRAZDNA01 +AREA578_PRAZDNA_PRAZDNA02 +AREA578_PRAZDNA_PRAZDNA03 +AREA578_PRAZDNA_PRAZDNA04 +AREA578_PRAZDNA_PRAZDNA05 +AREA579_PRAZDNA_PRAZDNA +AREA579_PRAZDNA_PRAZDNA01 +AREA579_PRAZDNA_PRAZDNA02 +AREA579_PRAZDNA_PRAZDNA03 +AREA579_PRAZDNA_PRAZDNA04 +AREA579_PRAZDNA_PRAZDNA05 +AREA580_PRAZDNA_PRAZDNA +AREA580_PRAZDNA_PRAZDNA01 +AREA580_PRAZDNA_PRAZDNA02 +AREA580_PRAZDNA_PRAZDNA03 +AREA580_PRAZDNA_PRAZDNA04 +AREA580_PRAZDNA_PRAZDNA05 +AREA581_PRAZDNA_PRAZDNA +AREA581_PRAZDNA_PRAZDNA01 +AREA581_PRAZDNA_PRAZDNA02 +AREA581_PRAZDNA_PRAZDNA03 +AREA581_PRAZDNA_PRAZDNA04 +AREA581_PRAZDNA_PRAZDNA05 +AREA582_PRAZDNA_PRAZDNA +AREA582_PRAZDNA_PRAZDNA01 +AREA582_PRAZDNA_PRAZDNA02 +AREA582_PRAZDNA_PRAZDNA03 +AREA582_PRAZDNA_PRAZDNA04 +AREA582_PRAZDNA_PRAZDNA05 +AREA583_PRAZDNA_PRAZDNA +AREA583_PRAZDNA_PRAZDNA01 +AREA583_PRAZDNA_PRAZDNA02 +AREA583_PRAZDNA_PRAZDNA03 +AREA583_PRAZDNA_PRAZDNA04 +AREA583_PRAZDNA_PRAZDNA05 +AREA584_PRAZDNA_PRAZDNA +AREA584_PRAZDNA_PRAZDNA01 +AREA584_PRAZDNA_PRAZDNA02 +AREA584_PRAZDNA_PRAZDNA03 +AREA584_PRAZDNA_PRAZDNA04 +AREA584_PRAZDNA_PRAZDNA05 +AREA585_PRAZDNA_PRAZDNA +AREA585_PRAZDNA_PRAZDNA01 +AREA585_PRAZDNA_PRAZDNA02 +AREA585_PRAZDNA_PRAZDNA03 +AREA585_PRAZDNA_PRAZDNA04 +AREA585_PRAZDNA_PRAZDNA05 +AREA585_PRAZDNA_PRAZDNA06 +AREA585_PRAZDNA_PRAZDNA07 +AREA585_PRAZDNA_PRAZDNA08 +AREA585_PRAZDNA_PRAZDNA09 +AREA585_PRAZDNA_PRAZDNA10 +AREA585_PRAZDNA_PRAZDNA11 +AREA585_PRAZDNA_PRAZDNA12 +AREA585_PRAZDNA_PRAZDNA13 +AREA585_PRAZDNA_PRAZDNA14 +AREA585_PRAZDNA_PRAZDNA15 +AREA585_PRAZDNA_PRAZDNA16 +AREA585_PRAZDNA_PRAZDNA17 +AREA600_SANDISLAND_TUNEL +AREA601_SANDISLAND_TUNEL +AREA602_DLCSUPERMARKET_sandisland +AREA603_MILLNEW_DLCDAMROAD +AREA604_DLCDAMROAD_MILLNEW +AREA605_DLCDAMROAD_millnew +AREA606_DLCDAMROAD_DLCYACHT +AREA607_DLCDAMROAD_RIVERSIDE +AREA608_DLCDAMROAD_RIVERSIDE +AREA609_DLCDAMROAD_RIVERSIDE +AREA610_RIVERSIDE_DLCDAMROAD +AREA611_DLCDAMROAD_RIVERSIDE +AREA612_RIVERSIDE_DIPTON +AREA613_RIVERSIDE_DLCDAMROAD +AREA614_DLCDAMROAD_RIVERSIDE +AREA615_DLCDAMROAD_RIVERSIDE +AREA616_DLCDAMROAD_RIVERSIDE +AREA617_WESTSIDE_DLCHOTEL +AREA618_DLCSUPERMARKET_SANDISLAND +AREA619_DLCSUPERMARKET_sandisland +AREA620_DLCDAMROAD_DLCYACHT +AREA621_DLCDAMROAD_DLCYACHT +AREA622_DLCDAMROAD_DLCYACHT +AREA623_DLCDAMROAD_DLCYACHT diff --git a/src/Illusion.Domain/EditRefusedException.cs b/src/Illusion.Domain/EditRefusedException.cs new file mode 100644 index 0000000..575a51b --- /dev/null +++ b/src/Illusion.Domain/EditRefusedException.cs @@ -0,0 +1,14 @@ +namespace Illusion.Domain; + +/// +/// Thrown by an whose undo or redo could not be carried out - a file it has to write +/// is locked, the thing it restores is gone. leaves its stacks as they were when a +/// step throws, so the step is still there to be taken once the obstacle is out of the way; whoever asked for +/// the step shows the message instead of treating it as done. +/// +public sealed class EditRefusedException : Exception +{ + public EditRefusedException(string message) : base(message) { } + + public EditRefusedException(string message, Exception inner) : base(message, inner) { } +} diff --git a/src/Illusion.Domain/Materials/IMaterialSlotEditor.cs b/src/Illusion.Domain/Materials/IMaterialSlotEditor.cs index e803e89..65b08e4 100644 --- a/src/Illusion.Domain/Materials/IMaterialSlotEditor.cs +++ b/src/Illusion.Domain/Materials/IMaterialSlotEditor.cs @@ -13,4 +13,14 @@ public interface IMaterialSlotEditor : ISceneSource /// Repoints a slot at another material hash. False when the slot is out of range. bool SetSlotMaterial(int slotIndex, ulong hash); + + /// + /// The hash every slot of every level binds, level by level. What an undo of a re-point puts back: the + /// mirror into further levels goes by the slot's OLD hash, and that cannot be run backwards - a level whose + /// slots were A and B is B and B after "A to B", and "B to A" makes it A and A. + /// + IReadOnlyList GetSlotTable(); + + /// Puts a table taken with back. False when it no longer fits the mesh. + bool SetSlotTable(IReadOnlyList table); } diff --git a/src/Illusion.Formats/CityAreas/CityAreasTable.cs b/src/Illusion.Formats/CityAreas/CityAreasTable.cs new file mode 100644 index 0000000..d581e6c --- /dev/null +++ b/src/Illusion.Formats/CityAreas/CityAreasTable.cs @@ -0,0 +1,174 @@ +using System.Text; + +namespace Illusion.Formats.CityAreas; + +/// +/// cityareas.bin read and written back as it stands: the table that says, for each AREA volume of +/// city_univers, which one or two districts the game keeps loaded while the player is inside it. +/// +/// Layout: "ratc", version (1), entry count, byte length of the district names; the district names, each +/// ended by a zero; then the entries - the volume's name ended by a zero, two 16-bit offsets into the district +/// names (0xFFFF for "none") and one byte. In the shipped file that byte is 1 on every entry that names two +/// districts but one, and 0 on every entry that names a single district. +/// +/// +public sealed class CityAreasTable +{ + private const uint Magic = 0x63746172; // "ratc" + private const ushort None = 0xFFFF; + private static readonly Encoding Text = Encoding.Latin1; + + private readonly uint _version; + private byte[] _districts; + private readonly List<(string Name, ushort Target1, ushort Target2, byte Flag)> _entries = []; + + /// One line of the table, its districts resolved to their names. + public sealed record Entry(string Name, string? Target1, string? Target2, byte Flag); + + private CityAreasTable(uint version, byte[] districts) + { + _version = version; + _districts = districts; + } + + /// The bytes are not a city areas table. + public static CityAreasTable Parse(ReadOnlySpan bytes) + { + if (bytes.Length < 16 || BitConverter.ToUInt32(bytes) != Magic) throw new FileFormatException("not a cityareas.bin (no \"ratc\" at its start)"); + uint version = BitConverter.ToUInt32(bytes[4..]); + int count = BitConverter.ToInt32(bytes[8..]), namesLength = BitConverter.ToInt32(bytes[12..]); + if (count < 0 || namesLength < 0 || namesLength > ushort.MaxValue || 16 + namesLength > bytes.Length) + { + throw new FileFormatException("cityareas.bin: its header does not fit the file"); + } + + var table = new CityAreasTable(version, bytes.Slice(16, namesLength).ToArray()); + int at = 16 + namesLength; + for (int i = 0; i < count; i++) + { + int end = at + bytes[at..].IndexOf((byte)0); + if (end < at || end + 6 > bytes.Length) throw new FileFormatException($"cityareas.bin: entry {i} runs past the end of the file"); + string name = Text.GetString(bytes[at..end]); + table._entries.Add((name, BitConverter.ToUInt16(bytes[(end + 1)..]), BitConverter.ToUInt16(bytes[(end + 3)..]), bytes[end + 5])); + at = end + 6; + } + if (at != bytes.Length) throw new FileFormatException($"cityareas.bin: {bytes.Length - at} byte(s) left over after its {count} entries"); + return table; + } + + public byte[] ToBytes() + { + using var stream = new MemoryStream(); + using (var writer = new BinaryWriter(stream, Text, leaveOpen: true)) + { + writer.Write(Magic); + writer.Write(_version); + writer.Write(_entries.Count); + writer.Write(_districts.Length); + writer.Write(_districts); + foreach ((string name, ushort target1, ushort target2, byte flag) in _entries) + { + writer.Write(Text.GetBytes(name)); + writer.Write((byte)0); + writer.Write(target1); + writer.Write(target2); + writer.Write(flag); + } + } + return stream.ToArray(); + } + + /// The district names the table holds, in its order. + public IReadOnlyList Districts + { + get + { + var names = new List(); + for (int at = 0; at < _districts.Length;) + { + int end = Array.IndexOf(_districts, (byte)0, at); + if (end < 0) end = _districts.Length; + names.Add(Text.GetString(_districts, at, end - at)); + at = end + 1; + } + return names; + } + } + + public IReadOnlyList Entries => [.. _entries.Select(e => new Entry(e.Name, NameAt(e.Target1), NameAt(e.Target2), e.Flag))]; + + public Entry? Find(string name) + { + int at = _entries.FindIndex(e => string.Equals(e.Name, name, StringComparison.OrdinalIgnoreCase)); + return at < 0 ? null : Entries[at]; + } + + /// + /// Adds a line for a volume: the one or two districts it keeps loaded. A district the table has not named + /// yet is added to its names. The byte is 1 for two districts and 0 for one - what the shipped table has. + /// + /// Null on success, otherwise why not; nothing is changed on a refusal. + public string? Add(string name, string target1, string? target2) + { + if (string.IsNullOrWhiteSpace(name) || name.Any(c => c is < ' ' or > '~')) return "the zone's name must be plain Latin text"; + if (Find(name) != null) return $"the table already has a line for '{name}'"; + if (string.IsNullOrWhiteSpace(target1)) return "a zone names at least one district"; + foreach (string district in new[] { target1, target2 ?? "" }) + { + if (district.Any(c => c is < ' ' or > '~')) return $"'{district}' is not a plain district name"; + } + if (string.Equals(target1, target2, StringComparison.OrdinalIgnoreCase)) return "the two districts are the same one"; + + byte[] grown = _districts; + ushort Offset(string district) + { + for (int at = 0; at < grown.Length;) + { + int end = Array.IndexOf(grown, (byte)0, at); + if (end < 0) end = grown.Length; + if (string.Equals(Text.GetString(grown, at, end - at), district, StringComparison.OrdinalIgnoreCase)) return (ushort)at; + at = end + 1; + } + int offset = grown.Length; + grown = [.. grown, .. Text.GetBytes(district), 0]; + return (ushort)offset; + } + + ushort first = Offset(target1); + ushort second = string.IsNullOrWhiteSpace(target2) ? None : Offset(target2); + if (grown.Length >= None) return "the table's district names would not fit its 16-bit offsets"; + _districts = grown; + _entries.Add((name, first, second, (byte)(second == None ? 0 : 1))); + return null; + } + + /// + /// Takes a volume's line out. A district only that line named is taken out of the names with it when it + /// stands at their end - where puts a new one - so a line added and removed leaves the + /// table as it was. False when the table had none. + /// + public bool Remove(string name) + { + int at = _entries.FindIndex(e => string.Equals(e.Name, name, StringComparison.OrdinalIgnoreCase)); + if (at < 0) return false; + (_, ushort target1, ushort target2, _) = _entries[at]; + _entries.RemoveAll(e => string.Equals(e.Name, name, StringComparison.OrdinalIgnoreCase)); + + // from the end backwards: of two districts added together, the second stands after the first + foreach (ushort offset in new[] { target1, target2 }.Where(o => o != None && o < _districts.Length).Distinct().OrderDescending()) + { + int end = Array.IndexOf(_districts, (byte)0, offset); + bool last = end == _districts.Length - 1 && (offset == 0 || _districts[offset - 1] == 0); + if (last && !_entries.Any(e => e.Target1 == offset || e.Target2 == offset)) _districts = _districts[..offset]; + } + return true; + } + + private string? NameAt(ushort offset) + { + if (offset == None || offset >= _districts.Length) return null; + int end = Array.IndexOf(_districts, (byte)0, offset); + if (end < 0) end = _districts.Length; + return Text.GetString(_districts, offset, end - offset); + } +} diff --git a/src/Illusion.Formats/Frames/ObjectTypes/FrameObjectModel.cs b/src/Illusion.Formats/Frames/ObjectTypes/FrameObjectModel.cs index 2c6b195..db8e90f 100644 --- a/src/Illusion.Formats/Frames/ObjectTypes/FrameObjectModel.cs +++ b/src/Illusion.Formats/Frames/ObjectTypes/FrameObjectModel.cs @@ -129,6 +129,15 @@ public FrameSkeletonHierarchy SkeletonHierarchy /// . public int SplitBlockSizeStored => physSplitSize; + /// The three stored counters as they stand — read to remember a split table exactly as it was, + /// written to put it back that way (an undo). Anything that BUILDS a table calls + /// instead. + public (int PhysSplitSize, int HitBoxSize, short NPhysSplits) StoredSplitCounters + { + get { return SplitCounters; } + set { SplitCounters = value; } + } + public int ComputeSplitBlockSize() { if (blendMeshSplits is not { Length: > 0 }) return 0; diff --git a/src/Illusion.Formats/StreamMap/StreamMapEditor.cs b/src/Illusion.Formats/StreamMap/StreamMapEditor.cs index 0c3bcc7..20a114e 100644 --- a/src/Illusion.Formats/StreamMap/StreamMapEditor.cs +++ b/src/Illusion.Formats/StreamMap/StreamMapEditor.cs @@ -89,6 +89,36 @@ public static class StreamMapEditor /// private static readonly Encoding PoolEncoding = Encoding.Latin1; + /// + /// Writes a patched StreamMap over , after putting what stood there beside it + /// as a backup, and returns the backup's path. The backup goes down first. + /// + /// It never overwrites an earlier one. The name is derived from the target, so it is the same on every + /// edit: a second pass over the same file would have replaced the pristine backup with the + /// already-patched file, leaving two copies of modified data and no way back. The first backup is the + /// valuable one, so it is kept as <name>_old.bin and later passes get _old.2.bin, + /// _old.3.bin and so on. + /// + /// + public static string WriteWithBackup(string filePath, byte[] original, byte[] patched) + { + ArgumentNullException.ThrowIfNull(filePath); + ArgumentNullException.ThrowIfNull(original); + ArgumentNullException.ThrowIfNull(patched); + string directory = Path.GetDirectoryName(filePath) ?? "."; + string stem = Path.GetFileNameWithoutExtension(filePath) + "_old"; + string extension = Path.GetExtension(filePath); + + string backup = Path.Combine(directory, stem + extension); + for (int n = 2; File.Exists(backup); n++) + { + backup = Path.Combine(directory, stem + "." + n.ToString(System.Globalization.CultureInfo.InvariantCulture) + extension); + } + File.WriteAllBytes(backup, original); + File.WriteAllBytes(filePath, patched); + return backup; + } + /// /// Replaces with in every selected string /// field. Returns what it found and, unless , the patched bytes. diff --git a/src/Illusion.Mcp/EditorModels.cs b/src/Illusion.Mcp/EditorModels.cs index f43e622..8f28afc 100644 --- a/src/Illusion.Mcp/EditorModels.cs +++ b/src/Illusion.Mcp/EditorModels.cs @@ -46,6 +46,27 @@ public sealed record CarCloneInfo(string Name, int VehicleId, int TrafficRows, i /// not in the archive, and no longer in the manifest. public sealed record PackedArchive(string Archive, string? Backup, IReadOnlyList? Dropped = null); +/// One triangle of a mesh, in world space, with the material of its slot and its level of detail. +public sealed record TriangleInfo(int Lod, string Material, float[] A, float[] B, float[] C); + +/// What a box takes out of a mesh: how many triangles on each level of detail, whether they were +/// hidden or only found, and the first of them. +public sealed record HiddenTrianglesInfo(int Triangles, IReadOnlyList PerLod, bool Applied, IReadOnlyList Sample); + +/// One material slot of a mesh: its index, the material it draws with (null when no loaded library +/// knows the hash), how many triangles it covers, and whether this call re-pointed it. +public sealed record MeshSlotInfo(int Slot, string? Material, int Triangles, bool Changed); + +/// One collision file of the open scene: its placements, the hulls it carried, how many of those no +/// placement referenced, and whether they were removed or only counted. +public sealed record UnusedHullsInfo(string Layer, int Placements, int Hulls, int Unused, bool Removed); + +/// One placement of a city_crash prop (a tree, a lamp, a bin): the prop's name, the placement's id in the +/// table, where it stands, whether the other season holds the same placement, and whether the two are kept in +/// step — an edit of a linked placement is made to its twin, and deleting it deletes both; one the user +/// unlinked ("In all seasons" off) loses only the season on screen. +public sealed record CrashPlacementInfo(string Prop, int Id, float[] Position, bool BothSeasons, bool SeasonLinked); + /// One car built under another's name: the model name the replaced archives are keyed by, each archive /// with the backup of what it held, and what was left as it was. public sealed record CarSubstituteInfo(string Model, IReadOnlyList Packed, IReadOnlyList Notes); @@ -126,3 +147,17 @@ public sealed record ObjectImportOutcome( /// Where the viewport camera is. Yaw and pitch are in radians, as the camera keeps them. public sealed record CameraInfo(float[] Position, float Yaw, float Pitch, float OrbitDistance); + +/// One load zone at a point: the districts it keeps loaded, whether the point is inside its volume +/// (and how far outside when not), its box in world space, whether it is of the kind that loads its districts +/// for a player who appears inside it - which its name decides (two words after the number) - and whether it +/// was added to the zones the game ships with. +public sealed record LoadZoneInfo(string Name, IReadOnlyList Districts, bool Inside, float OutsideBy, float[] BoxMin, float[] BoxMax, + bool LoadsOnArrival = false, bool Added = false); + +/// A face of a load zone moved - or, when not applied, what moving it would do: from where to where on +/// its axis (world), the zone's box afterwards, the districts it asks for, and the working-copy file written. +/// is which copy of city_univers.sds this is about: "base" or a DLC's folder name. +public sealed record LoadZoneMoveInfo( + string Copy, string Zone, string Face, float From, float To, float[] BoxMin, float[] BoxMax, IReadOnlyList Districts, bool Applied, + string? File, string Archive); diff --git a/src/Illusion.Mcp/IEditorSession.cs b/src/Illusion.Mcp/IEditorSession.cs index 031cb89..ce316f1 100644 --- a/src/Illusion.Mcp/IEditorSession.cs +++ b/src/Illusion.Mcp/IEditorSession.cs @@ -160,7 +160,60 @@ public interface IEditorSession /// nothing of the game is written. Null on success. string? ExportCarForM2o(string car, string? output, string? resource, out M2oExportInfo? result); + /// Finds — and with hides, as one undoable edit — the triangles of a mesh + /// whose corners all lie inside a world-space box, without rebuilding the mesh. + /// says what to do when other objects draw the same geometry: null refuses, "own" gives the mesh a copy of + /// its own first, "all" hides the triangles on every one of them. Null on success. + string? HideTriangles(string name, float[] boxMin, float[] boxMax, string? material, bool apply, int sample, string? shared, + out HiddenTrianglesInfo? result); + + /// The material slots of a mesh; with and , that + /// slot is first re-pointed at the named material, as one undoable edit. No geometry and no UV is touched. + /// Null on success. + string? MeshMaterials(string name, int? slot, string? material, out IReadOnlyList slots); + + /// Counts — and with removes, as one undoable edit — the hulls no placement + /// references, in every collision file of the open scene. Null on success. + string? UnusedHulls(bool apply, out IReadOnlyList result); + + /// Lists — and with removes, as one undoable edit and in both seasons where + /// the placement is linked — the city_crash props standing inside a world-space box. + /// is how many the box holds; the list stops at — except for a delete, which lists + /// every placement it removed. A delete of more than placements is refused + /// whole. Null on success. + string? CrashPlacements(float[] boxMin, float[] boxMax, string? nameContains, bool delete, int limit, int maxDelete, + out IReadOnlyList result, out int total); + string? Undo(); string? Redo(); + + /// The load zones that hold a world point, and those within metres of holding + /// it, read from city_univers; are the districts asked for there. Null on success. + /// Which copy of city_univers.sds: null or "base" for the base game's, or a DLC's folder name + /// (a DLC can ship a copy of its own - a different scene; free ride was measured to use the base one). + string? ZonesAt(float[] point, float near, string? copy, out IReadOnlyList zones, out IReadOnlyList districts, + out IReadOnlyList copies); + + /// A plan, north up, of where a district is asked for: one text row per step, '#' where a zone + /// holding the point names the district, '+' where zones hold it and none does, '.' where no zone holds it. + /// Null on success. + string? ZonesMap(string district, float[] from, float[] to, float step, float z, string? copy, out IReadOnlyList rows); + + /// Moves one face of a load zone to a world coordinate - its plane and its box together. Without + /// nothing is written. With it the scene of city_univers is saved to the working + /// copy; archive_build then takes it into the game. The face is moved in the base copy, in the copy named, or - + /// for "all" - in EVERY copy of city_univers.sds that has the zone, one result each. Null on success. + string? ZoneMoveFace(string zone, string face, float to, bool apply, string? copy, out IReadOnlyList results); + + /// Adds a new load zone to the base copy of city_univers: a box between two world corners, made like + /// an existing zone, keeping the one or two districts named loaded. Without nothing + /// is written. With it the scene, the frame name table and cityareas.bin are saved to the working copy. + /// Null on success. + string? ZoneCreate(string name, string like, float[] boxMin, float[] boxMax, string[] districts, bool apply, out LoadZoneInfo? zone); + + /// Takes a load zone that was ADDED to the game's own out of the base city_univers.sds again: its + /// volume, its place in the frame name table and its line in cityareas.bin. Without + /// nothing is written. A zone the game ships with is refused. Null on success. + string? ZoneDelete(string name, bool apply, out LoadZoneInfo? zone); } diff --git a/src/Illusion.Mcp/Tools/EditorTools.cs b/src/Illusion.Mcp/Tools/EditorTools.cs index 48bb6fe..159ccda 100644 --- a/src/Illusion.Mcp/Tools/EditorTools.cs +++ b/src/Illusion.Mcp/Tools/EditorTools.cs @@ -225,6 +225,110 @@ public static async Task ArchiveBuild( } } + [McpServerTool(Name = "mesh_hide_triangles")] + [Description("Cut an opening into a mesh WITHOUT rebuilding it: the triangles of the named mesh whose three corners all lie inside a world-space box are hidden on every level of detail (their indices are pointed at one vertex). No vertex is touched, so a stock facade keeps the channels Blender never sees (shadow-map UVs) — use this, not a Blender push, to open a painted door or garage shutter of a stock building. By default it only REPORTS what the box would take (count per LOD and the first triangles with their material and corners); pass apply=true to hide them as one undoable edit. Saved by editor_save, packed by editor_build. A mesh that shares its geometry with other objects is refused unless 'shared' says what to do about them.")] + public static async Task MeshHideTriangles( + IEditorSession editor, + IUiThreadMarshal ui, + [Description("The mesh, by name or path suffix (scene_find has the names).")] string name, + [Description("Lower corner of the box, world space [x, y, z].")] float[] boxMin, + [Description("Upper corner of the box, world space [x, y, z].")] float[] boxMax, + [Description("Only triangles whose material name contains this. Omit for any material.")] string? material = null, + [Description("Hide the triangles. Default false: report only.")] bool apply = false, + [Description("How many of the found triangles to list (0-500). Default 40.")] int sample = 40, + [Description("What to do when other objects draw the same geometry (the districts reuse it heavily): omit to refuse, 'own' gives this mesh a copy of the geometry of its own first and cuts it alone, 'all' hides the triangles on every object that draws it.")] string? shared = null) + { + try + { + HiddenTrianglesInfo? result = null; + string? refused = await ui.RunAsync(() => editor.HideTriangles(name, boxMin, boxMax, material, apply, sample, shared, out result)); + return refused != null ? ToolResult.Invalid(refused) : ToolResult.Json(new { success = true, hidden = result }); + } + catch (Exception ex) + { + return ToolResult.Fail(ex); + } + } + + [McpServerTool(Name = "mesh_materials")] + [Description("The material slots of a mesh - which material each part of it is drawn with, and how many triangles that part has. With 'slot' and 'material', that slot is first re-pointed at another material (what the Materials tab's 'assign to slot' does): one undoable edit that touches no geometry and no UV, so the new texture is laid out the way the old one was - look at the result. Further levels of detail follow wherever they used the slot's old material. The material is named exactly as search_materials spells it. A district's winter archive is edited separately. The hull's surface type (what footsteps sound like, where grass tufts grow) is the collision's own and does not change with this.")] + public static async Task MeshMaterials( + IEditorSession editor, + IUiThreadMarshal ui, + [Description("The mesh: its name, or its path when the name is not unique (scene_find gives both).")] string name, + [Description("Slot to re-point. Omit, with 'material', to only list the slots.")] int? slot = null, + [Description("Material to point the slot at, by exact name.")] string? material = null) + { + try + { + IReadOnlyList slots = []; + string? refused = await ui.RunAsync(() => editor.MeshMaterials(name, slot, material, out slots)); + return refused != null ? ToolResult.Invalid(refused) : ToolResult.Json(new { success = true, changed = slots.Any(s => s.Changed), slots }); + } + catch (Exception ex) + { + return ToolResult.Fail(ex); + } + } + + [McpServerTool(Name = "collision_unused_hulls")] + [Description("Count the collision hulls no placement references — dead weight a delete, a re-cook or a resize leaves behind in the .col (deleting a placement never removes its hull, so undo can put the file back). For every collision file of the open scene: its placements, its hulls and how many of those are unused. By default it only REPORTS; pass apply=true to remove them, all files as one undoable edit. Placements are never touched, so nothing changes in the game except the archive's size. Saved by editor_save, packed by editor_build; run editor_mirror_winter afterwards so the winter district loses them too.")] + public static async Task CollisionUnusedHulls( + IEditorSession editor, + IUiThreadMarshal ui, + [Description("Remove the unused hulls. Default false: count only.")] bool apply = false) + { + try + { + IReadOnlyList layers = []; + string? refused = await ui.RunAsync(() => editor.UnusedHulls(apply, out layers)); + return refused != null + ? ToolResult.Invalid(refused) + : ToolResult.Json(new { success = true, unused = layers.Sum(l => l.Unused), removed = apply, layers }); + } + catch (Exception ex) + { + return ToolResult.Fail(ex); + } + } + + [McpServerTool(Name = "crash_placements")] + [Description("The city_crash props — trees, bushes, lamps, bins, mailboxes — standing inside a world-space box: each one's prop name, placement id, position and whether the other season holds the same placement. scene_find only shows the prop TYPES of that layer ('lampLuxus — 31'); this is how to see where the copies stand. The layer must be in the scene (view_set crash=true). By default it only LISTS; pass delete=true to remove every placement the box (and the name filter) takes, as one undoable edit — a placement linked to the other season is removed there too. city_crash is ONE archive for the whole city: keep the box tight. Saved by editor_save, packed by editor_build.")] + public static async Task CrashPlacements( + IEditorSession editor, + IUiThreadMarshal ui, + [Description("Lower corner of the box, world space [x, y, z].")] float[] boxMin, + [Description("Upper corner of the box, world space [x, y, z].")] float[] boxMax, + [Description("Only props whose name contains this. Omit for every prop.")] string? nameContains = null, + [Description("Delete the placements found. Default false: list only.")] bool delete = false, + [Description("How many placements to list (the count is always complete; a delete lists everything it removed). Default 200.")] int limit = 200, + [Description("The most placements one delete may remove. With more than this in the box the call is refused and nothing is deleted. Default 100.")] int maxDelete = 100) + { + try + { + IReadOnlyList placements = []; + int total = 0; + string? refused = await ui.RunAsync(() => + editor.CrashPlacements(boxMin, boxMax, nameContains, delete, limit, maxDelete, out placements, out total)); + return refused != null + ? ToolResult.Invalid(refused) + : ToolResult.Json(new + { + success = true, + count = total, + deleted = delete ? total : 0, + // Gone from the OTHER season as well: only the linked ones. An unlinked placement that + // has a twin keeps it. + deletedInOtherSeasonToo = delete ? placements.Count(p => p.BothSeasons && p.SeasonLinked) : 0, + placements, + }); + } + catch (Exception ex) + { + return ToolResult.Fail(ex); + } + } + [McpServerTool(Name = "car_substitute")] [Description("Build one car under ANOTHER car's name: pc\\sds\\cars\\.sds is replaced by the source car's model, with the root frame, name table, prefab entry, entity data and buffers keyed by the target's model name. No table is touched — the game lists the target as before and finds the source's shape and tuning in its archive. For trying a car where nothing can be registered (a multiplayer that spawns from a fixed list of names). A timestamped backup of each replaced archive is kept in cars\\backups; the winter _z twin is replaced only where both cars have one. This OVERWRITES game files and the target's working copy: the game must not be running.")] public static async Task CarSubstitute( @@ -332,7 +436,7 @@ public static async Task OpenArea( } [McpServerTool(Name = "scene_find")] - [Description("Find objects in the loaded scene by a fragment of their name, by kind, and/or by a world-space box. Returns name, kind, tree path, position and bounds. With a box, a mesh is returned only when its TRIANGLES reach into the box (not merely its bounds), with TrianglesInBox and the extent of those triangles clipped to the box — the way to check that a volume is free before building in it. Objects without a mesh match a box by their position. Crash-layer copies (kind CrashInstance: trees, lamps, bins) are searched from the placement table whether or not their rows are expanded in the tree; a copy matches a name by its own label or by its prop's name, and a box by its prop's triangles at the copy's placement. Use it also to learn exact names before scene_select.")] + [Description("Find objects in the loaded scene by a fragment of their name, by kind, and/or by a world-space box. Returns name, kind, tree path, position and bounds. With a box, a mesh is returned only when its TRIANGLES reach into the box (not merely its bounds), with TrianglesInBox and the extent of those triangles clipped to the box — the way to check that a volume is free before building in it. Objects without a mesh match a box by their position. Crash-layer copies (kind CrashInstance: trees, lamps, bins) are searched from the placement table whether or not their rows are expanded in the tree; a copy matches a name by its own label or by its prop's name, and a box by its prop's triangles at the copy's placement. Use it also to learn exact names before scene_select. A collision placement (kind 'CollisionInstance', named 'instance N') is found by its HULL: its bounds, vertex and triangle counts are the hull's, and a box is tested against the hull's triangles.")] public static async Task Find( IEditorSession editor, IUiThreadMarshal ui, @@ -859,4 +963,128 @@ public static async Task Redo(IEditorSession editor, IUiThreadMarshal ui return ToolResult.Fail(ex); } } + + [McpServerTool(Name = "zones_at")] + [Description("Which LOAD ZONES hold a world point, and which districts the game is asked to keep loaded there. The city is tiled with AREA volumes (in city_univers.sds); while the camera is inside one, the districts its entry in cityareas.bin names stay loaded. A point that lies in no volume asks for nothing - a player teleported there stands in a district that does not stream in. Each zone is tested by its real volume (its planes), not only its box. 'near' also lists the zones that miss the point by up to that many metres, with the distance - which shows the edges of a gap.")] + public static async Task ZonesAt( + IEditorSession editor, + IUiThreadMarshal ui, + [Description("World position [x, y, z].")] float[] point, + [Description("Also list zones that miss the point by up to this many metres. Default 0: only the zones that hold it.")] float near = 0, + [Description("Which copy of city_univers.sds to read: 'base' (default) or a DLC's folder name, e.g. 'cnt_joes_adventures'. A DLC can ship a copy of its own - a different scene with fewer zones; free ride (a multiplayer client that mounts Joe's Adventures included) was measured to use the BASE copy. The result lists the copies there are.")] string? copy = null) + { + try + { + IReadOnlyList zones = []; + IReadOnlyList districts = []; + IReadOnlyList copies = []; + string? refused = await ui.RunAsync(() => editor.ZonesAt(point, near, copy, out zones, out districts, out copies)); + return refused != null + ? ToolResult.Invalid(refused) + : ToolResult.Json(new + { + success = true, + copy = string.IsNullOrWhiteSpace(copy) ? "base" : copy, + copies, + districtsAskedFor = districts, + inNoZone = !zones.Any(z => z.Inside), + zones, + }); + } + catch (Exception ex) + { + return ToolResult.Fail(ex); + } + } + + [McpServerTool(Name = "zones_map")] + [Description("A plan of where a district is asked for by the load zones, as rows of text, north up and x running left to right: '#' a zone holding the point names the district, '+' zones hold the point but none names it, '.' no zone holds the point. Use it to see a gap in the zones before sending players somewhere the stock game never let them stand, and to check a zone_move_face.")] + public static async Task ZonesMap( + IEditorSession editor, + IUiThreadMarshal ui, + [Description("District archive name, e.g. 'greenfield'.")] string district, + [Description("One corner of the plan [x, y].")] float[] from, + [Description("The opposite corner [x, y].")] float[] to, + [Description("Step between samples, metres. Default 20.")] float step = 20, + [Description("Height the samples are taken at. Default 0.")] float z = 0, + [Description("Which copy of city_univers.sds: 'base' (default) or a DLC's folder name. See zones_at.")] string? copy = null) + { + try + { + IReadOnlyList rows = []; + string? refused = await ui.RunAsync(() => editor.ZonesMap(district, from, to, step, z, copy, out rows)); + return refused != null ? ToolResult.Invalid(refused) : ToolResult.Json(new { success = true, district, step, z, rows }); + } + catch (Exception ex) + { + return ToolResult.Fail(ex); + } + } + + [McpServerTool(Name = "zone_move_face")] + [Description("Move one face of a load zone to a world coordinate, to close a gap in the zones or pull a zone back: the plane that bounds the volume on that side and its box are changed together. Only for a volume that stands square to the map and a face square to the axis. By default it only REPORTS what would change; apply=true writes the scene of city_univers.sds to its working copy - then editor_build or archive_build packs that archive (with a backup) and the game sees it. With the map editor open a move in the base copy is one step of its history (editor_undo takes it back); a move in a DLC's copy, or in every copy at once, is not. city_univers.sds is shared by the whole city and by both seasons, so every client of a multiplayer server needs the same file. What loads a district where a player appears is a zone naming TWO districts (zones_at lists them); a zone naming one did not load it by itself when measured in the game. The base copy is the one edited unless 'copy' says otherwise.")] + public static async Task ZoneMoveFace( + IEditorSession editor, + IUiThreadMarshal ui, + [Description("The zone's name, as zones_at lists it, e.g. 'AREA0019_GREENFIELD'.")] string zone, + [Description("Which face: '+x', '-x', '+y', '-y', '+z' or '-z' (the face on that side of the volume).")] string face, + [Description("Where that face is to stand, on its axis, in world coordinates.")] float to, + [Description("Write the change to the working copy. Default false: report only.")] bool apply = false, + [Description("Which copy of city_univers.sds: 'base' (default), a DLC's folder name, or 'all' for every copy that has the zone.")] string? copy = null) + { + try + { + IReadOnlyList results = []; + string? refused = await ui.RunAsync(() => editor.ZoneMoveFace(zone, face, to, apply, copy, out results)); + return refused != null ? ToolResult.Invalid(refused) : ToolResult.Json(new { success = true, moved = results }); + } + catch (Exception ex) + { + return ToolResult.Fail(ex); + } + } + + [McpServerTool(Name = "zone_create")] + [Description("Add a NEW load zone to city_univers.sds (the base copy): a box standing square to the map between two world corners that keeps one or two districts loaded while the player is inside it. It is made as a copy of an existing zone ('like' - its flags, parent and name-table membership) with a name, a place and a shape of its own, and gets a line in cityareas.bin. By default it only REPORTS; apply=true writes the scene, the frame name table and cityareas.bin to the working copy - then archive_build packs city_univers.sds (with a backup). With the map editor open the zone is one step of its history (editor_undo takes it out); zone_delete takes it out later. Refused while an editor holds city_univers (Whole map). Measured in the game: the NAME decides whether the zone loads its districts for a player who appears inside it - two words after the number ('AREA900_DOCK_SOUTH') and it does, one word ('AREA900_DOCK') and it does not, whatever districts it names; the result says which it is ('loadsOnArrival').")] + public static async Task ZoneCreate( + IEditorSession editor, + IUiThreadMarshal ui, + [Description("The new zone's name, e.g. 'AREA900_SANDISLAND_TUNEL' - two words after the number, or the game will not load the districts for a player who appears inside. No object of the scene may have it.")] string name, + [Description("An existing zone to make it like, as zones_at lists it - one that stands square to the map.")] string like, + [Description("One corner of the box, world [x, y, z].")] float[] boxMin, + [Description("The opposite corner, world [x, y, z].")] float[] boxMax, + [Description("The districts it keeps loaded: one or two archive names, e.g. ['sandisland', 'tunel'].")] string[] districts, + [Description("Write the change to the working copy. Default false: report only.")] bool apply = false) + { + try + { + LoadZoneInfo? made = null; + string? refused = await ui.RunAsync(() => editor.ZoneCreate(name, like, boxMin, boxMax, districts, apply, out made)); + return refused != null ? ToolResult.Invalid(refused) : ToolResult.Json(new { success = true, applied = apply, zone = made, loadsOnArrival = made?.LoadsOnArrival }); + } + catch (Exception ex) + { + return ToolResult.Fail(ex); + } + } + + [McpServerTool(Name = "zone_delete")] + [Description("Take a load zone that was ADDED (zone_create, or the editor's New loading zone button - zones_at marks such a zone 'Added') out of city_univers.sds again: its volume, its place in the frame name table and its line in cityareas.bin. A zone the game ships with is refused. By default it only REPORTS; apply=true writes the working copy - then archive_build packs city_univers.sds. With the map editor open it is one step of its history (editor_undo puts the zone back). Refused while an editor holds city_univers (Whole map).")] + public static async Task ZoneDelete( + IEditorSession editor, + IUiThreadMarshal ui, + [Description("The zone's name, as zones_at lists it.")] string name, + [Description("Write the change to the working copy. Default false: report only.")] bool apply = false) + { + try + { + LoadZoneInfo? gone = null; + string? refused = await ui.RunAsync(() => editor.ZoneDelete(name, apply, out gone)); + return refused != null ? ToolResult.Invalid(refused) : ToolResult.Json(new { success = true, applied = apply, zone = gone }); + } + catch (Exception ex) + { + return ToolResult.Fail(ex); + } + } } diff --git a/src/Illusion.Mcp/Tools/StreamMapTools.cs b/src/Illusion.Mcp/Tools/StreamMapTools.cs index f215cee..27e5080 100644 --- a/src/Illusion.Mcp/Tools/StreamMapTools.cs +++ b/src/Illusion.Mcp/Tools/StreamMapTools.cs @@ -1,5 +1,4 @@ using System.ComponentModel; -using System.Globalization; using Illusion.Formats.StreamMap; using ModelContextProtocol.Server; @@ -154,17 +153,9 @@ public static string EditStreamMap( bool written = false; if (!dryRun && patch.Patched is not null && applied > 0) { - // The backup goes down before the file does, and only when there is something to - // write — a no-op edit should not churn a backup the user may still need. - // - // It must also never overwrite an existing one. The name is derived from the target, - // so it is the same on every edit: a second pass over the same StreamMap would have - // replaced the pristine backup with the ALREADY-PATCHED file, leaving two copies of - // modified data and no way back to the original. The first backup is the valuable - // one, so it is kept and later passes get a numbered sibling instead. - backup = NextBackupPath(filePath); - File.WriteAllBytes(backup, original); - File.WriteAllBytes(filePath, patch.Patched); + // Only when there is something to write — a no-op edit should not churn a backup the user + // may still need. The write keeps the first backup and numbers the later ones. + backup = StreamMapEditor.WriteWithBackup(filePath, original, patch.Patched); written = true; } @@ -198,28 +189,6 @@ public static string EditStreamMap( } } - /// - /// A backup path that does not already exist: <name>_old.bin for the first edit, - /// then _old.2.bin, _old.3.bin and so on. Never returns a path that would clobber - /// an earlier backup — the whole point of the file is to still hold the original after the - /// second edit. - /// - private static string NextBackupPath(string filePath) - { - string directory = Path.GetDirectoryName(filePath) ?? "."; - string stem = Path.GetFileNameWithoutExtension(filePath) + "_old"; - string extension = Path.GetExtension(filePath); - - string candidate = Path.Combine(directory, stem + extension); - for (int n = 2; File.Exists(candidate); n++) - { - candidate = Path.Combine( - directory, - stem + "." + n.ToString(CultureInfo.InvariantCulture) + extension); - } - return candidate; - } - /// Parses the comma-separated field selector. An unknown name is refused rather than /// ignored — a typo that silently widened the edit to every field would be the worst outcome /// here, since this tool writes. diff --git a/src/Illusion.Rendering/Controls/BoxGizmo.cs b/src/Illusion.Rendering/Controls/BoxGizmo.cs new file mode 100644 index 0000000..80b9b9d --- /dev/null +++ b/src/Illusion.Rendering/Controls/BoxGizmo.cs @@ -0,0 +1,388 @@ +using System.Globalization; +using System.Numerics; +using System.Windows; +using System.Windows.Input; +using System.Windows.Media; + +using Illusion.Rendering.Gizmos; +using Illusion.Rendering.Scene; +using Vector = System.Windows.Vector; + +namespace Illusion.Rendering.Controls; + +/// +/// The gizmo of an axis-aligned box - a loading zone - overlaid on the viewport like . +/// With the shelf's Move tool three arrows stand at the box's centre and move it whole; with Scale an arrow +/// stands on each of its six faces and pulling one moves that face alone, the opposite one staying put. +/// The arrows stand where they do in any editor and stay there: the move arrows at the centre of the box, each +/// face's arrow at the centre of its face. +/// Hit-testing is limited to the arrows, so a click anywhere else falls through to the viewport. +/// +public sealed class BoxGizmo : FrameworkElement +{ + private const double MovePixels = 110; // an arrow's length on screen, Move tool + private const double FacePixels = 96; // ...and a face's arrow + private const double HitPx = 11; // pointer proximity to grab an arrow + private const double HeadPx = 18; // the arrow head + private const float MinThickness = 1f; // a face cannot be pulled through the opposite one + private const double MinArrowPx = 12; // shorter than this on screen, an arrow points at the viewer + + private static readonly Color[] AxisColors = + { + Color.FromRgb(0xE6, 0x46, 0x46), // X — red + Color.FromRgb(0x82, 0xC8, 0x3C), // Y — green + Color.FromRgb(0x46, 0x82, 0xDC), // Z — blue + }; + private static readonly Color HighlightColor = Color.FromRgb(0xFF, 0xD2, 0x4A); + private static readonly Pen[] AxisPens = new Pen[3]; + private static readonly Brush[] AxisBrushes = new Brush[3]; + private static readonly Pen HighlightPen; + private static readonly Brush HighlightBrush; + private static readonly Pen UnderPen; // a dark line under each arrow: the zone boxes behind it are every colour + private static readonly Brush UnderBrush; + private static readonly Brush LabelBack; + private static readonly Typeface LabelFace = new("Segoe UI"); + private static readonly Vector3[] Axes = [Vector3.UnitX, Vector3.UnitY, Vector3.UnitZ]; + private static readonly string[] AxisNames = ["X", "Y", "Z"]; + + static BoxGizmo() + { + for (int i = 0; i < 3; i++) + { + AxisPens[i] = Freeze(new Pen(new SolidColorBrush(AxisColors[i]), 3.6) { StartLineCap = PenLineCap.Round, EndLineCap = PenLineCap.Round }); + AxisBrushes[i] = Freeze(new SolidColorBrush(AxisColors[i])); + } + HighlightPen = Freeze(new Pen(new SolidColorBrush(HighlightColor), 4.4) { StartLineCap = PenLineCap.Round, EndLineCap = PenLineCap.Round }); + HighlightBrush = Freeze(new SolidColorBrush(HighlightColor)); + UnderBrush = Freeze(new SolidColorBrush(Color.FromArgb(0xD0, 0x10, 0x10, 0x10))); + UnderPen = Freeze(new Pen(UnderBrush, 7.0) { StartLineCap = PenLineCap.Round, EndLineCap = PenLineCap.Round }); + LabelBack = Freeze(new SolidColorBrush(Color.FromArgb(0xC8, 0x18, 0x18, 0x18))); + } + + private static T Freeze(T f) where T : Freezable { f.Freeze(); return f; } + + /// One arrow: along Axis; Side +1 / -1 is the upper / lower face (Scale), 0 the whole box (Move). + private readonly record struct Handle(int Axis, int Side); + + private readonly record struct Arrow(Handle Handle, Vector3 From, Vector3 To, Point A, Point B); + + private IBoxGizmoHost? _host; + private Handle? _hover; + private Handle? _active; + private Vector3 _startMin, _startMax, _lastMin, _lastMax; + private Vector3 _dragFrom; // where the grabbed arrow stood when the drag began + private float _dragStartT; + private bool _swallowRelease; // a drag was dropped on a right PRESS; its release must not reach the viewport + + public BoxGizmo() + { + HorizontalAlignment = HorizontalAlignment.Stretch; + VerticalAlignment = VerticalAlignment.Stretch; + Focusable = false; + ClipToBounds = true; + } + + /// True while an arrow is being dragged. + public bool IsDragging => _active != null; + + /// Binds the gizmo to a host; it repaints when the camera moves or the box changes. + public void Attach(IBoxGizmoHost host) + { + if (_host != null) return; + _host = host; + host.CameraMoved += InvalidateVisual; + host.BoxGizmoChanged += InvalidateVisual; + } + + // ── Layout (world arrows → screen) ── + + private List Arrows() + { + var arrows = new List(6); + if (_host?.BoxGizmoTarget is not { } box || ActualWidth <= 0 || ActualHeight <= 0) return arrows; + GizmoMode mode = _host.GizmoMode; + if (mode is not (GizmoMode.Move or GizmoMode.Scale)) return arrows; + + Matrix4x4 vp = _host.GizmoViewProjection; + Vector3 eye = _host.GizmoCameraPosition; + // Fixed places, as in any editor: the move arrows at the centre of the box, each face's arrow at the + // centre of its face. + Vector3 middle = (box.Min + box.Max) * 0.5f; + if (mode == GizmoMode.Move) + { + for (int axis = 0; axis < 3; axis++) Add(new Handle(axis, 0), middle, Axes[axis], MovePixels); + return arrows; + } + for (int axis = 0; axis < 3; axis++) + { + foreach (int side in (ReadOnlySpan)[1, -1]) + { + if (!_host.BoxGizmoFaceMoves(axis, side)) continue; + Vector3 at = With(middle, axis, side > 0 ? Component(box.Max, axis) : Component(box.Min, axis)); + Add(new Handle(axis, side), at, Axes[axis] * side, FacePixels); + } + } + return arrows; + + void Add(Handle handle, Vector3 from, Vector3 direction, double pixels) + { + if (!Project(vp, from, out Point a)) return; + // how long a metre along the arrow is on screen here, to give it a constant length in pixels; + // seen end-on an arrow has no length, and is kept to a share of the distance to it instead + float reach = MathF.Max(1f, (from - eye).Length()); + double metre = Project(vp, from + direction, out Point one) ? (one - a).Length : 0; + float length = metre > 1e-3 ? (float)(pixels / metre) : reach * 0.12f; + length = Math.Clamp(length, reach * 0.02f, reach * 0.35f); + Vector3 to = from + direction * length; + if (!Project(vp, to, out Point b)) return; + // Seen end-on an arrow is a dot: it is not drawn, and must not be grabbed either - a pixel of the + // pointer along a line that short is metres of the zone. + if ((b - a).Length < MinArrowPx) return; + arrows.Add(new Arrow(handle, from, to, a, b)); + } + } + + private static float Component(Vector3 v, int axis) => axis == 0 ? v.X : axis == 1 ? v.Y : v.Z; + + private static Vector3 With(Vector3 v, int axis, float value) => + axis == 0 ? new Vector3(value, v.Y, v.Z) : axis == 1 ? new Vector3(v.X, value, v.Z) : new Vector3(v.X, v.Y, value); + + // Row-vector projection in double precision, as TransformGizmo does it: at the city's coordinates a float + // clip-space product wobbles by a pixel as the camera moves. + private bool Project(Matrix4x4 m, Vector3 world, out Point screen) + { + double x = world.X, y = world.Y, z = world.Z; + double cx = x * m.M11 + y * m.M21 + z * m.M31 + m.M41; + double cy = x * m.M12 + y * m.M22 + z * m.M32 + m.M42; + double cw = x * m.M14 + y * m.M24 + z * m.M34 + m.M44; + if (cw <= 1e-4) { screen = default; return false; } + double inv = 1.0 / cw; + screen = new Point((cx * inv * 0.5 + 0.5) * ActualWidth, (0.5 - cy * inv * 0.5) * ActualHeight); + return true; + } + + // ── Drawing ── + + protected override void OnRender(DrawingContext dc) + { + DrawName(dc); + foreach (Arrow arrow in Arrows()) + { + bool lit = _active == arrow.Handle || (_active == null && _hover == arrow.Handle); + Pen pen = lit ? HighlightPen : AxisPens[arrow.Handle.Axis]; + Brush brush = lit ? HighlightBrush : AxisBrushes[arrow.Handle.Axis]; + Vector along = arrow.B - arrow.A; + along.Normalize(); + var across = new Vector(-along.Y, along.X); + Point neck = arrow.B - along * HeadPx; + var head = new StreamGeometry(); + using (StreamGeometryContext g = head.Open()) + { + g.BeginFigure(arrow.B, isFilled: true, isClosed: true); + g.LineTo(neck + across * (HeadPx * 0.5), true, false); + g.LineTo(neck - across * (HeadPx * 0.5), true, false); + } + head.Freeze(); + dc.DrawLine(UnderPen, arrow.A, neck); + dc.DrawGeometry(UnderBrush, UnderPen, head); + dc.DrawLine(pen, arrow.A, neck); + dc.DrawGeometry(brush, null, head); + dc.DrawEllipse(brush, null, arrow.A, 4.5, 4.5); // where the arrow stands on its face + + if (_active == arrow.Handle) DrawLabel(dc, arrow); + } + } + + // What the picked box is, over its top: with whichever tool, and with none. + private void DrawName(DrawingContext dc) + { + if (_host?.BoxGizmoTarget is not { } box || _host.BoxGizmoLabel is not { Length: > 0 } text || ActualWidth <= 0) return; + Vector3 top = (box.Min + box.Max) * 0.5f; + top.Z = box.Max.Z; + if (!Project(_host.GizmoViewProjection, top, out Point at)) return; + var label = new FormattedText(text, CultureInfo.InvariantCulture, FlowDirection.LeftToRight, LabelFace, 12.5, Brushes.White, + VisualTreeHelper.GetDpi(this).PixelsPerDip); + // kept inside the view: the top of a zone the camera stands in is as often as not off screen + double x = Math.Clamp(at.X - (label.Width / 2), 8, Math.Max(8, ActualWidth - label.Width - 8)); + double y = Math.Clamp(at.Y - label.Height - 14, 8, Math.Max(8, ActualHeight - label.Height - 8)); + dc.DrawRoundedRectangle(LabelBack, null, new Rect(x - 7, y - 3, label.Width + 14, label.Height + 6), 4, 4); + dc.DrawText(label, new Point(x, y)); + } + + // While a face is being pulled, where it stands; while the box is being moved, how far it has gone. + private void DrawLabel(DrawingContext dc, Arrow arrow) + { + int axis = arrow.Handle.Axis; + float value = arrow.Handle.Side switch + { + > 0 => Component(_lastMax, axis), + < 0 => Component(_lastMin, axis), + _ => Component(_lastMin, axis) - Component(_startMin, axis), + }; + string text = arrow.Handle.Side == 0 + ? $"{AxisNames[axis]} {(value >= 0 ? "+" : "")}{value.ToString("0.0", CultureInfo.InvariantCulture)} m" + : $"{AxisNames[axis]} {value.ToString("0.0", CultureInfo.InvariantCulture)}"; + var label = new FormattedText(text, CultureInfo.InvariantCulture, FlowDirection.LeftToRight, LabelFace, 12, Brushes.White, + VisualTreeHelper.GetDpi(this).PixelsPerDip); + var at = new Point(arrow.B.X + 12, arrow.B.Y - label.Height / 2); + dc.DrawRoundedRectangle(LabelBack, null, new Rect(at.X - 5, at.Y - 2, label.Width + 10, label.Height + 4), 3, 3); + dc.DrawText(label, at); + } + + // ── Input ── + + private Handle? HandleAt(Point p) + { + Handle? best = null; + double bestDistance = HitPx; + foreach (Arrow arrow in Arrows()) + { + double d = DistanceToSegment(p, arrow.A, arrow.B); + if (d < bestDistance) { bestDistance = d; best = arrow.Handle; } + } + return best; + } + + private static double DistanceToSegment(Point p, Point a, Point b) + { + Vector ab = b - a; + double lengthSquared = ab.LengthSquared; + if (lengthSquared < 1e-9) return (p - a).Length; + double t = Math.Clamp(Vector.Multiply(p - a, ab) / lengthSquared, 0, 1); + return (p - (a + ab * t)).Length; + } + + protected override HitTestResult? HitTestCore(PointHitTestParameters hitTestParameters) => + _active != null || HandleAt(hitTestParameters.HitPoint) != null ? new PointHitTestResult(this, hitTestParameters.HitPoint) : null; + + private (Vector3 Origin, Vector3 Dir) Ray(Point mouse) => + Picking.BuildRay(_host!.GizmoViewProjection, _host.GizmoCameraPosition, mouse.X, mouse.Y, ActualWidth, ActualHeight); + + protected override void OnMouseLeftButtonDown(MouseButtonEventArgs e) + { + base.OnMouseLeftButtonDown(e); + if (_host?.BoxGizmoTarget is not { } box) return; + Point p = e.GetPosition(this); + Arrow? grabbed = null; + double bestDistance = HitPx; + foreach (Arrow arrow in Arrows()) + { + double d = DistanceToSegment(p, arrow.A, arrow.B); + if (d < bestDistance) { bestDistance = d; grabbed = arrow; } + } + if (grabbed is not { } hit) return; + + _active = hit.Handle; + _startMin = _lastMin = box.Min; + _startMax = _lastMax = box.Max; + _dragFrom = hit.From; + (Vector3 origin, Vector3 dir) = Ray(p); + _dragStartT = GizmoRayMath.ClosestAxisParam(_dragFrom, Axes[hit.Handle.Axis], origin, dir); + _host.BoxGizmoBegin(); + CaptureMouse(); + e.Handled = true; + InvalidateVisual(); + } + + protected override void OnMouseMove(MouseEventArgs e) + { + base.OnMouseMove(e); + Point p = e.GetPosition(this); + if (_active is not { } handle) + { + Handle? over = HandleAt(p); + if (over != _hover) + { + _hover = over; + Cursor = over != null ? Cursors.SizeAll : null; + InvalidateVisual(); + } + return; + } + + (Vector3 origin, Vector3 dir) = Ray(p); + float moved = GizmoRayMath.ClosestAxisParam(_dragFrom, Axes[handle.Axis], origin, dir) - _dragStartT; + if (!float.IsFinite(moved)) return; + Vector3 min = _startMin, max = _startMax; + int axis = handle.Axis; + if (handle.Side == 0) + { + min = With(min, axis, Component(min, axis) + moved); + max = With(max, axis, Component(max, axis) + moved); + } + else if (handle.Side > 0) + { + max = With(max, axis, MathF.Max(Component(max, axis) + moved, Component(min, axis) + MinThickness)); + } + else + { + min = With(min, axis, MathF.Min(Component(min, axis) + moved, Component(max, axis) - MinThickness)); + } + _lastMin = min; + _lastMax = max; + _host!.BoxGizmoPreview(min, max); + e.Handled = true; + InvalidateVisual(); + } + + protected override void OnMouseLeftButtonUp(MouseButtonEventArgs e) + { + base.OnMouseLeftButtonUp(e); + if (_active == null) return; + End(commit: true); + e.Handled = true; + } + + // The right button drops a drag in progress, as Esc does in the transform gizmo. + protected override void OnMouseRightButtonDown(MouseButtonEventArgs e) + { + base.OnMouseRightButtonDown(e); + if (_active == null) return; + // The pointer stays captured until the button comes back up: let go here, its release would land on + // the viewport, which answers a right click with its menu and a selection of whatever lies under it. + _swallowRelease = true; + End(commit: false, releaseCapture: false); + e.Handled = true; + } + + protected override void OnMouseRightButtonUp(MouseButtonEventArgs e) + { + base.OnMouseRightButtonUp(e); + if (!_swallowRelease) return; + _swallowRelease = false; + if (IsMouseCaptured) ReleaseMouseCapture(); + e.Handled = true; + } + + /// Drops a drag in progress, the box going back to where it started. For the window's Esc. + public bool CancelDrag() + { + if (_active == null) return false; + End(commit: false); + return true; + } + + protected override void OnLostMouseCapture(MouseEventArgs e) + { + base.OnLostMouseCapture(e); + _swallowRelease = false; + if (_active != null) End(commit: false); // the capture went elsewhere mid-drag: nothing is kept + } + + protected override void OnMouseLeave(MouseEventArgs e) + { + base.OnMouseLeave(e); + if (_active != null || _hover == null) return; + _hover = null; + Cursor = null; + InvalidateVisual(); + } + + private void End(bool commit, bool releaseCapture = true) + { + _active = null; + if (releaseCapture && IsMouseCaptured) ReleaseMouseCapture(); + _host?.BoxGizmoEnd(commit); + InvalidateVisual(); + } +} diff --git a/src/Illusion.Rendering/Controls/ViewportControl.cs b/src/Illusion.Rendering/Controls/ViewportControl.cs index f692b66..492195f 100644 --- a/src/Illusion.Rendering/Controls/ViewportControl.cs +++ b/src/Illusion.Rendering/Controls/ViewportControl.cs @@ -341,11 +341,20 @@ private void OnRendering(object? sender, EventArgs e) UpdateCamera(dt); CameraMoved?.Invoke(); + // A hook above can pump the dispatcher (a dialog, a wait for a load), and a frame run inside that pump + // may have replaced the surface - at another size, the old one disposed. This frame's target is then + // gone: nothing is drawn into it, the next frame draws into the new one. + if (!ReferenceEquals(_surface.Target, target)) return; + _image.Lock(); try { Renderer.Render(target); - _image.AddDirtyRect(new Int32Rect(0, 0, target.Width, target.Height)); + // Never more than the image holds: a dirty rectangle larger than its back buffer is an exception + // out of the render pass, with nothing above it but the end of the program - seen when the window + // was resized while a district loaded ("dirtyRect 1194 must be less than or equal to 232"). + int width = Math.Min(target.Width, _image.PixelWidth), height = Math.Min(target.Height, _image.PixelHeight); + if (width > 0 && height > 0) _image.AddDirtyRect(new Int32Rect(0, 0, width, height)); } finally { diff --git a/src/Illusion.Rendering/Gizmos/IBoxGizmoHost.cs b/src/Illusion.Rendering/Gizmos/IBoxGizmoHost.cs new file mode 100644 index 0000000..9c40249 --- /dev/null +++ b/src/Illusion.Rendering/Gizmos/IBoxGizmoHost.cs @@ -0,0 +1,42 @@ +using System.Numerics; + +namespace Illusion.Rendering.Gizmos; + +/// +/// What the box gizmo (BoxGizmo) edits: one axis-aligned box in the world - a loading zone - with the +/// tool shelf's Move moving it whole and Scale moving one face at a time. The viewport host implements it; the +/// gizmo only turns drags into boxes and hands them back. +/// +public interface IBoxGizmoHost +{ + Matrix4x4 GizmoViewProjection { get; } + Vector3 GizmoCameraPosition { get; } + + /// The tool on the shelf: Move and Scale are the two this gizmo answers to. + GizmoMode GizmoMode { get; } + + /// The box to put handles on, or null when there is none. + (Vector3 Min, Vector3 Max)? BoxGizmoTarget { get; } + + /// Whether a face of the box can be pulled: axis 0-2, side +1 the upper face, -1 the lower. + /// One that cannot gets no arrow. + bool BoxGizmoFaceMoves(int axis, int side); + + /// What the box is, in a line - shown over it for as long as it is picked. Null: nothing shown. + string? BoxGizmoLabel { get; } + + /// Raised each frame the camera changes, so the overlay repaints. + event Action? CameraMoved; + + /// Raised when the box changes or another one is picked. + event Action? BoxGizmoChanged; + + /// A drag is starting: the box as it stands is what a cancelled drag goes back to. + void BoxGizmoBegin(); + + /// The box the drag has reached; shown, not yet kept. + void BoxGizmoPreview(Vector3 min, Vector3 max); + + /// The drag is over: keep the last previewed box, or go back to where it started. + void BoxGizmoEnd(bool commit); +} diff --git a/src/Illusion.Rendering/Passes/SceneRenderer.cs b/src/Illusion.Rendering/Passes/SceneRenderer.cs index 48439ea..3db9962 100644 --- a/src/Illusion.Rendering/Passes/SceneRenderer.cs +++ b/src/Illusion.Rendering/Passes/SceneRenderer.cs @@ -31,14 +31,15 @@ public sealed unsafe class SceneRenderer : IDisposable private readonly HelperGlyphRenderer _helperHighlightRenderer; private readonly LineOverlayRenderer _partShapeRenderer; private readonly LineOverlayRenderer _hitBoxRenderer; + private readonly LineOverlayRenderer _pickedRenderer; private readonly ActorMarkerRenderer _actorRenderer; private readonly ActorMarkerRenderer _actorSelectionRenderer; private readonly SelectionOutlineRenderer _selectionOutline; /// Whether to show debug loading-zone boxes (UI toggle). Off by default. public bool ShowZones { get; set; } - private IReadOnlyList<(Vector3 Min, Vector3 Max, Vector4 Color)>? _zoneBoxes; - public void SetZoneBoxes(IReadOnlyList<(Vector3 Min, Vector3 Max, Vector4 Color)> boxes) => _zoneBoxes = boxes; + private IReadOnlyList? _zoneBoxes; + public void SetZoneBoxes(IReadOnlyList boxes) => _zoneBoxes = boxes; /// Uploads/replaces one district's collision geometry (keyed so streaming can remove it alone). public void SetCollisionDistrict(object key, Domain.CollisionRenderData? data) => _collisionRenderer.SetDistrict(key, data); @@ -166,6 +167,16 @@ public void SetHitBoxLines(IReadOnlyList lineVertices, IReadOnlyListDrops the hit-box wireframe — what a scene reset owes this layer. public void ClearHitBoxes() => _hitBoxRenderer.Clear(); + /// + /// Replaces the outline of the triangles picked for a tool (Tools → Hide triangles marks the ones a click + /// chose). Drawn whenever there are any - it is not a layer to switch, it is the tool's own selection - and + /// gone with an empty list. + /// + public void SetPickedTriangleLines(IReadOnlyList lineVertices, IReadOnlyList colors) => + _pickedRenderer.SetDistrict(PickedKey, lineVertices, colors); + + private static readonly object PickedKey = new(); + private static readonly object HitBoxKey = new(); /// Removes one archive's rig (district unload). @@ -299,6 +310,7 @@ public SceneRenderer(GpuContext gpu) _helperHighlightRenderer = new HelperGlyphRenderer(_linePass); _partShapeRenderer = new LineOverlayRenderer(_linePass); _hitBoxRenderer = new LineOverlayRenderer(_linePass); + _pickedRenderer = new LineOverlayRenderer(_linePass); _actorRenderer = new ActorMarkerRenderer(_linePass); _actorSelectionRenderer = new ActorMarkerRenderer(_linePass); _selectionOutline = new SelectionOutlineRenderer(gpu); @@ -546,7 +558,7 @@ public void Render(SharedRenderTarget target) _collisionRenderer.Render(target, viewProj, Camera.Position, frustum); // Debug overlay of loading zones (on top of meshes, semi-transparent boxes). - if (ShowZones && _zoneBoxes != null) _zoneRenderer.Render(ctx, viewProj, _zoneBoxes); + if (ShowZones && _zoneBoxes != null) _zoneRenderer.Render(ctx, viewProj, Camera.Position, Camera.Orthographic, _depthReadOnly, _zoneBoxes); // Everything an overlay glyph needs to size itself in PIXELS rather than metres: the pixel scale is // metres-per-pixel per unit of clip W, read straight off the projection (M22 = 1/tan(fov/2)). @@ -597,6 +609,9 @@ public void Render(SharedRenderTarget target) _hitBoxRenderer.Render(ctx, overlayFrame, RigStyle(new Vector4(1f, 1f, 1f, 0.55f), 1.2f)); } + // The triangles a tool has picked, in their own colours. + if (_pickedRenderer.HasData) _pickedRenderer.Render(ctx, overlayFrame, RigStyle(Vector4.One, 2.2f)); + // Actor glyphs: everything the .act pack places that has no geometry of its own, coloured per segment // (per category), so the tint stays neutral. if (ShowActors) _actorRenderer.Render(ctx, overlayFrame, OverlayLineStyle.Default(Vector4.One)); @@ -903,6 +918,7 @@ public void Dispose() _helperHighlightRenderer.Dispose(); _partShapeRenderer.Dispose(); _hitBoxRenderer.Dispose(); + _pickedRenderer.Dispose(); _navMeshRenderer.Dispose(); _navRenderer.Dispose(); _linePass.Dispose(); diff --git a/src/Illusion.Rendering/Passes/ZoneRenderer.cs b/src/Illusion.Rendering/Passes/ZoneRenderer.cs index 94f5288..377a5d7 100644 --- a/src/Illusion.Rendering/Passes/ZoneRenderer.cs +++ b/src/Illusion.Rendering/Passes/ZoneRenderer.cs @@ -16,9 +16,17 @@ public struct ZoneConstants public Vector4 Color; // rgb + alpha } +/// One loading zone to draw: its world box, the colour of its district, whether it is of the kind that +/// loads a district for a player who appears inside it, whether it is the one picked, and whether it was added +/// to the game's own zones - drawn with a second frame round it, so that it is told from them at a glance. +public readonly record struct ZoneBox(Vector3 Min, Vector3 Max, Vector3 Color, bool Loads, bool Picked, bool Added = false); + /// -/// Debug pass: draws AREA load zones as translucent colored boxes (alpha blending, -/// no depth write and no depth-test — pure overlay). One box per zone, world from AABB. +/// Draws the AREA load zones so that they can be read: each zone as the EDGES of its box with a faint fill, both +/// hidden by the scene in front of them. Hundreds of filled boxes laid over one another - the city is tiled two +/// and three deep - came out as one coloured haze with no box to be told from the next. A zone's fill is left +/// out while the camera is inside it, zones fade with distance, the kind that does not load a district by itself +/// is drawn fainter, and the picked zone is drawn full and seen through walls. /// public sealed unsafe class ZoneRenderer : IDisposable { @@ -51,6 +59,16 @@ struct PSIn { float4 pos : SV_POSITION; }; 0,3,7, 0,7,4, 1,5,6, 1,6,2, 0,4,5, 0,5,1, 3,2,6, 3,6,7, }; + private static readonly uint[] EdgeIndices = + { + 0,1, 1,2, 2,3, 3,0, 4,5, 5,6, 6,7, 7,4, 0,4, 1,5, 2,6, 3,7, + }; + + private const float FadeFrom = 300f; // metres from the camera at which a zone starts to fade... + private const float FadeTo = 1500f; // ...and is no longer drawn + private const float AddedFrame = 2.5f; // metres between an added zone's box and the second frame round it + + private ComPtr _edges; public ZoneRenderer(GpuContext gpu) { @@ -74,6 +92,8 @@ public ZoneRenderer(GpuContext gpu) _vb = GpuBuffers.CreateImmutable(gpu, pv, (uint)(CubeVerts.Length * sizeof(float)), BindFlag.VertexBuffer); fixed (uint* pi = CubeIndices) _ib = GpuBuffers.CreateImmutable(gpu, pi, (uint)(CubeIndices.Length * sizeof(uint)), BindFlag.IndexBuffer); + fixed (uint* pe = EdgeIndices) + _edges = GpuBuffers.CreateImmutable(gpu, pe, (uint)(EdgeIndices.Length * sizeof(uint)), BindFlag.IndexBuffer); _cb = GpuBuffers.CreateConstant(gpu); @@ -106,13 +126,18 @@ public ZoneRenderer(GpuContext gpu) _raster = rs; } - public void Render(ComPtr ctx, Matrix4x4 viewProj, - IReadOnlyList<(Vector3 Min, Vector3 Max, Vector4 Color)> boxes) + /// Where the camera stands: zones fade with their distance from it. + /// The view is a parallel one (Top, Front...): the camera then stands wherever the + /// projection was framed from, which says nothing about how far a zone is - nothing fades. + /// The scene's own "test, never write" depth state, so the scene hides what lies + /// behind it. + public void Render(ComPtr ctx, Matrix4x4 viewProj, Vector3 eye, bool parallel, + ComPtr depthRead, IReadOnlyList boxes) { if (boxes == null || boxes.Count == 0) return; ctx.OMSetBlendState(_blend, (float*)null, 0xffffffff); - ctx.OMSetDepthStencilState(_noDepth, 0); + ctx.OMSetDepthStencilState(depthRead, 0); ctx.RSSetState(_raster); ctx.IASetInputLayout(_layout); ctx.VSSetShader(_vs, (ID3D11ClassInstance**)null, 0); @@ -127,13 +152,58 @@ public void Render(ComPtr ctx, Matrix4x4 viewProj, ctx.IASetVertexBuffers(0, 1, &vb, &stride, &offset); ctx.IASetIndexBuffer(_ib, Format.FormatR32Uint, 0); - foreach ((Vector3 min, Vector3 max, Vector4 color) in boxes) + // How much of each zone is left at its distance, and whether the camera stands inside it. + Span fade = boxes.Count <= 2048 ? stackalloc float[boxes.Count] : new float[boxes.Count]; + Span inside = boxes.Count <= 2048 ? stackalloc bool[boxes.Count] : new bool[boxes.Count]; + for (int i = 0; i < boxes.Count; i++) { - Vector3 size = max - min; - Matrix4x4 world = Matrix4x4.CreateScale(size) * Matrix4x4.CreateTranslation(min); - var c = new ZoneConstants { Wvp = world * viewProj, Color = color }; + ZoneBox box = boxes[i]; + float distance = Vector3.Distance(eye, Vector3.Clamp(eye, box.Min, box.Max)); + inside[i] = !parallel && distance <= 0f; + fade[i] = box.Picked || parallel ? 1f : Math.Clamp(1f - ((distance - FadeFrom) / (FadeTo - FadeFrom)), 0f, 1f); + } + + void Draw(ZoneBox box, float alpha, float lighten, uint indices) + { + Matrix4x4 world = Matrix4x4.CreateScale(box.Max - box.Min) * Matrix4x4.CreateTranslation(box.Min); + Vector3 colour = Vector3.Lerp(box.Color, Vector3.One, lighten); + var c = new ZoneConstants { Wvp = world * viewProj, Color = new Vector4(colour, alpha) }; GpuBuffers.UpdateConstant(ctx, _cb, ref c); - ctx.DrawIndexed((uint)CubeIndices.Length, 0, 0); + ctx.DrawIndexed(indices, 0, 0); + } + + // Fills: faint, and none for a zone the camera is in - its faces are all round the viewer, and their + // tint would lie over the whole picture. + for (int i = 0; i < boxes.Count; i++) + { + ZoneBox box = boxes[i]; + if (inside[i] && !box.Picked) continue; + float alpha = box.Picked ? (inside[i] ? 0.05f : 0.20f) : (box.Loads ? 0.055f : 0.025f) * fade[i]; + if (alpha > 0.004f) Draw(box, alpha, 0f, (uint)CubeIndices.Length); + } + + // Edges: what tells one zone from the next. + ctx.IASetPrimitiveTopology(D3DPrimitiveTopology.D3DPrimitiveTopologyLinelist); + ctx.IASetIndexBuffer(_edges, Format.FormatR32Uint, 0); + for (int i = 0; i < boxes.Count; i++) + { + ZoneBox box = boxes[i]; + float alpha = box.Picked ? 1f : (box.Loads ? 0.9f : 0.4f) * fade[i]; + if (alpha > 0.02f) Draw(box, alpha, box.Picked ? 0.55f : 0.25f, (uint)EdgeIndices.Length); + // A zone that was added: a second, near-white frame a little way out. It fades half as much as the + // rest - what one has made oneself is what one looks for from afar. + if (box.Added) + { + var grown = box with { Min = box.Min - new Vector3(AddedFrame), Max = box.Max + new Vector3(AddedFrame) }; + Draw(grown, box.Picked ? 1f : 0.5f + (0.5f * fade[i]), 0.85f, (uint)EdgeIndices.Length); + } + } + + // The picked zone once more, through whatever stands in front of it. + ctx.OMSetDepthStencilState(_noDepth, 0); + for (int i = 0; i < boxes.Count; i++) + { + if (boxes[i].Picked) Draw(boxes[i], 0.45f, 0.55f, (uint)EdgeIndices.Length); } // Restore opaque blending for the next frame. @@ -145,6 +215,7 @@ public void Dispose() _raster.Dispose(); _noDepth.Dispose(); _blend.Dispose(); + _edges.Dispose(); _ib.Dispose(); _vb.Dispose(); _cb.Dispose(); diff --git a/src/Illusion/BlenderAddon/scripts/addons/illusion_bridge/importer.py b/src/Illusion/BlenderAddon/scripts/addons/illusion_bridge/importer.py index 6cfca8e..8a478f8 100644 --- a/src/Illusion/BlenderAddon/scripts/addons/illusion_bridge/importer.py +++ b/src/Illusion/BlenderAddon/scripts/addons/illusion_bridge/importer.py @@ -262,7 +262,16 @@ def _bind_skin(obj, desc, header, blocks, armatures, warnings): group = groups[b] = obj.vertex_groups.new(name=bone_names[b]) group.add((v,), w, 'REPLACE') + # Parenting keeps the mesh's own matrix as its offset FROM the armature — and that matrix is already the + # model's world, the same one the armature stands at. Left alone the mesh is carried there twice. On a + # car nobody sees it (a car's model sits at the identity); a person's model is turned a quarter turn — + # its geometry is stored along the spine — so the body arrived lying down beside a rig that stood, and + # a push from that scene read as "the object was moved". The offset is what is left of the mesh's world + # once the armature's is taken out: nothing, for a mesh on its own rig. + world = obj.matrix_world.copy() obj.parent = armature + obj.matrix_parent_inverse = Matrix.Identity(4) + obj.matrix_basis = armature.matrix_world.inverted_safe() @ world modifier = obj.modifiers.new(name="Illusion Rig", type='ARMATURE') modifier.object = armature modifier.use_vertex_groups = True diff --git a/src/Illusion/Bridge/BridgeSessionController.cs b/src/Illusion/Bridge/BridgeSessionController.cs index cbda9c3..737efc7 100644 --- a/src/Illusion/Bridge/BridgeSessionController.cs +++ b/src/Illusion/Bridge/BridgeSessionController.cs @@ -655,6 +655,13 @@ private void ApplyPush(PushMessage push) var sharedMeshNotes = new List(); var editedBuffers = new List<(string Name, ulong Hash, string Archive)>(); var geometry = new List(); + // Results already worked out in this push, per frame. Both levels of a mesh are one frame: they + // share its quantization lattice, its bounds and — on a model — its pools, and a result is + // computed against the frame as it stands. Two computed side by side against the SAME state each + // carry "the other level as it was", so applying the second put the first level's edit back + // where it came from. The second is therefore computed with the first in place. + var perFrame = new Dictionary>(); + var levelsMoved = new List<(string Name, FrameObjectSingleMesh Frame)>(); var transforms = new List(); var reshapes = new List(); var newHulls = new List(); @@ -755,9 +762,14 @@ private void ApplyPush(PushMessage push) continue; } - // Back into the level the row stands for — the same one it was exported from. + // Back into the level the row stands for — the same one it was exported from. What this + // push has already decided for the same frame goes in first and comes out again after: + // the batch below applies the results in this order, and undoes them in reverse. + FrameObjectSingleMesh? levelFrame = fn is FrameNodeAdapter { Frame: FrameObjectSingleMesh ofRow } ? ofRow : null; + List earlier = + levelFrame != null && perFrame.TryGetValue(levelFrame, out List? held) ? held : []; BridgeMeshApplier.ApplyResult? result = - BridgeMeshApplier.TryApply(fn, payload, out string? reason, node.Lod); + BridgeMeshApplier.TryApplyAfter(earlier, fn, payload, out string? reason, node.Lod); if (result == null) { ack.Skipped.Add(new PushSkip { Id = payload.Id, Reason = reason ?? "not applicable" }); @@ -785,6 +797,11 @@ private void ApplyPush(PushMessage push) if (!result.Unchanged) { geometry.Add(new GeometryEditController.GeometryItem(node, result)); + if (levelFrame != null) + { + if (!perFrame.TryGetValue(levelFrame, out List? sofar)) perFrame[levelFrame] = sofar = []; + sofar.Add(result); + } touchedTotal += result.TouchedVertices; // A frame references its mesh rather than owning it, and the shipped districts reuse @@ -815,11 +832,23 @@ private void ApplyPush(PushMessage push) // Object moved in Blender's Object Mode → re-localize against the CURRENT // parent and ride the same undoable batch. + // + // The FRAME is what stands somewhere, and its first level speaks for it. A coarser level + // is a second Blender object of the same frame: left where it was while the first one + // moved, it still carries the old matrix, and taken at its word it moved the frame back + // on the very next push — and forth again on the one after. if (!MatrixNear(payload.World, fn.WorldTransform)) { - transforms.Add(new GeometryEditController.TransformItem( - node, fn.LocalTransform, - TransformMath.ComputeLocalTransform(payload.World, fn.ParentWorldTransform))); + if (node.Lod == 0) + { + transforms.Add(new GeometryEditController.TransformItem( + node, fn.LocalTransform, + TransformMath.ComputeLocalTransform(payload.World, fn.ParentWorldTransform))); + } + else if (levelFrame != null) + { + levelsMoved.Add((payload.Name, levelFrame)); + } } ack.Applied.Add(payload.Id); @@ -867,6 +896,18 @@ private void ApplyPush(PushMessage push) }); continue; } + // A coarser LEVEL is not an object. Its row stands for the same frame as the first + // level's, so deleting "it" deleted the whole mesh, near level and all. + if (node.Lod > 0) + { + ack.Skipped.Add(new PushSkip + { + Id = id, + Reason = "a level of detail cannot be deleted on its own — the mesh keeps the level " + + "it had; delete the object itself (its first level) to remove it", + }); + continue; + } deleteNodes.Add(node); } var set = new HashSet(deleteNodes); @@ -1043,6 +1084,14 @@ private void ApplyPush(PushMessage push) }); var notes = new List(notesEarly); + // A coarser level moved by itself: said, since nothing happened. (Moved together with the first + // level it simply follows the frame, and there is nothing to say.) + foreach ((string name, FrameObjectSingleMesh frame) in levelsMoved) + { + if (transforms.Any(t => t.Node.Source is FrameNodeAdapter { Frame: FrameObjectSingleMesh moved } && ReferenceEquals(moved, frame))) continue; + notes.Add($"{name}: a coarser level was moved on its own — a mesh is placed by its first level, " + + "so the object stayed where it is. Move the LOD 0 object to move it."); + } if (skinNotSent.Count > 0) { notes.Add($"{string.Join(", ", skinNotSent.Take(3))}: no vertex weights came back — every " diff --git a/src/Illusion/Diagnostics/ProbeRunner.cs b/src/Illusion/Diagnostics/ProbeRunner.cs index 653a3fd..b0431b4 100644 --- a/src/Illusion/Diagnostics/ProbeRunner.cs +++ b/src/Illusion/Diagnostics/ProbeRunner.cs @@ -39,6 +39,18 @@ public static bool TryRun(string[] args) case "--probe-stream": WorldProbes.RunStreamProbe(); return true; + // The city's load zones: with no arguments a self-check on the real city (nothing written to the + // working copy); with "x y z", "dump ", "map x0 y0 x1 y1 step z", "move + // " a look around. Output: %TEMP%\illusion_zones.txt + case "--probe-zones": + WorldProbes.RunZonesProbe(args.Skip(1).ToArray()); + return true; + // What a Build would put into the game: an untouched working copy shows nothing, a changed and a + // removed file are listed, a loose file is not. Arguments: archives under pc\sds (default: + // city_univers and one district). Reads the game only. Output: %TEMP%\illusion_build_list.txt + case "--probe-build-list": + BuildListProbes.Run(args.Skip(1).ToArray()); + return true; // Dump of district scenes + their categories (proxy/snow/normal). case "--probe-scenes": SceneProbes.RunScenesProbe(args.Length >= 2 ? args[1] : "eastside"); @@ -173,6 +185,21 @@ public static bool TryRun(string[] args) case "--probe-prop-catalog": PropCatalogProbes.RunPropCatalogProbe(); return true; + // What a remap pool is, over every skinned model: the pools against the skeleton's own account + // of them. Optional arg = the car to dump. Output: %TEMP%\illusion_remap_pools.txt + case "--probe-remap-pools": + RemapPoolProbes.RunRemapPoolProbe(args.Length >= 2 ? args[1] : "shubert_38"); + return true; + // Where a person's stored geometry and rig stand, against where they are drawn. + // Optional arg = archive under pc\sds. Output: %TEMP%\illusion_person_rig.txt + case "--probe-person-rig": + PersonRigProbes.RunPersonRigProbe(args.Length >= 2 ? args[1] : "traffic/cirand.sds"); + return true; + // The content browser's archive pictures: people (head and shoulders, from both sides) and a car. + // Optional args = archives under pc\sds. Output: %TEMP%\illusion_archive_thumbs.txt + pictures. + case "--probe-archive-thumbs": + ArchiveThumbnailProbes.RunArchiveThumbnailProbe(args.Skip(1).ToArray()); + return true; // A car cloned under a new name, on scratch copies of the working copies. // Output: %TEMP%\illusion_car_clone.txt // The real clone against the install with a pack made to fail, and everything it wrote taken @@ -193,6 +220,12 @@ public static bool TryRun(string[] args) case "--probe-car-m2o": CarM2oExportProbes.RunCarM2oExportProbe(); return true; + // Hiding triangles of a stock mesh without a rebuild, in memory only: what a box or a click finds, + // that nothing but the found triangles' indices changes, a mesh given geometry of its own. + // Output: %TEMP%\illusion_hide_triangles.txt + case "--probe-hide-triangles": + HideTrianglesProbes.RunHideTrianglesProbe(args.Length >= 2 ? args[1] : "uppertown"); + return true; // Buffers nothing draws from: none in the archives as shipped, how many in the working copies. // Output: %TEMP%\illusion_geometry_sweep.txt case "--probe-geometry-sweep": diff --git a/src/Illusion/Diagnostics/Probes/ArchiveThumbnailProbes.cs b/src/Illusion/Diagnostics/Probes/ArchiveThumbnailProbes.cs new file mode 100644 index 0000000..9704378 --- /dev/null +++ b/src/Illusion/Diagnostics/Probes/ArchiveThumbnailProbes.cs @@ -0,0 +1,267 @@ +using System.IO; +using System.Text; +using System.Windows.Media.Imaging; +using Illusion.Assets; +using Illusion.Assets.Library; +using Illusion.Viewport; + +namespace Illusion.Diagnostics.Probes; + +/// +/// The content browser's archive pictures: a handful of people and a car drawn the way a tile draws them, the +/// pictures written beside the report so they can be looked at — a face seen from behind passes every count. +/// +internal static class ArchiveThumbnailProbes +{ + // Output: %TEMP%\illusion_archive_thumbs.txt, pictures in %TEMP%\illusion_archive_thumbs\ + // Args: archive paths under pc\sds (traffic/ccerb2.sds ...); none = a default handful. + internal static void RunArchiveThumbnailProbe(string[] archives) + { + string outFile = Path.Combine(Path.GetTempPath(), "illusion_archive_thumbs.txt"); + string pictures = Path.Combine(Path.GetTempPath(), "illusion_archive_thumbs"); + var sb = new StringBuilder(); + int pass = 0, fail = 0; + void Check(string name, bool ok, string detail = "") + { + if (ok) pass++; else fail++; + sb.AppendLine($"[{(ok ? "PASS" : "FAIL")}] {name}{(detail == "" ? "" : " — " + detail)}"); + } + + try + { + if (!ProbeAssert.InitEnv(out string? err)) { sb.AppendLine("INIT FAIL: " + err); return; } + if (archives.Length == 0) + { + archives = ["traffic/ccerb2.sds", "traffic/cbarma.sds", "hchar/vitarmy.sds", "player/vitvez2.sds", + "police_char/polmur.sds", "cars/shubert_38.sds"]; + } + if (Directory.Exists(pictures)) Directory.Delete(pictures, recursive: true); + Directory.CreateDirectory(pictures); + + using var renderer = new ArchiveThumbnailRenderer(); + int drawn = 0, tried = 0, keptAfter = 0, olderLeft = 0; + foreach (string relative in archives) + { + var file = new FileInfo(Path.Combine(MafiaEnvironment.PcFolder, "sds", relative.Replace('/', Path.DirectorySeparatorChar))); + if (!file.Exists) { sb.AppendLine($" (not in this install: {relative})"); continue; } + string folder = "sds/" + Path.GetDirectoryName(relative)!.Replace('\\', '/'); + var entry = new LibraryEntry + { + Name = Path.GetFileNameWithoutExtension(file.Name), + File = file, + Size = file.Length, + FolderPath = folder, + Resource = LibraryResourceKinds.Of(folder), + }; + foreach (float side in new[] { 1f, -1f }) // both sides of a person, so the right one can be told by eye + { + if (side < 0 && entry.Resource == LibraryResourceKind.Car) continue; + ArchiveThumbnailRenderer.PersonFrontY = side; + string kept = Path.Combine( + Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "Illusion", "archive_thumbs"); + if (Directory.Exists(kept)) Directory.Delete(kept, recursive: true); // draw, do not read back + // A picture kept for this archive in an earlier state of it — what every rebuild used to leave behind. + Directory.CreateDirectory(kept); + string older = Path.Combine(kept, ArchiveThumbnailRenderer.KeyOf(entry)[..17] + "0000000000000000.png"); + File.WriteAllBytes(older, [1]); + tried++; + var clock = System.Diagnostics.Stopwatch.StartNew(); + if (renderer.Render(entry) is not BitmapSource picture) + { + sb.AppendLine($" {relative}: nothing drawn"); + continue; + } + drawn++; + // Found again under the key computed NOW — after the archive was read, and so extracted. + if (ArchiveThumbnailRenderer.TryKept(entry, out System.Windows.Media.ImageSource? again) && again != null) keptAfter++; + if (File.Exists(older)) olderLeft++; + var encoder = new PngBitmapEncoder(); + encoder.Frames.Add(BitmapFrame.Create(picture)); + string name = $"{entry.Name}_{(side > 0 ? "plusY" : "minusY")}.png"; + using FileStream png = File.Create(Path.Combine(pictures, name)); + encoder.Save(png); + sb.AppendLine($" {relative} ({entry.Resource}) from {(side > 0 ? "+Y" : "-Y")}: {clock.ElapsedMilliseconds} ms, detail {Colours(picture)} colours -> {name}"); + } + } + Check("every archive asked for gets a picture", drawn == tried && tried > 0, $"{drawn} of {tried}"); + Check("each picture is kept and found again by the next look-up", keptAfter == drawn, $"{keptAfter} of {drawn}"); + Check("and the one kept for the archive's earlier state is dropped", olderLeft == 0, $"{olderLeft} left behind"); + + // An archive that cannot be read: a working copy whose manifest is cut short, which is not one of + // the errors anybody would think to name. It has to come back as "no picture", not as an exception — + // the tile asks from a timer tick, where an exception ends the program. + var broken = new FileInfo(Path.Combine(MafiaEnvironment.PcFolder, "sds", "cars", "illusion_probe_thumb.sds")); + string brokenCopy = MafiaEnvironment.ExtractedDir(broken); + try + { + File.WriteAllBytes(broken.FullName, [0]); + Directory.CreateDirectory(brokenCopy); + File.WriteAllText(Path.Combine(brokenCopy, "SDSContent.xml"), "FrameRes"); + broken.Refresh(); + var entry = new LibraryEntry + { + Name = "illusion_probe_thumb", File = broken, Size = broken.Length, FolderPath = "sds/cars", + Resource = LibraryResourceKind.Car, + }; + string outcome; + bool quiet; + try + { + ArchiveThumbnailRenderer.Staged staged = ArchiveThumbnailRenderer.Stage(entry); + System.Windows.Media.ImageSource? picture = renderer.Draw(staged); + quiet = staged.Empty && !staged.Passing && picture == null; + outcome = $"empty {staged.Empty}, passing {staged.Passing}"; + } + catch (Exception ex) + { + quiet = false; + outcome = ex.GetType().Name + ": " + ex.Message; + } + Check("an archive whose working copy cannot be read gets no picture and throws nothing", quiet, outcome); + Check("and that is written down, so the next run does not read it again to find out", + ArchiveThumbnailRenderer.TryKept(entry, out System.Windows.Media.ImageSource? none) && none == null); + } + finally + { + if (File.Exists(broken.FullName)) File.Delete(broken.FullName); + if (Directory.Exists(brokenCopy)) Directory.Delete(brokenCopy, recursive: true); + } + + // What a car is made of, part by part: which materials are paintwork is a question about their + // shaders and slots, and this is where it can be read off. + foreach (string relative in archives.Where(a => a.StartsWith("cars/", StringComparison.OrdinalIgnoreCase)).Take(3)) + { + var file = new FileInfo(Path.Combine(MafiaEnvironment.PcFolder, "sds", relative.Replace('/', Path.DirectorySeparatorChar))); + if (!file.Exists) continue; + sb.AppendLine($"\n parts of {relative}:"); + foreach (Illusion.Domain.MeshData mesh in Illusion.Assets.Sds.SdsMeshLoader.LoadSds(file, lod: 0)) + { + sb.AppendLine($" mesh {mesh.Name}: {mesh.Positions.Length} vertices, skinned {mesh.BoneIndices != null}"); + foreach (Illusion.Domain.MeshPart part in mesh.Parts) + { + Illusion.Domain.Materials.MaterialInfo? info = + Illusion.Assets.Materials.MafiaMaterialCatalog.Instance.GetMaterial(part.MaterialHash); + string c002 = info?.Parameters.FirstOrDefault(x => x.ParamId == "C002") is { } colour + ? string.Join(",", colour.Values.Select(v => v.ToString("0.##", System.Globalization.CultureInfo.InvariantCulture))) : "-"; + sb.AppendLine($" {part.IndexCount / 3,5} tris {info?.Name ?? "?",-26} shader {info?.ShaderId:X16}/{info?.ShaderHash:X8} " + + $"blended {part.Blended,-5} C002 {c002,-18} slots " + + string.Join(" ", (info?.TextureSlots ?? []).Select(t => $"{t.SlotId}={t.TextureName}")) + + " flags " + string.Join("|", info?.Flags ?? [])); + } + } + } + ArchiveThumbnailRenderer.PersonFrontY = 1f; + string keptDir = Path.Combine( + Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "Illusion", "archive_thumbs"); + if (Directory.Exists(keptDir)) Directory.Delete(keptDir, recursive: true); // the -Y pictures must not be read back as the tiles' + + // The pane itself: the street people's folder open in a real content browser, left to fill in. + LibraryCatalog catalog = LibraryCatalog.Build(Path.Combine(MafiaEnvironment.PcFolder, "sds")); + LibraryFolder? people = Find(catalog.Roots, "traffic"); + Check("the catalog has the street people's folder", people != null, people == null ? "" : $"{people.Entries.Count} archives"); + if (people != null) + { + var browser = new Views.ContentBrowser(); + var window = new System.Windows.Window + { + Width = 1280, Height = 420, Left = -4000, Top = 0, ShowInTaskbar = false, ShowActivated = false, + WindowStyle = System.Windows.WindowStyle.None, Content = browser, + }; + window.Show(); + browser.SetCatalog(catalog); + browser.OpenFolder(people); + // The pane has to stay alive while it fills: the longest the UI thread is held between two + // turns of the pump. Reading an archive used to happen on it, a second or more at a time. + var beat = System.Diagnostics.Stopwatch.StartNew(); + long longest = 0, last = 0; + var gaps = new List<(long At, long Gap)>(); + var pulse = new System.Windows.Threading.DispatcherTimer(System.Windows.Threading.DispatcherPriority.Send) + { + Interval = TimeSpan.FromMilliseconds(5), + }; + pulse.Tick += (_, _) => + { + long now = beat.ElapsedMilliseconds; + // The first three seconds hold two things that happen once per run and are not what is + // being asked about: the pane laying out a hundred and fifty tiles while the GPU stack is + // made, and the first picture drawn through shaders nothing has compiled yet. Both are in + // the list the check prints. + if (last > 0 && now - last > 60) gaps.Add((last, now - last)); + if (last > 3000) longest = Math.Max(longest, now - last); + last = now; + }; + pulse.Start(); + Pump(TimeSpan.FromSeconds(30)); + pulse.Stop(); + var images = Images(browser).ToList(); + int tiles = images.Count; + int looked = images.Count(Views.ArchiveThumb.OnScreen); + int filled = images.Count(i => Views.ArchiveThumb.OnScreen(i) && i.Source != null); + int unseenFilled = images.Count(i => !Views.ArchiveThumb.OnScreen(i) && i.Source != null); + Check("the tiles on screen get their pictures while the pane is open", looked > 0 && filled == looked, + $"{filled} of {looked} on screen ({tiles} tiles in the folder) after 30 s"); + Check("archives of tiles nobody is looking at are not read", unseenFilled == 0, $"{unseenFilled} off-screen tiles have a picture"); + Check("the UI thread is never held for long while the folder fills", longest < 400, + $"longest gap between two pumps after the first 3 s: {longest} ms; every gap over 60 ms (at ms: length): " + + string.Join(", ", gaps.Take(40).Select(g => $"{g.At}: {g.Gap}"))); + var shot = new RenderTargetBitmap((int)browser.ActualWidth, (int)browser.ActualHeight, 96, 96, + System.Windows.Media.PixelFormats.Pbgra32); + shot.Render(browser); + var png = new PngBitmapEncoder(); + png.Frames.Add(BitmapFrame.Create(shot)); + using (FileStream file = File.Create(Path.Combine(pictures, "_browser.png"))) png.Save(file); + window.Close(); + } + } + catch (Exception ex) + { + fail++; + sb.AppendLine("[FAIL] unexpected exception — " + ex); + } + finally + { + ArchiveThumbnailRenderer.PersonFrontY = 1f; + sb.Insert(0, $"ARCHIVE THUMBNAIL PROBE: {pass} passed, {fail} failed\n\n"); + File.WriteAllText(outFile, sb.ToString()); + } + } + + private static LibraryFolder? Find(IEnumerable folders, string name) + { + foreach (LibraryFolder folder in folders) + { + if (string.Equals(folder.Name, name, StringComparison.OrdinalIgnoreCase) && folder.Entries.Count > 0) return folder; + if (Find(folder.Folders, name) is { } found) return found; + } + return null; + } + + private static IEnumerable Images(System.Windows.DependencyObject root) + { + int n = System.Windows.Media.VisualTreeHelper.GetChildrenCount(root); + for (int i = 0; i < n; i++) + { + System.Windows.DependencyObject child = System.Windows.Media.VisualTreeHelper.GetChild(root, i); + if (child is System.Windows.Controls.Image image && Views.ArchiveThumb.GetEntry(image) != null) yield return image; + foreach (System.Windows.Controls.Image inner in Images(child)) yield return inner; + } + } + + private static void Pump(TimeSpan span) + { + var frame = new System.Windows.Threading.DispatcherFrame(); + var timer = new System.Windows.Threading.DispatcherTimer { Interval = span }; + timer.Tick += (_, _) => { timer.Stop(); frame.Continue = false; }; + timer.Start(); + System.Windows.Threading.Dispatcher.PushFrame(frame); + } + + private static int Colours(BitmapSource picture) + { + var bgra = new byte[picture.PixelWidth * picture.PixelHeight * 4]; + picture.CopyPixels(bgra, picture.PixelWidth * 4, 0); + var colours = new HashSet(); + for (int i = 0; i < bgra.Length; i += 4 * 7) colours.Add(bgra[i] | bgra[i + 1] << 8 | bgra[i + 2] << 16); + return colours.Count; + } +} diff --git a/src/Illusion/Diagnostics/Probes/BridgeSkinProbes.cs b/src/Illusion/Diagnostics/Probes/BridgeSkinProbes.cs index f469a96..7a3c801 100644 --- a/src/Illusion/Diagnostics/Probes/BridgeSkinProbes.cs +++ b/src/Illusion/Diagnostics/Probes/BridgeSkinProbes.cs @@ -495,8 +495,10 @@ Formats.Frames.ObjectTypes.FrameObjectModel probed Check("a skinned body with a face removed is accepted", applied != null, why ?? ""); if (applied != null) { - // The REBUILT mesh's face count, not the model's index buffer: TryApply leaves the buffers - // untouched until the caller commits with ApplyNew, so the model still holds the old ones. + // Working a push out leaves the MODEL untouched as well as its buffers — the face ranges and + // pools ride in the result — so it is committed before the model is asked about them, and + // taken back at the end of this block for the cases that follow. + applied.ApplyNew(); int triangles = (applied.NewMesh?.Indices.Length ?? 0) / 3; // The whole point: no range may name a face the mesh no longer has. A stale one is what @@ -562,6 +564,7 @@ Formats.Frames.ObjectTypes.FrameObjectModel probed applied.NewMesh == null ? "no replacement mesh built" : $"ids {applied.NewMesh.BoneIndices?.Length ?? 0}, rig " + $"{applied.NewMesh.Skeleton?.Bones.Count ?? 0} bones"); + applied.RestoreOriginal(); } // ── The mirror case: geometry ADDED ── @@ -585,6 +588,7 @@ Formats.Frames.ObjectTypes.FrameObjectModel probed Check("a skinned body with a face ADDED is accepted", grown != null, growWhy ?? ""); if (grown != null) { + grown.ApplyNew(); // the model is asked about its face ranges below int triangles = (grown.NewMesh?.Indices.Length ?? 0) / 3; int verts = grown.NewMesh?.Positions.Length ?? 0; Check("the added geometry really arrived", @@ -645,7 +649,6 @@ Formats.Frames.ObjectTypes.FrameObjectModel probed $"{onWanted} of {checkedNew} added vertices on bone {wantBone} " + $"(nearest would have given {nearBone})"); - grown.ApplyNew(); Check("the grown model survives the writer and comes back", SurvivesRoundTrip(model3, verts, triangles, out string? tripWhy), tripWhy ?? ""); @@ -853,7 +856,7 @@ private static bool IsAdded(System.Numerics.Vector3 position, MeshObjectPayload /// is perfectly fine in memory and simply makes the car vanish in game. /// /// - private static bool SurvivesRoundTrip(FrameObjectModel model, int verts, int faces, out string? why) + internal static bool SurvivesRoundTrip(FrameObjectModel model, int verts, int faces, out string? why) { why = null; try @@ -910,7 +913,7 @@ private static bool SurvivesRoundTrip(FrameObjectModel model, int verts, int fac /// id read against the wrong pool binds geometry to whatever bone happens to sit at that offset. /// /// - private static (int Checked, int Wrong, int Groups) ResolvesThroughPools( + internal static (int Checked, int Wrong, int Groups) ResolvesThroughPools( FrameObjectModel model, BridgeMeshApplier.ApplyResult applied, byte[] pools, byte[] remap) { MeshData? mesh = applied.NewMesh; @@ -1000,7 +1003,7 @@ private static System.Numerics.Matrix4x4 Affine(System.Numerics.Matrix4x4 m) return m; } - private static IFrameNode? FindModelNode(SdsFrameNode node) + internal static IFrameNode? FindModelNode(SdsFrameNode node) { if (node.Source is IFrameNode f && IsModel(f)) return f; foreach (SdsFrameNode c in node.Children) diff --git a/src/Illusion/Diagnostics/Probes/BuildListProbes.cs b/src/Illusion/Diagnostics/Probes/BuildListProbes.cs new file mode 100644 index 0000000..a21a872 --- /dev/null +++ b/src/Illusion/Diagnostics/Probes/BuildListProbes.cs @@ -0,0 +1,215 @@ +using System.Diagnostics; +using System.IO; +using System.Text; +using Illusion.Assets; +using Illusion.Assets.Sds; +using Illusion.Formats.Archive; +using static Illusion.Diagnostics.Probes.ProbeAssert; + +namespace Illusion.Diagnostics.Probes; + +/// Probes of what a Build would put into the game (). +internal static class BuildListProbes +{ + // The list a Build window shows, against real archives. For each archive named (default: city_univers, a + // small one, and a district) the archive is unpacked into %TEMP% and compared with itself - an untouched + // working copy must show NOTHING, or every Build window would cry wolf. Then one file is changed, one the + // manifest names is removed, one loose file is dropped in beside them and one named piece is added: the + // list must hold exactly the changed one, the removed one (also as a missing manifest entry) and the + // manifest - and not the loose file, which a pack does not take. The game is only read. + // With "real ..." it lists instead what a Build of the toolkit's own working copies would change. + // Output: %TEMP%\illusion_build_list.txt + internal static void Run(string[] args) + { + string outFile = Path.Combine(Path.GetTempPath(), "illusion_build_list.txt"); + var sb = new StringBuilder(); + int pass = 0, fail = 0; + void Check(string label, bool ok, string detail = "") + { + if (ok) pass++; else fail++; + sb.AppendLine($"[{(ok ? "PASS" : "FAIL")}] {label}{(detail.Length > 0 ? " - " + detail : "")}"); + } + var scratches = new List(); + try + { + if (!InitEnv(out string? err)) { sb.AppendLine("INIT FAIL: " + err); return; } + // "real ..." lists, for the working copies the toolkit actually keeps, what a Build of + // each would change in the game - the same list the Build window shows. Nothing is checked. + if (args.Length > 1 && args[0] == "real") + { + foreach (string name in args.Skip(1)) + { + var real = new FileInfo(Path.IsPathRooted(name) ? name : Path.Combine(MafiaEnvironment.PcFolder, "sds", name)); + var watch = Stopwatch.StartNew(); + WorkingCopyComparison found = WorkingCopyDiff.Compare(real); + sb.AppendLine($"{real.Name}: {found.Changes.Count} file(s) differ from the game, {found.MissingEntries.Count} manifest entr(ies) without a file ({watch.ElapsedMilliseconds} ms)"); + foreach (WorkingCopyChange change in found.Changes.Take(40)) + { + sb.AppendLine($" {change.Kind,-8} {change.Path} {change.Size} B {change.Modified:yyyy-MM-dd HH:mm}"); + } + } + return; + } + List archives = args.Length > 0 + ? [.. args.Select(a => new FileInfo(Path.IsPathRooted(a) ? a : Path.Combine(MafiaEnvironment.PcFolder, "sds", a)))] + : [new FileInfo(MafiaEnvironment.CityUniversSds), new FileInfo(Path.Combine(MafiaEnvironment.CityFolder, "arpradelna.sds"))]; + foreach (FileInfo sds in archives) + { + if (!sds.Exists) { Check($"{sds.Name} is in the game", false, sds.FullName); continue; } + string scratch = Path.Combine(Path.GetTempPath(), "illusion_buildlist_probe_" + Guid.NewGuid().ToString("N")); + scratches.Add(scratch); + var clock = Stopwatch.StartNew(); + SdsArchive.Open(sds.FullName).Extract(scratch); + int files = Directory.EnumerateFiles(scratch, "*", SearchOption.AllDirectories).Count(); + long unpack = clock.ElapsedMilliseconds; + + clock.Restart(); + WorkingCopyComparison same = WorkingCopyDiff.Compare(sds, scratch); + Check($"{sds.Name}: a working copy nobody touched shows nothing ({files} files; unpack {unpack} ms, compare {clock.ElapsedMilliseconds} ms)", + same.InGame && same.Changes.Count == 0 && same.MissingEntries.Count == 0, + string.Join(", ", same.Changes.Take(6).Select(c => $"{c.Kind} {c.Path}"))); + + // a resource the archive does not name is unpacked under its index, and the index moves when a + // resource is added before it: the same bytes under another number are the same resource + string? numbered = Directory.EnumerateFiles(scratch).Select(Path.GetFileName) + .FirstOrDefault(n => n != null && System.Text.RegularExpressions.Regex.IsMatch(n, @"^[A-Za-z_]+_\d+\.(fr|fnt|act|col|ibp|vbp)$")); + if (numbered != null) + { + string renumbered = System.Text.RegularExpressions.Regex.Replace(numbered, @"\d+(\.[a-z]+)$", "9999$1"); + File.Move(Path.Combine(scratch, numbered), Path.Combine(scratch, renumbered)); + string manifestPath = Path.Combine(scratch, "SDSContent.xml"); + File.WriteAllText(manifestPath, File.ReadAllText(manifestPath).Replace(">" + numbered + "<", ">" + renumbered + "<")); + WorkingCopyComparison moved = WorkingCopyDiff.Compare(sds, scratch); + Check($"{sds.Name}: {numbered} renumbered to {renumbered} is still the same resource - nothing listed", + moved.Changes.Count == 0 && moved.MissingEntries.Count == 0, string.Join(", ", moved.Changes.Take(6).Select(c => $"{c.Kind} {c.Path}"))); + File.AppendAllText(Path.Combine(scratch, renumbered), "x"); + WorkingCopyComparison both = WorkingCopyDiff.Compare(sds, scratch); + Check($"{sds.Name}: renumbered AND changed, it is listed once, as changed", + both.Changes.Count == 1 && both.Changes[0] is { Kind: WorkingCopyChangeKind.Changed } only && only.Path == renumbered, + string.Join(", ", both.Changes.Take(6).Select(c => $"{c.Kind} {c.Path}"))); + // put back, for the checks below + File.Delete(Path.Combine(scratch, renumbered)); + Directory.Delete(scratch, recursive: true); + SdsArchive.Open(sds.FullName).Extract(scratch); + } + + // one changed, one removed, one loose + List plain = [.. Directory.EnumerateFiles(scratch).Select(Path.GetFileName).Where(n => n != "SDSContent.xml").Select(n => n!) + .OrderBy(n => n, StringComparer.Ordinal)]; + if (plain.Count < 2) { sb.AppendLine($"({sds.Name} has fewer than two files beside its manifest - the change checks are skipped)"); continue; } + string changed = plain[0], removed = plain[^1]; + File.AppendAllText(Path.Combine(scratch, changed), "x"); + File.Delete(Path.Combine(scratch, removed)); + File.WriteAllText(Path.Combine(scratch, "note to self.txt"), "not part of the archive"); + WorkingCopyComparison edited = WorkingCopyDiff.Compare(sds, scratch); + List got = [.. edited.Changes.Select(c => $"{c.Kind} {c.Path}")]; + Check($"{sds.Name}: a changed file and a removed one are listed, and nothing else", + got.SequenceEqual(new[] { $"Changed {changed}", $"Removed {removed}" }.OrderBy(x => x[(x.IndexOf(' ') + 1)..], StringComparer.OrdinalIgnoreCase)), + string.Join(", ", got)); + Check($"{sds.Name}: the removed file is also named as an entry the pack would leave out", + edited.MissingEntries.Any(m => m.TrimStart('/', '\\').Equals(removed, StringComparison.OrdinalIgnoreCase)), string.Join(", ", edited.MissingEntries)); + Check($"{sds.Name}: a loose file the manifest does not name is not listed", + !edited.Changes.Any(c => c.Path.Contains("note to self"))); + Check($"{sds.Name}: the changed file carries its size and the time it was written", + edited.Changes.FirstOrDefault(c => c.Kind == WorkingCopyChangeKind.Changed) is { Size: > 0 } c1 && (DateTime.Now - c1.Modified).TotalMinutes < 5); + + // A resource deleted in the content browser: its entry is unsaid and its payload stays where it + // lay, byte for byte what the archive has. The pack leaves it out, so it is a removal - and was + // once paired with the archive's copy as "the same", which made the deletion invisible. + if (plain.Count >= 3) + { + string unsaid = plain[1]; + string entry = Path.GetExtension(unsaid).Equals(".xml", StringComparison.OrdinalIgnoreCase) ? unsaid[..^4] : unsaid; + var manifest = Formats.Archive.SdsManifest.Load(scratch); + string? listed = manifest.Entries.Select(e => e.File).FirstOrDefault(f => f.TrimStart('/', '\\').Equals(entry, StringComparison.OrdinalIgnoreCase)); + if (listed != null && manifest.RemoveEntry(listed)) + { + WorkingCopyComparison dropped = WorkingCopyDiff.Compare(sds, scratch); + Check($"{sds.Name}: a resource unsaid in the list, its file still lying there, is listed as removed", + dropped.Changes.Any(c => c.Kind == WorkingCopyChangeKind.Removed && c.Path.Equals(unsaid, StringComparison.OrdinalIgnoreCase)) + && dropped.Changes.Count == edited.Changes.Count + 1, string.Join(", ", dropped.Changes.Select(c => $"{c.Kind} {c.Path}"))); + } + else + { + sb.AppendLine($"({sds.Name}: {unsaid} is not a plain entry of the list - the unsaid-resource check is skipped)"); + } + } + + // A texture's top level lies beside it as "MIP_" and is a resource with an entry of its own. + // Unsaid alone - its texture still listed - it used to pass as named by the texture's entry. + { + var manifest = Formats.Archive.SdsManifest.Load(scratch); + string? mip = Directory.EnumerateFiles(scratch).Select(Path.GetFileName).Select(n => n!) + .FirstOrDefault(n => n.StartsWith("MIP_", StringComparison.OrdinalIgnoreCase) && File.Exists(Path.Combine(scratch, n[4..])) + && manifest.HasFile(n) && manifest.HasFile(n[4..])); + if (mip != null) + { + int before = WorkingCopyDiff.Compare(sds, scratch).Changes.Count; + manifest.RemoveEntry(mip); + WorkingCopyComparison without = WorkingCopyDiff.Compare(sds, scratch); + Check($"{sds.Name}: a Mipmap unsaid alone, its texture still listed, is listed as removed", + without.Changes.Count == before + 1 && without.Changes.Any(c => c.Kind == WorkingCopyChangeKind.Removed && c.Path.Equals(mip, StringComparison.OrdinalIgnoreCase)), + $"{mip}: {before} -> {without.Changes.Count} change(s)"); + } + else + { + sb.AppendLine($"({sds.Name} has no texture with a Mipmap of its own entry - the Mipmap check is skipped)"); + } + } + } + + // an archive the game does not have yet: everything the manifest names is an addition + if (scratches.Count > 0) + { + var absent = new FileInfo(Path.Combine(Path.GetTempPath(), "illusion_no_such_archive_" + Guid.NewGuid().ToString("N") + ".sds")); + WorkingCopyComparison fresh = WorkingCopyDiff.Compare(absent, scratches[0]); + Check("an archive not in the game yet: every packed file is an addition", + !fresh.InGame && fresh.Changes.Count > 0 && fresh.Changes.All(c => c.Kind == WorkingCopyChangeKind.Added) + && !fresh.Changes.Any(c => c.Path.Contains("note to self")), $"{fresh.Changes.Count} file(s)"); + } + // The window itself, with a row in it. A wrong style or binding in a row's template is not a build + // error: it throws when the first row is made - and took the application down the first time. + { + var window = new Views.BuildWindow([new FileInfo(MafiaEnvironment.CityUniversSds)]) { ShowInTaskbar = false, Left = -4000, Top = -4000 }; + window.Show(); + var limit = Stopwatch.StartNew(); + while (window.Rows.Any(r => r.Comparing) && limit.ElapsedMilliseconds < 20000) + { + var frame = new System.Windows.Threading.DispatcherFrame(); + System.Windows.Threading.Dispatcher.CurrentDispatcher.BeginInvoke(System.Windows.Threading.DispatcherPriority.Background, + new Action(() => frame.Continue = false)); + System.Windows.Threading.Dispatcher.PushFrame(frame); + Thread.Sleep(20); + } + window.UpdateLayout(); + Views.BuildWindow.ArchiveRow row = window.Rows[0]; + Check("the Build window makes a row for an archive and finishes comparing it", !row.Comparing && row.Summary.Length > 0, row.Summary); + row.IsOpen = true; + window.UpdateLayout(); + Check("...and the row can be opened and ticked without the window falling over", window.IsLoaded); + window.Close(); + } + + bool refused = false; + try { WorkingCopyDiff.Compare(new FileInfo(MafiaEnvironment.CityUniversSds), Path.Combine(Path.GetTempPath(), "illusion_no_such_folder")); } + catch (FileNotFoundException) { refused = true; } + Check("a folder that is not a working copy is refused", refused); + Check("nothing is left in %TEMP% by the comparisons themselves", + !Directory.EnumerateDirectories(Path.GetTempPath(), "illusion_compare_*").Any()); + } + catch (Exception ex) + { + fail++; + sb.AppendLine("unexpected exception: " + ex); + } + finally + { + foreach (string scratch in scratches) + { + try { if (Directory.Exists(scratch)) Directory.Delete(scratch, recursive: true); } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) { sb.AppendLine("(scratch left behind: " + scratch + ")"); } + } + File.WriteAllText(outFile, $"BUILD LIST PROBE: {pass} passed, {fail} failed\n" + sb); + } + } +} diff --git a/src/Illusion/Diagnostics/Probes/HideTrianglesProbes.cs b/src/Illusion/Diagnostics/Probes/HideTrianglesProbes.cs new file mode 100644 index 0000000..2dff357 --- /dev/null +++ b/src/Illusion/Diagnostics/Probes/HideTrianglesProbes.cs @@ -0,0 +1,234 @@ +using System.IO; +using System.Numerics; +using System.Text; +using Illusion.Assets; +using Illusion.Assets.Sds; +using Illusion.Formats.Frames; +using Illusion.Formats.Frames.ObjectTypes; + +namespace Illusion.Diagnostics.Probes; + +/// +/// Hiding triangles of a stock mesh without rebuilding it (), on a district read +/// from its working copy and never written: what a box finds, that only the found triangles' indices change +/// and nothing else of the buffer does, that a second look finds nothing, and what a material filter leaves. +/// +internal static class HideTrianglesProbes +{ + // Output: %TEMP%\illusion_hide_triangles.txt + internal static void RunHideTrianglesProbe(string district) + { + string outFile = Path.Combine(Path.GetTempPath(), "illusion_hide_triangles.txt"); + var sb = new StringBuilder(); + int pass = 0, fail = 0; + void Check(string name, bool ok, string detail = "") + { + if (ok) pass++; else fail++; + sb.AppendLine($"[{(ok ? "PASS" : "FAIL")}] {name}{(detail == "" ? "" : " — " + detail)}"); + } + + try + { + if (!ProbeAssert.InitEnv(out string? err)) { sb.AppendLine("INIT FAIL: " + err); return; } + var sds = new FileInfo(Path.Combine(MafiaEnvironment.CityFolder, district + ".sds")); + if (!sds.Exists) { sb.AppendLine("no such district: " + sds.FullName); return; } + ExtractedSds scene = ExtractedSds.Load(SdsMeshLoader.EnsureExtracted(sds)); + + // What the district's frames share with one another - which is what an edit of ONE mesh has to refuse + // or carry to the others: a geometry block, a material block, an index buffer under two blocks. + { + List all = [.. scene.FrameResource!.FrameObjects.Values.OfType()]; + int Shared(Func of) => all.Where(m => of(m) != null) + .GroupBy(m => of(m)!, ReferenceEqualityComparer.Instance).Where(g => g.Count() > 1).Sum(g => g.Count()); + int materialAcrossGeometry = all.Where(m => m.Material != null).GroupBy(m => (object)m.Material, ReferenceEqualityComparer.Instance) + .Count(g => g.Select(m => (object?)m.Geometry).Distinct(ReferenceEqualityComparer.Instance).Count() > 1); + var blocksOfBuffer = new Dictionary>(); + foreach (FrameObjectSingleMesh m in all.Where(m => m.Geometry?.LOD != null)) + { + for (int lod = 0; lod < m.Geometry.LOD.Length; lod++) + { + if (m.GetIndexBuffer(lod) is not { } buffer) continue; + if (!blocksOfBuffer.TryGetValue(buffer.Hash, out HashSet? blocks)) + blocksOfBuffer[buffer.Hash] = blocks = new HashSet(ReferenceEqualityComparer.Instance); + blocks.Add(m.Geometry); + } + } + sb.AppendLine($"INFO {district}: {all.Count} mesh frames; {Shared(m => m.Geometry)} on a geometry block another frame uses, " + + $"{Shared(m => m.Material)} on a material block another frame uses ({materialAcrossGeometry} material block(s) under frames " + + $"of different geometry); {blocksOfBuffer.Count(b => b.Value.Count > 1)} of {blocksOfBuffer.Count} index buffers drawn by more than one geometry block"); + sb.AppendLine("INFO frames on a shared material block: " + string.Join(", ", all.Where(m => m.Refs.ContainsKey(FrameEntryRefTypes.Material)) + .GroupBy(m => (object)m.Material, ReferenceEqualityComparer.Instance).Where(g => g.Count() > 1).SelectMany(g => g).Take(12).Select(m => m.Name.ToString()))); + } + + // The mesh with the most levels of detail and materials the district has, so both are exercised + // where the district has them at all. + FrameObjectSingleMesh? mesh = scene.FrameResource!.FrameObjects.Values.OfType() + .Where(m => m.Geometry?.LOD is { Length: > 0 } && m.Material?.Materials is { Count: > 0 } slots + && slots[0].Length > 0 && slots[0][0].NumFaces > 0) + .OrderByDescending(m => m.Geometry.LOD.Length) + .ThenByDescending(m => m.Material.Materials[0].Length) + .ThenByDescending(m => m.Geometry.LOD[0].NumVerts) + .FirstOrDefault(m => SdsMeshLoader.DecodeLod(m, 0) != null); + Check("a mesh to try", mesh != null, mesh == null ? "" : + $"{mesh.Name.String}: {mesh.Geometry.LOD.Length} level(s), {mesh.Material.Materials[0].Length} material(s)"); + if (mesh == null) return; + Matrix4x4 world = mesh.WorldTransform; + DecodedMesh decoded = SdsMeshLoader.DecodeLod(mesh, 0)!; + uint[] original = (uint[])decoded.Indices.Clone(); + + // A box around the first triangle of the first slot, a little larger than the triangle itself. + var slot = mesh.Material.Materials[0][0]; + Vector3[] corners = [.. Enumerable.Range(0, 3).Select(k => Vector3.Transform(decoded.Positions[original[slot.StartIndex + k]], world))]; + Vector3 min = Vector3.Min(corners[0], Vector3.Min(corners[1], corners[2])) - new Vector3(0.05f); + Vector3 max = Vector3.Max(corners[0], Vector3.Max(corners[1], corners[2])) + new Vector3(0.05f); + + TriangleHider.Plan plan = TriangleHider.Find(mesh, world, min, max); + Check("the box finds its triangle", plan.Triangles.Any(t => t.Lod == 0 && t.A == corners[0] && t.B == corners[1] && t.C == corners[2]), + $"{plan.Triangles.Count} triangles over {plan.Changes.Count} buffers"); + Check("every triangle found lies inside the box", plan.Triangles.All(t => + new[] { t.A, t.B, t.C }.All(p => p.X >= min.X - 0.01f && p.X <= max.X + 0.01f && p.Y >= min.Y - 0.01f + && p.Y <= max.Y + 0.01f && p.Z >= min.Z - 0.01f && p.Z <= max.Z + 0.01f))); + Check("finding changes nothing", decoded.Indices.AsSpan().SequenceEqual(original) + && mesh.GetIndexBuffer(0)!.GetData().AsSpan().SequenceEqual(original)); + + TriangleHider.Change? first = plan.Changes.FirstOrDefault(c => ReferenceEquals(c.Buffer, mesh.GetIndexBuffer(0))); + Check("the first level's buffer is among the changes", first != null); + if (first == null) return; + int differing = 0, wholeTriangles = 0, degenerate = 0; + for (int i = 0; i + 2 < first.After.Length; i += 3) + { + bool changed = first.After[i] != first.Before[i] || first.After[i + 1] != first.Before[i + 1] || first.After[i + 2] != first.Before[i + 2]; + if (!changed) continue; + differing++; + if (first.After[i] == first.Before[i]) wholeTriangles++; + if (first.After[i] == first.After[i + 1] && first.After[i + 1] == first.After[i + 2]) degenerate++; + } + int lod0 = plan.Triangles.Count(t => t.Lod == 0); + Check("only the found triangles' indices change, each to one vertex of its own", + first.After.Length == first.Before.Length && differing == lod0 && wholeTriangles == lod0 && degenerate == lod0, + $"{differing} triangles differ of {lod0} found"); + + // Applied in memory (nothing is saved): a second look finds nothing, and putting it back restores every index. + foreach (TriangleHider.Change change in plan.Changes) change.Buffer.SetData(change.After); + Check("hidden triangles are not found again", TriangleHider.Find(mesh, world, min, max).Triangles.Count == 0); + Check("the vertex count is what it was", SdsMeshLoader.DecodeLod(mesh, 0)!.NumVerts == decoded.NumVerts); + foreach (TriangleHider.Change change in plan.Changes) change.Buffer.SetData(change.Before); + Check("restoring puts every index back", mesh.GetIndexBuffer(0)!.GetData().AsSpan().SequenceEqual(original) + && TriangleHider.Find(mesh, world, min, max).Triangles.Count == plan.Triangles.Count); + + string wanted = slot.MaterialName ?? ""; + Check("a material filter keeps to that material", + TriangleHider.Find(mesh, world, min, max, wanted).Triangles.All(t => t.Material == wanted) + && TriangleHider.Find(mesh, world, min, max, "no_such_material_name").Triangles.Count == 0, wanted); + Check("a box elsewhere finds nothing", + TriangleHider.Find(mesh, world, max + new Vector3(5000f), max + new Vector3(5001f)).Changes.Count == 0); + + // Picked by a click: a ray from just off the first triangle, straight at its middle, meets it; the + // plan made of that one pick hides it and - on the other levels - only what lies on it. + Vector3 middle = (corners[0] + corners[1] + corners[2]) / 3f; + Vector3 normal = Vector3.Normalize(Vector3.Cross(corners[1] - corners[0], corners[2] - corners[0])); + TriangleHider.Picked? hit = TriangleHider.Pick(mesh, world, middle + (normal * 0.02f), -normal); + Check("a ray at a triangle's middle meets it, from either side", + hit != null && MathF.Abs(hit.Distance - 0.02f) < 0.005f + && TriangleHider.Pick(mesh, world, middle - (normal * 0.02f), normal) is { } behind && MathF.Abs(behind.Distance - 0.02f) < 0.005f, + hit == null ? "no hit" : $"index {hit.Index} at {hit.Distance:F3} m (the first slot starts at {slot.StartIndex})"); + Check("a ray pointing away from the mesh meets nothing", + TriangleHider.Pick(mesh, world, max + new Vector3(5000f), Vector3.Normalize(Vector3.One)) == null); + if (hit != null) + { + TriangleHider.Plan picked = TriangleHider.FindPicked(mesh, world, new HashSet { hit.Index }); + Check("one pick is one triangle of the first level, the one that was met", + picked.Triangles.Count(t => t.Lod == 0) == 1 && picked.Triangles.First(t => t.Lod == 0) == hit.Triangle, + string.Join(", ", Enumerable.Range(0, mesh.Geometry.LOD.Length).Select(l => $"LOD {l}: {picked.Triangles.Count(t => t.Lod == l)}"))); + Vector3 lo = Vector3.Min(hit.Triangle.A, Vector3.Min(hit.Triangle.B, hit.Triangle.C)) - new Vector3(0.06f); + Vector3 hi = Vector3.Max(hit.Triangle.A, Vector3.Max(hit.Triangle.B, hit.Triangle.C)) + new Vector3(0.06f); + Check("what goes with it on the other levels lies on it", + picked.Triangles.Where(t => t.Lod > 0).All(t => new[] { t.A, t.B, t.C }.All(p => + p.X >= lo.X && p.Y >= lo.Y && p.Z >= lo.Z && p.X <= hi.X && p.Y <= hi.Y && p.Z <= hi.Z))); + Check("picking changes nothing", mesh.GetIndexBuffer(0)!.GetData().AsSpan().SequenceEqual(original)); + Check("no pick, or a pick that is no triangle's start, hides nothing", + TriangleHider.FindPicked(mesh, world, new HashSet()).Changes.Count == 0 + && TriangleHider.FindPicked(mesh, world, new HashSet { hit.Index + 1, -3, int.MaxValue }).Triangles.Count == 0); + foreach (TriangleHider.Change change in picked.Changes) change.Buffer.SetData(change.After); + Check("a hidden triangle is not met by the same ray again", + TriangleHider.Pick(mesh, world, middle + (normal * 0.02f), -normal)?.Index != hit.Index + && TriangleHider.FindPicked(mesh, world, new HashSet { hit.Index }).Triangles.Count == 0); + foreach (TriangleHider.Change change in picked.Changes) change.Buffer.SetData(change.Before); + Check("...and restoring puts every index back", mesh.GetIndexBuffer(0)!.GetData().AsSpan().SequenceEqual(original)); + } + + // A mesh whose geometry other frames draw too, given a block and buffers of its own - what lets its + // triangles be hidden without the others losing theirs. + var document = new Illusion.Assets.Adapters.SceneDocumentAdapter(scene.FrameResource!, sds); + FrameObjectSingleMesh? shared = scene.FrameResource!.FrameObjects.Values.OfType() + .FirstOrDefault(m => m.GetType() == typeof(FrameObjectSingleMesh) && m.Geometry?.LOD is { Length: > 0 } + && m.GetIndexBuffer(0) != null && document.GeometrySharers(m).Count > 0 && SdsMeshLoader.DecodeLod(m, 0) != null); + Check("a mesh that shares its geometry to try", shared != null, shared == null ? "none in this district" : + $"{shared.Name.String}: {document.GeometrySharers(shared).Count} other frame(s) on its block"); + if (shared != null) + { + byte[] sceneBefore = scene.FrameResource!.WriteToStream(); + Illusion.Formats.Frames.Resources.FrameGeometry block = shared.Geometry; + FrameObjectSingleMesh other = document.GeometrySharers(shared)[0]; + DecodedMesh was = SdsMeshLoader.DecodeLod(shared, 0)!; + uint[] sharedIndices = (uint[])shared.GetIndexBuffer(0)!.GetData().Clone(); + int vertexBuffers = scene.FrameResource.VertexBuffers.Buffers.Count, indexBuffers = scene.FrameResource.IndexBuffers.Buffers.Count; + + Illusion.Assets.Frames.FrameDuplicator.OwnedGeometry? owned = + Illusion.Assets.Frames.FrameDuplicator.TryOwnGeometry(document, document.Node(shared), out string? noCopy); + Check("it is given a block of its own, and the others stay on theirs", + owned != null && !ReferenceEquals(shared.Geometry, block) && ReferenceEquals(other.Geometry, block) + && document.GeometrySharers(shared).Count == 0, noCopy ?? ""); + if (owned != null) + { + DecodedMesh now = SdsMeshLoader.DecodeLod(shared, 0)!; + Check("...which holds what the shared one held, in buffers of its own", + now.Positions.AsSpan().SequenceEqual(was.Positions) && now.Indices.AsSpan().SequenceEqual(was.Indices) + && !ReferenceEquals(shared.GetIndexBuffer(0), other.GetIndexBuffer(0)) + && !ReferenceEquals(shared.GetVertexBuffer(0), other.GetVertexBuffer(0)) + && ReferenceEquals(owned.CopyOf(other.GetIndexBuffer(0)!), shared.GetIndexBuffer(0))); + Vector3[] tri = [.. Enumerable.Range(0, 3).Select(k => Vector3.Transform(now.Positions[now.Indices[k]], shared.WorldTransform))]; + TriangleHider.Plan alone = TriangleHider.Find(shared, shared.WorldTransform, + Vector3.Min(tri[0], Vector3.Min(tri[1], tri[2])) - new Vector3(0.05f), Vector3.Max(tri[0], Vector3.Max(tri[1], tri[2])) + new Vector3(0.05f)); + foreach (TriangleHider.Change change in alone.Changes) change.Buffer.SetData(change.After); + Check("triangles hidden in it are not hidden in the shared block", + alone.Changes.Count > 0 && other.GetIndexBuffer(0)!.GetData().AsSpan().SequenceEqual(sharedIndices) + && !shared.GetIndexBuffer(0)!.GetData().AsSpan().SequenceEqual(sharedIndices), $"{alone.Triangles.Count} triangle(s)"); + Check("the scene with the new block writes and reads back with the mesh on it", + Reread(scene.FrameResource.WriteToStream(), shared.Name.String) is { } again && again.Geometry.LOD.Length == shared.Geometry.LOD.Length + && again.Geometry.LOD[0].IndexBufferRef.Hash == shared.Geometry.LOD[0].IndexBufferRef.Hash); + owned.Revert(); + Check("taking it back leaves the scene and the pools as they were, byte for byte", + ReferenceEquals(shared.Geometry, block) && scene.FrameResource.WriteToStream().AsSpan().SequenceEqual(sceneBefore) + && scene.FrameResource.VertexBuffers.Buffers.Count == vertexBuffers && scene.FrameResource.IndexBuffers.Buffers.Count == indexBuffers + && shared.GetIndexBuffer(0)!.GetData().AsSpan().SequenceEqual(sharedIndices)); + } + } + } + catch (Exception ex) + { + fail++; + sb.AppendLine("[FAIL] unexpected exception — " + ex); + } + finally + { + sb.Insert(0, $"HIDE TRIANGLES PROBE: {pass} passed, {fail} failed\n\n"); + File.WriteAllText(outFile, sb.ToString()); + } + } + + // A scene written to memory, read again, and the mesh of that name in it. + private static FrameObjectSingleMesh? Reread(byte[] sceneBytes, string name) + { + string file = Path.Combine(Path.GetTempPath(), "illusion_hide_triangles_scene.fr"); + File.WriteAllBytes(file, sceneBytes); + try + { + return new FrameResource(file).FrameObjects.Values.OfType().FirstOrDefault(m => m.Name.String == name); + } + finally + { + File.Delete(file); + } + } +} diff --git a/src/Illusion/Diagnostics/Probes/MaterialEditorProbes.cs b/src/Illusion/Diagnostics/Probes/MaterialEditorProbes.cs index 393e0e0..b2f7a17 100644 --- a/src/Illusion/Diagnostics/Probes/MaterialEditorProbes.cs +++ b/src/Illusion/Diagnostics/Probes/MaterialEditorProbes.cs @@ -385,6 +385,44 @@ private static void CheckSlotReassign(Action check, String check("out-of-range slot is refused", !editor.SetSlotMaterial(9999, other), ""); editor.SetSlotMaterial(0, original); check("slot restored", editor.GetSlotMaterial(0) == original, ""); + + // A re-point mirrors itself into further levels by the slot's old material, and that cannot be run + // backwards: a level with slots A and B is B and B after "A to B", and "B to A" makes it A and A. So an + // undo puts the table back. Tried on a mesh whose further level holds the first level's material and + // another one beside it. + foreach (FrameObjectSingleMesh mesh in fr.FrameObjects.Values.OfType()) + { + if (doc.Node(mesh) is not IMaterialSlotEditor levels) continue; + IReadOnlyList shipped = levels.GetSlotTable(); + if (shipped.Count < 2 || shipped[0].Length == 0) continue; + ulong a = shipped[0][0]; + // Such a level is made where the district ships none: its first two slots are given the first + // level's material and another one. The shipped table goes back at the end. + int at = Enumerable.Range(1, shipped.Count - 1).FirstOrDefault(l => shipped[l].Length >= 2, -1); + if (at < 0) continue; + ulong b = a ^ 0x5555UL; + ulong[][] made = [.. shipped.Select(l => (ulong[])l.Clone())]; + (made[at][0], made[at][1]) = (a, b); + if (!levels.SetSlotTable(made)) continue; + IReadOnlyList before = levels.GetSlotTable(); + ulong[] further = before[at]; + + static bool Same(IReadOnlyList x, IReadOnlyList y) => x.Count == y.Count && x.Zip(y).All(p => p.First.SequenceEqual(p.Second)); + levels.SetSlotMaterial(0, b); + IReadOnlyList after = levels.GetSlotTable(); + levels.SetSlotMaterial(0, a); + bool backwardsLoses = !Same(levels.GetSlotTable(), before); + check("a re-point run backwards does not give the further level its two materials back", backwardsLoses, + $"{mesh.Name}: a further level bound {further.Distinct().Count()} material(s), {levels.GetSlotTable().Skip(1).First(l => l.Length == further.Length).Distinct().Count()} after"); + check("the table put back does", levels.SetSlotTable(after) && Same(levels.GetSlotTable(), after) + && levels.SetSlotTable(before) && Same(levels.GetSlotTable(), before), ""); + check("a table of another shape is refused and changes nothing", !levels.SetSlotTable([.. before.Skip(1)]) && Same(levels.GetSlotTable(), before), ""); + check("the shipped table is back", levels.SetSlotTable(shipped) && Same(levels.GetSlotTable(), shipped), ""); + return; + } + var shapes = fr.FrameObjects.Values.OfType().Select(m => doc.Node(m) is IMaterialSlotEditor e ? e.GetSlotTable() : []) + .GroupBy(t => string.Join("/", t.Select(l => l.Length))).OrderByDescending(g => g.Count()).Take(8).Select(g => $"{g.Key} x{g.Count()}"); + sb.AppendLine("(no mesh with a further level of two slots - the table check is skipped; slot counts per level: " + string.Join(", ", shapes) + ")"); } // The FULL assign path exactly as the user drives it: editor opened from a mesh tile (ShowMaterial diff --git a/src/Illusion/Diagnostics/Probes/McpProbes.cs b/src/Illusion/Diagnostics/Probes/McpProbes.cs index ab2e5dd..3f970cd 100644 --- a/src/Illusion/Diagnostics/Probes/McpProbes.cs +++ b/src/Illusion/Diagnostics/Probes/McpProbes.cs @@ -69,6 +69,7 @@ internal static class McpProbes "archive_build", "car_export_m2o", "car_substitute", + "mesh_hide_triangles", }; /// Records one assertion. A delegate rather than an Action so the optional diff --git a/src/Illusion/Diagnostics/Probes/PersonRigProbes.cs b/src/Illusion/Diagnostics/Probes/PersonRigProbes.cs new file mode 100644 index 0000000..e503b7c --- /dev/null +++ b/src/Illusion/Diagnostics/Probes/PersonRigProbes.cs @@ -0,0 +1,163 @@ +using System.IO; +using System.Numerics; +using System.Text; +using Illusion.Assets; +using Illusion.Assets.Sds; +using Illusion.Formats.Frames; +using Illusion.Formats.Frames.ObjectTypes; +using Illusion.Formats.Frames.Resources; +using Illusion.Formats.Hashing; +using static Illusion.Diagnostics.Probes.ProbeAssert; + +namespace Illusion.Diagnostics.Probes; + +/// +/// Where a person's mesh and rig really stand. A car is authored in the pose its bones rest in, so the +/// geometry as stored and the geometry as drawn are the same thing; a person need not be — and anything that +/// hands the stored geometry to another program (the Blender bridge) then shows it lying down beside a rig +/// that stands. This prints both, bone by bone. +/// Args: an archive under pc\sds (default traffic/cirand.sds). Output: %TEMP%\illusion_person_rig.txt +/// +internal static class PersonRigProbes +{ + internal static void RunPersonRigProbe(string archive) + { + string outFile = Path.Combine(Path.GetTempPath(), "illusion_person_rig.txt"); + var sb = new StringBuilder(); + try + { + if (!InitEnv(out string? err)) { sb.AppendLine("INIT FAIL: " + err); return; } + var sds = new FileInfo(Path.Combine(MafiaEnvironment.PcFolder, "sds", archive.Replace('/', Path.DirectorySeparatorChar))); + string extracted = SdsMeshLoader.EnsureExtracted(sds); + FrameResource? fr = SdsMeshLoader.OpenScene(extracted).FrameResource; + if (fr?.FrameObjects == null) { sb.AppendLine("no frame resource"); return; } + + foreach (FrameObjectModel model in fr.FrameObjects.Values.OfType()) + { + sb.AppendLine($"MODEL {model.Name}"); + sb.AppendLine(" local " + Describe(model.LocalTransform)); + sb.AppendLine(" world " + Describe(model.WorldTransform)); + + FrameSkeleton skeleton = model.GetSkeletonObject(); + HashName[] names = skeleton.BoneNames ?? []; + byte[] parents = model.GetSkeletonHierarchyObject().ParentIndices ?? []; + Matrix4x4[] rest = [.. (model.RestTransform ?? []).Select(Affine)]; + Matrix4x4[] worlds = [.. (skeleton.WorldTransforms ?? []).Select(Affine)]; + Matrix4x4[] joints = [.. (skeleton.JointTransforms ?? []).Select(Affine)]; + sb.AppendLine($" bones {names.Length}, rest {rest.Length}, skeleton.World {worlds.Length}, skeleton.Joint {joints.Length}"); + + // Accumulated down the hierarchy — the reading in which a rest transform is local to its parent. + var chained = new Matrix4x4[rest.Length]; + for (int i = 0; i < rest.Length; i++) + { + int p = i < parents.Length ? parents[i] : i; + chained[i] = p == i || p >= i ? rest[i] : rest[i] * chained[p]; + } + + // A car's wheels stand on these: where the axles and the brake drums are says how big a wheel is. + for (int i = 0; i < rest.Length && i < names.Length; i++) + { + string bone = names[i].ToString() ?? ""; + if (bone.StartsWith("axle", StringComparison.OrdinalIgnoreCase) + || bone.StartsWith("brake", StringComparison.OrdinalIgnoreCase) + || bone.Contains("wheel", StringComparison.OrdinalIgnoreCase)) + { + sb.AppendLine($" wheel bone {bone,-16} {V(rest[i].Translation)}"); + } + } + + int show = Math.Min(rest.Length, 14); + sb.AppendLine("\n bone parent rest (as stored) chained down the hierarchy"); + for (int i = 0; i < show; i++) + { + int p = i < parents.Length ? parents[i] : -1; + sb.AppendLine($" {i,2} {names[i],-18} {p,3} {V(rest[i].Translation)} {V(chained[i].Translation)}"); + } + + // The palette each reading gives: bind (the skeleton's own table) times the pose. + sb.AppendLine("\n palette = skeleton.World[i] * pose[i] (identity = stored geometry is drawn as stored)"); + for (int i = 0; i < show && i < worlds.Length; i++) + { + sb.AppendLine($" {i,2} {names[i],-18} rest: {Describe(worlds[i] * rest[i])}"); + sb.AppendLine($" {"",-18} chain: {Describe(worlds[i] * chained[i])}"); + } + + // inverse(skeleton.World[i]) is the pose the geometry was skinned in: where does it put bones? + sb.AppendLine("\n bind pose = inverse(skeleton.World[i])"); + for (int i = 0; i < show && i < worlds.Length; i++) + { + sb.AppendLine(Matrix4x4.Invert(worlds[i], out Matrix4x4 bind) + ? $" {i,2} {names[i],-18} {Describe(bind)}" + : $" {i,2} {names[i],-18} does not invert"); + } + + DecodedMesh? decoded = SdsMeshLoader.DecodeLod(model, 0); + if (decoded == null) { sb.AppendLine(" no LOD0"); continue; } + Bounds(decoded.Positions, out Vector3 min, out Vector3 max); + sb.AppendLine($"\n stored geometry {V(min)} .. {V(max)}"); + + byte[]? ids = SdsMeshLoader.ResolveBoneRemap( + model, SdsMeshLoader.BuildParts(model, decoded.Indices.Length, decoded.Lod), decoded); + if (ids == null || decoded.BoneWeights is not { } weights) { sb.AppendLine(" skin does not resolve"); continue; } + + foreach ((string label, Matrix4x4[] pose) in new[] { ("rest", rest), ("chain", chained) }) + { + var skinned = new Vector3[decoded.Positions.Length]; + for (int v = 0; v < skinned.Length; v++) + { + Vector3 sum = Vector3.Zero; + float total = 0; + for (int k = 0; k < 4; k++) + { + float w = weights[(v * 4) + k]; + int b = ids[(v * 4) + k]; + if (w <= 0 || b >= pose.Length || b >= worlds.Length) continue; + sum += Vector3.Transform(decoded.Positions[v], worlds[b] * pose[b]) * w; + total += w; + } + skinned[v] = total > 0 ? sum / total : decoded.Positions[v]; + } + Bounds(skinned, out Vector3 smin, out Vector3 smax); + sb.AppendLine($" skinned, pose={label,-6} {V(smin)} .. {V(smax)}"); + Bounds([.. skinned.Select(p => Vector3.Transform(p, Affine(model.WorldTransform)))], out smin, out smax); + sb.AppendLine($" ... in the world {V(smin)} .. {V(smax)}"); + } + sb.AppendLine(); + } + } + catch (Exception ex) + { + sb.AppendLine("unexpected exception — " + ex); + } + finally + { + File.WriteAllText(outFile, sb.ToString()); + } + } + + private static void Bounds(Vector3[] points, out Vector3 min, out Vector3 max) + { + min = new Vector3(float.MaxValue); + max = new Vector3(float.MinValue); + foreach (Vector3 p in points) + { + min = Vector3.Min(min, p); + max = Vector3.Max(max, p); + } + } + + private static string V(Vector3 v) => FormattableString.Invariant($"({v.X,7:F3} {v.Y,7:F3} {v.Z,7:F3})"); + + // Translation plus where the three axes point — enough to read a quarter turn off the page. + private static string Describe(Matrix4x4 m) => FormattableString.Invariant( + $"t {V(m.Translation)} x→({m.M11,5:F2} {m.M12,5:F2} {m.M13,5:F2}) y→({m.M21,5:F2} {m.M22,5:F2} {m.M23,5:F2}) z→({m.M31,5:F2} {m.M32,5:F2} {m.M33,5:F2})"); + + private static Matrix4x4 Affine(Matrix4x4 m) + { + m.M14 = 0; + m.M24 = 0; + m.M34 = 0; + m.M44 = 1; + return m; + } +} diff --git a/src/Illusion/Diagnostics/Probes/RemapPoolProbes.cs b/src/Illusion/Diagnostics/Probes/RemapPoolProbes.cs new file mode 100644 index 0000000..3a97ae6 --- /dev/null +++ b/src/Illusion/Diagnostics/Probes/RemapPoolProbes.cs @@ -0,0 +1,960 @@ +using System.IO; +using System.Numerics; +using System.Text; +using Illusion.Assets; +using Illusion.Assets.Adapters; +using Illusion.Assets.Bridge; +using Illusion.Assets.Frames; +using Illusion.Assets.Sds; +using Illusion.Bridge.Payload; +using Illusion.Domain; +using Illusion.Formats.Frames; +using Illusion.Formats.Frames.ObjectTypes; +using Illusion.Formats.Frames.Resources; +using Illusion.Formats.Hashing; +using static Illusion.Diagnostics.Probes.ProbeAssert; + +namespace Illusion.Diagnostics.Probes; + +/// +/// What a remap pool IS, measured rather than assumed — the census a pool REBUILD has to stand on. +/// +/// A skinned vertex names its bones through the pool of the face group that draws it. The pools live in the +/// blend info; the skeleton carries a second description of the same thing (a count per level, a usage array +/// per level, a per-bone reference into it, a per-bone level mask), and a rebuild that rewrites one half and +/// not the other gives a model the editor draws right and the game does not. Each hypothesis below is scored +/// over every skinned model in the install, so what the rebuild writes is what the shipped data does. +/// +/// Args: a focus archive name (default shubert_38). Output: %TEMP%\illusion_remap_pools.txt +/// +internal static class RemapPoolProbes +{ + internal static void RunRemapPoolProbe(string focus) + { + string outFile = Path.Combine(Path.GetTempPath(), "illusion_remap_pools.txt"); + var sb = new StringBuilder(); + var dump = new StringBuilder(); + try + { + if (!InitEnv(out string? err)) { sb.AppendLine("INIT FAIL: " + err); return; } + + var score = new Dictionary(); + var notes = new Dictionary>(); + void Score(string name, bool ok, string where) + { + (int Ok, int Of) s = score.GetValueOrDefault(name); + score[name] = (s.Ok + (ok ? 1 : 0), s.Of + 1); + if (ok) return; + if (!notes.TryGetValue(name, out List? list)) notes[name] = list = []; + if (list.Count < 6) list.Add(where); + } + var histogram = new Dictionary>(); + void Tally(string name, string value) + { + if (!histogram.TryGetValue(name, out Dictionary? h)) histogram[name] = h = []; + h[value] = h.GetValueOrDefault(value) + 1; + } + + int models = 0; + int longestTable = 0, widestPool = 0; + string longestTableAt = "", widestPoolAt = ""; + foreach (string kind in new[] { "cars", "traffic", "hchar", "player", "police_char" }) + { + string folder = Path.Combine(MafiaEnvironment.PcFolder, "sds", kind); + if (!Directory.Exists(folder)) continue; + foreach (FileInfo sds in new DirectoryInfo(folder).GetFiles("*.sds").OrderBy(f => f.Name)) + { + string extracted = MafiaEnvironment.ExtractedDir(sds); + if (!File.Exists(Path.Combine(extracted, "SDSContent.xml"))) continue; + FrameResource? fr; + try { fr = SdsMeshLoader.OpenScene(extracted).FrameResource; } + catch (Exception) { continue; } + if (fr?.FrameObjects == null) continue; + + // The whole file through the writer, before and after every model's skeleton is re-derived + // from its pools: an untouched model must come out byte for byte the file it was. + try + { + byte[] before = fr.WriteToStream(); + int synced = 0; + foreach (FrameObjectModel each in fr.FrameObjects.Values.OfType()) + if (BlendPoolTables.Sync(each)) synced++; + if (synced > 0) + { + Score("REBUILD: re-deriving the skeleton's tables leaves the frame file byte-identical", + before.AsSpan().SequenceEqual(fr.WriteToStream()), $"{kind}/{sds.Name}"); + } + } + catch (Exception ex) + { + Score("REBUILD: re-deriving the skeleton's tables leaves the frame file byte-identical", + false, $"{kind}/{sds.Name}: {ex.GetType().Name}"); + } + + foreach (FrameObjectModel model in fr.FrameObjects.Values.OfType()) + { + FrameSkeleton skeleton; + FrameBlendInfo blend; + try { skeleton = model.GetSkeletonObject(); blend = model.GetBlendInfoObject(); } + catch (Exception) { continue; } + HashName[] names = skeleton.BoneNames ?? []; + FrameBlendInfo.BoneIndexInfo[] lods = blend.BoneIndexInfos ?? []; + if (names.Length == 0 || lods.Length == 0) continue; + models++; + string who = $"{kind}/{sds.Name}:{model.Name}"; + bool isFocus = Path.GetFileNameWithoutExtension(sds.Name).Equals(focus, StringComparison.OrdinalIgnoreCase); + string Bone(int id) => id < names.Length ? names[id].ToString() ?? $"#{id}" : $"#{id}"; + + int[] perLod = skeleton.LodRemapIDCount ?? []; + FrameSkeleton.MappingForBlendingInfo[] maps = skeleton.MappingForBlendingInfos ?? []; + byte[] lodUsage = skeleton.BoneLODUsage ?? []; + Tally(kind + ": levels (pools / skeleton counts / mappings / geometry)", + $"{lods.Length}/{perLod.Length}/{maps.Length}/{model.Geometry?.LOD?.Length ?? -1}"); + Tally(kind + ": NumBones shape", string.Join("/", (skeleton.NumBones ?? []).Select(n => + n == names.Length ? "bones" : n.ToString(System.Globalization.CultureInfo.InvariantCulture)))); + Tally(kind + ": IDType", skeleton.IDType.ToString(System.Globalization.CultureInfo.InvariantCulture)); + Score("BoneLODUsage is one byte per bone", lodUsage.Length == names.Length, who); + + int total0 = (lods[0].BonesPerRemapPool ?? []).Sum(s => (int)s); + int totalMax = lods.Max(l => (l.BonesPerRemapPool ?? []).Sum(s => (int)s)); + int totalSum = lods.Sum(l => (l.BonesPerRemapPool ?? []).Sum(s => (int)s)); + Score("NumBlendIDs == level 0's pool total", skeleton.NumBlendIDs == total0, who); + Score("NumBlendIDs == the largest level's pool total", skeleton.NumBlendIDs == totalMax, who); + Score("NumBlendIDs == the pool totals of all levels added up", skeleton.NumBlendIDs == totalSum, who); + Score("NumBlendIDs == the bone count", skeleton.NumBlendIDs == names.Length, who); + + if (isFocus) + { + dump.AppendLine($"\n════ {who}: {names.Length} bones, NumBlendIDs {skeleton.NumBlendIDs}, " + + $"NumBones {string.Join("/", skeleton.NumBones ?? [])}, IDType {skeleton.IDType}, " + + $"LodRemapIDCount {string.Join("/", perLod)} ════"); + dump.AppendLine(" BoneLODUsage: " + string.Join(" ", lodUsage.Select((u, i) => $"{Bone(i)}={u}"))); + } + + var inLevel = new HashSet[lods.Length]; + var weightedInLevel = new HashSet[lods.Length]; + for (int lod = 0; lod < lods.Length; lod++) + { + FrameBlendInfo.BoneIndexInfo info = lods[lod]; + byte[] sizes = info.BonesPerRemapPool ?? []; + byte[] remap = info.BoneRemapIDs ?? []; + FrameBlendInfo.SkinnedMaterialInfo[] groups = info.SkinnedMaterialInfo ?? []; + int total = sizes.Sum(s => (int)s); + string at = $"{who} lod {lod}"; + inLevel[lod] = [.. remap.Take(total).Select(b => (int)b)]; + weightedInLevel[lod] = []; + + Tally(kind + ": pools in use per level", sizes.Count(s => s > 0).ToString(System.Globalization.CultureInfo.InvariantCulture)); + Tally(kind + ": BonesPerRemapPool array length", sizes.Length.ToString(System.Globalization.CultureInfo.InvariantCulture)); + if (total > longestTable) { longestTable = total; longestTableAt = at; } + if (sizes.Length > 0 && sizes.Max() > widestPool) { widestPool = sizes.Max(); widestPoolAt = at; } + Score("the remap table is exactly as long as the pools add up to", remap.Length == total, at); + Score("skeleton.LodRemapIDCount[level] == the pool total", + lod < perLod.Length && perLod[lod] == total, at); + + var poolStart = new int[sizes.Length]; + for (int p = 0, run = 0; p < sizes.Length; p++) { poolStart[p] = run; run += sizes[p]; } + bool usedPoolsFirst = true; + for (int p = 1; p < sizes.Length; p++) if (sizes[p] > 0 && sizes[p - 1] == 0) usedPoolsFirst = false; + Score("pools in use come first, with no empty pool between them", usedPoolsFirst, at); + + for (int p = 0; p < sizes.Length; p++) + { + if (sizes[p] == 0 || poolStart[p] + sizes[p] > remap.Length) continue; + byte[] slice = remap[poolStart[p]..(poolStart[p] + sizes[p])]; + Score("a pool names each bone once", slice.Distinct().Count() == slice.Length, $"{at} pool {p}"); + Score("a pool lists its bones in ascending order", + slice.Zip(slice.Skip(1)).All(pair => pair.First < pair.Second), $"{at} pool {p}"); + } + var appearances = remap.Take(total).GroupBy(b => b).ToDictionary(g => g.Key, g => g.Count()); + Tally(kind + ": most pools one bone sits in", + (appearances.Count == 0 ? 0 : appearances.Values.Max()).ToString(System.Globalization.CultureInfo.InvariantCulture)); + + // ── the skeleton's own description of this level ── + if (lod < maps.Length) + { + byte[] usage = maps[lod].UsageArray ?? []; + byte[] refs = maps[lod].RefToUsageArray ?? []; + Score("UsageArray is as long as the pool total", usage.Length == total, at); + Score("RefToUsageArray is one byte per bone", refs.Length == names.Length, at); + Score("Bounds is one box per bone", (maps[lod].Bounds?.Length ?? -1) == names.Length, at); + Score("UsageArray == BoneRemapIDs, entry for entry", + usage.Length == total && usage.AsSpan().SequenceEqual(remap.AsSpan(0, Math.Min(total, remap.Length))), at); + + (byte[] usageWant, byte[] refsWant) = BlendPoolTables.Describe( + remap.AsSpan(0, Math.Min(total, remap.Length)), names.Length); + Score("REBUILD: the usage array comes out of the pools exactly as shipped", + usage.AsSpan().SequenceEqual(usageWant), at); + Score("REBUILD: the reference array comes out of the pools exactly as shipped", + refs.AsSpan().SequenceEqual(refsWant), at); + + // Other readings of UsageArray, scored entry by entry. + for (int i = 0; i < Math.Min(usage.Length, total); i++) + { + Score(" UsageArray[i] == remap[i] (per entry)", usage[i] == remap[i], $"{at} entry {i}: usage {usage[i]}, remap {remap[i]}"); + Score(" UsageArray[i] is a bone id (< bone count)", usage[i] < names.Length, $"{at} entry {i}: {usage[i]}"); + } + // RefToUsageArray: per bone, an index into the level's table? + for (int b = 0; b < Math.Min(refs.Length, names.Length); b++) + { + bool present = inLevel[lod].Contains(b); + int first = Array.IndexOf(remap, (byte)b, 0, Math.Min(total, remap.Length)); + int last = Array.LastIndexOf(remap, (byte)b, Math.Min(total, remap.Length) - 1); + int firstUsage = Array.IndexOf(usage, (byte)b); + if (present) + { + Score(" a pooled bone's ref == its FIRST place in the remap table", refs[b] == first, $"{at} {Bone(b)}: ref {refs[b]}, first {first}, last {last}"); + Score(" a pooled bone's ref == its LAST place in the remap table", refs[b] == last, $"{at} {Bone(b)}: ref {refs[b]}, first {first}, last {last}"); + Score(" a pooled bone's ref == its first place in UsageArray", refs[b] == firstUsage, $"{at} {Bone(b)}: ref {refs[b]}, in usage {firstUsage}"); + Score(" usage[ref[bone]] == bone, for a pooled bone", refs[b] < usage.Length && usage[refs[b]] == b, $"{at} {Bone(b)}: ref {refs[b]}"); + } + else + { + Tally(kind + ": ref of a bone the level's pools do not name", refs[b].ToString(System.Globalization.CultureInfo.InvariantCulture)); + } + } + if (isFocus) + { + dump.AppendLine($"\n lod {lod}: pools {string.Join("+", sizes.Where(s => s > 0))} = {total}"); + for (int p = 0; p < sizes.Length; p++) + { + if (sizes[p] == 0) continue; + dump.AppendLine($" pool {p}: " + string.Join(", ", + remap.Skip(poolStart[p]).Take(sizes[p]).Select(b => $"{b}:{Bone(b)}"))); + } + dump.AppendLine(" UsageArray: " + string.Join(" ", usage)); + dump.AppendLine(" BoneRemapIDs: " + string.Join(" ", remap)); + dump.AppendLine(" RefToUsageArray: " + string.Join(" ", refs.Select((r, b) => $"{b}>{r}"))); + } + } + + // ── what the geometry actually asks of the pools ── + DecodedMesh? decoded; + try { decoded = SdsMeshLoader.DecodeLod(model, lod); } + catch (Exception) { decoded = null; } + if (decoded?.BoneIndices is not { } ids || decoded.BoneWeights is not { } weights) continue; + MeshPart[] parts = SdsMeshLoader.BuildParts(model, decoded.Indices.Length, lod); + Score("one face group per material slot", groups.Length == parts.Length, $"{at}: {groups.Length} groups, {parts.Length} slots"); + if (groups.Length < parts.Length) continue; + + MaterialStruct[] mats = model.Material?.Materials is { } all && lod < all.Count ? all[lod] : []; + Score("no material is drawn by two slots of one level", + mats.Select(m => m.MaterialHash).Distinct().Count() == mats.Length, at); + + var usedOfPool = new Dictionary>(); + var poolOfVertex = new int[decoded.Positions.Length]; + Array.Fill(poolOfVertex, -1); + int sharedAcrossPools = 0; + for (int part = 0; part < parts.Length; part++) + { + int pool = groups[part].AssignedPoolIndex; + if (pool >= sizes.Length) continue; + if (!usedOfPool.TryGetValue(pool, out HashSet? used)) usedOfPool[pool] = used = []; + int most = 0; + var bonesOfPart = new HashSet(); + int end = Math.Min(parts[part].StartIndex + parts[part].IndexCount, decoded.Indices.Length); + for (int i = parts[part].StartIndex; i < end; i++) + { + int v = (int)decoded.Indices[i]; + if (v < 0 || (v * 4) + 3 >= ids.Length) continue; + if (poolOfVertex[v] >= 0 && poolOfVertex[v] != pool) sharedAcrossPools++; + poolOfVertex[v] = pool; + int here = 0; + for (int k = 0; k < 4; k++) + { + if (weights[(v * 4) + k] <= 0f) continue; + here++; + int slot = poolStart[pool] + ids[(v * 4) + k]; + if (slot < remap.Length) + { + used.Add(remap[slot]); + bonesOfPart.Add(remap[slot]); + weightedInLevel[lod].Add(remap[slot]); + } + Score(" a weighted id stays inside its pool", ids[(v * 4) + k] < sizes[pool], $"{at} slot {part}"); + } + most = Math.Max(most, here); + } + Score("a face group declares exactly as many weights as its heaviest vertex carries", + groups[part].NumWeightsPerVertex == most, $"{at} slot {part}: declares {groups[part].NumWeightsPerVertex}, carries {most}"); + Score("a face group declares at least as many weights as its heaviest vertex carries", + groups[part].NumWeightsPerVertex >= most, $"{at} slot {part}: declares {groups[part].NumWeightsPerVertex}, carries {most}"); + if (isFocus) + { + string material = part < mats.Length + ? MafiaMaterials.GetMaterialName(mats[part].MaterialHash) ?? $"0x{mats[part].MaterialHash:X16}" : "?"; + dump.AppendLine($" slot {part,2} {material,-28} pool {pool}, declares {groups[part].NumWeightsPerVertex}, " + + $"{parts[part].IndexCount / 3} faces, {bonesOfPart.Count} bones: " + + string.Join(", ", bonesOfPart.Order().Select(Bone))); + } + } + Score("no vertex is drawn through two different pools", sharedAcrossPools == 0, $"{at}: {sharedAcrossPools} index hits"); + for (int p = 0; p < sizes.Length; p++) + { + if (sizes[p] == 0) continue; + var named = new HashSet(remap.Skip(poolStart[p]).Take(sizes[p]).Select(b => (int)b)); + HashSet used = usedOfPool.GetValueOrDefault(p) ?? []; + Score("a pool names ONLY bones its face groups put weight on", named.IsSubsetOf(used), + $"{at} pool {p}: names {named.Count}, weighted {used.Count}, idle: " + + string.Join(", ", named.Except(used).Order().Take(8).Select(Bone))); + Score("every pool in use is drawn by some face group", usedOfPool.ContainsKey(p), $"{at} pool {p}"); + } + } + + // ── BoneLODUsage against what the levels name ── + for (int b = 0; b < Math.Min(lodUsage.Length, names.Length); b++) + { + int maskNamed = 0, maskWeighted = 0; + for (int lod = 0; lod < lods.Length; lod++) + { + if (inLevel[lod].Contains(b)) maskNamed |= 1 << lod; + if (weightedInLevel[lod]?.Contains(b) == true) maskWeighted |= 1 << lod; + } + int levelsNamed = System.Numerics.BitOperations.PopCount((uint)maskNamed); + Score(" BoneLODUsage == bit mask of the levels whose pools name the bone", lodUsage[b] == maskNamed, $"{who} {Bone(b)}: {lodUsage[b]} vs mask {maskNamed}"); + Score(" BoneLODUsage == how many levels' pools name the bone", lodUsage[b] == levelsNamed, $"{who} {Bone(b)}: {lodUsage[b]} vs {levelsNamed}"); + Score(" BoneLODUsage == bit mask of the levels that put weight on the bone", lodUsage[b] == maskWeighted, $"{who} {Bone(b)}: {lodUsage[b]} vs mask {maskWeighted}"); + Tally(kind + ": BoneLODUsage value / levels naming the bone", $"{lodUsage[b]} / mask {maskNamed}"); + } + + Score("REBUILD: the level masks come out of the pools exactly as shipped", + lodUsage.AsSpan().SequenceEqual(BlendPoolTables.LevelMasks(lods, names.Length)), who); + + // ── a split's BlendIndex against the level 0 table ── + byte[] remap0 = lods[0].BoneRemapIDs ?? []; + byte[] refs0 = maps.Length > 0 ? maps[0].RefToUsageArray ?? [] : []; + foreach (FrameObjectModel.WeightedByMeshSplit split in model.BlendMeshSplits ?? []) + { + int index = split.BlendIndex; + if (index >= remap0.Length) { Score(" a split's BlendIndex is inside level 0's remap table", false, who); continue; } + Score(" a split's BlendIndex is inside level 0's remap table", true, who); + int bone = remap0[index]; + Score(" a split's BlendIndex is its bone's FIRST place in the table", + Array.IndexOf(remap0, (byte)bone) == index, $"{who}: index {index}, bone {Bone(bone)}"); + Score(" a split's BlendIndex == RefToUsageArray[its bone]", + bone < refs0.Length && refs0[bone] == index, $"{who}: index {index}, bone {Bone(bone)}, ref {(bone < refs0.Length ? refs0[bone] : -1)}"); + } + var splitIndices = (model.BlendMeshSplits ?? []).Select(s => (int)s.BlendIndex).ToList(); + Score("one split per entry of level 0's remap table", + splitIndices.Count == total0 && splitIndices.Distinct().Count() == total0, $"{who}: {splitIndices.Count} splits, {total0} entries"); + Score("no two splits share a BlendIndex", splitIndices.Distinct().Count() == splitIndices.Count, who); + Score("splits are stored in ascending BlendIndex", splitIndices.Zip(splitIndices.Skip(1)).All(p => p.First < p.Second), who); + if (isFocus) + { + dump.AppendLine("\n splits (BlendIndex → bone, pieces): " + string.Join(", ", + (model.BlendMeshSplits ?? []).Select(s => $"{s.BlendIndex}→{(s.BlendIndex < remap0.Length ? Bone(remap0[s.BlendIndex]) : "?")}×{s.Data?.Length ?? 0}"))); + } + } + } + } + + sb.AppendLine($"REMAP POOL CENSUS: {models} skinned models"); + sb.AppendLine($" longest table: {longestTable} entries ({longestTableAt}); widest pool: {widestPool} ({widestPoolAt})\n"); + foreach ((string name, (int ok, int of)) in score.OrderBy(p => p.Key.TrimStart(), StringComparer.Ordinal)) + { + sb.AppendLine($"{(ok == of ? "ALWAYS" : ok == 0 ? "NEVER " : " ")} {ok,7} of {of,-7} {name}"); + if (ok != of && ok != 0 && notes.TryGetValue(name, out List? list)) + foreach (string n in list.Take(of - ok < 4 ? 6 : 3)) sb.AppendLine(" · " + n); + } + sb.AppendLine(); + foreach ((string name, Dictionary h) in histogram.OrderBy(p => p.Key, StringComparer.Ordinal)) + { + sb.AppendLine(name + ": " + string.Join(", ", h.OrderByDescending(p => p.Value).Take(14).Select(p => $"[{p.Key}] ×{p.Value}"))); + } + PushCases(sb, focus); + UndoAndLevels(sb, focus); + sb.Append(dump); + } + catch (Exception ex) + { + sb.AppendLine("unexpected exception — " + ex); + } + finally + { + File.WriteAllText(outFile, sb.ToString()); + } + } + + // ───────────────────────── the rebuild itself, on a car held in memory ───────────────────────── + + /// + /// Pushes that the shipped pools cannot answer for, each on a freshly loaded copy of the focus car: a + /// triangle added to one face group and weighted to bones that group's pool does not hold. Nothing is + /// written to the game — the model is edited in memory and put through the writer and back. + /// + private static void PushCases(StringBuilder sb, string focus) + { + sb.AppendLine("\n════ pushes the shipped pools cannot answer for ════"); + int pass = 0, fail = 0; + void Check(string name, bool ok, string detail = "") + { + if (ok) pass++; else fail++; + sb.AppendLine($" [{(ok ? "PASS" : "FAIL")}] {name}{(detail == "" ? "" : " — " + detail)}"); + } + + var car = new FileInfo(Path.Combine(MafiaEnvironment.PcFolder, "sds", "cars", focus + ".sds")); + if (!car.Exists) { sb.AppendLine(" no such car: " + focus); return; } + + // (label, how to pick the face group and the bones it is newly weighted to) + var cases = new (string Label, Func Pick)[] + { + ("the LAST pool takes one bone more", shape => + { + int last = shape.PoolCount - 1; + if (last < 1) return null; + int slot = shape.SlotOfPool(last); + byte? bone = shape.BoneOnlyIn(pool: 0, notIn: last); + return slot < 0 || bone == null ? null : (slot, new[] { bone.Value }, -1); + }), + ("an EARLIER pool takes one bone more, so every later pool moves along", shape => + { + if (shape.PoolCount < 2 || shape.Sizes[0] >= 60) return null; + int slot = shape.SlotOfPool(0); + byte? bone = shape.BoneOnlyIn(pool: 1, notIn: 0); + return slot < 0 || bone == null ? null : (slot, new[] { bone.Value }, -1); + }), + ("a face group needs more than its pool has room for, so it moves to the pool that does", shape => + { + if (shape.PoolCount < 2) return null; + int slot = shape.SlotOfPool(0); + int need = 60 - shape.Sizes[0] + 1; + byte[] bones = shape.BonesOnlyIn(pool: 1, notIn: 0, count: need); + return slot < 0 || bones.Length < need || need > 4 ? null : (slot, bones, -1); + }), + ("one new vertex drawn by two face groups that read different pools", shape => + { + if (shape.PoolCount < 2) return null; + int slot = shape.SlotOfPool(0), also = shape.SlotOfPool(1); + byte[] common = [.. shape.PoolSlice(0).Where(b => shape.PoolSlice(1).Contains(b)).Take(1)]; + return slot < 0 || also < 0 || common.Length == 0 ? null : (slot, common, also); + }), + ("a bone no pool of the level names at all", shape => + { + int slot = shape.SlotOfPool(shape.PoolCount - 1); + byte? bone = shape.BoneInNoPool(); + return slot < 0 || bone == null ? null : (slot, new[] { bone.Value }, -1); + }), + }; + + foreach ((string label, Func pick) in cases) + { + sb.AppendLine($"\n ── {label} ──"); + try + { + (List roots, _, ISceneDocument? document) = SdsMeshLoader.LoadHierarchy(car); + IFrameNode? node = null; + foreach (SdsFrameNode r in roots) node ??= BridgeSkinProbes.FindModelNode(r); + MeshObjectPayload? mesh = node == null || document == null + ? null : BridgeMeshExporter.TryExport(node, document, out _); + if (node == null || mesh == null) { Check("the car rides the bridge", false); continue; } + var model = (FrameObjectModel)((FrameNodeAdapter)node).Frame; + + Shape before = Shape.Of(model); + (int Slot, byte[] Bones, int Also)? picked = pick(before); + if (picked == null) { sb.AppendLine(" (this car's pools do not offer the case)"); continue; } + (int slot, byte[] bones, int also) = picked.Value; + string[] names = [.. (model.GetSkeletonObject().BoneNames ?? []).Select(n => n.ToString() ?? "?")]; + int poolWas = before.Groups[slot].AssignedPoolIndex; + sb.AppendLine($" slot {slot} (pool {poolWas}, {before.Sizes[poolWas]} bones) " + + "is given a triangle weighted to " + string.Join(" + ", bones.Select(b => names[b])) + + (also < 0 ? "" : $", and slot {also} (pool {before.Groups[also].AssignedPoolIndex}) a second one on the SAME three corners")); + + byte[] pristine = model.Resource.WriteToStream(); + byte[] lod1Before = [.. before.Lod1Remap]; + int[] splitBones = [.. (model.BlendMeshSplits ?? []).Select(s => (int)before.Remap[s.BlendIndex])]; + int wasVerts = mesh.Positions.Length; + MeshObjectPayload more = WithTriangle(mesh, slot, bones, also); + + BridgeMeshApplier.ApplyResult? applied = BridgeMeshApplier.TryApply(node, more, out string? why); + if (applied == null && why != null && why.Contains("different bones", StringComparison.Ordinal)) + { + // Not a failure of the rebuild: this group is at the ceiling already, and saying so is + // the right answer. + sb.AppendLine(" (refused, as it should be: " + why + ")"); + continue; + } + Check("the push is accepted", applied != null, why ?? ""); + if (applied == null) continue; + + // Working a push out no longer writes into the model — that is the caller's to do. + byte[] untouched = model.Resource.WriteToStream(); + Check("accepting a push leaves the model as it was until it is applied", + untouched.AsSpan().SequenceEqual(pristine)); + applied.ApplyNew(); + Shape after = Shape.Of(model); + int poolNow = after.Groups[slot].AssignedPoolIndex; + sb.AppendLine($" pools {string.Join("+", before.Sizes.Where(x => x > 0))} = {before.Remap.Length} → " + + $"{string.Join("+", after.Sizes.Where(x => x > 0))} = {after.Remap.Length}; " + + $"slot {slot} now draws from pool {poolNow}; " + + $"{wasVerts} welded vertices → {applied.NewMesh?.Positions.Length} in the buffer"); + + Check("every old entry kept its pool and its place in it", + Enumerable.Range(0, before.PoolCount).All(p => + after.PoolSlice(p).Take(before.Sizes[p]).SequenceEqual(before.PoolSlice(p)))); + Check("no pool passes 60 bones", after.Sizes.All(x => x <= 60), + string.Join("+", after.Sizes.Where(x => x > 0))); + Check("the new bones are in the pool the face group draws from", + bones.All(b => after.PoolSlice(poolNow).Contains(b))); + + FrameSkeleton skeleton = model.GetSkeletonObject(); + (byte[] usage, byte[] refs) = BlendPoolTables.Describe(after.Remap, names.Length); + Check("the skeleton's count for the level is the new table length", + skeleton.LodRemapIDCount[0] == after.Remap.Length && skeleton.NumBlendIDs == after.Remap.Length, + $"{skeleton.LodRemapIDCount[0]} / {skeleton.NumBlendIDs} vs {after.Remap.Length}"); + Check("its usage and reference arrays describe the new table", + skeleton.MappingForBlendingInfos[0].UsageArray.AsSpan().SequenceEqual(usage) + && skeleton.MappingForBlendingInfos[0].RefToUsageArray.AsSpan().SequenceEqual(refs)); + Check("its level masks follow", + skeleton.BoneLODUsage.AsSpan().SequenceEqual( + BlendPoolTables.LevelMasks(model.GetBlendInfoObject().BoneIndexInfos, names.Length))); + Check("the other level's pools were left alone", after.Lod1Remap.AsSpan().SequenceEqual(lod1Before)); + + // Splits may be DROPPED by the face-range rebuild (a piece left with no faces), never + // re-pointed: every split still there must name the bone it named before, in the same order. + int[] splitBonesAfter = [.. (model.BlendMeshSplits ?? []).Select(s => + s.BlendIndex < after.Remap.Length ? (int)after.Remap[s.BlendIndex] : -1)]; + Check("every split still names the bone it named before the table moved", + splitBonesAfter.All(b => b >= 0) && IsSubsequence(splitBonesAfter, splitBones), + $"{splitBonesAfter.Length} splits (was {splitBones.Length})"); + + (int Checked, int Wrong, int Groups) resolve = + BridgeSkinProbes.ResolvesThroughPools(model, applied, after.Sizes, after.Remap); + Check("every bone id in the new buffer resolves through its own group's pool", + resolve.Checked > 0 && resolve.Wrong == 0, $"{resolve.Checked - resolve.Wrong} of {resolve.Checked}"); + + int added = 0, right = 0; + if (applied.NewMesh?.BoneIndices is { } ids) + { + for (int v = 0; v < applied.NewMesh.Positions.Length; v++) + { + bool isNew = false; + for (int i = wasVerts; i < more.Positions.Length && !isNew; i++) + isNew = Vector3.Distance(more.Positions[i], applied.NewMesh.Positions[v]) < 1e-4f; + if (!isNew) continue; + added++; + if (bones.Contains(ids[v * 4])) right++; + } + } + Check("the added corners ride the bones they were weighted to", added > 0 && right == added, + $"{right} of {added}"); + if (also >= 0) + { + Check("the three shared corners came out once per pool, six vertices in all", added == 6, + $"{added} vertices"); + Check("…and the table did not have to grow for it", after.Remap.Length == before.Remap.Length, + $"{before.Remap.Length} → {after.Remap.Length}"); + } + + byte[]? readsBack = SdsMeshLoader.GlobalBoneIds(model); + Check("the skin reads back through the pools once the buffers are in place", readsBack != null, + readsBack == null ? SdsMeshLoader.DescribeBoneRemap(model) : ""); + int verts = applied.NewMesh?.Positions.Length ?? 0; + int faces = (applied.NewMesh?.Indices.Length ?? 0) / 3; + Check("the model survives the writer and comes back", + BridgeSkinProbes.SurvivesRoundTrip(model, verts, faces, out string? trip), trip ?? ""); + + // The file as the game would read it: written, read back, and its skeleton compared with + // what the pools it came back with imply. + var back = new FrameResource(); + using (var stream = new MemoryStream(model.Resource.WriteToStream())) back.ReadFromFile(stream); + FrameObjectModel? same = back.FrameObjects?.Values.OfType() + .FirstOrDefault(m => m.Name.ToString() == model.Name.ToString()); + bool agrees = false; + if (same != null) + { + Shape reread = Shape.Of(same); + FrameSkeleton sk = same.GetSkeletonObject(); + (byte[] u, byte[] r) = BlendPoolTables.Describe(reread.Remap, names.Length); + agrees = reread.Remap.AsSpan().SequenceEqual(after.Remap) + && sk.LodRemapIDCount[0] == reread.Remap.Length + && sk.MappingForBlendingInfos[0].UsageArray.AsSpan().SequenceEqual(u) + && sk.MappingForBlendingInfos[0].RefToUsageArray.AsSpan().SequenceEqual(r); + } + Check("…and the file that comes back still agrees with itself, pools against skeleton", agrees); + + // A SECOND pull and push on top: the grown model must be as editable as a stock one. + MeshObjectPayload? again = document == null ? null : BridgeMeshExporter.TryExport(node, document, out _); + Check("the grown model can be pulled into Blender again", + again != null && again.BoneIndices.Length > 0 && again.SkinWarning == null, again?.SkinWarning ?? ""); + if (again != null) + { + BridgeMeshApplier.ApplyResult? second = + BridgeMeshApplier.TryApply(node, WithTriangle(again, slot, bones, also), out string? secondWhy); + second?.ApplyNew(); + Shape third = Shape.Of(model); + Check("…and pushed again without the table growing a second time", + second != null && third.Remap.Length == after.Remap.Length, + secondWhy ?? $"{after.Remap.Length} → {third.Remap.Length}"); + + // …and both taken back, newest first: the model is the file it was before either. + second?.RestoreOriginal(); + applied.RestoreOriginal(); + Check("undoing both pushes gives back the model byte for byte", + model.Resource.WriteToStream().AsSpan().SequenceEqual(pristine)); + } + } + catch (Exception ex) + { + Check("no exception", false, ex.ToString()); + } + } + // ── the FAR level: its own table, its own count; level 0 and the splits must not notice ── + sb.AppendLine("\n ── the far level's pool takes a bone it never named ──"); + try + { + (List roots, _, ISceneDocument? document) = SdsMeshLoader.LoadHierarchy(car); + IFrameNode? node = null; + foreach (SdsFrameNode r in roots) node ??= BridgeSkinProbes.FindModelNode(r); + var model = node == null ? null : (FrameObjectModel)((FrameNodeAdapter)node).Frame; + FrameBlendInfo.BoneIndexInfo[] levels = model?.GetBlendInfoObject().BoneIndexInfos ?? []; + MeshObjectPayload? far = node == null || document == null || levels.Length < 2 + ? null : BridgeMeshExporter.TryExport(node, document, out _, lod: 1); + if (model == null || far == null) + { + sb.AppendLine(" (this car has no second level to push into)"); + } + else + { + byte[] table0 = [.. levels[0].BoneRemapIDs ?? []]; + byte[] table1 = [.. levels[1].BoneRemapIDs ?? []]; + ushort[] blendBefore = [.. (model.BlendMeshSplits ?? []).Select(s => s.BlendIndex)]; + int bonesInRig = model.GetSkeletonObject().BoneNames?.Length ?? 0; + int stranger = -1; + for (int b = 0; b < table0.Length && stranger < 0; b++) + if (!table1.Contains(table0[b])) stranger = table0[b]; + if (stranger < 0) + { + sb.AppendLine(" (the far level already names every bone the near one does)"); + } + else + { + BridgeMeshApplier.ApplyResult? applied = BridgeMeshApplier.TryApply( + node!, WithTriangle(far, far.FaceMaterials[0], [(byte)stranger]), out string? why, lod: 1); + byte[] pristine = model.Resource.WriteToStream(); + Check("the push into the far level is accepted", applied != null, why ?? ""); + if (applied != null) + { + applied.ApplyNew(); + levels = model.GetBlendInfoObject().BoneIndexInfos; + FrameSkeleton skeleton = model.GetSkeletonObject(); + byte[] grown = levels[1].BoneRemapIDs ?? []; + sb.AppendLine($" far table {table1.Length} → {grown.Length} entries; near table {table0.Length} → {(levels[0].BoneRemapIDs ?? []).Length}"); + Check("the far table grew by the one bone", grown.Length == table1.Length + 1 && grown.Contains((byte)stranger)); + Check("the near level's table was left alone", (levels[0].BoneRemapIDs ?? []).AsSpan().SequenceEqual(table0)); + Check("no split was re-pointed — they index the near table", + (model.BlendMeshSplits ?? []).Select(s => s.BlendIndex).SequenceEqual(blendBefore)); + Check("the skeleton counts each level by its own table, and the longest overall", + skeleton.LodRemapIDCount[1] == grown.Length && skeleton.LodRemapIDCount[0] == table0.Length + && skeleton.NumBlendIDs == Math.Max(grown.Length, table0.Length), + $"{string.Join("/", skeleton.LodRemapIDCount)}, blend ids {skeleton.NumBlendIDs}"); + (byte[] usage, byte[] refs) = BlendPoolTables.Describe(grown, bonesInRig); + Check("its far usage and reference arrays describe the grown table", + skeleton.MappingForBlendingInfos[1].UsageArray.AsSpan().SequenceEqual(usage) + && skeleton.MappingForBlendingInfos[1].RefToUsageArray.AsSpan().SequenceEqual(refs)); + Check("the bone is now marked as used by both levels", skeleton.BoneLODUsage[stranger] == 3, + skeleton.BoneLODUsage[stranger].ToString(System.Globalization.CultureInfo.InvariantCulture)); + Check("the far skin reads back through its pools", SdsMeshLoader.GlobalBoneIds(model, 1) != null); + Check("…and the near one still does", SdsMeshLoader.GlobalBoneIds(model, 0) != null); + var back = new FrameResource(); + using (var stream = new MemoryStream(model.Resource.WriteToStream())) back.ReadFromFile(stream); + FrameObjectModel? same = back.FrameObjects?.Values.OfType() + .FirstOrDefault(m => m.Name.ToString() == model.Name.ToString()); + Check("the file comes back through the writer with the grown far table", + same != null && (same.GetBlendInfoObject().BoneIndexInfos[1].BoneRemapIDs ?? []).AsSpan().SequenceEqual(grown) + && same.GetSkeletonObject().LodRemapIDCount[1] == grown.Length); + applied.RestoreOriginal(); + Check("undoing it gives back the model byte for byte", + model.Resource.WriteToStream().AsSpan().SequenceEqual(pristine)); + } + } + } + } + catch (Exception ex) + { + Check("no exception", false, ex.ToString()); + } + + // ── and the push that cannot be answered: refused, with the model exactly as it was ── + // A face group already naming many bones is weighted to bones of the other pool too. Its own pool is + // full, the other would pass 60 taking it in, and a third pool would carry the table past the longest + // one any shipped model has — so there is no arrangement left that the game is known to read. + sb.AppendLine("\n ── a push no arrangement of pools can answer is refused, and nothing is written ──"); + try + { + bool reached = false; + for (int slot = 0; slot < 16 && !reached; slot++) + { + (List roots, _, ISceneDocument? document) = SdsMeshLoader.LoadHierarchy(car); + IFrameNode? node = null; + foreach (SdsFrameNode r in roots) node ??= BridgeSkinProbes.FindModelNode(r); + MeshObjectPayload? mesh = node == null || document == null + ? null : BridgeMeshExporter.TryExport(node, document, out _); + if (node == null || mesh == null) break; + var model = (FrameObjectModel)((FrameNodeAdapter)node).Frame; + Shape shape = Shape.Of(model); + if (slot >= shape.Groups.Length) break; + if (shape.PoolCount < 2 || shape.Groups[slot].AssignedPoolIndex != 0) continue; + byte[] bones = shape.BonesOnlyIn(pool: 1, notIn: 0, count: 4); + if (bones.Length < 60 - shape.Sizes[0] + 1) break; + + byte[] before = model.Resource.WriteToStream(); + BridgeMeshApplier.ApplyResult? applied = + BridgeMeshApplier.TryApply(node, WithTriangle(mesh, slot, bones), out string? why); + if (applied != null) continue; // this group still fitted somewhere — try a busier one + reached = true; + sb.AppendLine($" slot {slot}: {why}"); + Check("the refusal names the limit it ran into", + why != null && (why.Contains("remap table", StringComparison.Ordinal) + || why.Contains("no remap pool", StringComparison.Ordinal) + || why.Contains("different bones", StringComparison.Ordinal))); + Check("the model is byte for byte what it was before the push", + before.AsSpan().SequenceEqual(model.Resource.WriteToStream())); + } + if (!reached) sb.AppendLine(" (every face group of this car still fits somewhere)"); + } + catch (Exception ex) + { + Check("no exception", false, ex.ToString()); + } + + sb.AppendLine($"\n PUSH CASES: {pass} passed, {fail} failed"); + } + + /// + /// Two things a self-review of the pool rebuild said go wrong, each run for real: what is left in the + /// model when a push whose pools grew is UNDONE, and what two levels of one frame pushed together do to + /// each other. + /// + private static void UndoAndLevels(StringBuilder sb, string focus) + { + sb.AppendLine("\n════ undo of a push that changed the pools; two levels in one push ════"); + int pass = 0, fail = 0; + void Check(string name, bool ok, string detail = "") + { + if (ok) pass++; else fail++; + sb.AppendLine($" [{(ok ? "PASS" : "FAIL")}] {name}{(detail == "" ? "" : " — " + detail)}"); + } + + var car = new FileInfo(Path.Combine(MafiaEnvironment.PcFolder, "sds", "cars", focus + ".sds")); + if (!car.Exists) return; + try + { + foreach (bool movesPool in new[] { false, true }) + { + (List roots, _, ISceneDocument? document) = SdsMeshLoader.LoadHierarchy(car); + IFrameNode? node = null; + foreach (SdsFrameNode r in roots) node ??= BridgeSkinProbes.FindModelNode(r); + MeshObjectPayload? mesh = node == null || document == null ? null : BridgeMeshExporter.TryExport(node, document, out _); + if (node == null || mesh == null) continue; + var model = (FrameObjectModel)((FrameNodeAdapter)node).Frame; + Shape shape = Shape.Of(model); + if (shape.PoolCount < 2) { sb.AppendLine(" (one pool only — nothing to move between)"); break; } + + int slot; + byte[] bones; + if (movesPool) + { + int need = 60 - shape.Sizes[0] + 1; + slot = shape.SlotOfPool(0); + bones = shape.BonesOnlyIn(pool: 1, notIn: 0, count: need); + if (slot < 0 || bones.Length < need || need > 4) { sb.AppendLine(" (no face group has to change pool on this car)"); continue; } + } + else + { + slot = shape.SlotOfPool(shape.PoolCount - 1); + byte? bone = shape.BoneOnlyIn(pool: 0, notIn: shape.PoolCount - 1); + if (slot < 0 || bone == null) continue; + bones = [bone.Value]; + } + + byte[]? before = SdsMeshLoader.GlobalBoneIds(model); + byte[] fileBefore = model.Resource.WriteToStream(); + BridgeMeshApplier.ApplyResult? applied = BridgeMeshApplier.TryApply(node, WithTriangle(mesh, slot, bones), out string? why); + if (applied == null && why != null && why.Contains("different bones", StringComparison.Ordinal)) + { + sb.AppendLine(" (this face group is at the ceiling already and is refused, as it should be)"); + continue; + } + if (applied == null || before == null) { Check("the push is accepted", false, why ?? "no skin"); continue; } + applied.ApplyNew(); + applied.RestoreOriginal(); + Check(movesPool + ? "UNDO after a face group moved to another pool: the model is byte for byte the file it was" + : "UNDO after a pool merely grew: the model is byte for byte the file it was", + model.Resource.WriteToStream().AsSpan().SequenceEqual(fileBefore)); + byte[]? after = SdsMeshLoader.GlobalBoneIds(model); + int wrong = after == null ? -1 : Enumerable.Range(0, Math.Min(before.Length, after.Length)).Count(i => before[i] != after[i]) + + Math.Abs(before.Length - after.Length); + Check(movesPool + ? "UNDO after a face group moved to another pool: every vertex still names the bone it named" + : "UNDO after a pool merely grew: every vertex still names the bone it named", + after != null && wrong == 0, + after == null ? "the skin no longer resolves: " + SdsMeshLoader.DescribeBoneRemap(model)[..Math.Min(200, SdsMeshLoader.DescribeBoneRemap(model).Length)] + : $"{wrong} of {before.Length} bone ids differ"); + } + + // Two levels, one push: each result is computed against the frame as it stood, then both are applied. + { + (List roots, _, ISceneDocument? document) = SdsMeshLoader.LoadHierarchy(car); + IFrameNode? node = null; + foreach (SdsFrameNode r in roots) node ??= BridgeSkinProbes.FindModelNode(r); + MeshObjectPayload? near = node == null || document == null ? null : BridgeMeshExporter.TryExport(node, document, out _, lod: 0); + MeshObjectPayload? far = node == null || document == null ? null : BridgeMeshExporter.TryExport(node, document, out _, lod: 1); + if (node == null || near == null || far == null || near.Id == far.Id) + { + sb.AppendLine(" (this car has no second level)"); + } + else + { + var model = (FrameObjectModel)((FrameNodeAdapter)node).Frame; + // One vertex of each level lifted five metres — outside the box both levels are packed against. + Vector3 liftedNear = near.Positions[0] + new Vector3(0, 0, 5f); + Vector3 liftedFar = far.Positions[0] + new Vector3(0, 0, 6f); + near.Positions[0] = liftedNear; + far.Positions[0] = liftedFar; + // As the session does it: the second level is worked out with the first one in place. + byte[] fileBefore = model.Resource.WriteToStream(); + BridgeMeshApplier.ApplyResult? r0 = BridgeMeshApplier.TryApply(node, near, out string? why0, lod: 0); + BridgeMeshApplier.ApplyResult? r1 = r0 == null ? null + : BridgeMeshApplier.TryApplyAfter([r0], node, far, out why0, lod: 1); + Check("both levels are accepted", r0 != null && r1 != null, why0 ?? ""); + if (r0 != null && r1 != null) + { + Check("working both out leaves the model untouched", + model.Resource.WriteToStream().AsSpan().SequenceEqual(fileBefore)); + r0.ApplyNew(); + r1.ApplyNew(); + bool Has(int lod, Vector3 want) => + SdsMeshLoader.DecodeLod(model, lod)?.Positions.Any(p => Vector3.Distance(p, want) < 0.05f) == true; + Check("TWO LEVELS in one push: the near level still holds its own edit", Has(0, liftedNear), + $"requantized near {r0.Requantized}, far {r1.Requantized}"); + Check("TWO LEVELS in one push: the far level holds its edit", Has(1, liftedFar)); + Check("…and both skins still read through their pools", + SdsMeshLoader.GlobalBoneIds(model, 0) != null && SdsMeshLoader.GlobalBoneIds(model, 1) != null); + r1.RestoreOriginal(); + r0.RestoreOriginal(); + Check("undone in reverse, the model is byte for byte the file it was", + model.Resource.WriteToStream().AsSpan().SequenceEqual(fileBefore)); + } + } + } + } + catch (Exception ex) + { + Check("no exception", false, ex.ToString()); + } + sb.AppendLine($"\n UNDO AND LEVELS: {pass} passed, {fail} failed"); + } + + private static bool IsSubsequence(int[] part, int[] whole) + { + int at = 0; + foreach (int item in whole) + { + if (at < part.Length && part[at] == item) at++; + } + return at == part.Length; + } + + /// Level 0's pools of a model, as the cases above ask about them. + private sealed class Shape + { + internal byte[] Sizes = []; + internal byte[] Remap = []; + internal byte[] Lod1Remap = []; + internal FrameBlendInfo.SkinnedMaterialInfo[] Groups = []; + internal int Bones; + + internal int PoolCount => Sizes.Count(s => s > 0); + + internal static Shape Of(FrameObjectModel model) + { + FrameBlendInfo.BoneIndexInfo[] lods = model.GetBlendInfoObject().BoneIndexInfos; + return new Shape + { + Sizes = lods[0].BonesPerRemapPool ?? [], + Remap = lods[0].BoneRemapIDs ?? [], + Lod1Remap = lods.Length > 1 ? lods[1].BoneRemapIDs ?? [] : [], + Groups = lods[0].SkinnedMaterialInfo ?? [], + Bones = model.GetSkeletonObject().BoneNames?.Length ?? 0, + }; + } + + internal byte[] PoolSlice(int pool) + { + int start = 0; + for (int p = 0; p < pool; p++) start += Sizes[p]; + return Remap[start..(start + Sizes[pool])]; + } + + internal int SlotOfPool(int pool) => Array.FindIndex(Groups, g => g.AssignedPoolIndex == pool); + + internal byte[] BonesOnlyIn(int pool, int notIn, int count) => + [.. PoolSlice(pool).Where(b => !PoolSlice(notIn).Contains(b)).Take(count)]; + + internal byte? BoneOnlyIn(int pool, int notIn) => + BonesOnlyIn(pool, notIn, 1) is { Length: > 0 } found ? found[0] : null; + + internal byte? BoneInNoPool() + { + for (int b = Bones - 1; b >= 0; b--) + { + if (!Remap.Contains((byte)b)) return (byte)b; + } + return null; + } + } + + /// The same mesh with one triangle more: three new corners beside the first face of + /// , drawn with that slot's material and weighted evenly to . + /// With , a second triangle on the SAME three corners is drawn with + /// that other slot's material — one welded vertex, one normal, one UV, two face groups. + private static MeshObjectPayload WithTriangle(MeshObjectPayload mesh, int slot, byte[] bones, int alsoSlot = -1) + { + int face = Array.IndexOf(mesh.FaceMaterials, (ushort)slot); + if (face < 0) face = 0; + int at = mesh.Positions.Length; + var offset = new Vector3(0.013f, 0.011f, 0.012f); + + byte[] ids = new byte[(at + 3) * 4]; + float[] weights = new float[(at + 3) * 4]; + Array.Copy(mesh.BoneIndices, ids, Math.Min(mesh.BoneIndices.Length, at * 4)); + Array.Copy(mesh.BoneWeights, weights, Math.Min(mesh.BoneWeights.Length, at * 4)); + int influences = Math.Min(bones.Length, 4); + for (int i = 0; i < 3; i++) + { + for (int k = 0; k < influences; k++) + { + ids[((at + i) * 4) + k] = bones[k]; + weights[((at + i) * 4) + k] = 1f / influences; + } + } + + int l = face * 3; + T[] Twice(T[] corners) => alsoSlot < 0 ? corners : [.. corners, .. corners]; + return new MeshObjectPayload + { + BoneIndices = ids, + BoneWeights = weights, + Id = mesh.Id, + Name = mesh.Name, + World = mesh.World, + Local = mesh.Local, + Positions = [.. mesh.Positions, + mesh.Positions[mesh.LoopVertexIndices[l]] + offset, + mesh.Positions[mesh.LoopVertexIndices[l + 1]] + offset, + mesh.Positions[mesh.LoopVertexIndices[l + 2]] + offset], + LoopVertexIndices = [.. mesh.LoopVertexIndices, .. Twice(new[] { (uint)at, (uint)(at + 1), (uint)(at + 2) })], + LoopNormals = [.. mesh.LoopNormals, .. Twice(new[] { mesh.LoopNormals[l], mesh.LoopNormals[l + 1], mesh.LoopNormals[l + 2] })], + LoopUvs = [.. mesh.LoopUvs, .. Twice(new[] { mesh.LoopUvs[l], mesh.LoopUvs[l + 1], mesh.LoopUvs[l + 2] })], + LoopOrigIndex = [.. mesh.LoopOrigIndex, .. Twice(new[] { -1, -1, -1 })], + FaceMaterials = alsoSlot < 0 + ? [.. mesh.FaceMaterials, (ushort)slot] + : [.. mesh.FaceMaterials, (ushort)slot, (ushort)alsoSlot], + Materials = mesh.Materials, + VertexDeclaration = mesh.VertexDeclaration, + DecompressionOffset = mesh.DecompressionOffset, + DecompressionFactor = mesh.DecompressionFactor, + }; + } +} diff --git a/src/Illusion/Diagnostics/Probes/WorldProbes.cs b/src/Illusion/Diagnostics/Probes/WorldProbes.cs index 057c8a4..2ea64df 100644 --- a/src/Illusion/Diagnostics/Probes/WorldProbes.cs +++ b/src/Illusion/Diagnostics/Probes/WorldProbes.cs @@ -224,4 +224,349 @@ internal static void RunStreamMapProbe() File.WriteAllText(outFile, sb.ToString()); } } + + // The load zones (Illusion.Assets.World.LoadZones) against the real city. With no arguments it checks itself: + // the scene of city_univers comes back out of the writer byte for byte; a point known to lie in a gap asks + // for nothing; moving one face closes the gap and nothing else moves; the changed scene survives a write to + // %TEMP% and a read back. Nothing is written to the working copy or to the game. + // Arguments, in any order, for looking around: "x y z" triples; "dump "; "map x0 y0 x1 + // y1 step z"; "move " (in memory, shows in the maps after it). + // Output: %TEMP%\illusion_zones.txt + internal static void RunZonesProbe(string[] args) + { + string outFile = Path.Combine(Path.GetTempPath(), "illusion_zones.txt"); + var sb = new StringBuilder(); + int pass = 0, fail = 0; + void Check(string label, bool ok, string detail = "") + { + if (ok) pass++; else fail++; + sb.AppendLine($"[{(ok ? "PASS" : "FAIL")}] {label}{(detail.Length > 0 ? " - " + detail : "")}"); + } + try + { + if (!InitEnv(out string? err)) { sb.AppendLine("INIT FAIL: " + err); return; } + MapCatalog map = MapCatalog.Build(MafiaEnvironment.CityFolder, f => SdsMeshLoader.EnsureExtracted(f)); + List districts = [.. map.Areas.Select(a => a.BaseName)]; + // "copy " as the first two arguments looks at another copy of city_univers (a DLC's). + FileInfo? which = null; + if (args.Length >= 2 && args[0] == "copy") + { + which = LoadZones.Copies().FirstOrDefault(c => string.Equals(LoadZones.CopyName(c), args[1], StringComparison.OrdinalIgnoreCase)); + if (which == null) { sb.AppendLine($"no copy named {args[1]}; there are: {string.Join(", ", LoadZones.Copies().Select(LoadZones.CopyName))}"); return; } + args = args[2..]; + } + LoadZones zones = LoadZones.Open(f => SdsMeshLoader.EnsureExtracted(f), districts, which); + sb.AppendLine($"copy '{LoadZones.CopyName(zones.Archive)}' of {string.Join(", ", LoadZones.Copies().Select(LoadZones.CopyName))}: " + + $"{zones.Volumes.Count} volumes, {zones.Volumes.Keys.Count(n => zones.DistrictsOf(n).Count > 0)} of them with districts"); + + void Plan(LoadZones of, string district, float x0, float y0, float x1, float y1, float step, float z) + { + sb.AppendLine($"MAP of '{district}' at z {z}: x {x0}..{x1} left to right, y {y1}..{y0} top to bottom, step {step} (# asked for, + other zones only, . no zone)"); + for (float y = y1; y >= y0 - 0.001f; y -= step) + { + var row = new StringBuilder($"{y,8:F0} "); + for (float x = x0; x <= x1 + 0.001f; x += step) + { + var holding = of.At(new Vector3(x, y, z)).ToList(); + row.Append(holding.Any(h => of.DistrictsOf(h.Name).Contains(district, StringComparer.OrdinalIgnoreCase)) ? '#' : holding.Count > 0 ? '+' : '.'); + } + sb.AppendLine(row.ToString()); + } + } + + if (args.Length == 0) + { + string file = Directory.GetFiles(SdsMeshLoader.EnsureExtracted(zones.Archive), "FrameResource_*").First(); + byte[] original = File.ReadAllBytes(file); + byte[] written = new FrameResource(file).WriteToStream(); + Check("the scene of city_univers comes back out of the writer byte for byte", + original.AsSpan().SequenceEqual(written), $"{original.Length} bytes in, {written.Length} out"); + + // The plateau of the Greenfield hill: the stock game never lets the player stand there. + var gap = new Vector3(-1520f, 1300f, 5f); + var slope = new Vector3(-1620f, 1300f, 0f); + const string South = "AREA0019_GREENFIELD"; + if (zones.DistrictsAt(gap).Count > 0) + { + // Closed already in this working copy: every zone that was stretched over the gap is put + // back where the game shipped it, in memory, so the checks below run against the gap all the same. + (string Zone, float North)[] shipped = + [ + (South, 1222.3011f), ("AREA341_GREENFIELD_KINGSTONE", 1195.4f), ("AREA519_GREENFIELDF_KINGSTONE", 1195.4f), + ]; + bool back = true; + foreach ((string zone, float north) in shipped) + { + if (zones.Volumes.ContainsKey(zone) && LoadZones.Contains(zones.Volumes[zone], gap, out _)) + back &= zones.MoveFace(zone, "+y", north, out _) == null; + } + Check("the gap is closed in this working copy; its zones go back to the shipped places in memory", + back && zones.DistrictsAt(gap).Count == 0); + } + { + Check("a point on the hill's plateau lies in no zone", !zones.At(gap).Any()); + Check("the slope beside it asks for greenfield", zones.DistrictsAt(slope).Contains("greenfield")); + FrameObjectArea south = zones.Volumes[South]; + (Vector3 min0, Vector3 max0) = LoadZones.WorldBox(south); + Vector4[] planes0 = [.. south.Planes]; + + Check("a face that does not exist is refused", zones.MoveFace(South, "+q", 0, out _) != null); + Check("a move that would turn the zone inside out is refused", zones.MoveFace(South, "+y", min0.Y - 10f, out _) != null); + Check("…and neither refusal changed the zone", south.Planes.SequenceEqual(planes0) && LoadZones.WorldBox(south) == (min0, max0)); + + string? refused = zones.MoveFace(South, "+y", 1381f, out LoadZoneFaceMove? move); + Check("the south zone's north face moves to y 1381", refused == null && move != null + && MathF.Abs(move.From - max0.Y) < 0.05f && MathF.Abs(move.BoxMax.Y - 1381f) < 0.01f, refused ?? $"from {move!.From:F2}"); + (Vector3 min1, Vector3 max1) = LoadZones.WorldBox(south); + Check("…and only that face: the other five stand where they stood", + MathF.Abs(min1.X - min0.X) < 1e-3f && MathF.Abs(max1.X - max0.X) < 1e-3f && MathF.Abs(min1.Y - min0.Y) < 1e-3f + && MathF.Abs(min1.Z - min0.Z) < 1e-3f && MathF.Abs(max1.Z - max0.Z) < 1e-3f + && south.Planes.Where((p, i) => i != move!.Plane).SequenceEqual(planes0.Where((p, i) => i != move!.Plane))); + Check("the plateau now asks for greenfield", zones.DistrictsAt(gap).Contains("greenfield")); + int holes = 0; + for (float x = -1570f; x <= -1470f; x += 10f) + { + for (float y = 1230f; y <= 1375f; y += 10f) + { + if (!zones.DistrictsAt(new Vector3(x, y, 5f)).Contains("greenfield")) holes++; + } + } + Check("…everywhere in what was the gap", holes == 0, $"{holes} sample(s) still ask for nothing"); + + string trial = Path.Combine(Path.GetTempPath(), "illusion_zones_trial.fr"); + File.WriteAllBytes(trial, zones.Frame.WriteToStream()); + var back = new FrameResource(trial); + int differ = back.FrameObjects.Values.OfType().Count(read => + zones.Volumes.TryGetValue(read.Name.ToString(), out FrameObjectArea? mine) + && !(read.Planes.SequenceEqual(mine.Planes) && read.Bounds.Min == mine.Bounds.Min && read.Bounds.Max == mine.Bounds.Max)); + Check("the changed scene reads back as it was written", differ == 0 && new FileInfo(trial).Length == original.Length, + $"{differ} zone(s) differ, {new FileInfo(trial).Length} bytes"); + File.Delete(trial); + + // What the box gizmo does: a zone moved whole goes by its place, a pulled face by its plane, + // and a side sliced off by a slanted plane offers no face to pull. + const string Slanted = "AREA341_GREENFIELD_KINGSTONE"; + FrameObjectArea cut = zones.Volumes[Slanted]; + IReadOnlySet faces = zones.SquareFaces(Slanted); + Check("a zone with a corner sliced off offers the faces square to an axis, and not the sliced side", + faces.Contains("+y") && faces.Contains("+z") && faces.Contains("-z") && faces.Count < 6 + && new[] { "+x", "-x", "+y", "-y", "+z", "-z" }.All(f => faces.Contains(f) == (zones.MoveFace(Slanted, f, FaceAt(cut, f), out _) == null)), + string.Join(" ", faces.Order())); + Check("the zone moved under it has all six", zones.SquareFaces(South).Count == 6, string.Join(" ", zones.SquareFaces(South).Order())); + + (Vector3 cutMin, Vector3 cutMax) = LoadZones.WorldBox(cut); + Vector4[] cutPlanes = [.. cut.Planes]; + Vector3 inside = (cutMin + cutMax) * 0.5f; + var by = new Vector3(450f, -7.25f, 3f); // further than the zone is wide: it leaves where it stood + bool held = LoadZones.Contains(cut, inside, out _) && !LoadZones.Contains(cut, inside + by, out _); + string? unmoved = zones.Move(Slanted, by); + (Vector3 movedMin, Vector3 movedMax) = LoadZones.WorldBox(cut); + Check("a zone moves whole: its box goes by the offset and its planes are not touched", + unmoved == null && Vector3.Distance(movedMin, cutMin + by) < 0.01f && Vector3.Distance(movedMax, cutMax + by) < 0.01f + && cut.Planes.SequenceEqual(cutPlanes), unmoved ?? $"{movedMin - cutMin} / {movedMax - cutMax}"); + Check("…and what it held, it holds where it went, and no longer where it was", + held && LoadZones.Contains(cut, inside + by, out _) && !LoadZones.Contains(cut, inside, out _)); + Check("an offset that is not a number is refused", zones.Move(Slanted, new Vector3(float.NaN, 0, 0)) != null); + File.WriteAllBytes(trial, zones.Frame.WriteToStream()); + FrameObjectArea? reread = new FrameResource(trial).FrameObjects.Values.OfType().FirstOrDefault(a => a.Name.ToString() == Slanted); + Check("the moved zone reads back where it was put", reread != null && Vector3.Distance(LoadZones.WorldBox(reread).Min, movedMin) < 0.01f + && Vector3.Distance(LoadZones.WorldBox(reread).Max, movedMax) < 0.01f); + File.Delete(trial); + Check("…and moves back", zones.Move(Slanted, -by) == null && Vector3.Distance(LoadZones.WorldBox(cut).Min, cutMin) < 0.01f); + + Check("no two volumes of the scene share a name", zones.AmbiguousNames.Count == 0, string.Join(", ", zones.AmbiguousNames.Take(8))); + + // An editor that holds city_univers (the map editor in Whole map mode) has a scene of its own + // and writes it whole on its next save. A zone written from the disk's copy is carried into + // that scene, or the save would put the zone back - and a zone the editor changed and has + // not saved is not written over at all. + { + LoadZones disk = LoadZones.Open(f => SdsMeshLoader.EnsureExtracted(f), districts, which); + var editor = new Assets.Adapters.SceneDocumentAdapter(new FrameResource(file), zones.Archive); + Check("a scene an editor holds starts in step with the disk", disk.InStepWith(editor, South) && disk.InStepWith(editor, Slanted)); + // A matrix read from disk ends in a column of zeros - the file keeps three columns - and + // one an editor composed ends in a one. A volume moved in the editor and SAVED is the + // disk's volume with that one difference, and has to count as in step. + { + FrameObjectArea theirs = editor.Frame.FrameObjects.Values.OfType().First(a => a.Name.ToString() == South); + Matrix4x4 composed = theirs.LocalTransform; + bool zeros = composed.M44 == 0f; + composed.M44 = 1f; + theirs.LocalTransform = composed; + Check("a volume whose matrix an editor composed (same place, a one in the corner) is in step", disk.InStepWith(editor, South), + zeros ? "the disk's ends in zeros" : "the disk's already ends in a one here"); + composed.M42 += 3f; + theirs.LocalTransform = composed; + Check("...and one the editor has moved is not", !disk.InStepWith(editor, South)); + composed.M42 -= 3f; + theirs.LocalTransform = composed; + } + (Vector3 lo0, Vector3 hi0) = LoadZones.WorldBox(disk.Volumes[South]); + bool faceMoved = disk.MoveFace(South, "+y", hi0.Y + 37.5f, out _) == null; + bool carried = disk.Move(Slanted, new Vector3(5f, -3f, 0f)) == null; + Check("a zone changed on disk is out of step with the editor's copy", faceMoved && carried + && !disk.InStepWith(editor, South) && !disk.InStepWith(editor, Slanted)); + Check("…and is carried into it", disk.MirrorInto(editor, South) && disk.MirrorInto(editor, Slanted) + && disk.InStepWith(editor, South) && disk.InStepWith(editor, Slanted)); + Check("…so that the editor's own save writes the scene the zone write wrote", + editor.Frame.WriteToStream().AsSpan().SequenceEqual(disk.Frame.WriteToStream())); + Check("a zone the scene does not have is nothing to disagree about", disk.InStepWith(editor, "no such zone") && !disk.MirrorInto(editor, "no such zone")); + } + + static float FaceAt(FrameObjectArea zone, string face) + { + (Vector3 lo, Vector3 hi) = LoadZones.WorldBox(zone); + Vector3 side = face[0] == '+' ? hi : lo; + return face[1] == 'x' ? side.X : face[1] == 'y' ? side.Y : side.Z; + } + + // A NEW zone: the districts table read and written back as it stands, a volume added to the scene + // and a line to the table, and all of it found again in what would be written. In memory only. + { + string tableFile = Path.Combine(SdsMeshLoader.EnsureExtracted(zones.Archive), "missions", "CITY", "cityareas.bin"); + byte[] shipped = File.ReadAllBytes(tableFile); + var table = Illusion.Formats.CityAreas.CityAreasTable.Parse(shipped); + Check("cityareas.bin comes back out of its writer byte for byte", shipped.AsSpan().SequenceEqual(table.ToBytes()), + $"{table.Entries.Count} entries, {table.Districts.Count} district names"); + Check("a line added to it reads back, its districts by name", + table.Add("AREA900_PROBE", "greenfield", "kingstone") == null + && Illusion.Formats.CityAreas.CityAreasTable.Parse(table.ToBytes()).Find("AREA900_PROBE") is { Target1: "greenfield", Target2: "kingstone", Flag: 1 }); + Check("a second line of that name is refused, and so is a name that is not plain text", + table.Add("AREA900_PROBE", "greenfield", null) != null && table.Add("ЗОНА", "greenfield", null) != null); + Check("a district the table has not named is added to its names", + table.Add("AREA901_PROBE", "a_new_district", null) == null + && Illusion.Formats.CityAreas.CityAreasTable.Parse(table.ToBytes()).Find("AREA901_PROBE") is { Target1: "a_new_district", Target2: null, Flag: 0 }); + + LoadZones adding = LoadZones.Open(f => SdsMeshLoader.EnsureExtracted(f), districts, which, forNewZones: true); + Check("the scene read with its name table comes back out of the writer byte for byte too", + original.AsSpan().SequenceEqual(adding.Frame.WriteToStream())); + Check("zones opened for moving refuse to be added to", zones.Create("AREA900_PROBE", South, new Vector3(0), new Vector3(10), "greenfield", null) != null); + var far = new Vector3(9000f, 9000f, 0f); // nothing of the city is out there + int volumes = adding.Volumes.Count; + string? unmade = adding.Create("AREA900_PROBE", South, far - new Vector3(50, 40, 10), far + new Vector3(50, 40, 30), "greenfield", "kingstone"); + Check("a new zone is added where no zone was", unmade == null && adding.Volumes.Count == volumes + 1 + && adding.At(far).Select(z => z.Name).SequenceEqual(["AREA900_PROBE"]) && adding.DistrictsAt(far).SequenceEqual(["greenfield", "kingstone"]), + unmade ?? ""); + Check("...with the box it was asked for", unmade == null && Vector3.Distance(LoadZones.WorldBox(adding.Volumes["AREA900_PROBE"]).Min, far - new Vector3(50, 40, 10)) < 0.01f + && Vector3.Distance(LoadZones.WorldBox(adding.Volumes["AREA900_PROBE"]).Max, far + new Vector3(50, 40, 30)) < 0.01f); + Check("...and a point just outside it is outside", !adding.At(far + new Vector3(51, 0, 0)).Any()); + Check("a second zone of that name is refused", adding.Create("AREA900_PROBE", South, far, far + new Vector3(10), "greenfield", null) != null); + string made = Path.Combine(Path.GetTempPath(), "illusion_zones_created.fr"); + File.WriteAllBytes(made, adding.Frame.WriteToStream()); + FrameObjectArea? found = new FrameResource(made).FrameObjects.Values.OfType().FirstOrDefault(a => a.Name.ToString() == "AREA900_PROBE"); + Check("the scene with the new volume reads back with it in place, six planes and all", found != null && found.Planes.Length == 6 + && Vector3.Distance(LoadZones.WorldBox(found).Min, far - new Vector3(50, 40, 10)) < 0.01f && LoadZones.Contains(found, far, out _), + $"{new FileInfo(made).Length - original.Length} bytes longer"); + File.Delete(made); + var names = new FrameNameTable(); + names.BuildDataFromResource(adding.Frame); + Check("the name table built from it lists the new volume", (names.FrameData ?? []).Any(d => d.FrameIndex >= 0 + && adding.Frame.FrameObjects.Values.ElementAt(d.FrameIndex) is FrameObjectArea a && a.Name.ToString() == "AREA900_PROBE")); + + // Taken out again - what undoing the creation is: the scene is the shipped one once more. + Check("a zone that is not there cannot be taken out", adding.Delete("AREA999_NOT_THERE") != null); + string? kept = adding.Delete("AREA900_PROBE"); + Check("the new zone is taken out again, and the scene is what it was, byte for byte", + kept == null && adding.Volumes.Count == volumes && !adding.At(far).Any() + && original.AsSpan().SequenceEqual(adding.Frame.WriteToStream()), kept ?? ""); + var again = Illusion.Formats.CityAreas.CityAreasTable.Parse(shipped); + Check("...and so is the districts table with its line added and removed", + again.Add("AREA900_PROBE", "greenfield", "kingstone") == null && again.Remove("AREA900_PROBE") && !again.Remove("AREA900_PROBE") + && shipped.AsSpan().SequenceEqual(again.ToBytes())); + Check("...and with a line that brought a district name of its own", + again.Add("AREA901_PROBE", "a_new_district", "another_one") == null && again.Remove("AREA901_PROBE") + && shipped.AsSpan().SequenceEqual(again.ToBytes())); + + Check("a zone loads its districts on arrival by its name: two words after the number, not one", + LoadZones.LoadsOnArrival("AREA341_GREENFIELD_KINGSTONE") && LoadZones.LoadsOnArrival("AREA903_A_B") + && !LoadZones.LoadsOnArrival("AREA0019_GREENFIELD") && !LoadZones.LoadsOnArrival("AREA902_FOOXBAR") + && zones.Volumes.Keys.Count(LoadZones.LoadsOnArrival) > 500, + $"{zones.Volumes.Keys.Count(LoadZones.LoadsOnArrival)} of {zones.Volumes.Count} volumes are named as seams"); + + Check("a zone the game ships with is told from one that was added", + LoadZones.IsShipped(South) && LoadZones.IsShipped("AREA0223-DIPTON-KINGSTONE") && !LoadZones.IsShipped("AREA900_PROBE") + && !LoadZones.IsShipped(null), + $"{zones.Volumes.Keys.Count(z => !LoadZones.IsShipped(z))} added zone(s) in this copy"); + + // The districts are said as the table says them, and what is no district is refused whole. + string? junk = adding.Create("AREA902_PROBE", South, far, far + new Vector3(10), "no_such_district", null); + string? twice = adding.Create("AREA902_PROBE", South, far, far + new Vector3(10), "kingstone", "kingston"); + Check("a district that is no archive of the city is refused, and so is one district named twice in two spellings", + junk != null && twice != null && adding.Volumes.Count == volumes && original.AsSpan().SequenceEqual(adding.Frame.WriteToStream()), + $"{junk} / {twice}"); + string? spelled = adding.Create("AREA902_PROBE", South, far, far + new Vector3(10), "GREENFIELD", "kingston"); + Check("a district is taken in either spelling and listed as the archive it is", + spelled == null && adding.DistrictsOf("AREA902_PROBE").SequenceEqual(["greenfield", "kingstone"]), spelled ?? ""); + Check("...and taken out again leaves the scene as it was", + adding.Delete("AREA902_PROBE") == null && original.AsSpan().SequenceEqual(adding.Frame.WriteToStream())); + } + + Plan(zones, "greenfield", -1760, 1120, -1380, 1600, 20, 5); + } + return; + } + + var numbers = new List(); + for (int i = 0; i < args.Length; i++) + { + if (args[i] == "map" && i + 7 < args.Length) + { + float[] v = [.. args.Skip(i + 2).Take(6).Select(a => float.Parse(a, System.Globalization.CultureInfo.InvariantCulture))]; + sb.AppendLine(); + Plan(zones, args[i + 1], v[0], v[1], v[2], v[3], v[4], v[5]); + i += 7; + } + else if (args[i] == "dump" && i + 1 < args.Length) + { + foreach ((string name, FrameObjectArea area) in zones.Volumes.Where(z => z.Key.Contains(args[i + 1], StringComparison.OrdinalIgnoreCase)) + .OrderBy(z => z.Key, StringComparer.Ordinal)) + { + Matrix4x4 w = area.WorldTransform; + sb.AppendLine(); + sb.AppendLine($"DUMP {name} -> {string.Join(", ", zones.DistrictsOf(name))} unk01={area.Unk01} onTable={area.IsOnFrameTable} flags={area.SecondaryFlags}"); + sb.AppendLine($" at ({w.M41:F2}, {w.M42:F2}, {w.M43:F2}) axes ({w.M11:F3}, {w.M12:F3}, {w.M13:F3}) ({w.M21:F3}, {w.M22:F3}, {w.M23:F3}) ({w.M31:F3}, {w.M32:F3}, {w.M33:F3})"); + sb.AppendLine($" local box ({area.Bounds.Min.X:F2}, {area.Bounds.Min.Y:F2}, {area.Bounds.Min.Z:F2}) .. ({area.Bounds.Max.X:F2}, {area.Bounds.Max.Y:F2}, {area.Bounds.Max.Z:F2})"); + foreach (Vector4 plane in area.Planes) sb.AppendLine($" plane n ({plane.X:F3}, {plane.Y:F3}, {plane.Z:F3}) d {plane.W:F2}"); + } + i += 1; + } + else if (args[i] == "move" && i + 3 < args.Length) + { + string? refused = zones.MoveFace(args[i + 1], args[i + 2], + float.Parse(args[i + 3], System.Globalization.CultureInfo.InvariantCulture), out LoadZoneFaceMove? move); + sb.AppendLine(); + sb.AppendLine(refused != null ? $"MOVE refused: {refused}" + : $"MOVE (in memory) {move!.Zone} {move.Face}: {move.From:F2} -> {move.To:F2}; box now x {move.BoxMin.X:F1}..{move.BoxMax.X:F1} y {move.BoxMin.Y:F1}..{move.BoxMax.Y:F1} z {move.BoxMin.Z:F1}..{move.BoxMax.Z:F1}"); + i += 3; + } + else if (float.TryParse(args[i], System.Globalization.NumberStyles.Float, System.Globalization.CultureInfo.InvariantCulture, out float n)) + { + numbers.Add(n); + } + } + for (int i = 0; i + 2 < numbers.Count; i += 3) + { + var at = new Vector3(numbers[i], numbers[i + 1], numbers[i + 2]); + sb.AppendLine(); + sb.AppendLine($"POINT {at.X:F1}, {at.Y:F1}, {at.Z:F1}"); + foreach (var z in zones.At(at, 60f)) + { + (Vector3 min, Vector3 max) = LoadZones.WorldBox(z.Zone); + sb.AppendLine($" {(z.Inside ? "IN " : "out")} {z.Name,-36} -> {string.Join(", ", zones.DistrictsOf(z.Name)),-28} " + + $"x {min.X:F0}..{max.X:F0} y {min.Y:F0}..{max.Y:F0} z {min.Z:F0}..{max.Z:F0}" + (z.Inside ? "" : $" (outside by {z.OutsideBy:F1} m)")); + } + IReadOnlyList asked = zones.DistrictsAt(at); + sb.AppendLine(" districts asked for here: " + (asked.Count > 0 ? string.Join(", ", asked) : "NONE")); + } + } + catch (Exception ex) + { + sb.AppendLine("EXCEPTION: " + ex); + fail++; + } + finally + { + File.WriteAllText(outFile, $"LOAD ZONES PROBE: {pass} passed, {fail} failed\n" + sb); + } + } } diff --git a/src/Illusion/Mcp/AppEditorSession.cs b/src/Illusion/Mcp/AppEditorSession.cs index 2e9b54c..acf785e 100644 --- a/src/Illusion/Mcp/AppEditorSession.cs +++ b/src/Illusion/Mcp/AppEditorSession.cs @@ -263,17 +263,18 @@ public IReadOnlyList Find( if (nameContains != null && !node.Name.Contains(nameContains, StringComparison.OrdinalIgnoreCase)) continue; if (kind != null && !string.Equals(node.Kind, kind, StringComparison.OrdinalIgnoreCase)) continue; - // An instanced mesh's bounds span every copy across the map — not where this row is. - bool bounded = node.Mesh is { Instanced: false }; + // What the row stands where: a drawn mesh, or — for a collision placement — its hull. An instanced + // mesh has neither: its bounds span every copy across the map, not where this row is. + Shape? shape = ShapeOf(node, lo is { } boxLo && hi is { } boxHi ? (boxLo, boxHi) : null); Vector3? position = node.Source is IFrameNode frame ? frame.WorldTransform.Translation : null; int? triangles = null; Vector3 insideMin = default, insideMax = default; if (lo is { } min && hi is { } max) { - if (bounded) + if (shape is { } solid) { - if (!Overlaps(node.Mesh!.BoundsMin, node.Mesh.BoundsMax, min, max)) continue; - triangles = TrianglesInBox(node.Mesh, min, max, out insideMin, out insideMax); + if (!Overlaps(solid.Min, solid.Max, min, max)) continue; + triangles = TrianglesInBox(solid, min, max, out insideMin, out insideMax); if (triangles == 0) continue; // its box reaches in; its geometry does not } else if (position is not { } p || !Overlaps(p, p, min, max)) @@ -285,14 +286,14 @@ public IReadOnlyList Find( found.Add(new SceneObjectInfo( node.Name, node.Kind, PathOf(node), position is { } at ? [at.X, at.Y, at.Z] : null, - bounded ? [node.Mesh!.BoundsMin.X, node.Mesh.BoundsMin.Y, node.Mesh.BoundsMin.Z] : null, - bounded ? [node.Mesh!.BoundsMax.X, node.Mesh.BoundsMax.Y, node.Mesh.BoundsMax.Z] : null, + shape is { } b0 ? [b0.Min.X, b0.Min.Y, b0.Min.Z] : null, + shape is { } b1 ? [b1.Max.X, b1.Max.Y, b1.Max.Z] : null, node.IsSelected, triangles, triangles > 0 ? [insideMin.X, insideMin.Y, insideMin.Z] : null, triangles > 0 ? [insideMax.X, insideMax.Y, insideMax.Z] : null, - node.Mesh?.PickPositions?.Length, - node.Mesh?.PickIndices?.Length / 3)); + shape?.Positions?.Length ?? node.Mesh?.PickPositions?.Length, + (shape?.Indices?.Length ?? node.Mesh?.PickIndices?.Length) / 3)); } if (kind == null || string.Equals(kind, CrashCopyKind, StringComparison.OrdinalIgnoreCase)) @@ -396,16 +397,87 @@ private static int TrianglesInBox( return count; } + /// The triangles a row occupies and their world-space bounds. Positions are null for a mesh that + /// keeps no CPU geometry — its bounds still stand. + private readonly record struct Shape(Vector3[]? Positions, uint[]? Indices, Matrix4x4 World, Vector3 Min, Vector3 Max); + + // Decoded hulls, by the cooked bytes they came from: a re-cooked hull is another array and is decoded again. + private static readonly System.Runtime.CompilerServices.ConditionalWeakTable Hulls = new(); + + // A decoded hull and the box of its vertices in its own space - what a placement of it is first tried against. + private sealed record Hull(Vector3[] Vertices, uint[] Triangles, Vector3 Min, Vector3 Max); + + /// A world box the caller is asking about: a hull whose own box, stood where the placement + /// stands, does not reach it is answered with that looser box and without its vertices being put through + /// the placement's matrix - a search of a district tries every placement of every collision file. + private static Shape? ShapeOf(SceneNode node, (Vector3 Min, Vector3 Max)? box = null) + { + if (node.Mesh is { Instanced: false } mesh) + { + return new Shape(mesh.PickPositions, mesh.PickIndices, mesh.World, mesh.BoundsMin, mesh.BoundsMax); + } + if (node.Source is not Assets.Adapters.CollisionInstanceAdapter placement) return null; + + // by the file's own index of its hulls: a scan of the list for every placement was hulls x placements + byte[]? cooked = placement.Document.MeshFor(placement.Instance.Hash)?.CookedMesh; + if (cooked == null) return null; + Hull hull = Hulls.GetValue(cooked, static bytes => + { + try + { + Formats.Collisions.CookedTriangleMesh decoded = Formats.Collisions.CookedTriangleMesh.Decode(bytes); + var lo = new Vector3(float.MaxValue); + var hi = new Vector3(float.MinValue); + foreach (Vector3 vertex in decoded.Vertices) + { + lo = Vector3.Min(lo, vertex); + hi = Vector3.Max(hi, vertex); + } + return new Hull(decoded.Vertices, Array.ConvertAll(decoded.Triangles, i => (uint)i), lo, hi); + } + catch (Formats.Collisions.CollisionDecodeException) + { + return new Hull([], [], default, default); + } + }); + if (hull.Vertices.Length == 0) return null; + + Matrix4x4 world = placement.WorldTransform; + var min = new Vector3(float.MaxValue); + var max = new Vector3(float.MinValue); + if (box is { } asked) + { + // the eight corners of the hull's own box: it holds every vertex, so a miss here is a miss + for (int i = 0; i < 8; i++) + { + Vector3 corner = Vector3.Transform(new Vector3((i & 1) == 0 ? hull.Min.X : hull.Max.X, + (i & 2) == 0 ? hull.Min.Y : hull.Max.Y, (i & 4) == 0 ? hull.Min.Z : hull.Max.Z), world); + min = Vector3.Min(min, corner); + max = Vector3.Max(max, corner); + } + if (!Overlaps(min, max, asked.Min, asked.Max)) return new Shape(hull.Vertices, hull.Triangles, world, min, max); + min = new Vector3(float.MaxValue); + max = new Vector3(float.MinValue); + } + foreach (Vector3 vertex in hull.Vertices) + { + Vector3 at = Vector3.Transform(vertex, world); + min = Vector3.Min(min, at); + max = Vector3.Max(max, at); + } + return new Shape(hull.Vertices, hull.Triangles, world, min, max); + } + /// How many of a mesh's triangles reach into a world-space box, and the extent of those /// triangles clipped to it. Null when the mesh keeps no CPU geometry to ask. private static int? TrianglesInBox( - Rendering.Gpu.GpuMesh mesh, Vector3 boxMin, Vector3 boxMax, out Vector3 insideMin, out Vector3 insideMax) + Shape shape, Vector3 boxMin, Vector3 boxMax, out Vector3 insideMin, out Vector3 insideMax) { insideMin = new Vector3(float.MaxValue); insideMax = new Vector3(float.MinValue); - if (mesh.PickPositions is not { } positions || mesh.PickIndices is not { } indices) return null; + if (shape.Positions is not { } positions || shape.Indices is not { } indices) return null; - Matrix4x4 world = mesh.World; + Matrix4x4 world = shape.World; int count = 0; for (int i = 0; i + 2 < indices.Length; i += 3) { @@ -1126,20 +1198,209 @@ private static IEnumerable CarArchives(string car) return null; } + public string? HideTriangles(string name, float[] boxMin, float[] boxMax, string? material, bool apply, int sample, string? shared, + out HiddenTrianglesInfo? result) + { + result = null; + if (TargetHost is not { } host) return TargetNotOpen; + GeometryEditController.SharedGeometry sharing; + if (string.IsNullOrWhiteSpace(shared)) sharing = GeometryEditController.SharedGeometry.Refuse; + else if (string.Equals(shared, "own", StringComparison.OrdinalIgnoreCase)) sharing = GeometryEditController.SharedGeometry.OwnCopy; + else if (string.Equals(shared, "all", StringComparison.OrdinalIgnoreCase)) sharing = GeometryEditController.SharedGeometry.All; + else return "shared is 'own' or 'all', or left out"; + if (boxMin is not { Length: 3 } || boxMax is not { Length: 3 } || boxMin.Concat(boxMax).Any(v => !float.IsFinite(v))) + return "boxMin and boxMax are [x, y, z], finite numbers"; + if (Resolve(host, name, out SceneNode? node) is { } unresolved) return unresolved; + if (apply && host.BridgeEditedCount > 0) return "a Blender session is open — blender_end first"; + if (PlanHiddenTriangles(node!, new Vector3(boxMin[0], boxMin[1], boxMin[2]), new Vector3(boxMax[0], boxMax[1], boxMax[2]), + material, out Assets.Sds.TriangleHider.Plan? found) is { } notAMesh) + { + return notAMesh; + } + Assets.Sds.TriangleHider.Plan plan = found!; + if (apply && plan.Changes.Count > 0 && host.GeometryEditing.HideTriangles(node!, plan.Changes, sharing) is { } refused) + { + return sharing == GeometryEditController.SharedGeometry.Refuse && host.GeometryEditing.GeometrySharersOf(node!) is { Count: > 0 } others + ? $"{refused} ({others.Count}: {others.Names}) — shared: 'own' cuts this mesh alone, 'all' cuts every one of them" + : refused; + } + + static float[] P(Vector3 v) => [v.X, v.Y, v.Z]; + int levels = plan.Triangles.Count == 0 ? 0 : plan.Triangles.Max(t => t.Lod) + 1; + result = new HiddenTrianglesInfo( + plan.Triangles.Count, + [.. Enumerable.Range(0, levels).Select(l => plan.Triangles.Count(t => t.Lod == l))], + apply && plan.Changes.Count > 0, + [.. plan.Triangles.Take(Math.Clamp(sample, 0, 500)).Select(t => new TriangleInfo(t.Lod, t.Material, P(t.A), P(t.B), P(t.C)))]); + return null; + } + + /// The triangles of a scene node's mesh whose corners all lie inside a world-space box, and the index + /// edits that hide them. One implementation for the tool and for Tools → Hide triangles. Null on success. + internal static string? PlanHiddenTriangles(SceneNode node, Vector3 min, Vector3 max, string? material, + out Assets.Sds.TriangleHider.Plan? plan) + { + plan = null; + if (node.Source is not Assets.Adapters.FrameNodeAdapter { Frame: Formats.Frames.ObjectTypes.FrameObjectSingleMesh mesh } adapter) + { + return $"'{node.Name}' is a {node.Kind} — only a mesh has triangles to hide"; + } + plan = Assets.Sds.TriangleHider.Find(mesh, ((IFrameNode)adapter).WorldTransform, min, max, + string.IsNullOrWhiteSpace(material) ? null : material); + return null; + } + + /// The same for triangles picked one by one on the mesh's first level of detail (Tools → Hide + /// triangles, by clicking): the plan that hides them and what lies on them on the other levels, and how many + /// levels the mesh has. Null on success. + internal static string? PlanPickedTriangles(SceneNode node, IReadOnlyCollection picked, out Assets.Sds.TriangleHider.Plan? plan, + out int levels) + { + plan = null; + levels = 0; + if (node.Source is not Assets.Adapters.FrameNodeAdapter { Frame: Formats.Frames.ObjectTypes.FrameObjectSingleMesh mesh } adapter) + { + return $"'{node.Name}' is a {node.Kind} — only a mesh has triangles to hide"; + } + levels = mesh.Geometry?.LOD?.Length ?? 0; + plan = Assets.Sds.TriangleHider.FindPicked(mesh, ((IFrameNode)adapter).WorldTransform, picked); + return null; + } + + public string? MeshMaterials(string name, int? slot, string? material, out IReadOnlyList slots) + { + slots = []; + if (TargetHost is not { } host) return TargetNotOpen; + if (Resolve(host, name, out SceneNode? node) is { } unresolved) return unresolved; + if (node!.Source is not Assets.Adapters.FrameNodeAdapter adapter || adapter.GetMaterials() is not { Count: > 0 } before) + { + return $"'{name}' is a {node.Kind} with no material slots"; + } + + int changed = -1; + if (slot != null || !string.IsNullOrWhiteSpace(material)) + { + if (slot is not { } index || string.IsNullOrWhiteSpace(material)) return "to re-point a slot give both: slot and material"; + if (index < 0 || index >= before.Count) return $"'{name}' has slots 0 to {before.Count - 1}"; + if (host.BridgeEditedCount > 0) return "a Blender session is open — blender_end first"; + if (Assets.MafiaMaterials.FindHashByName(material) is not { } hash) + { + return $"no material named '{material}' in the loaded libraries — search_materials has the names (the spelling is exact)"; + } + // The slot is a row of the mesh's material block, and a block can serve several objects: all of them + // would change in the file and in the game, while this answer - and the viewport - showed one. + if (host.SlotAssignObstacle(node) is { } shared) return shared; + if (!host.AssignSlotMaterial(node, index, hash)) return $"slot {index} of '{name}' could not be re-pointed"; + changed = index; + } + slots = [.. adapter.GetMaterials().Select((m, i) => new MeshSlotInfo(i, m.Name, m.TriangleCount, i == changed))]; + return null; + } + + public string? UnusedHulls(bool apply, out IReadOnlyList result) + { + result = []; + if (TargetHost is not { } host) return TargetNotOpen; + if (apply && host.BridgeEditedCount > 0) return "a Blender session is open — blender_end first"; + + // Counted before the sweep: afterwards a layer's hull list no longer says what it carried. + var layers = AllNodes(host) + .Where(n => n.Source is Assets.Adapters.CollisionDocumentAdapter) + .Select(n => (Node: n, File: ((Assets.Adapters.CollisionDocumentAdapter)n.Source!).Collision)) + .Select(l => (l.Node, Placements: l.File.Instances.Count, Hulls: l.File.Meshes.Count)) + .ToList(); + if (layers.Count == 0) return "the open scene has no collision file"; + + Dictionary unused = host.CollisionEditing.SweepUnusedHulls(layers.Select(l => l.Node), apply) + .ToDictionary(l => l.Layer, l => l.Unused); + result = [.. layers.Select(l => + { + int n = unused.GetValueOrDefault(l.Node); + return new UnusedHullsInfo(PathOf(l.Node), l.Placements, l.Hulls, n, apply && n > 0); + })]; + return null; + } + + public string? CrashPlacements(float[] boxMin, float[] boxMax, string? nameContains, bool delete, int limit, int maxDelete, + out IReadOnlyList result, out int total) + { + result = []; + total = 0; + if (TargetHost is not { } host) return TargetNotOpen; + if (boxMin is not { Length: 3 } || boxMax is not { Length: 3 }) return "boxMin and boxMax are [x, y, z]"; + // A box that holds nothing by construction answers "count 0", and that reads as "nothing stands here". + if (boxMin.Concat(boxMax).Any(v => !float.IsFinite(v))) return "boxMin and boxMax must be finite numbers"; + for (int axis = 0; axis < 3; axis++) + { + if (boxMin[axis] > boxMax[axis]) + { + return $"boxMin is above boxMax on {"xyz"[axis]} ({boxMin[axis]} > {boxMax[axis]}) — such a box holds " + + "nothing; give the lower corner first"; + } + } + if (host.Streamer.CrashLayer is not { } layer) return "the crash layer is not in the scene — view_set crash=true first"; + if (delete && host.BridgeEditedCount > 0) return "a Blender session is open — blender_end first"; + + var lo = new Vector3(boxMin[0], boxMin[1], boxMin[2]); + var hi = new Vector3(boxMax[0], boxMax[1], boxMax[2]); + var hits = new List<(Formats.Translokator.Object Row, Formats.Translokator.Instance Placement)>(); + foreach (Formats.Translokator.Object row in layer.Rows) + { + if (!string.IsNullOrWhiteSpace(nameContains) + && !row.Name.String.Contains(nameContains, StringComparison.OrdinalIgnoreCase)) continue; + foreach (Formats.Translokator.Instance placement in row.Instances) + { + Vector3 p = placement.Position; + if (p.X >= lo.X && p.X <= hi.X && p.Y >= lo.Y && p.Y <= hi.Y && p.Z >= lo.Z && p.Z <= hi.Z) + hits.Add((row, placement)); + } + } + + total = hits.Count; + // A wide box with no name is the whole city table — tens of thousands of placements and their twins as + // one edit, of which the list would show the first two hundred. Refused whole instead: the caller + // says how many it means to remove. + if (delete && hits.Count > Math.Max(0, maxDelete)) + { + return $"{hits.Count} placements are in the box — more than maxDelete ({maxDelete}); nothing was deleted. " + + "List them first (delete=false), narrow the box or the name, or raise maxDelete"; + } + // A delete lists everything it removed, whatever the listing limit: what went has to be readable. + result = [.. hits.Take(delete ? hits.Count : Math.Max(0, limit)).Select(h => new CrashPlacementInfo( + h.Row.Name.String, h.Placement.ID, [h.Placement.Position.X, h.Placement.Position.Y, h.Placement.Position.Z], + layer.Document.HasTwinOf(h.Placement, h.Row), layer.Document.Node(h.Placement, h.Row).SeasonLinked))]; + if (!delete || hits.Count == 0) return null; + + // The viewport's own delete, so the edit is the one the Delete key makes: undoable, the streaming grid + // kept in step, the twin in the other season gone with a linked placement. + List nodes = [.. hits.Select(h => host.Streamer.CrashNodeFor(h.Placement, h.Row)).OfType()]; + if (nodes.Count != hits.Count) return "some placements could not be given a tree node — nothing was deleted"; + // That delete works on the selection. What the user had selected is put back afterwards, less + // whatever of it was just deleted — a tool that lists and removes props has no business leaving the + // editor with nothing selected. + List selectedBefore = [.. host.Selection.Selected]; + SceneNode? activeBefore = host.Selection.Active; + host.Selection.SetSelection(nodes, nodes[^1]); + host.CrashEditing.DeleteSelected(); + var gone = new HashSet(nodes); + List kept = [.. selectedBefore.Where(n => !gone.Contains(n))]; + host.Selection.SetSelection(kept, + activeBefore != null && kept.Contains(activeBefore) ? activeBefore : kept.Count > 0 ? kept[^1] : null); + return null; + } + public string? Undo() { if (TargetHost is not { } host) return TargetNotOpen; if (!host.History.CanUndo) return "nothing to undo"; - host.Undo(); - return null; + return host.TryUndo(); } public string? Redo() { if (TargetHost is not { } host) return TargetNotOpen; if (!host.History.CanRedo) return "nothing to redo"; - host.Redo(); - return null; + return host.TryRedo(); } // ── Scene tree helpers ── @@ -1212,4 +1473,248 @@ private static IEnumerable Descendants(DependencyObject root) foreach (T deeper in Descendants(element)) yield return deeper; } } + + // The city's load zones, read afresh from the working copy of city_univers on every call: they are a few + // hundred small objects, and a tool that moves one must not answer from what stood there before. + private string? OpenLoadZones(string? copy, out Assets.World.LoadZones? zones) + { + zones = null; + if (EnsureEnvironment() is { } notReady) return notReady; + try + { + IReadOnlyList copies = Assets.World.LoadZones.Copies(); + FileInfo? which = string.IsNullOrWhiteSpace(copy) + ? copies[0] + : copies.FirstOrDefault(c => string.Equals(Assets.World.LoadZones.CopyName(c), copy, StringComparison.OrdinalIgnoreCase)); + if (which == null) + { + return $"no copy of city_univers.sds named '{copy}' - there are: " + string.Join(", ", copies.Select(Assets.World.LoadZones.CopyName)); + } + // The districts' names: the open map editor has them already; without one the city folder is read. + IReadOnlyCollection districts = Window?.Viewport.Catalogs.DistrictNames is { Count: > 0 } known + ? known + : [.. Assets.World.MapCatalog.Build(Assets.MafiaEnvironment.CityFolder, f => Assets.Sds.SdsMeshLoader.EnsureExtracted(f)) + .Areas.Select(a => a.BaseName)]; + zones = Assets.World.LoadZones.Open(f => Assets.Sds.SdsMeshLoader.EnsureExtracted(f), districts, which); + return null; + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + return "could not read the load zones of city_univers: " + ex.Message; + } + } + + private static float[] Xyz(Vector3 v) => [v.X, v.Y, v.Z]; + + public string? ZonesAt(float[] point, float near, string? copy, out IReadOnlyList zones, out IReadOnlyList districts, + out IReadOnlyList copies) + { + zones = []; + districts = []; + copies = []; + if (point is not { Length: 3 } || point.Any(v => !float.IsFinite(v))) return "point is [x, y, z], finite numbers"; + if (!float.IsFinite(near) || near < 0) return "near is a distance in metres, 0 or more"; + if (OpenLoadZones(copy, out Assets.World.LoadZones? all) is { } failed) return failed; + copies = [.. Assets.World.LoadZones.Copies().Select(Assets.World.LoadZones.CopyName)]; + + var at = new Vector3(point[0], point[1], point[2]); + zones = [.. all!.At(at, near).Select(z => + { + (Vector3 min, Vector3 max) = Assets.World.LoadZones.WorldBox(z.Zone); + return new LoadZoneInfo(z.Name, all.DistrictsOf(z.Name), z.Inside, z.OutsideBy, Xyz(min), Xyz(max), + Assets.World.LoadZones.LoadsOnArrival(z.Name), !Assets.World.LoadZones.IsShipped(z.Name)); + })]; + districts = all.DistrictsAt(at); + return null; + } + + public string? ZonesMap(string district, float[] from, float[] to, float step, float z, string? copy, out IReadOnlyList rows) + { + rows = []; + if (string.IsNullOrWhiteSpace(district)) return "name the district, e.g. greenfield"; + if (from is not { Length: 2 } || to is not { Length: 2 } || from.Concat(to).Any(v => !float.IsFinite(v))) + return "from and to are [x, y], finite numbers"; + // Below a tenth of a metre a step is no longer a step at the city's coordinates: a float there is + // spaced a quarter of a millimetre apart, and a column "one step on" came out as the same column. + if (!float.IsFinite(step) || step < 0.1f || !float.IsFinite(z)) return "step is a distance of 0.1 m or more and z a height"; + float x0 = MathF.Min(from[0], to[0]), x1 = MathF.Max(from[0], to[0]); + float y0 = MathF.Min(from[1], to[1]), y1 = MathF.Max(from[1], to[1]); + if ((x1 - x0) / step > 200 || (y1 - y0) / step > 200) return "that is more than 200 steps a side - take a larger step or a smaller box"; + if (OpenLoadZones(copy, out Assets.World.LoadZones? all) is { } failed) return failed; + + // Rows and columns are counted, and each stands at its own multiple of the step: adding the step to a + // running float drifts, and with a step too small to change the float it never arrives. + int columns = (int)MathF.Floor(((x1 - x0) / step) + 0.001f) + 1, lineCount = (int)MathF.Floor(((y1 - y0) / step) + 0.001f) + 1; + var lines = new List(lineCount); + for (int line = 0; line < lineCount; line++) + { + float y = y1 - (line * step); + var row = new System.Text.StringBuilder($"{y,8:F0} "); + for (int column = 0; column < columns; column++) + { + float x = x0 + (column * step); + var holding = all!.At(new Vector3(x, y, z)).ToList(); + row.Append(holding.Any(h => all.DistrictsOf(h.Name).Contains(district, StringComparer.OrdinalIgnoreCase)) ? '#' + : holding.Count > 0 ? '+' : '.'); + } + lines.Add(row.ToString()); + } + rows = lines; + return null; + } + + public string? ZoneCreate(string name, string like, float[] boxMin, float[] boxMax, string[] districts, bool apply, out LoadZoneInfo? zone) + { + zone = null; + if (EnsureEnvironment() is { } notReady) return notReady; + if (boxMin is not { Length: 3 } || boxMax is not { Length: 3 } || boxMin.Concat(boxMax).Any(v => !float.IsFinite(v))) + return "boxMin and boxMax are [x, y, z], finite numbers"; + if (districts is not { Length: 1 or 2 } || districts.Any(string.IsNullOrWhiteSpace)) return "districts is one or two district names"; + var archive = new FileInfo(Assets.MafiaEnvironment.CityUniversSds); + // asked of a dry run too: one that says "it can be made" while the write would be refused says nothing + if (ZoneWrites.StructureBlocked(archive) is { } blocked) return blocked; + try + { + IReadOnlyCollection known = Window?.Viewport.Catalogs.DistrictNames is { Count: > 0 } names + ? names + : [.. Assets.World.MapCatalog.Build(Assets.MafiaEnvironment.CityFolder, f => Assets.Sds.SdsMeshLoader.EnsureExtracted(f)) + .Areas.Select(a => a.BaseName)]; + Assets.World.LoadZones zones = Assets.World.LoadZones.Open(f => Assets.Sds.SdsMeshLoader.EnsureExtracted(f), known, archive, + forNewZones: true); + var min = new Vector3(boxMin[0], boxMin[1], boxMin[2]); + var max = new Vector3(boxMax[0], boxMax[1], boxMax[2]); + // worked out in memory first: what the zone would be is the answer of a dry run, and of a write too + if (zones.Create(name, like, min, max, districts[0], districts.Length > 1 ? districts[1] : null) is { } refused) return refused; + (Vector3 lo, Vector3 hi) = Assets.World.LoadZones.WorldBox(zones.Volumes[name]); + zone = new LoadZoneInfo(name, zones.DistrictsOf(name), true, 0f, Xyz(lo), Xyz(hi), Assets.World.LoadZones.LoadsOnArrival(name), Added: true); + if (!apply) return null; + if (Window?.Viewport is { } viewport) + { + // With the map editor open: written the way its own button writes, so the zone is a step of the + // editor's undo history - and the step is on the MAP editor's history (see ZoneMoveFace). + if (viewport.ZoneEditing.Create(name, like, min, max, districts[0], districts.Length > 1 ? districts[1] : null) is { } unmade) return unmade; + _resourceTarget = false; + return null; + } + zones.Save(); + ZoneWrites.Landed(zones, name); + return null; + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + return "could not add the zone to city_univers: " + ex.Message; + } + } + + public string? ZoneDelete(string name, bool apply, out LoadZoneInfo? zone) + { + zone = null; + if (EnsureEnvironment() is { } notReady) return notReady; + if (string.IsNullOrWhiteSpace(name)) return "name is the zone to take out"; + if (Assets.World.LoadZones.IsShipped(name)) return $"{name} came with the game - only a zone that was added is taken out"; + var archive = new FileInfo(Assets.MafiaEnvironment.CityUniversSds); + if (ZoneWrites.StructureBlocked(archive) is { } blocked) return blocked; + try + { + IReadOnlyCollection known = Window?.Viewport.Catalogs.DistrictNames is { Count: > 0 } names + ? names + : [.. Assets.World.MapCatalog.Build(Assets.MafiaEnvironment.CityFolder, f => Assets.Sds.SdsMeshLoader.EnsureExtracted(f)) + .Areas.Select(a => a.BaseName)]; + Assets.World.LoadZones zones = Assets.World.LoadZones.Open(f => Assets.Sds.SdsMeshLoader.EnsureExtracted(f), known, archive, + forNewZones: true); + if (!zones.Volumes.TryGetValue(name, out Formats.Frames.ObjectTypes.FrameObjectArea? volume)) return $"no load zone named '{name}'"; + (Vector3 lo, Vector3 hi) = Assets.World.LoadZones.WorldBox(volume); + zone = new LoadZoneInfo(name, zones.DistrictsOf(name), false, 0f, Xyz(lo), Xyz(hi), Assets.World.LoadZones.LoadsOnArrival(name), Added: true); + if (zones.Delete(name) is { } refused) return refused; + if (!apply) return null; + if (Window?.Viewport is { } viewport) + { + if (viewport.ZoneEditing.Remove(name) is { } kept) return kept; + _resourceTarget = false; + return null; + } + zones.Save(); + ZoneWrites.Landed(zones, name); + return null; + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + return "could not take the zone out of city_univers: " + ex.Message; + } + } + + public string? ZoneMoveFace(string zone, string face, float to, bool apply, string? copy, out IReadOnlyList results) + { + results = []; + if (EnsureEnvironment() is { } notReady) return notReady; + List names = string.IsNullOrWhiteSpace(copy) ? ["base"] + : string.Equals(copy, "all", StringComparison.OrdinalIgnoreCase) + ? [.. Assets.World.LoadZones.Copies().Select(Assets.World.LoadZones.CopyName)] + : [copy]; + + // Everything is worked out, in memory, for every copy before any of them is written: a face that can + // be moved in one copy and not in another must not leave the two telling the game different things. + var moved = new List<(Assets.World.LoadZones Zones, Assets.World.LoadZoneFaceMove Move, string Copy)>(); + var absent = new List(); + foreach (string name in names) + { + if (OpenLoadZones(name, out Assets.World.LoadZones? all) is { } failed) return failed; + if (!all!.Volumes.ContainsKey(zone ?? "")) + { + absent.Add(name); + continue; + } + // The scene is written whole, from what is on disk. An editor that holds the archive has a copy of + // its own and writes THAT whole on its next save: its copy of the zone must be what the disk has + // now, and is brought in step after the write (ZoneWrites). + if (apply && ZoneWrites.Blocked(all, zone!) is { } blocked) return $"{name}: {blocked} (editor_save)"; + if (all.MoveFace(zone!, face, to, out Assets.World.LoadZoneFaceMove? move) is { } refused) return $"{name}: {refused}"; + moved.Add((all, move!, name)); + } + if (moved.Count == 0) return $"no load zone named '{zone}' in " + string.Join(", ", absent); + + // The base copy alone, with the map editor open: written through the editor's own zone editing, the way + // its gizmo and its Loading zones window write - so the move is a step of the editor's undo history. + bool viaEditor = apply && moved.Count == 1 && Window?.Viewport is not null + && string.Equals(moved[0].Zones.Archive.FullName, new FileInfo(Assets.MafiaEnvironment.CityUniversSds).FullName, StringComparison.OrdinalIgnoreCase); + if (viaEditor) + { + if (Window!.Viewport.ZoneEditing.MoveFace(zone!, face, to) is { } unmoved) return unmoved; + // The step is on the MAP editor's history. Left pointing at the resource editor, the editor_undo + // that follows took back whatever that one did last and called it done. + _resourceTarget = false; + } + + // Written one copy after another - and all of them, or none: the scene each copy had is kept until the + // last one is down, and a copy that fails puts the ones before it back. + var written = new List<(string File, byte[] Before)>(); + if (apply && !viaEditor) + { + foreach ((Assets.World.LoadZones all, _, string name) in moved) + { + try + { + byte[] before = File.ReadAllBytes(all.SceneFile); + all.Save(); + written.Add((all.SceneFile, before)); + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + var stuck = new List(); + foreach ((string file, byte[] before) in written) + { + try { File.WriteAllBytes(file, before); } + catch (Exception back) when (back is IOException or UnauthorizedAccessException) { stuck.Add(file); } + } + return $"could not write the scene of city_univers ({name}): {ex.Message} - nothing was changed" + + (stuck.Count > 0 ? ", EXCEPT that these could not be put back: " + string.Join(", ", stuck) : ""); + } + } + foreach ((Assets.World.LoadZones all, _, _) in moved) ZoneWrites.Landed(all, zone!); + } + + results = [.. moved.Select(m => new LoadZoneMoveInfo(m.Copy, m.Move.Zone, m.Move.Face, m.Move.From, m.Move.To, Xyz(m.Move.BoxMin), + Xyz(m.Move.BoxMax), m.Zones.DistrictsOf(m.Move.Zone), apply, apply ? m.Zones.SceneFile : null, m.Zones.Archive.FullName))]; + return null; + } } diff --git a/src/Illusion/Viewport/ArchiveThumbnailRenderer.cs b/src/Illusion/Viewport/ArchiveThumbnailRenderer.cs new file mode 100644 index 0000000..9d74819 --- /dev/null +++ b/src/Illusion/Viewport/ArchiveThumbnailRenderer.cs @@ -0,0 +1,641 @@ +using System.IO; +using System.Numerics; +using System.Windows.Media; +using System.Windows.Media.Imaging; +using Illusion.Assets.Library; +using Illusion.Assets.Sds; +using Illusion.Domain; +using Illusion.Formats.Hashing; +using Illusion.Rendering.Gpu; +using Illusion.Rendering.Passes; + +namespace Illusion.Viewport; + +/// +/// The picture on an archive tile of the content browser: what the archive holds, drawn on its own with its own +/// textures. A person is framed head and shoulders — at tile size that is the part that tells one skin from +/// the next; a car is shown whole and on its wheels, from the front quarter. +/// +/// The same shape as : a headless GPU stack created on first use, an +/// offscreen target that is read back, and the picture kept on disk — keyed by the archive's path, size and +/// time stamp, so a rebuilt archive draws again and an untouched one never does. Drawing one costs reading the +/// archive (and extracting it the first time), which is why the browser asks for pictures one at a time, for +/// the kinds named by only. +/// +/// +/// In two halves, so that the costly one stays off the UI thread: reads the archive and +/// works out what to show and from where — any thread, nothing of the GPU — and puts that +/// on the offscreen target, UI thread only. Neither throws: a tile whose archive cannot be read keeps its +/// icon, and a timer tick is no place for an exception to surface — there it takes the program down. +/// +/// +internal sealed class ArchiveThumbnailRenderer : IDisposable +{ + public const int Width = 192; + public const int Height = 128; + + private GpuContext? _gpu; + private SceneRenderer? _renderer; + private SharedRenderTarget? _target; + private bool _failed; + + private static string CacheDir => Path.Combine( + Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData), "Illusion", "archive_thumbs"); + + /// Whether an archive of this kind is one model worth a picture. A district or a sound bank is + /// not: the first is a scene of thousands of objects, the second has nothing to draw. + public static bool Draws(LibraryResourceKind kind) => + IsPerson(kind) || kind is LibraryResourceKind.Wardrobe or LibraryResourceKind.Car; + + // A figure standing upright — as against a wardrobe archive, which is a garment on a hanger. + private static bool IsPerson(LibraryResourceKind kind) => kind is LibraryResourceKind.Character + or LibraryResourceKind.Player or LibraryResourceKind.Police or LibraryResourceKind.Traffic; + + /// What is kept for an archive, without drawing anything. False when nothing is kept yet; true + /// with a null picture for an archive already found to hold nothing a tile can show. + public static bool TryKept(LibraryEntry entry, out ImageSource? picture) + { + picture = null; + string stem = StemOf(entry); + if (File.Exists(stem + ".png")) + { + picture = Load(stem + ".png"); + if (picture != null) return true; + } + return File.Exists(stem + NothingSuffix); + } + + /// What names the state of an archive a kept picture was made from: its path, size and time + /// stamp and its working copy's. A picture shown under another key is out of date. + public static string KeyOf(LibraryEntry entry) => Path.GetFileName(StemOf(entry)); + + /// What worked out for one archive: what to draw, where its textures are + /// and where the camera stands. + public sealed class Staged + { + public required LibraryEntry Entry { get; init; } + public List Meshes { get; init; } = []; + public List TextureFolders { get; init; } = []; + public Vector3 Centre { get; init; } + public Vector3 From { get; init; } + public float Radius { get; init; } + public float Fov { get; init; } + + /// Nothing to draw. + public bool Empty => Meshes.Count == 0; + + /// Empty because the archive could not be READ this time (a file held by something else) — + /// as against an archive that holds nothing to show, which stays so until it changes. + public bool Passing { get; init; } + } + + /// The picture for one archive: the kept one, or a fresh one drawn and kept. Null when the archive + /// holds nothing that can be drawn — the tile then keeps its icon. + public ImageSource? Render(LibraryEntry entry) => + TryKept(entry, out ImageSource? kept) ? kept : Draw(Stage(entry)); + + /// + /// Reads an archive (extracting it the first time) and works out its picture: the meshes, the folders + /// their textures are in, the camera. Any thread; touches nothing of the GPU. Never throws — an archive + /// that cannot be read, or holds nothing to show, comes back . + /// + public static Staged Stage(LibraryEntry entry) + { + if (!Draws(entry.Resource)) return new Staged { Entry = entry }; + try + { + Assets.MafiaMaterials.EnsureLoaded(); + List meshes = SdsMeshLoader.LoadSds(entry.File, lod: 0); + if (meshes.Count == 0) return new Staged { Entry = entry }; + + bool car = entry.Resource == LibraryResourceKind.Car; + var folders = new List { Assets.MafiaEnvironment.ExtractedDir(entry.File) }; + // A car takes its chrome, glass and lamp textures from the shared car library (see + // StageCompanions); without that folder two thirds of what its materials name is white. + foreach (FileInfo companion in StageCompanions.For(entry.File)) + { + try { folders.Add(SdsMeshLoader.EnsureExtracted(companion)); } + catch (Exception ex) when (ex is not OutOfMemoryException) + { + // a missing library is a duller car, not a missing picture + } + } + + // A car is shown the way it stands in the street: on its wheels, without the parts that are + // only there some of the time, and without the helper volumes its archive also holds. + if (car) meshes = CarForShow(entry.File, meshes); + if (meshes.Count == 0) return new Staged { Entry = entry }; + + Vector3 min = new(float.MaxValue), max = new(float.MinValue); + foreach (MeshData mesh in meshes) + { + (Vector3 lo, Vector3 hi) = BoundsOf(mesh); + min = Vector3.Min(min, lo); + max = Vector3.Max(max, hi); + } + if (min.X > max.X) return new Staged { Entry = entry }; + + Vector3 centre; + float radius; + Vector3 from; + float fov = MathF.PI / 3f; // the viewport's own; a car narrows it below + if (IsPerson(entry.Resource)) + { + // Head and shoulders: the top fifth of the figure is the head, the shoulders end a little + // under a third of the way down. Seen from the front, a touch to one side and from above — + // dead ahead flattens a face into a mask. + float height = max.Z - min.Z; + centre = new Vector3((min.X + max.X) / 2f, (min.Y + max.Y) / 2f, max.Z - height * 0.155f); + radius = MathF.Max(0.05f, height * 0.17f); + from = Vector3.Normalize(new Vector3(0.32f, PersonFrontY, 0.10f)); + } + else if (car) + { + // The whole car, from the front quarter and a little above — the angle a car is recognized + // from — through a long lens: at the viewport's wide angle the near wing swells and the tail + // shrinks away, and a tile of that is mostly empty. + fov = CarLens; + from = Vector3.Normalize(new Vector3(0.66f, 0.75f, 0.27f)); + Vector3 right = Vector3.Normalize(Vector3.Cross(Vector3.UnitZ, from)); + Vector3 up = Vector3.Cross(from, right); + float tan = MathF.Tan(CarLens / 2f), aspect = (float)Width / Height; + + // Fitted to what is DRAWN, vertex by vertex, not to a box round it: the nearest the camera + // can stand with every vertex in the picture, then the aim moved to the middle of what that + // shows, and again — a point nearer the camera takes up more of the frame, so the middle of + // the car is not the middle of its picture. + List drawn = DrawnPoints(meshes); + centre = (min + max) / 2f; + float fit = 1f; + for (int pass = 0; pass < 4; pass++) + { + fit = 0f; + foreach (Vector3 point in drawn) + { + Vector3 offset = point - centre; + float across = MathF.Abs(Vector3.Dot(offset, right)) / (tan * aspect); + float upward = MathF.Abs(Vector3.Dot(offset, up)) / tan; + fit = MathF.Max(fit, Vector3.Dot(offset, from) + MathF.Max(across, upward)); + } + float left = float.MaxValue, rightmost = float.MinValue, low = float.MaxValue, high = float.MinValue; + foreach (Vector3 point in drawn) + { + Vector3 offset = point - centre; + float depth = fit - Vector3.Dot(offset, from); + if (depth <= 0.01f) continue; + float x = Vector3.Dot(offset, right) / depth, y = Vector3.Dot(offset, up) / depth; + left = MathF.Min(left, x); + rightmost = MathF.Max(rightmost, x); + low = MathF.Min(low, y); + high = MathF.Max(high, y); + } + if (left > rightmost) break; + centre += (right * ((left + rightmost) / 2f * fit)) + (up * ((low + high) / 2f * fit)); + } + // Turned back into the "radius" the shared camera code in Draw works from (which adds its + // own margin). + radius = MathF.Max(0.05f, fit * MathF.Sin(CarLens / 2f)); + } + else + { + // A garment on its hanger: all of it, from the same side a person is looked at from. + centre = (min + max) / 2f; + radius = MathF.Max(0.05f, (max - min).Length() / 2f * 0.85f); + from = Vector3.Normalize(new Vector3(0.32f, PersonFrontY, 0.10f)); + } + + if (!float.IsFinite(radius) || !float.IsFinite(centre.X + centre.Y + centre.Z)) return new Staged { Entry = entry }; + return new Staged + { + Entry = entry, Meshes = meshes, TextureFolders = folders, Centre = centre, From = from, Radius = radius, Fov = fov, + }; + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) + { + return new Staged { Entry = entry, Passing = true }; + } + catch (Exception ex) when (ex is not OutOfMemoryException) + { + // Everything else a broken archive can throw on the way in — a manifest that is not XML, a frame + // type nobody knows, an index past the end of a buffer. The list is not knowable in advance, and + // the answer to each is the same: this archive has no picture. + return new Staged { Entry = entry }; + } + } + + /// Draws what worked out and keeps the picture. UI thread. Null — and the + /// tile keeps its icon — when there is nothing to draw or the GPU would not have it; never throws. + public ImageSource? Draw(Staged staged) + { + ArgumentNullException.ThrowIfNull(staged); + if (staged.Empty) + { + if (!staged.Passing) NoteNothing(staged.Entry); + return null; + } + if (!EnsureContext()) return null; + + try + { + // One archive at a time, and only ITS folders: a texture is taken from the first folder that + // has the name, so folders left from the archive before would dress this one in its textures. + _renderer!.Clear(); + _renderer.Textures.ClearFolders(); + foreach (string folder in staged.TextureFolders) _renderer.Textures.AddFolder(folder); + foreach (MeshData mesh in staged.Meshes) _renderer.AddMesh(mesh); + + _renderer.Camera.Fov = staged.Fov; + float distance = staged.Radius / MathF.Sin(staged.Fov / 2f) * 1.05f; + _renderer.Camera.Near = MathF.Max(0.01f, distance - staged.Radius * 3f); + _renderer.Camera.Far = distance + staged.Radius * 12f; + _renderer.Camera.LookAt(staged.Centre + staged.From * distance, staged.Centre); + _renderer.Render(_target!); + + byte[] bgra = RenderTargetReadback.Read(_gpu!, _target!); + BitmapSource bmp = BitmapSource.Create(Width, Height, 96, 96, PixelFormats.Bgra32, null, bgra, Width * 4); + bmp.Freeze(); + Save(bmp, staged.Entry); + return bmp; + } + catch (Exception ex) when (ex is not OutOfMemoryException) + { + // A texture that will not decode, a device the driver took away. Not written down: the next run + // may well draw it. + return null; + } + finally + { + try { _renderer?.Clear(); } // the meshes are not held until the next tile comes along + catch (Exception ex) when (ex is not OutOfMemoryException) { /* nothing more to release */ } + } + } + + // About a 100 mm lens on a 35 mm frame — what a car is photographed with. + private const float CarLens = 0.36f; + + /// Every vertex some triangle actually draws, in the world — a part emptied by + /// and a triangle collapsed there contribute nothing. + private static List DrawnPoints(List meshes) + { + var points = new List(); + foreach (MeshData mesh in meshes) + { + var seen = new bool[mesh.Positions.Length]; + foreach (MeshPart part in mesh.Parts) + { + int end = Math.Min(part.StartIndex + part.IndexCount, mesh.Indices.Length); + for (int i = part.StartIndex; i + 2 < end; i += 3) + { + if (mesh.Indices[i] == mesh.Indices[i + 1] && mesh.Indices[i] == mesh.Indices[i + 2]) continue; + for (int corner = 0; corner < 3; corner++) + { + int vertex = (int)mesh.Indices[i + corner]; + if (vertex >= seen.Length || seen[vertex]) continue; + seen[vertex] = true; + points.Add(Vector3.Transform(mesh.Positions[vertex], mesh.World)); + } + } + } + } + return points; + } + + private static (Vector3 Min, Vector3 Max) BoundsOf(MeshData mesh) + { + Vector3 min = new(float.MaxValue), max = new(float.MinValue); + foreach (Vector3 p in mesh.Positions) + { + Vector3 w = Vector3.Transform(p, mesh.World); + min = Vector3.Min(min, w); + max = Vector3.Max(max, w); + } + return (min, max); + } + + // Glass and snow, by shader: a car's windows carry the broken-glass texture the game only shows after a + // crash, and the snow layer is a second skin the game puts on in winter. Neither is the car. + private const ulong GlassShader = 0x23767C2D47662E58; + private const ulong SnowShader = 0x53BA752812971810; + + /// + /// A car as it stands in the street, from what its archive holds. + /// + /// The archive is the car AND its states: the snow it wears in winter, the supercharger and the side + /// exhaust a tuning shop may fit, windows textured as broken glass — all present at once, all on bones or + /// materials of their own. It also holds volumes that are not geometry at all (the rain shelter, emitters), + /// which name no material and draw as white blocks over the roof. And it holds no wheels: those live in + /// the shared car library and are put on by the game. + /// + /// So: meshes with no known material are left out, the sometimes-parts are dropped triangle by triangle, + /// and a wheel from the library is stood on every axle bone. + /// + private static List CarForShow(FileInfo archive, List meshes) + { + var shown = new List(); + MeshData? body = null; + foreach (MeshData mesh in meshes) + { + if (!mesh.Parts.Any(part => Assets.MafiaMaterials.KnowsMaterial(part.MaterialHash))) continue; + + var parts = new MeshPart[mesh.Parts.Length]; + for (int i = 0; i < parts.Length; i++) + { + MeshPart part = mesh.Parts[i]; + ulong shader = Assets.Materials.MafiaMaterialCatalog.Instance.GetMaterial(part.MaterialHash)?.ShaderId ?? 0; + parts[i] = shader is GlassShader or SnowShader + ? new MeshPart(part.StartIndex, 0, part.DiffuseTexture, part.NormalTexture, part.SpecularTexture, + part.MaterialHash, part.Tint, part.Blended) + : part; + } + + // The sometimes-parts that share a material with the rest: by the bone their vertices ride. + uint[] indices = mesh.Indices; + if (mesh.Skeleton is { } rig && mesh.BoneIndices is { } ids && mesh.BoneWeights is { } weights) + { + var optional = new bool[rig.Bones.Count]; + bool any = false; + for (int bone = 0; bone < optional.Length; bone++) + { + // "turbo_charger" on one car is "turbo charger" on the next. + string name = rig.Bones[bone].Name.Replace("_", "", StringComparison.Ordinal) + .Replace(" ", "", StringComparison.Ordinal); + optional[bone] = name.StartsWith("snow", StringComparison.OrdinalIgnoreCase) + || name.Equals("turbocharger", StringComparison.OrdinalIgnoreCase) + || name.Equals("exhaustcustom", StringComparison.OrdinalIgnoreCase); + any |= optional[bone]; + } + if (any) + { + indices = (uint[])indices.Clone(); + for (int i = 0; i + 2 < indices.Length; i += 3) + { + int vertex = (int)indices[i]; + int best = 0; + for (int k = 1; k < 4; k++) + if (weights[(vertex * 4) + k] > weights[(vertex * 4) + best]) best = k; + int bone = ids[(vertex * 4) + best]; + // A triangle with no area is a triangle not drawn. + if (bone < optional.Length && optional[bone]) indices[i + 1] = indices[i + 2] = indices[i]; + } + } + } + + var copy = new MeshData + { + Name = mesh.Name, + World = mesh.World, + Positions = mesh.Positions, + Normals = mesh.Normals, + UVs = mesh.UVs, + Tangents = mesh.Tangents, + Binormals = mesh.Binormals, + Indices = indices, + Parts = parts, + Lod = mesh.Lod, + BoneIndices = mesh.BoneIndices, + BoneWeights = mesh.BoneWeights, + Skeleton = mesh.Skeleton, + LiveRest = mesh.LiveRest, + }; + shown.Add(copy); + if (copy.Skeleton != null && (body == null || copy.Positions.Length > body.Positions.Length)) body = copy; + } + + if (body?.Skeleton is { } bones) shown.AddRange(Wheels(archive, body, bones)); + return shown; + } + + private static List? _wheelLibrary; + private static readonly object WheelSync = new(); + + // The shared car library's meshes, read once and kept: every car's wheels come out of it. + private static List WheelLibrary(FileInfo archive) + { + lock (WheelSync) + { + if (_wheelLibrary != null) return _wheelLibrary; + var library = new List(); + foreach (FileInfo file in StageCompanions.For(archive)) + { + try { library.AddRange(SdsMeshLoader.LoadSds(file, lod: 0)); } + catch (Exception ex) when (ex is not OutOfMemoryException) + { + // no library, no wheels — the car is still worth a picture + } + } + return _wheelLibrary = library; + } + } + + /// + /// One wheel per hub. A car's rig marks each hub with a brake-drum bone (the axle bones sit on the car's + /// centre line — they are the pivots, not where a wheel goes), and the shared library holds the wheels at + /// their real size. Which of them a car wears is a row of its tuning table; a tile only has to say "a car + /// on its wheels", so the choice here is by the kind of vehicle — a plain road wheel, the jeep's, a + /// lorry's — and the wheel is not scaled. + /// + private static IEnumerable Wheels(FileInfo archive, MeshData body, SkeletonData rig) + { + List wheels = WheelLibrary(archive); + + (Vector3 lo, Vector3 hi) = BoundsOf(body); + string name = Path.GetFileNameWithoutExtension(archive.Name); + bool lorry = name.StartsWith("hank", StringComparison.OrdinalIgnoreCase); + string[] front, rear; + if (lorry) (front, rear) = (["wheel hankF"], ["wheel HankB"]); + else if (name.Contains("jeep", StringComparison.OrdinalIgnoreCase)) front = rear = ["pneu_jeep", "wheel jeep1"]; + else if (hi.Y - lo.Y > 6.5f) front = rear = ["wheel_truck01", "wheel_truck02"]; + else front = rear = ["wheel_civil01"]; + + MeshData? Find(string[] wanted) => + wanted.Select(w => wheels.FirstOrDefault(m => m.Name.Equals(w, StringComparison.OrdinalIgnoreCase))) + .FirstOrDefault(m => m != null) + ?? wheels.FirstOrDefault(m => m.Name.StartsWith("wheel_civil", StringComparison.OrdinalIgnoreCase)); + + foreach (BoneData bone in rig.Bones) + { + // "brake_drumFL", "brake drumBR2", "BRAKE DRUM RL" — one bone per hub, spelled three ways. + string plain = bone.Name.Replace("_", "", StringComparison.Ordinal).Replace(" ", "", StringComparison.Ordinal); + if (!plain.StartsWith("brakedrum", StringComparison.OrdinalIgnoreCase)) continue; + Vector3 at = bone.Rest.Translation; + if (Find(at.Y >= (lo.Y + hi.Y) / 2f ? front : rear) is not { } wheel) continue; + + // The wheel's own shape: centred on its hub, thinnest along its axle. + Vector3 wheelLo = new(float.MaxValue), wheelHi = new(float.MinValue); + foreach (Vector3 p in wheel.Positions) + { + wheelLo = Vector3.Min(wheelLo, p); + wheelHi = Vector3.Max(wheelHi, p); + } + Vector3 hub = (wheelLo + wheelHi) / 2f, extent = wheelHi - wheelLo; + Matrix4x4 upright = extent.X <= extent.Y && extent.X <= extent.Z ? Matrix4x4.Identity + : extent.Y <= extent.Z ? Matrix4x4.CreateRotationZ(MathF.PI / 2f) + : Matrix4x4.CreateRotationY(MathF.PI / 2f); + + Matrix4x4 place = Matrix4x4.CreateTranslation(-hub) * upright + // The far side's wheel is the same wheel turned round, so its face looks outward too. + * (at.X < 0f ? Matrix4x4.CreateRotationZ(MathF.PI) : Matrix4x4.Identity) + * Matrix4x4.CreateTranslation(at) * body.World; + yield return new MeshData + { + Name = "wheel " + bone.Name, + World = place, + Positions = wheel.Positions, + Normals = wheel.Normals, + UVs = wheel.UVs, + Tangents = wheel.Tangents, + Binormals = wheel.Binormals, + Indices = wheel.Indices, + Parts = wheel.Parts, + Lod = 0, + }; + } + } + + /// Which way a person's model faces along Y: the camera stands on that side. + internal static float PersonFrontY { get; set; } = 1f; + + // Bumped when what a tile shows changes — the framing, what is left off a car — so that pictures kept by + // an older build are not handed out as this one's. + private const int Look = 2; + private const string NothingSuffix = ".none"; + + // "_": the first half is the archive, the second its size and time stamp, its working + // copy's time stamp (the picture is drawn from the working copy, which a save changes without touching + // the packed archive) and the look. One archive's pictures share the first half, which is how the ones + // a new picture replaces are found. + private static string StemOf(LibraryEntry entry) + { + var file = new FileInfo(entry.File.FullName); + long copy = 0; + try + { + string extracted = Assets.MafiaEnvironment.ExtractedDir(file); + if (Directory.Exists(extracted)) copy = Directory.GetLastWriteTimeUtc(extracted).Ticks; + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or ArgumentException + or InvalidOperationException or NullReferenceException) + { + // no environment, or a path outside the game: then there is no working copy to speak of + } + string state = file.Exists ? $"{file.Length}|{file.LastWriteTimeUtc.Ticks}|{copy}|{Look}" : $"gone|{Look}"; + return Path.Combine(CacheDir, $"{Fnv64.Hash(file.FullName.ToLowerInvariant()):x16}_{Fnv64.Hash(state):x16}"); + } + + // The files kept for the same archive in an earlier state: each rebuild used to leave one behind for good. + private static void DropOlder(string stem) + { + string whose = Path.GetFileName(stem)[..17]; // sixteen digits and the underscore + foreach (string other in Directory.GetFiles(CacheDir, whose + "*")) + { + if (!Path.GetFileNameWithoutExtension(other).Equals(Path.GetFileName(stem), StringComparison.OrdinalIgnoreCase)) + { + File.Delete(other); + } + } + if (_legacyDropped) return; + _legacyDropped = true; + // Pictures under the name the first version of this gave them (no underscore): nothing reads them. + foreach (string old in Directory.GetFiles(CacheDir, "*.png")) + { + if (!Path.GetFileName(old).Contains('_')) File.Delete(old); + } + } + + private static bool _legacyDropped; + + // An archive with nothing a tile can show: written down, so that the next run does not extract and read + // it again to find that out. + private static void NoteNothing(LibraryEntry entry) + { + try + { + Directory.CreateDirectory(CacheDir); + string stem = StemOf(entry); + File.WriteAllBytes(stem + NothingSuffix, []); + DropOlder(stem); + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) + { + // not written down: it is found out again next time + } + } + + private static ImageSource? Load(string path) + { + try + { + var bmp = new BitmapImage(); + bmp.BeginInit(); + bmp.CacheOption = BitmapCacheOption.OnLoad; // read now: the file must not stay locked + bmp.UriSource = new Uri(path); + bmp.EndInit(); + bmp.Freeze(); + return bmp; + } + catch (Exception ex) when (ex is IOException or NotSupportedException or UriFormatException) + { + return null; + } + } + + // Keyed AFTER the archive was read: reading it the first time extracts it, and the working copy that + // then exists is part of the key the next look-up will compute. + private static void Save(BitmapSource bmp, LibraryEntry entry) + { + try + { + Directory.CreateDirectory(CacheDir); + string stem = StemOf(entry); + var encoder = new PngBitmapEncoder(); + encoder.Frames.Add(BitmapFrame.Create(bmp)); + using (FileStream file = File.Create(stem + ".png")) encoder.Save(file); + DropOlder(stem); + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) + { + // not keeping the picture costs drawing it again next time, nothing more + } + } + + /// Whether there is a GPU to draw on — false once creating one has failed, and then there is + /// no point reading archives for pictures nobody can draw. + public bool Usable => EnsureContext(); + + private bool EnsureContext() + { + if (_renderer != null) return true; + if (_failed) return false; + try + { + _gpu = new GpuContext(); + _renderer = new SceneRenderer(_gpu) + { + Mode = RenderMode.MaterialPreview, + ShowSky = false, + // Nothing behind the figure: the tile's own background, hover and selection show through, + // where a filled picture would sit on the tile as a darker box. + ClearColor = new Vector4(0f, 0f, 0f, 0f), + }; + _renderer.Textures.SetFallbackResolver(Assets.Textures.TextureSearchIndex.FindPath); + _target = new SharedRenderTarget(_gpu, Width, Height); + return true; + } + catch + { + Dispose(); + _failed = true; + return false; + } + } + + public void Dispose() + { + _renderer?.Dispose(); + _target?.Dispose(); + _gpu?.Dispose(); + _renderer = null; + _target = null; + _gpu = null; + } +} diff --git a/src/Illusion/Viewport/CollisionEditController.cs b/src/Illusion/Viewport/CollisionEditController.cs index 4f6073a..c6972f4 100644 --- a/src/Illusion/Viewport/CollisionEditController.cs +++ b/src/Illusion/Viewport/CollisionEditController.cs @@ -34,13 +34,8 @@ internal sealed class CollisionEditController : ICollisionEditSink /// hulls" command would sweep. Zero disables it, so a modder can read the file's state off the menu instead /// of guessing whether a delete left anything behind. /// - public int UnusedHullCount() - { - int total = 0; - foreach ((CollisionDocumentAdapter doc, SceneNode layer) in SelectedCollisionLayers()) - total += CollisionOrphanEdit.Build(this, doc, layer)?.Count ?? 0; - return total; - } + public int UnusedHullCount() => + SweepUnusedHulls(SelectedCollisionLayers().Select(l => l.Layer), apply: false).Sum(l => l.Unused); /// /// Removes every hull no placement references, as one undoable edit per layer. Never runs on its own: a @@ -49,20 +44,33 @@ public int UnusedHullCount() /// public void RemoveUnusedHulls() { - int removed = 0; + if (SweepUnusedHulls(SelectedCollisionLayers().Select(l => l.Layer), apply: true).Sum(l => l.Unused) == 0) + _host.RaiseNotice("no unused hulls — every hull in this .col is placed"); + } + + /// + /// The same sweep for layers named outright instead of reached through the selection — what the MCP tool + /// drives, having no selection to go by. Reports each layer's unused hulls; with + /// removes them, all layers as one undoable edit. + /// + public IReadOnlyList<(SceneNode Layer, int Unused)> SweepUnusedHulls(IEnumerable layers, bool apply) + { + var found = new List<(SceneNode Layer, int Unused)>(); var edits = new List(); - foreach ((CollisionDocumentAdapter doc, SceneNode layer) in SelectedCollisionLayers()) + foreach (SceneNode layer in layers) { + if (layer.Source is not CollisionDocumentAdapter doc) continue; CollisionOrphanEdit? edit = CollisionOrphanEdit.Build(this, doc, layer); - if (edit == null) continue; + found.Add((layer, edit?.Count ?? 0)); + if (edit == null || !apply) continue; edit.Redo(); edits.Add(edit); - removed += edit.Count; } - if (edits.Count == 0) { _host.RaiseNotice("no unused hulls — every hull in this .col is placed"); return; } + if (edits.Count == 0) return found; History.Push(edits.Count == 1 ? edits[0] : new CompositeEdit(edits.ToArray())); - _host.RaiseNotice($"removed {removed} unused hull(s) — Ctrl+Z restores them"); + _host.RaiseNotice($"removed {found.Sum(l => l.Unused)} unused hull(s) — Ctrl+Z restores them"); + return found; } // The collision layers the selection reaches: the layer node itself when it is selected, or the layer behind diff --git a/src/Illusion/Viewport/D3DImageHost.cs b/src/Illusion/Viewport/D3DImageHost.cs index 76e76c7..b6025dd 100644 --- a/src/Illusion/Viewport/D3DImageHost.cs +++ b/src/Illusion/Viewport/D3DImageHost.cs @@ -32,10 +32,11 @@ namespace Illusion.Viewport; /// (save/build tracking). This class wires them to the base control and /// exposes the single facade the UI binds to, including the transform-gizmo host (). /// -public sealed class D3DImageHost : ViewportControl, ITransformGizmoHost +public sealed class D3DImageHost : ViewportControl, ITransformGizmoHost, IBoxGizmoHost { internal readonly SceneTree Tree; internal readonly ViewportCatalogs Catalogs; + internal readonly ZoneEditController ZoneEditing; internal readonly DistrictStreamer Streamer; internal readonly SelectionController Selection; internal readonly TransformEditController Editing; @@ -57,6 +58,7 @@ public D3DImageHost() { Tree = new SceneTree(); Catalogs = new ViewportCatalogs(this); + ZoneEditing = new ZoneEditController(this); Streamer = new DistrictStreamer(this); Selection = new SelectionController(this); Editing = new TransformEditController(this); @@ -242,10 +244,17 @@ public void ToggleSelect(SceneNode node) public EditHistory History => Editing.History; /// Reverts the last transform edit (gizmo drag or numeric field). - public void Undo() => Unwind(Editing.History.Undo, "Undo"); + public void Undo() => TryUndo(); /// Re-applies the last undone transform edit. - public void Redo() => Unwind(Editing.History.Redo, "Redo"); + public void Redo() => TryRedo(); + + /// Undo, saying why when the step could not be taken (null: it was). The step then stays in the + /// history; the reason has been shown as a notice already. + public string? TryUndo() => Unwind(Editing.History.Undo, "Undo"); + + /// + public string? TryRedo() => Unwind(Editing.History.Redo, "Redo"); /// /// One step of the history, taken while a Blender push cannot be landing. @@ -257,14 +266,25 @@ public void ToggleSelect(SceneNode node) /// reason the gate itself gives: this thread is the one the push comes back to. /// /// - private void Unwind(Action step, string what) + private string? Unwind(Action step, string what) { if (!BridgeSession.TryHoldForEdit()) { - RaiseNotice($"a push from Blender is landing — {what} again in a moment"); - return; + string busy = $"a push from Blender is landing — {what} again in a moment"; + RaiseNotice(busy); + return busy; + } + try + { + step(); + return null; + } + catch (EditRefusedException refused) + { + // the history kept the step (it leaves its stacks alone when one throws): it can be taken again + RaiseNotice(refused.Message, isError: true); + return refused.Message; } - try { step(); } finally { BridgeSession.ReleaseAfterEdit(); } } @@ -737,7 +757,7 @@ public IReadOnlyDictionary LiveFrameNames(System.IO.FileInfo arch return Staged(archive)?.BoneWorlds() ?? new Dictionary(); } - private Assets.Adapters.SceneDocumentAdapter? Staged(System.IO.FileInfo archive) + internal Assets.Adapters.SceneDocumentAdapter? Staged(System.IO.FileInfo archive) { foreach (Assets.Adapters.SceneDocumentAdapter document in CarCollisionEditing.StageDocuments()) { @@ -864,6 +884,9 @@ public bool AddMaterialParameter(ulong hash, string paramId, IReadOnlyList MaterialEditing.AssignSlotMaterial(node, slotIndex, newHash); + /// + public string? SlotAssignObstacle(SceneNode node) => MaterialEditing.SlotAssignObstacle(node); + /// Whether is still part of the loaded scene tree — actions pinned /// to a node across scene reloads (the material editor's assign target) validate with this. public bool IsNodeInScene(SceneNode node) => Tree.IsInScene(node); @@ -1142,6 +1165,10 @@ public readonly record struct BuildReport( /// public BuildReport BuildEdits(bool createBackup = true) => Persistence.BuildEdits(createBackup); + /// + public BuildReport BuildArchives(IReadOnlyList archives, bool createBackup = true) => + Persistence.BuildArchives(archives, createBackup); + /// public void PrepareForArchiveRestore() => Streamer.ResetForExternalChange(); @@ -1171,6 +1198,19 @@ public readonly record struct BuildReport( // multi-selection (a miss is ignored); a plain click replaces the selection (or clears it on a miss). protected override void OnViewportLeftClick(Point pos) { + // A tool that is picking on the scene (Tools → Hide triangles picks triangles) is offered the click + // first; what it does not take is an ordinary click. + if (SceneClickTaker is { } taker && taker(pos)) return; + + // With the Loading zones layer up a click picks a ZONE and nothing else: the zone is then what the tool + // shelf's Move and Scale act on, and an object picked by the same click would be a second thing under + // the same gizmo. Objects are picked with the layer off. + if (ShowZones && Catalogs.Zones != null) + { + Select(null); + PickZone(pos); + return; + } SceneNode? hit = PickNode(pos); if ((Keyboard.Modifiers & ModifierKeys.Control) != 0) { @@ -1179,6 +1219,104 @@ protected override void OnViewportLeftClick(Point pos) else Select(hit); } + /// While set, a left click on the scene is offered to it before anything is selected; true means + /// the click was its own. One tool at a time: the tool that sets it clears it. + internal Func? SceneClickTaker { get; set; } + + /// + /// The triangle of a mesh node under a screen pixel, as the eye has it: of the first level of detail, and + /// only when nothing of the scene stands in front of it there. Null when the node is not a mesh, the pixel + /// misses it, or something covers it - then says which, for the user. + /// + internal TriangleHider.Picked? PickTriangle(SceneNode node, Point pos, out string? missed) + { + missed = null; + if (node.Source is not Assets.Adapters.FrameNodeAdapter { Frame: Formats.Frames.ObjectTypes.FrameObjectSingleMesh mesh } adapter) return null; + (Vector3 origin, Vector3 dir) = BuildViewportRay(pos); + TriangleHider.Picked? hit = TriangleHider.Pick(mesh, ((IFrameNode)adapter).WorldTransform, origin, dir); + if (hit == null) + { + missed = "That spot is not on this mesh."; + return null; + } + // Nearer scenery at this pixel means the triangle is behind it: what cannot be seen is not picked. The + // mesh's own rows do not count as scenery - from a distance the editor draws a coarser level of it, whose + // surface need not lie where the first level's does, and that level stood "in front" of every triangle. + SceneNode? front = PickNode(pos); + if (front != null && (ReferenceEquals(front, node) || ReferenceEquals(front.Source, node.Source))) return hit; + float nearest = PickMesh(pos, out float meshT) != null ? meshT : float.PositiveInfinity; + if (nearest >= hit.Distance - MathF.Max(0.05f, hit.Distance * 0.002f)) return hit; + missed = $"{front?.Name ?? "Something"} stands in front of the mesh there."; + return null; + } + + /// Marks triangles in the viewport as a tool's pick - their edges, and a stroke to each corner so a + /// small one still reads as filled. An empty list takes the marks away. + internal void ShowPickedTriangles(IReadOnlyList triangles) + { + if (Rnd == null) return; + var lines = new List(triangles.Count * 12); + var colours = new List(triangles.Count * 12); + var edge = new Vector4(1f, 0.62f, 0.22f, 1f); + var inner = new Vector4(1f, 0.62f, 0.22f, 0.45f); + foreach (TriangleHider.Triangle t in triangles) + { + Vector3 centre = (t.A + t.B + t.C) / 3f; + foreach ((Vector3 from, Vector3 to, Vector4 colour) in new[] + { + (t.A, t.B, edge), (t.B, t.C, edge), (t.C, t.A, edge), (centre, t.A, inner), (centre, t.B, inner), (centre, t.C, inner), + }) + { + lines.Add(from); + lines.Add(to); + colours.Add(colour); // one for each end of the line + colours.Add(colour); + } + } + Rnd.SetPickedTriangleLines(lines, colours); + } + + /// Raised by a click on the scene while the Loading zones layer is shown: the world point clicked + /// and the zone picked there, or null when no zone's box holds the point. + public event Action? ZonePicked; + + private Point _zoneClick; + private Vector3 _zoneClickAt; + + // A zone is not picked by where the ray enters its box: the camera stands inside most of them, and they lie + // two and three deep over the whole city. It is picked by the point clicked ON THE SCENE - "which zone is + // this spot in" - the smallest box first; a second click on the same spot takes the next one out. A click + // on the sky picks none. + private void PickZone(Point pos) + { + (Vector3 origin, Vector3 dir) = BuildViewportRay(pos); + float t = PickMesh(pos, out float meshT) != null ? meshT : float.PositiveInfinity; + if (Streamer.PickCrash(origin, dir, out float crashT) != null) t = MathF.Min(t, crashT); + if (!float.IsFinite(t) || t <= 0f) + { + Catalogs.SelectZone(null); + return; + } + + Vector3 at = origin + dir * t; + IReadOnlyList here = Catalogs.ZonesHolding(at); + // "The same spot again" is the same spot of the WORLD, and the next zone is the one after the zone that + // is picked now: the camera can have moved under a still pointer, and the pick can have been made in + // the Loading zones window since - a count of clicks at a pixel knew of neither. + int current = -1; + for (int i = 0; i < here.Count; i++) + { + if (here[i].Name == Catalogs.SelectedZone) current = i; + } + bool again = current >= 0 && here.Count > 1 && (pos - _zoneClick).Length < 5.0 + && Vector3.Distance(at, _zoneClickAt) < MathF.Max(0.5f, t * 0.01f); + _zoneClick = pos; + _zoneClickAt = at; + string? zone = here.Count > 0 ? here[again ? (current + 1) % here.Count : 0].Name : null; + Catalogs.SelectZone(zone); + ZonePicked?.Invoke(at, zone); + } + // ── Glyph hover: what the cursor is over, highlighted and named ── /// The glyph object under the cursor (helper node or bone), or null. Highlighted in the viewport @@ -1330,6 +1468,35 @@ public override void Dispose() base.Dispose(); // Renderer.Dispose releases the attached meshes } + // ── Box gizmo host (IBoxGizmoHost): the picked loading zone under the tool shelf's Move and Scale ── + + public (Vector3 Min, Vector3 Max)? BoxGizmoTarget => ZoneEditing.Target; + + public string? BoxGizmoLabel => ShowZones ? Catalogs.SelectedZoneLabel : null; + + /// The point of the scene the middle of the view looks at - where a thing made "here" goes - or, + /// with nothing under it, a point some way in front of the camera. + internal Vector3 ViewPoint(float ahead = 60f) + { + var centre = new Point(ActualWidth / 2, ActualHeight / 2); + (Vector3 origin, Vector3 dir) = BuildViewportRay(centre); + float t = PickMesh(centre, out float meshT) != null ? meshT : float.PositiveInfinity; + if (Streamer.PickCrash(origin, dir, out float crashT) != null) t = MathF.Min(t, crashT); + return origin + (dir * (float.IsFinite(t) && t > 0f ? t : ahead)); + } + + public bool BoxGizmoFaceMoves(int axis, int side) => Catalogs.SelectedZoneFaces.Contains((side > 0 ? "+" : "-") + "xyz"[axis]); + + public event Action? BoxGizmoChanged; + + internal void RaiseBoxGizmoChanged() => BoxGizmoChanged?.Invoke(); + + public void BoxGizmoBegin() => ZoneEditing.Begin(); + + public void BoxGizmoPreview(Vector3 min, Vector3 max) => ZoneEditing.Preview(min, max); + + public void BoxGizmoEnd(bool commit) => ZoneEditing.End(commit); + // ── Transform gizmo host (ITransformGizmoHost) ── public Matrix4x4 GizmoViewProjection => Renderer?.Camera.ViewProjection ?? Matrix4x4.Identity; diff --git a/src/Illusion/Viewport/DistrictStreamer.cs b/src/Illusion/Viewport/DistrictStreamer.cs index 5f8cdc8..4baff41 100644 --- a/src/Illusion/Viewport/DistrictStreamer.cs +++ b/src/Illusion/Viewport/DistrictStreamer.cs @@ -168,6 +168,17 @@ private sealed class PreparedLoad /// having its meshes attached. What a caller that asked for an area waits on. public bool IsBusy => _building || _loadTask != null || _loadQueue.Count > 0; + /// + /// Whether an archive is queued for loading, being read or being put into the tree right now. Its scene is + /// then read from disk - or about to be - and is not registered as held yet: a change written to the + /// archive in that window would not be in the scene that arrives. + /// + internal bool IsLoading(FileInfo archive) + { + bool Same(FileInfo? file) => file != null && string.Equals(file.FullName, archive.FullName, StringComparison.OrdinalIgnoreCase); + return ((_building || _loadTask != null) && (Same(_loadCtx.file) || Same(_buildCtx.file))) || _loadQueue.Any(q => Same(q.File)); + } + private Queue<(SceneNode Leaf, GpuMesh Mesh)> _buildQueue = null!; // prepared meshes awaiting attach private (string label, string? district, string folder, int gen, FileInfo file) _buildCtx; private List _buildMeshes = null!; diff --git a/src/Illusion/Viewport/GeometryEditController.cs b/src/Illusion/Viewport/GeometryEditController.cs index 36e3b44..706f67b 100644 --- a/src/Illusion/Viewport/GeometryEditController.cs +++ b/src/Illusion/Viewport/GeometryEditController.cs @@ -1,6 +1,8 @@ using System.Numerics; using Illusion.Assets.Adapters; using Illusion.Assets.Bridge; +using Illusion.Assets.Frames; +using Illusion.Assets.Sds; using Illusion.Domain; using Illusion.Formats.Frames.ObjectTypes; using Illusion.Rendering.Gpu; @@ -171,6 +173,208 @@ public List ApplyPushBatch( return outcomes; } + /// What hiding triangles does about a mesh whose geometry other objects draw too. + public enum SharedGeometry + { + /// Nothing is hidden: the triangles would vanish from every object drawing it. + Refuse, + + /// The mesh gets a geometry block and buffers of its own first, and loses the triangles alone. + OwnCopy, + + /// The triangles vanish from every object drawing the geometry - one shape, cut once. + All, + } + + /// How many other objects draw the geometry of this node's mesh, and the first few by name. + public (int Count, string Names) GeometrySharersOf(SceneNode node) + { + if (node.Kind == "Lod" && node.Parent is { } owner && ReferenceEquals(owner.Source, node.Source)) node = owner; + if (node.Source is not FrameNodeAdapter { Frame: FrameObjectSingleMesh mesh } adapter) return (0, ""); + IReadOnlyList others = adapter.Document.GeometrySharers(mesh); + return (others.Count, string.Join(", ", others.Take(4).Select(o => o.Name.String)) + (others.Count > 4 ? ", ..." : "")); + } + + /// + /// Hides triangles of a mesh () as one undoable edit: the index buffers take + /// their planned contents and every level of detail the tree shows is redrawn. A mesh that shares its + /// geometry is refused, given a copy of its own first, or cut together with the others, as + /// says; an instanced one is refused. + /// + /// Null when applied, or why not. + public string? HideTriangles(SceneNode node, IReadOnlyList changes, SharedGeometry shared = SharedGeometry.Refuse) + { + // A click in the viewport on a mesh of several levels picks the row of the level that is drawn ("LOD 0"), + // not the mesh's own row. The edit is the mesh's: every level's buffer changes, and every level's row + // has to be redrawn - from the level's row alone the others went on drawing triangles the data had lost. + if (node.Kind == "Lod" && node.Parent is { } owner && ReferenceEquals(owner.Source, node.Source)) node = owner; + if (node.Source is not FrameNodeAdapter { Frame: FrameObjectSingleMesh mesh } adapter) return "not a mesh"; + if (changes.Count == 0) return "nothing to hide"; + // Asked of every row that draws the mesh: one of several levels keeps its GPU meshes on the level rows, + // and its own row has none to ask. + if (LevelRows(node).Any(r => r.Mesh is { Instanced: true })) + return "an instanced mesh is drawn many times over — its triangles cannot be hidden in one place"; + bool sharing = adapter.Document.GeometrySharers(mesh).Any(); + if (sharing && shared == SharedGeometry.Refuse) return "other objects draw the same geometry — the triangles would vanish from all of them"; + // Two geometry blocks can stand on one index buffer: an object imported twice, with a level's distance + // changed in between, gets a block of its own over the buffers it already brought. A mesh about to get + // copies of its buffers leaves that buffer to the other block. + if (!(sharing && shared == SharedGeometry.OwnCopy) + && adapter.Document.IndexBufferSharers(mesh, changes.Select(c => c.Buffer.Hash).ToHashSet()).Any()) + return "another object draws from the same index buffer — the triangles would vanish from it as well"; + // The plan is the buffers as they were when it was made, and what they become. Made earlier and applied + // now - by a window that counted before something else changed the mesh - it would write over that change. + if (changes.Any(c => !ReferenceEquals(c.Buffer.GetData(), c.Before))) + return "the mesh changed since these triangles were counted — count them again"; + if (_host.Rnd == null) return "the viewport is not rendering yet"; + + if (sharing && shared == SharedGeometry.OwnCopy) + { + // Its own block and buffers first - what a duplicate of the object gets - and the plan carried over + // to them: the copies hold what the shared buffers held, index for index. + if (FrameDuplicator.TryOwnGeometry(adapter.Document, adapter, out string? noCopy) is not { } owned) + return "the mesh could not be given geometry of its own: " + noCopy; + var carried = new List(); + foreach (TriangleHider.Change change in changes) + { + if (owned.CopyOf(change.Buffer) is not { } copy) + { + owned.Revert(); + return "the mesh changed since these triangles were counted — count them again"; + } + carried.Add(new TriangleHider.Change(copy, copy.GetData(), change.After)); + } + var alone = new HiddenTrianglesEdit(this, node, adapter.Document, mesh, carried, []); + alone.Redo(); + _host.Editing.History.Push(new CompositeEdit([new OwnGeometryEdit(this, node, owned), alone])); + _host.RaiseSceneChanged(); + return null; + } + + // Every object drawing the block is redrawn with it: there is one buffer in the file. Their rows are + // taken whether or not the row itself carries a mesh - an object of several levels draws from its level + // rows - and an instanced one among them cannot be redrawn as a single mesh. + List others = []; + if (sharing) + { + foreach (FrameObjectSingleMesh other in adapter.Document.GeometrySharers(mesh)) + { + if (_host.Streamer.Actors.RowOf(other) is { } row && _host.Tree.IsInScene(row)) others.Add(row); + } + if (others.SelectMany(LevelRows).Any(r => r.Mesh is { Instanced: true })) + return "one of the objects drawing this geometry is instanced — cut this object alone instead"; + } + var edit = new HiddenTrianglesEdit(this, node, adapter.Document, mesh, changes, others); + edit.Redo(); + _host.Editing.History.Push(edit); + _host.RaiseSceneChanged(); + return null; + } + + /// A mesh put on a geometry block of its own, and back on the shared one. Nothing is redrawn: the + /// copy holds what the shared block held. + private sealed class OwnGeometryEdit(GeometryEditController owner, SceneNode node, FrameDuplicator.OwnedGeometry owned) : INodeEdit + { + public IEnumerable Nodes { get { yield return node; } } + + public void Undo() + { + owned.Revert(); + owner._host.Persistence.MarkFrameModified(node); + } + + public void Redo() + { + owned.Apply(); + owner._host.Persistence.MarkFrameModified(node); + } + } + + // The rows that draw one mesh: its own (the first level) and a row per further level under it. + private static IEnumerable LevelRows(SceneNode node) + { + yield return node; + foreach (SceneNode child in node.Children) + { + if (child.Kind == "Lod" && ReferenceEquals(child.Source, node.Source)) yield return child; + } + } + + /// Index buffers swapped between two states, and the GPU meshes that draw each. + private sealed class HiddenTrianglesEdit : INodeEdit + { + private readonly GeometryEditController _owner; + private readonly SceneNode _node; + private readonly SceneDocumentAdapter _document; + private readonly FrameObjectSingleMesh _mesh; + private readonly IReadOnlyList _changes; + private readonly List<(SceneNode Row, GpuMesh? Shown, GpuMesh? Hidden)> _rows = []; + private bool _applied; + + /// The rows of the other objects that draw the same geometry block, when the + /// triangles are hidden in the shared block: they lose them too, and are redrawn with this one. + public HiddenTrianglesEdit(GeometryEditController owner, SceneNode node, SceneDocumentAdapter document, + FrameObjectSingleMesh mesh, IReadOnlyList changes, IReadOnlyList sharers) + { + _owner = owner; + _node = node; + _document = document; + _mesh = mesh; + _changes = changes; + foreach (SceneNode row in LevelRows(node)) _rows.Add((row, row.Mesh, null)); + foreach (SceneNode other in sharers) + { + foreach (SceneNode row in LevelRows(other)) _rows.Add((row, row.Mesh, null)); + } + } + + public IEnumerable Nodes { get { yield return _node; } } + + public void Undo() => Apply(hidden: false); + + public void Redo() => Apply(hidden: true); + + private void Apply(bool hidden) + { + D3DImageHost host = _owner._host; + if (!host.Tree.IsInScene(_node) || host.Rnd == null) return; // streamed out — pruning backstop + foreach (TriangleHider.Change change in _changes) + { + change.Buffer.SetData(hidden ? change.After : change.Before); + _document.MarkIndexBufferDirty(change.Buffer.Hash); // a save may have written the other state + } + for (int i = 0; i < _rows.Count; i++) + { + (SceneNode row, GpuMesh? shown, GpuMesh? gone) = _rows[i]; + if (shown == null) continue; // a level the tree never drew + // a row of another object on the same block is built from ITS frame: the block is shared, the + // materials need not be + FrameObjectSingleMesh drawn = row.Source is FrameNodeAdapter { Frame: FrameObjectSingleMesh own } ? own : _mesh; + if (gone == null && hidden && SdsMeshLoader.BuildMeshData(drawn, row.Lod) is { } data) + { + gone = host.Rnd.CreateMeshGpu(data); + gone.Owner = row; + _rows[i] = (row, shown, gone); + } + if (gone == null) continue; + _owner.SwapMesh(row, hidden ? shown : gone, hidden ? gone : shown); + } + host.Persistence.MarkFrameModified(_node); + _applied = hidden; + } + + // Dropped from history: whichever mesh of each pair is DETACHED belongs to this edit alone. + public void Discard() + { + foreach ((SceneNode _, GpuMesh? shown, GpuMesh? gone) in _rows) + { + if (gone == null) continue; + if (_applied) shown?.Dispose(); + else gone.Dispose(); + } + } + } + /// The other tree rows drawing the same geometry block as this node's mesh — the frames that just /// changed with it. Empty when the mesh is the block's only user, or when the sharers are not resident. private IEnumerable SharerNodes(SceneNode node) diff --git a/src/Illusion/Viewport/MaterialEditController.cs b/src/Illusion/Viewport/MaterialEditController.cs index f2b698f..363b2d3 100644 --- a/src/Illusion/Viewport/MaterialEditController.cs +++ b/src/Illusion/Viewport/MaterialEditController.cs @@ -264,19 +264,40 @@ public bool DeleteMaterial(ulong hash) public bool AssignSlotMaterial(SceneNode node, int slotIndex, ulong newHash) { if (node.Source is not IMaterialSlotEditor editor || !_host.Tree.IsInScene(node)) return false; + if (SlotAssignObstacle(node) != null) return false; ulong? before = editor.GetSlotMaterial(slotIndex); if (before == null) return false; if (before == newHash) return true; + IReadOnlyList tableBefore = editor.GetSlotTable(); if (!editor.SetSlotMaterial(slotIndex, newHash)) return false; - _host.Editing.History.Push(new SlotEdit(this, node, slotIndex, before.Value, newHash)); + _host.Editing.History.Push(new SlotEdit(this, node, slotIndex, before.Value, newHash, tableBefore, editor.GetSlotTable())); AfterSlotChanged(node, slotIndex, newHash); return true; } - private void ApplySlot(SceneNode node, int slotIndex, ulong hash) + // The whole table goes back, every level of it: the re-point mirrored itself into further levels by the + // slot's old hash, and the same rule run the other way turns a level that had two materials into one. + /// + /// Why a slot of this node's mesh cannot be re-pointed, or null. A slot is a row of the mesh's material + /// block, and a block can serve several objects: all of them would change in the file and in the game, + /// while the viewport redraws - and the caller is told about - one. + /// + public string? SlotAssignObstacle(SceneNode node) + { + if (node.Source is not Assets.Adapters.FrameNodeAdapter { Frame: Formats.Frames.ObjectTypes.FrameObjectSingleMesh mesh } adapter + || adapter.Document.MaterialSharers(mesh) is not { Count: > 0 } sharers) + { + return null; + } + return $"'{node.Name}' draws with a material block {sharers.Count} other object(s) use too (" + + string.Join(", ", sharers.Take(5).Select(s => s.Name.ToString())) + (sharers.Count > 5 ? ", …" : "") + + ") — re-pointing the slot would change them all"; + } + + private void ApplySlot(SceneNode node, int slotIndex, ulong hash, IReadOnlyList table) { if (node.Source is not IMaterialSlotEditor editor || !_host.Tree.IsInScene(node)) return; - if (editor.SetSlotMaterial(slotIndex, hash)) AfterSlotChanged(node, slotIndex, hash); + if (editor.SetSlotTable(table) || editor.SetSlotMaterial(slotIndex, hash)) AfterSlotChanged(node, slotIndex, hash); } // ── Shared after-effects ── @@ -545,18 +566,23 @@ private sealed class SlotEdit : INodeEdit private readonly int _slotIndex; private readonly ulong _before; private readonly ulong _after; + private readonly IReadOnlyList _tableBefore; + private readonly IReadOnlyList _tableAfter; - public SlotEdit(MaterialEditController owner, SceneNode node, int slotIndex, ulong before, ulong after) + public SlotEdit(MaterialEditController owner, SceneNode node, int slotIndex, ulong before, ulong after, + IReadOnlyList tableBefore, IReadOnlyList tableAfter) { _owner = owner; _node = node; _slotIndex = slotIndex; _before = before; _after = after; + _tableBefore = tableBefore; + _tableAfter = tableAfter; } public IEnumerable Nodes { get { yield return _node; } } - public void Undo() => _owner.ApplySlot(_node, _slotIndex, _before); - public void Redo() => _owner.ApplySlot(_node, _slotIndex, _after); + public void Undo() => _owner.ApplySlot(_node, _slotIndex, _before, _tableBefore); + public void Redo() => _owner.ApplySlot(_node, _slotIndex, _after, _tableAfter); } } diff --git a/src/Illusion/Viewport/ScenePersistence.cs b/src/Illusion/Viewport/ScenePersistence.cs index 52b64ed..5b9a692 100644 --- a/src/Illusion/Viewport/ScenePersistence.cs +++ b/src/Illusion/Viewport/ScenePersistence.cs @@ -145,14 +145,19 @@ public int SaveEdits(List? notSaved = null) /// backup first (all archives in this build share one timestamp). Each archive is packed independently: a /// failure is captured (not thrown) so the remaining archives still build, and only the ones that packed /// leave the list — the rest stay buildable for a retry. - public D3DImageHost.BuildReport BuildEdits(bool createBackup = true) + public D3DImageHost.BuildReport BuildEdits(bool createBackup = true) => BuildArchives(_editedArchives.Values.ToList(), createBackup); + + /// for a chosen set: the archives ticked in the Build window. One that is + /// on the build list and is not among them stays on it; one that is not on the list at all - a working + /// copy changed by hand, added in the window - is packed like the rest. + public D3DImageHost.BuildReport BuildArchives(IReadOnlyList archives, bool createBackup = true) { SaveEdits(); // ensure the extracted folders match memory before we pack them var packed = new List(); var failed = new List(); DateTime when = DateTime.Now; // one stamp for the whole build, so co-packed archives group in backups\ - foreach (FileInfo sds in _editedArchives.Values.ToList()) + foreach (FileInfo sds in archives) { try { diff --git a/src/Illusion/Viewport/ViewportCatalogs.cs b/src/Illusion/Viewport/ViewportCatalogs.cs index 9120166..67b8eaa 100644 --- a/src/Illusion/Viewport/ViewportCatalogs.cs +++ b/src/Illusion/Viewport/ViewportCatalogs.cs @@ -113,6 +113,110 @@ public void InitAsync(bool withMap = true) }); } + /// + /// Reads the loading zones again from the working copy of city_univers and redraws their boxes — after one + /// was moved (Tools → Loading zones), so the layer does not go on showing where it stood. False when the + /// catalog is not up yet or the zones could not be read; the boxes then stay as they were. + /// + public bool ReloadZones() + { + if (Map == null || _host.Rnd == null) return false; + try + { + Zones = AreaZones.Load(f => SdsMeshLoader.EnsureExtracted(f), Map.Areas.Select(a => a.BaseName).ToList()); + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + Debug.WriteLine("Zone reload error: " + ex); + return false; + } + _preview = null; + SelectedZoneFaces = FacesOf(SelectedZone); + BuildZoneBoxes(); + _host.RaiseBoxGizmoChanged(); + ZonesChanged?.Invoke(); + return true; + } + + /// Raised when the zones were read again from the working copy - after any write to one, by the + /// gizmo, the Loading zones window, an undo or the zone_move_face tool. For whoever shows them. + public event Action? ZonesChanged; + + /// The zone picked - by a click on the scene with the layer up, or in the Loading zones window. It is + /// drawn bright and the others are dimmed, so that among boxes lying two and three deep it is plain which one + /// is being looked at. + public string? SelectedZone { get; private set; } + + /// Picks a zone by name, or none (null); the boxes are redrawn. + public void SelectZone(string? name) + { + if (string.Equals(SelectedZone, name, StringComparison.Ordinal)) return; + SelectedZone = name; + _preview = null; + SelectedZoneFaces = FacesOf(name); + if (_host.Rnd != null) BuildZoneBoxes(); + _host.RaiseBoxGizmoChanged(); + } + + /// The picked zone in a line: its name, the districts it keeps loaded, and - for one whose name is + /// not a seam's () - that it does not load them by itself. + public string? SelectedZoneLabel + { + get + { + if (SelectedZone == null || Zones?.FirstOrDefault(z => z.Name == SelectedZone) is not { } zone) return null; + string label = $"{zone.Name} · {string.Join(" + ", zone.Districts)}"; + if (!LoadZones.IsShipped(zone.Name)) label += " · added"; + return LoadZones.LoadsOnArrival(zone.Name) ? label : label + " · does not load it by itself (one word in its name)"; + } + } + + /// The faces of the picked zone that can be pulled ("+x", "-z", ...): a zone with a corner sliced off + /// has a side with no face square to its axis, and that side gets no arrow. + public IReadOnlySet SelectedZoneFaces { get; private set; } = new HashSet(); + + private IReadOnlySet FacesOf(string? zone) + { + if (zone == null) return new HashSet(); + try + { + return LoadZones.Open(f => SdsMeshLoader.EnsureExtracted(f), DistrictNames).SquareFaces(zone); + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + Debug.WriteLine("Zone faces error: " + ex); + return new HashSet(); + } + } + + private (Vector3 Min, Vector3 Max)? _preview; + + /// The picked zone's box - the one a gizmo drag has reached while one is in progress. + public (Vector3 Min, Vector3 Max)? SelectedZoneBox => + SelectedZone != null && Zones?.FirstOrDefault(z => z.Name == SelectedZone) is { } zone ? _preview ?? (zone.Min, zone.Max) : null; + + /// Draws the picked zone as the box given instead of its own, until a null puts it back: what a + /// drag shows before it is let go. Nothing is written. + public void PreviewZone((Vector3 Min, Vector3 Max)? box) + { + _preview = box; + if (_host.Rnd != null) BuildZoneBoxes(); + _host.RaiseBoxGizmoChanged(); + } + + /// The zones whose drawn box holds a world point, the smallest first: a click picks the most + /// particular zone there, and the next click at the same spot the next one out. + public IReadOnlyList ZonesHolding(Vector3 point) + { + if (Zones == null) return []; + const float Slack = 0.3f; // a point ON the ground of a zone whose floor is that ground + static float Volume(AreaZone z) => (z.Max.X - z.Min.X) * (z.Max.Y - z.Min.Y) * (z.Max.Z - z.Min.Z); + return [.. Zones.Where(z => point.X >= z.Min.X - Slack && point.X <= z.Max.X + Slack + && point.Y >= z.Min.Y - Slack && point.Y <= z.Max.Y + Slack + && point.Z >= z.Min.Z - Slack && point.Z <= z.Max.Z + Slack) + .OrderBy(Volume).ThenBy(z => z.Name, StringComparer.Ordinal)]; + } + // Zone boxes for the debug overlay: world AABB of the zone + color by its (first) district. public void BuildZoneBoxes() { @@ -122,12 +226,24 @@ public void BuildZoneBoxes() foreach (MapArea a in Areas) if (!hue.ContainsKey(a.BaseName)) hue[a.BaseName] = hue.Count; int count = Math.Max(1, hue.Count); - var boxes = new List<(Vector3 Min, Vector3 Max, Vector4 Color)>(Zones.Count); + var boxes = new List(Zones.Count); + AreaZone? picked = SelectedZone == null ? null : Zones.FirstOrDefault(z => z.Name == SelectedZone); foreach (AreaZone z in Zones) { + if (ReferenceEquals(z, picked)) continue; string? d = z.Districts.Count > 0 ? z.Districts[0] : null; float h = d != null && hue.TryGetValue(d, out int i) ? (float)i / count : 0.5f; - boxes.Add((z.Min, z.Max, HueToColor(h, 0.16f))); + Vector4 colour = HueToColor(h, 1f); + // a zone named as a seam is the kind that loads its districts for a player who appears in it + boxes.Add(new Rendering.Passes.ZoneBox(z.Min, z.Max, new Vector3(colour.X, colour.Y, colour.Z), LoadZones.LoadsOnArrival(z.Name), Picked: false, + Added: !LoadZones.IsShipped(z.Name))); + } + // last, so it is laid over the rest: the accent the editor selects with + if (picked != null) + { + (Vector3 min, Vector3 max) = _preview ?? (picked.Min, picked.Max); + boxes.Add(new Rendering.Passes.ZoneBox(min, max, new Vector3(0.91f, 0.53f, 0.24f), LoadZones.LoadsOnArrival(picked.Name), Picked: true, + Added: !LoadZones.IsShipped(picked.Name))); } _host.Rnd!.SetZoneBoxes(boxes); } diff --git a/src/Illusion/Viewport/ZoneEditController.cs b/src/Illusion/Viewport/ZoneEditController.cs new file mode 100644 index 0000000..6592b5f --- /dev/null +++ b/src/Illusion/Viewport/ZoneEditController.cs @@ -0,0 +1,435 @@ +using System.IO; +using System.Numerics; +using Illusion.Assets; +using Illusion.Assets.Sds; +using Illusion.Assets.World; +using Illusion.Domain; + +namespace Illusion.Viewport; + +/// +/// The picked loading zone as something the tool shelf can move and stretch (BoxGizmo): the box the +/// gizmo stands on, the box a drag has reached, and - when the drag is let go - the zone written to +/// city_univers, as one step of the editor's undo history. The Loading zones window moves a face through the +/// same door, so its moves are in the history too. +/// +/// A zone is not edited as a scene object: it is changed in the working copy of city_univers at once, the +/// archive is queued for a Build, and undo writes it back ( keeps an editor that has +/// that archive loaded - the map editor in Whole map mode - in step). A step records WHAT was changed - which +/// faces, from where to where, or how far the zone was carried - and undoing it takes back that and nothing +/// else: a face moved since by another path stays where that path put it. +/// +/// +internal sealed class ZoneEditController +{ + private const float Same = 0.002f; // faces closer than this stand in the same place + private const float Stale = 0.05f; // the box on disk against the box the layer shows + + private readonly D3DImageHost _host; + private string? _zone; + private (Vector3 Min, Vector3 Max) _start; + private (Vector3 Min, Vector3 Max)? _reached; + + public ZoneEditController(D3DImageHost host) => _host = host; + + /// The picked zone's box while the Loading zones layer is shown; null otherwise. + public (Vector3 Min, Vector3 Max)? Target => _host.ShowZones ? _host.Catalogs.SelectedZoneBox : null; + + /// What a zone edit changed: the faces that moved (each by its own plane, from where to where) or + /// the offset the whole zone was carried by. + private sealed record Change(string Zone, IReadOnlyList<(string Face, float From, float To)> Faces, Vector3 By); + + public void Begin() + { + _zone = _host.Catalogs.SelectedZone; + _reached = null; + if (_host.Catalogs.SelectedZoneBox is { } box) _start = box; + } + + public void Preview(Vector3 min, Vector3 max) + { + if (_zone == null) return; + _reached = (min, max); + _host.Catalogs.PreviewZone(_reached); + } + + public void End(bool commit) + { + string? zone = _zone; + (Vector3 Min, Vector3 Max)? reached = _reached; + _zone = null; + _reached = null; + if (zone == null) return; + // the layer was switched off under the drag (a tool can do that): what cannot be seen is not written + if (!_host.ShowZones) commit = false; + if (!commit || reached is not { } to || (Vector3.Distance(to.Min, _start.Min) < 0.005f && Vector3.Distance(to.Max, _start.Max) < 0.005f)) + { + _host.Catalogs.PreviewZone(null); + return; + } + + // What the drag did, said as what it changed: a box of the same size somewhere else was carried, and + // anything else is the faces that left their places. Said against the box the drag STARTED from - the + // write below checks that the disk still has that box. + Vector3 grew = (to.Max - to.Min) - (_start.Max - _start.Min); + var faces = new List<(string Face, float From, float To)>(); + Vector3 by = Vector3.Zero; + if (MathF.Abs(grew.X) < Same && MathF.Abs(grew.Y) < Same && MathF.Abs(grew.Z) < Same) + { + by = to.Min - _start.Min; + } + else + { + foreach ((string face, float value, float was) in new[] + { + ("+x", to.Max.X, _start.Max.X), ("-x", to.Min.X, _start.Min.X), ("+y", to.Max.Y, _start.Max.Y), + ("-y", to.Min.Y, _start.Min.Y), ("+z", to.Max.Z, _start.Max.Z), ("-z", to.Min.Z, _start.Min.Z), + }) + { + if (MathF.Abs(value - was) >= Same) faces.Add((face, was, value)); + } + } + + string? refused = Write(new Change(zone, faces, by), forward: true, expect: _start, out Change? done); + _host.Catalogs.PreviewZone(null); + if (refused != null || done == null) + { + _host.RaiseNotice($"{zone} was not changed: {refused}", isError: true); + return; + } + _host.History.Push(new ZoneEdit(this, done)); + _host.RaiseNotice($"{zone} changed - Build packs city_univers; every player of a server needs that archive."); + } + + /// + /// Moves one face of a zone to a world coordinate, as the Loading zones window asks for it: written at once, + /// one step of the undo history. Null on success - on a refusal nothing is written. + /// + public string? MoveFace(string zone, string face, float to) + { + string? refused = Write(new Change(zone, [(face.ToLowerInvariant(), float.NaN, to)], Vector3.Zero), forward: true, expect: null, out Change? done); + if (refused != null) return refused; + // the face stands there already: nothing was written, and there is no step to take back + if (done != null) _host.History.Push(new ZoneEdit(this, done)); + return null; + } + + /// + /// What a new zone made where the view looks would be: a name not taken yet and the two districts of the + /// nearest zone that loads districts - the neighbours a zone standing here most likely joins. The name has + /// two words after its number either way: that is what makes the game load the districts for a player who + /// appears inside (). + /// + public (string Name, string District1, string? District2) Suggest() + { + Vector3 at = _host.ViewPoint(); + AreaZone? near = Nearest(at, z => z.Districts.Count >= 2) ?? Nearest(at, _ => true); + string first = near?.Districts[0] ?? _host.Catalogs.DistrictNames.FirstOrDefault() ?? ""; + string? second = near is { Districts.Count: >= 2 } ? near.Districts[1] : null; + var taken = new HashSet((_host.Catalogs.Zones ?? []).Select(z => z.Name), StringComparer.OrdinalIgnoreCase); + for (int number = 900; ; number++) + { + string name = $"AREA{number}_{first.ToUpperInvariant()}_{(second ?? "new").ToUpperInvariant()}"; + if (!taken.Contains(name)) return (name, first, second); + } + } + + private AreaZone? Nearest(Vector3 at, Func wanted) => (_host.Catalogs.Zones ?? []) + .Where(wanted).OrderBy(z => Vector3.DistanceSquared(at, Vector3.Clamp(at, z.Min, z.Max))).FirstOrDefault(); + + /// + /// Makes a new zone where the view looks: a box 120 m square, as tall as its nearest neighbour, written to + /// city_univers at once (the scene, its name table and cityareas.bin), queued for a Build, picked - so the + /// shelf's Move and Scale put it where it belongs - and one step of the undo history. Null on success. + /// + public string? CreateInView(string name, string district1, string? district2) + { + if (!_host.ShowZones) return "switch the Loading zones layer on first"; + Vector3 at = _host.ViewPoint(); + AreaZone? pattern = Nearest(at, z => z.Districts.Count >= 2) ?? Nearest(at, _ => true); + if (pattern == null) return "the scene has no load zone to make it like"; + var min = new Vector3(at.X - 60f, at.Y - 60f, pattern.Min.Z); + var max = new Vector3(at.X + 60f, at.Y + 60f, pattern.Max.Z); + var made = new Created(name.Trim(), pattern.Name, min, max, district1.Trim(), string.IsNullOrWhiteSpace(district2) ? null : district2.Trim()); + if (Structure(made, create: true) is { } refused) return refused; + _host.History.Push(new ZoneCreateEdit(this, made)); + _host.Catalogs.SelectZone(made.Name); + _host.RaiseNotice($"{made.Name} made - Move puts it in place, Scale pulls its faces; Build packs city_univers."); + return null; + } + + /// + /// Makes a new zone between two world corners, like an existing one - what the zone_create tool asks for: + /// written to city_univers at once, queued for a Build, one step of the undo history. Null on success. + /// + public string? Create(string name, string like, Vector3 min, Vector3 max, string district1, string? district2) + { + var made = new Created(name.Trim(), like, min, max, district1.Trim(), string.IsNullOrWhiteSpace(district2) ? null : district2.Trim()); + if (Structure(made, create: true) is { } refused) return refused; + _host.History.Push(new ZoneCreateEdit(this, made)); + _host.RaiseNotice($"{made.Name} made - Build packs city_univers; every player of a server needs that archive."); + return null; + } + + /// + /// Takes a zone that was added to the game's own out of city_univers again - written at once, queued for a + /// Build, one step of the undo history (undo puts it back as it was). Null on success. + /// + public string? Remove(string name) + { + if (LoadZones.IsShipped(name)) return $"{name} came with the game - only a zone that was added is taken out"; + var gone = new Removed(name); + if (Unstructure(gone, remove: true) is { } refused) return refused; + _host.History.Push(new ZoneRemoveEdit(this, gone)); + _host.RaiseNotice($"{name} taken out - Build packs city_univers; every player of a server needs that archive."); + return null; + } + + private sealed record Removed(string Name) + { + // The three files as they stood with the zone and as taking it out left them: undo puts the first back + // while the files are still the second, and refuses once something else has written to them. + public IReadOnlyDictionary? Before { get; set; } + public IReadOnlyDictionary? After { get; set; } + } + + // Takes a zone out of the scene on disk, or - undoing that - puts the files back as they stood with it. + private string? Unstructure(Removed zone, bool remove) + { + var archive = new FileInfo(MafiaEnvironment.CityUniversSds); + if (ZoneWrites.StructureBlocked(archive) is { } blocked) return blocked; + LoadZones zones; + try + { + zones = LoadZones.Open(f => SdsMeshLoader.EnsureExtracted(f), _host.Catalogs.DistrictNames, archive, forNewZones: true); + IReadOnlyDictionary now = zones.StructureFiles(); + if (remove) + { + if (zones.Delete(zone.Name) is { } refused) return refused; + zones.Save(); + zone.Before = now; + zone.After = zones.StructureFiles(); + } + else if (zone.Before != null && zone.After != null && SameFiles(now, zone.After)) + { + LoadZones.Restore(zone.Before); + } + else + { + return "city_univers was written to since the zone was taken out - make the zone again instead"; + } + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + return "city_univers could not be written: " + ex.Message; + } + if (remove && _host.Catalogs.SelectedZone == zone.Name) _host.Catalogs.SelectZone(null); + try + { + ZoneWrites.Landed(zones, zone.Name); + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + _host.RaiseNotice($"{zone.Name} is written, but the editor could not be brought up to date: {ex.Message}", isError: true); + } + return null; + } + + private sealed class ZoneRemoveEdit(ZoneEditController owner, Removed zone) : IEditAction + { + public void Undo() => Apply(remove: false); + + public void Redo() => Apply(remove: true); + + private void Apply(bool remove) + { + if (owner.Unstructure(zone, remove) is { } refused) + { + throw new EditRefusedException($"{zone.Name} was not {(remove ? "taken out again" : "put back")}: {refused}"); + } + } + } + + private sealed record Created(string Name, string Like, Vector3 Min, Vector3 Max, string District1, string? District2) + { + // The three files the zone was written into, as they stood before it was made and as making it left + // them. Taking a zone out rebuilds the name table, and a rebuilt table is the same names in another + // order: while the files are still what the making left, undo puts back what they held before - byte + // for byte - and only when something else has written to them since is the zone taken out of the + // scene as it now stands. + public IReadOnlyDictionary? Before { get; set; } + public IReadOnlyDictionary? After { get; set; } + } + + private static bool SameFiles(IReadOnlyDictionary a, IReadOnlyDictionary b) => + a.Count == b.Count && a.All(file => b.TryGetValue(file.Key, out byte[]? other) + && (file.Value == null ? other == null : other != null && file.Value.AsSpan().SequenceEqual(other))); + + // A zone made, moved, moved back and taken out: the scene is what it was before the zone, and the name table + // Save has just rebuilt says what the one from before said, in another order. The one from before is put + // back - when nothing but the making has written the name table since (`was` is the files as the undo found + // them). The zone is out either way: a table that cannot be put back is left as rebuilt. + private static void KeepNameTable(LoadZones zones, Created zone, IReadOnlyDictionary was) + { + if (zone.Before == null || zone.After == null || zones.NameTableFile is not { } names) return; + if (!zone.Before.TryGetValue(names, out byte[]? before) || before == null) return; + if (!zone.After.TryGetValue(names, out byte[]? made) || made == null) return; + if (!was.TryGetValue(names, out byte[]? found) || found == null || !found.AsSpan().SequenceEqual(made)) return; + if (!zone.Before.TryGetValue(zones.SceneFile, out byte[]? scene) || scene == null) return; + try + { + if (File.ReadAllBytes(zones.SceneFile).AsSpan().SequenceEqual(scene)) LoadZones.Restore(new Dictionary { [names] = before }); + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) + { + // the rebuilt table stays + } + } + + // Adds a zone to the scene on disk, or takes it out again (ZoneWrites.StructureBlocked says when not). + private string? Structure(Created zone, bool create) + { + var archive = new FileInfo(MafiaEnvironment.CityUniversSds); + if (ZoneWrites.StructureBlocked(archive) is { } blocked) return blocked; + LoadZones zones; + try + { + zones = LoadZones.Open(f => SdsMeshLoader.EnsureExtracted(f), _host.Catalogs.DistrictNames, archive, forNewZones: true); + IReadOnlyDictionary now = zones.StructureFiles(); + if (create) + { + if (zones.Create(zone.Name, zone.Like, zone.Min, zone.Max, zone.District1, zone.District2) is { } refused) return refused; + zones.Save(); + zone.Before = now; + zone.After = zones.StructureFiles(); + } + else if (zone.Before != null && zone.After != null && SameFiles(now, zone.After)) + { + LoadZones.Restore(zone.Before); + } + else + { + if (zones.Delete(zone.Name) is { } refused) return refused; + zones.Save(); + KeepNameTable(zones, zone, now); + } + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + return "city_univers could not be written: " + ex.Message; + } + if (!create && _host.Catalogs.SelectedZone == zone.Name) _host.Catalogs.SelectZone(null); + try + { + ZoneWrites.Landed(zones, zone.Name); + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + _host.RaiseNotice($"{zone.Name} is written, but the editor could not be brought up to date: {ex.Message}", isError: true); + } + return null; + } + + private sealed class ZoneCreateEdit(ZoneEditController owner, Created zone) : IEditAction + { + public void Undo() => Apply(create: false); + + public void Redo() => Apply(create: true); + + private void Apply(bool create) + { + if (owner.Structure(zone, create) is { } refused) + { + throw new EditRefusedException($"{zone.Name} was not {(create ? "made again" : "taken out")}: {refused}"); + } + } + } + + // Carries a change out (forward) or takes it back, in a copy of the scene read from disk, and saves it. A + // face goes to the change's To (forward) or back to its From; the From of a change made forward is read off + // the plane as it stands, and comes back in `done` - that is what an undo returns the face to, not the box + // drawn around the planes. `expect` is the box the caller believes the zone has: when the disk says + // otherwise the zone was changed behind the layer's back, and nothing is written. + private string? Write(Change change, bool forward, (Vector3 Min, Vector3 Max)? expect, out Change? done) + { + done = null; + LoadZones written; + try + { + LoadZones zones = LoadZones.Open(f => SdsMeshLoader.EnsureExtracted(f), _host.Catalogs.DistrictNames); + if (!zones.Volumes.TryGetValue(change.Zone, out Formats.Frames.ObjectTypes.FrameObjectArea? volume)) + { + return "there is no such zone in city_univers any more"; + } + if (expect is { } box) + { + (Vector3 Min, Vector3 Max) now = LoadZones.WorldBox(volume); + if (Vector3.Distance(now.Min, box.Min) > Stale || Vector3.Distance(now.Max, box.Max) > Stale) + { + _host.Catalogs.ReloadZones(); + return "it was changed elsewhere since the layer was drawn - the layer has been read again, pull it once more"; + } + } + if (ZoneWrites.Blocked(zones, change.Zone) is { } blocked) return blocked; + + var moved = new List<(string Face, float From, float To)>(); + if (change.Faces.Count == 0 && change.By.Length() < Same) + { + return null; // carried nowhere + } + if (change.Faces.Count == 0) + { + // the same size somewhere else: the zone is moved by its own place, whatever its shape + if (zones.Move(change.Zone, forward ? change.By : -change.By) is { } refused) return refused; + } + else + { + // only the faces that were pulled: a zone with a slanted side has no face to rewrite there + foreach ((string face, float from, float to) in change.Faces) + { + if (zones.MoveFace(change.Zone, face, forward ? to : from, out LoadZoneFaceMove? move) is { } refused) return refused; + moved.Add((face, forward ? move!.From : from, to)); + } + // every face stands where it was asked to stand: the scene is not rewritten, the archive is + // not queued for a Build and no step goes into the history + if (moved.All(m => MathF.Abs(m.From - m.To) < Same)) return null; + } + zones.Save(); + written = zones; + done = new Change(change.Zone, moved, change.By); + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + return "city_univers could not be written: " + ex.Message; + } + + // The zone is written. What follows only tells the editors about it; a failure there is not a failure + // of the write, and reporting it as one would leave a change on disk with no step to take it back. + try + { + ZoneWrites.Landed(written, change.Zone); + } + catch (Exception ex) when (ZoneWrites.IsFileTrouble(ex)) + { + _host.RaiseNotice($"{change.Zone} is written, but the editor could not be brought up to date: {ex.Message}", isError: true); + } + return null; + } + + private sealed class ZoneEdit(ZoneEditController owner, Change change) : IEditAction + { + public void Undo() => Apply(forward: false); + + public void Redo() => Apply(forward: true); + + // A step that cannot be taken says so and stays where it was in the history (EditHistory leaves its + // stacks alone when a step throws): reporting it done would leave the zone moved with no way back. + private void Apply(bool forward) + { + if (owner.Write(change, forward, expect: null, out _) is { } refused) + { + throw new EditRefusedException($"{change.Zone} was not {(forward ? "changed again" : "put back")}: {refused}"); + } + } + } +} diff --git a/src/Illusion/Viewport/ZoneWrites.cs b/src/Illusion/Viewport/ZoneWrites.cs new file mode 100644 index 0000000..5f7652c --- /dev/null +++ b/src/Illusion/Viewport/ZoneWrites.cs @@ -0,0 +1,137 @@ +using System.IO; +using System.Windows; +using Illusion.Assets.Adapters; +using Illusion.Assets.Sds; +using Illusion.Assets.World; +using Illusion.Scene; +using Illusion.Views; + +namespace Illusion.Viewport; + +/// +/// What every writer of a loading zone has to do around the write itself - the viewport's gizmo, the Loading +/// zones window and the zone_move_face tool alike. +/// +/// A zone is changed in a copy of the scene read from disk and that copy is written whole. An editor can hold the +/// same archive as a document of its own (the map editor in Whole map mode loads city_univers; the resource +/// editor can be handed it), and that editor writes ITS scene whole on its next save. So before a write the +/// editor's copy of the zone must be what the disk has - or the write would be made from a state the editor is +/// about to replace - and after it the editor's copy is brought in step, or its next save would put the zone back. +/// +/// +internal static class ZoneWrites +{ + /// What can go wrong reading or writing a working copy, as opposed to a fault in the program: a + /// file that is locked, missing or cut short, a contents list that is not XML. + public static bool IsFileTrouble(Exception ex) => + ex is IOException or UnauthorizedAccessException or InvalidDataException or Formats.FileFormatException + or System.Xml.XmlException or InvalidOperationException; + + private static IEnumerable Viewports() + { + if (Application.Current is not { } app) yield break; + foreach (Window window in app.Windows) + { + if (window is MainWindow map) yield return map.Viewport; + else if (window is ResourceEditorWindow resources) yield return resources.Stage; + } + } + + // The row of an editor's tree that stands for a volume of the given document. + private static SceneNode? NodeOf(D3DImageHost host, SceneDocumentAdapter document, string zone) + { + var stack = new Stack(host.Tree.Roots); + while (stack.Count > 0) + { + SceneNode node = stack.Pop(); + if (node.Source is FrameNodeAdapter { Frame: Formats.Frames.ObjectTypes.FrameObjectArea area } adapter + && ReferenceEquals(adapter.Document, document) + && string.Equals(area.Name?.ToString(), zone, StringComparison.OrdinalIgnoreCase)) + { + return node; + } + foreach (SceneNode child in node.Children) stack.Push(child); + } + return null; + } + + private static IEnumerable Holding(FileInfo archive) + { + foreach (D3DImageHost host in OpenArchives.HoldersOf(archive).OfType()) + { + if (host.Staged(archive) is { } document) yield return document; + } + } + + /// + /// Why a zone cannot be written now, or null: an open editor holds the archive and its own copy of this + /// zone is not what is on disk (the volume was moved there as an object and not saved yet). Asked BEFORE the + /// zone is changed in - it compares what was read from disk. + /// + public static string? Blocked(LoadZones zones, string zone) + { + // An editor that is still loading the archive (Whole map just switched on) has read its scene, or is + // about to, and does not hold it yet: a write made now would be missing from the scene that arrives, + // with nothing to carry it in. + if (Viewports().Any(host => host.Streamer.IsLoading(zones.Archive))) + { + return "city_univers is still being loaded into an editor - try again when it is in"; + } + foreach (SceneDocumentAdapter document in Holding(zones.Archive)) + { + if (!zones.InStepWith(document, zone)) + { + return $"{zone} has changes in an open editor that are not saved yet - save there first"; + } + } + return null; + } + + /// + /// Why a zone cannot be ADDED to an archive or taken out of it now, or null. Not while an editor holds the + /// archive as a document (Whole map, or the resource editor handed it) or is still loading it: a volume + /// cannot be carried into a scene that is already loaded the way a moved one is, and that editor's next + /// save would write its own scene - without the volume, or with it - over this one. + /// + public static string? StructureBlocked(FileInfo archive) + { + if (Viewports().Any(host => host.Streamer.IsLoading(archive))) + { + return "city_univers is still being loaded into an editor - try again when it is in"; + } + return OpenArchives.HoldersOf(archive).Count > 0 + ? "city_univers is open in an editor (Whole map, or the resource editor) - a zone is added or taken out with a single district loaded" + : null; + } + + /// + /// After was saved with changed: every editor holding the + /// archive gets the zone as it now is, and every viewport queues the archive for a Build and reads its + /// Loading zones layer again. + /// + public static void Landed(LoadZones zones, string zone) + { + foreach (D3DImageHost host in OpenArchives.HoldersOf(zones.Archive).OfType()) + { + if (host.Staged(zones.Archive) is not { } document || !zones.MirrorInto(document, zone)) continue; + // The volume is a frame of that editor's scene, and the editor shows it: its glyph, the outline of + // a selection, the numbers of the property panel. Those are caches of the frame, and the panel + // writes its cache back on the next number typed - the old place, over the new one. + if (NodeOf(host, document, zone) is { } node) host.Editing.CommitNodeTransform(node); + host.RaiseSelectionPropertiesChanged(); + } + + bool main = string.Equals(zones.Archive.FullName, new FileInfo(Assets.MafiaEnvironment.CityUniversSds).FullName, + StringComparison.OrdinalIgnoreCase); + IReadOnlyList holders = OpenArchives.HoldersOf(zones.Archive); + foreach (D3DImageHost host in Viewports()) + { + // the map editor builds the city's archives; a resource editor only the one it was handed + if (!host.IsMapViewport && !holders.Contains(host)) continue; + host.MarkArchiveModified(zones.Archive); + // the layer draws the base game's zones: those are the ones in force + if (main && host.IsMapViewport) host.Catalogs.ReloadZones(); + host.RaiseDirtyChanged(); + } + } +} diff --git a/src/Illusion/Views/ArchiveThumb.cs b/src/Illusion/Views/ArchiveThumb.cs new file mode 100644 index 0000000..7170f06 --- /dev/null +++ b/src/Illusion/Views/ArchiveThumb.cs @@ -0,0 +1,246 @@ +using System.Runtime.CompilerServices; +using System.Windows; +using System.Windows.Controls; +using System.Windows.Media; +using System.Windows.Threading; +using Illusion.Assets.Library; +using Illusion.Viewport; + +namespace Illusion.Views; + +/// +/// Puts an archive's picture on an of a content browser tile: +/// v:ArchiveThumb.Entry="{Binding}". The picture arrives later than the tile — a kept one within a tick +/// or two, a new one once its archive has been read and drawn, one archive at a time. +/// +/// Only tiles that are on screen have their archives read. Reading one the first time extracts it — a working +/// copy of tens of megabytes — and a folder of a hundred cars opened for a glance is not a request to extract +/// a hundred cars; the rest are read when they are scrolled to. The reading itself runs off the UI thread +/// (), so the pane and the viewport behind it stay alive while it +/// does; only the drawing happens here. +/// +/// +/// The entry is a plain catalog row with nowhere to hold a picture and no way to say it changed, which is why +/// this is an attached property on the Image rather than a binding to the row. +/// +/// +public static class ArchiveThumb +{ + public static readonly DependencyProperty EntryProperty = DependencyProperty.RegisterAttached( + "Entry", typeof(LibraryEntry), typeof(ArchiveThumb), new PropertyMetadata(null, OnEntryChanged)); + + public static LibraryEntry? GetEntry(DependencyObject o) => (LibraryEntry?)o.GetValue(EntryProperty); + + public static void SetEntry(DependencyObject o, LibraryEntry? value) => o.SetValue(EntryProperty, value); + + // What this run already knows about an archive: its picture — null for one with nothing to show, which is + // not tried again — and the state of the archive that picture was made from. + private sealed class Slot(string key, ImageSource? picture) + { + public readonly string Key = key; + public readonly ImageSource? Picture = picture; + } + + // Held by the catalog row, weakly: a catalog that is rebuilt (every time a resource window opens) takes its + // rows' pictures with it, where a dictionary keyed by the row kept every catalog's pictures for good. + private static readonly ConditionalWeakTable Known = new(); + private static readonly List Waiting = new(); + + // A tile waiting for its picture. A tile that left the pane (another folder was opened) keeps its Image + // alive for a while and never says so — it is simply no longer loaded, and after a second of that it is + // dropped rather than asked about for ever. One that comes back is queued again by its Loaded event. + private sealed class Pending(Image image) + { + public readonly WeakReference Image = new(image); + public int Unloaded; + } + + private static readonly TimeSpan Busy = TimeSpan.FromMilliseconds(20); + private static readonly TimeSpan Idle = TimeSpan.FromMilliseconds(250); // only tiles off screen are waiting + + private static ArchiveThumbnailRenderer? _renderer; + private static DispatcherTimer? _timer; + + // The one archive being read, on a pool thread, and whose it is. + private static Task? _reading; + private static LibraryEntry? _readingFor; + + private static void OnEntryChanged(DependencyObject d, DependencyPropertyChangedEventArgs e) + { + if (d is not Image image) return; + image.Source = null; + image.Loaded -= OnLoaded; + if (e.NewValue is not LibraryEntry entry || !ArchiveThumbnailRenderer.Draws(entry.Resource)) return; + image.Loaded += OnLoaded; + if (Known.TryGetValue(entry, out Slot? slot)) + { + // Checked each time a tile is made for it: an archive rebuilt since its picture was taken is a + // different picture, and the row it hangs on is the same row. + if (slot.Key == ArchiveThumbnailRenderer.KeyOf(entry)) + { + image.Source = slot.Picture; + return; + } + Known.Remove(entry); + } + Queue(image); + } + + private static void OnLoaded(object sender, RoutedEventArgs e) + { + if (sender is Image { Source: null } image && GetEntry(image) is { } entry && !Known.TryGetValue(entry, out _)) + { + Queue(image); + } + } + + private static void Queue(Image image) + { + foreach (Pending pending in Waiting) + { + if (pending.Image.TryGetTarget(out Image? queued) && ReferenceEquals(queued, image)) return; + } + Waiting.Add(new Pending(image)); + DispatcherTimer timer = Timer(); + timer.Interval = Busy; + timer.Start(); + } + + private static DispatcherTimer Timer() + { + if (_timer != null) return _timer; + _timer = new DispatcherTimer(DispatcherPriority.Background) { Interval = Busy }; + _timer.Tick += (_, _) => Tick(); + if (Application.Current is { } app) + { + app.Exit += (_, _) => + { + _timer.Stop(); + _renderer?.Dispose(); + _renderer = null; + }; + } + return _timer; + } + + private static void Tick() + { + try + { + Work(); + } + catch (Exception ex) when (ex is not OutOfMemoryException) + { + // An exception out of a timer tick has nothing above it: it ends the program, with whatever is + // unsaved in the editor. Pictures are not worth that. The queue is dropped rather than kept — + // what threw would throw again twenty milliseconds from now. + Waiting.Clear(); + _timer?.Stop(); + } + } + + private static void Work() + { + var live = new List<(Image Image, LibraryEntry Entry)>(); + for (int i = Waiting.Count - 1; i >= 0; i--) + { + Pending pending = Waiting[i]; + if (!pending.Image.TryGetTarget(out Image? image) || GetEntry(image) is not { } entry) + { + Waiting.RemoveAt(i); + continue; + } + if (Known.TryGetValue(entry, out Slot? done)) + { + image.Source = done.Picture; + Waiting.RemoveAt(i); + continue; + } + if (image.IsLoaded) + { + pending.Unloaded = 0; + live.Add((image, entry)); + } + else if (++pending.Unloaded > 50) + { + Waiting.RemoveAt(i); + } + } + + // An archive that has been read: draw it. Its tile takes the picture at the top of the next tick. + if (_reading is { IsCompleted: true } read && _readingFor is { } readFor) + { + _reading = null; + _readingFor = null; + ImageSource? picture = read.IsCompletedSuccessfully + ? (_renderer ??= new ArchiveThumbnailRenderer()).Draw(read.Result) + : null; + Remember(readFor, picture); + return; + } + + if (Waiting.Count == 0) + { + if (_reading == null) _timer?.Stop(); + return; + } + if (live.Count == 0) return; + live.Reverse(); // back in the order the tiles were made + + // Kept pictures cost a file read each: a handful per tick, the ones on screen first. + var onScreen = live.Where(l => OnScreen(l.Image)).ToList(); + int handed = 0; + foreach ((Image image, LibraryEntry entry) in onScreen.Concat(live)) + { + if (handed >= 10) break; + if (Known.TryGetValue(entry, out _)) continue; + if (!ArchiveThumbnailRenderer.TryKept(entry, out ImageSource? kept)) continue; + Remember(entry, kept); + image.Source = kept; + handed++; + } + if (handed > 0 || _reading != null) + { + _timer!.Interval = Busy; + return; + } + + // Nothing kept is left to hand out: read ONE archive — one that is being looked at. + if (onScreen.Count == 0) + { + _timer!.Interval = Idle; + return; + } + _renderer ??= new ArchiveThumbnailRenderer(); + if (!_renderer.Usable) + { + Waiting.Clear(); // no GPU to draw on: reading archives would be for nothing + _timer!.Stop(); + return; + } + LibraryEntry next = onScreen[0].Entry; + _readingFor = next; + _reading = Task.Run(() => ArchiveThumbnailRenderer.Stage(next)); + _timer!.Interval = Busy; + } + + private static void Remember(LibraryEntry entry, ImageSource? picture) => + Known.AddOrUpdate(entry, new Slot(ArchiveThumbnailRenderer.KeyOf(entry), picture)); + + internal static bool OnScreen(Image image) + { + if (!image.IsVisible) return false; + DependencyObject? parent = VisualTreeHelper.GetParent(image); + while (parent != null && parent is not ScrollViewer) parent = VisualTreeHelper.GetParent(parent); + if (parent is not ScrollViewer scroller) return true; + try + { + Point at = image.TransformToAncestor(scroller).Transform(new Point(0, 0)); + return at.Y > -120 && at.Y < scroller.ViewportHeight + 20; + } + catch (InvalidOperationException) + { + return false; // not in that tree any more + } + } +} diff --git a/src/Illusion/Views/BuildWindow.xaml b/src/Illusion/Views/BuildWindow.xaml new file mode 100644 index 0000000..b04d390 --- /dev/null +++ b/src/Illusion/Views/BuildWindow.xaml @@ -0,0 +1,121 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +