Add matching session caching and native codegen improvements

- Introduced a new matching session caching mechanism in the Lua matching module to enhance performance with native code generation. - Rewrote the Lua matching algorithm to utilize session-based caching, reducing redundant computations and improving efficiency. - Updated Rust implementations to include session caching, ensuring consistency across both Lua and Rust. - Enhanced the `hydrate` function to support session management during instance hydration. - Added comprehensive tests to validate the new caching mechanism and its integration with existing functionality. These changes significantly improve the performance and maintainability of the matching process in the application.
2026-10-23 05:51:39 +00:00 · 2026-02-18 04:55:31 -06:00 · 2026-02-18 04:55:31 -06:00 · b39c01e5e9
commit b39c01e5e9
parent 307ac81aec
14 changed files with 1674 additions and 461 deletions
--- a/.cursor/plans/matching_session_caching_cbfa7bda.plan.md
+++ b/.cursor/plans/matching_session_caching_cbfa7bda.plan.md
@ -0,0 +1,609 @@
+---
+name: Matching session caching
+overview: Rewrite the Lua matching module for native codegen performance (--!strict, --!native, --!optimize 2) with session-based caching, clean 2-function mutual recursion, concrete types, and local functions for inlining. Add session caching to both Rust implementations for consistency.
+todos:
+  - id: lua-trueEquals
+    content: "Rewrite trueEquals.lua: --!strict/--!native/--!optimize 2, single typeof dispatch, dedicated helper functions per type (no temp arrays), adopt DeepEqualsPureUnsafe pattern for table comparison (rawlen/rawget/next, no checkedKeys set), keep fuzzy equality + null-ref + EnumItem cross-type"
+    status: completed
+  - id: lua-rewrite
+    content: "Rewrite matching.lua: --!strict/--!native/--!optimize 2, all functions as local function in dependency order (one forward declare for mutual recursion), 2-function mutual recursion (matchChildren + computePairCost), session caching, totalCost return, O(n) removeMatched, remove pcall in grouping"
+    status: completed
+  - id: lua-hydrate-threading
+    content: Thread session through hydrate.lua, Reconciler/init.lua, and ServeSession.lua (create session before hydrate call at line 628)
+    status: completed
+  - id: lua-verify-codegen
+    content: Run luau-compile --remarks on trueEquals.lua and matching.lua to verify inlining and native codegen quality, iterate until clean
+    status: completed
+  - id: lua-test
+    content: Build plugin, test Lua changes in Studio (hydrate + matching works correctly with session caching) before proceeding to Rust
+    status: completed
+  - id: rust-forward-session
+    content: "AFTER Lua passes: Add MatchingSession to src/snapshot/matching.rs with cost_cache, add total_cost to ForwardMatchResult, thread session through all functions and patch_compute.rs"
+    status: completed
+  - id: rust-syncback-session
+    content: "AFTER Lua passes: Add MatchingSession to src/syncback/matching.rs with cost_cache, add total_cost to MatchResult, thread session through all functions and dir.rs"
+    status: completed
+  - id: rust-test-updates
+    content: Update tests/tests/matching_fixtures.rs, run cargo test to verify Rust changes
+    status: completed
+isProject: false
+---
+
+# Matching Session Caching + Native Codegen Rewrite
+
+## Problem
+
+1. **Redundant computation**: Scoring recurses into children, then hydrate re-calls matchChildren on the same children. The intermediate `_computeChildrenCost` calls matchChildren, iterates matched pairs calling `_computeChangeCount`, which recurses again -- all discarded after scoring.
+2. **Poor native codegen structure**: All functions are on the `Matching` table (table lookup per call, uninlineable). No type annotations. No `--!strict`/`--!native`/`--!optimize 2`. A `pcall` with closure allocation in the hot grouping loop.
+3. **4-function recursion chain** (`matchChildren -> _computeChildrenCost -> _computeChangeCount -> _computeChildrenCost -> matchChildren`) is complex and hard for both the optimizer and the developer to reason about.
+
+## Design
+
+### Conceptual Model
+
+`matchChildren` is fundamentally `computeSubtreeCost` -- it finds the cost-minimizing pairing of children and returns the total cost. The actual match assignments are a bonus output. The 2-function mutual recursion reflects this:
+
+- `**matchChildren(session, vChildren, sChildren, virtualInstances, parentVId, parentSInst)`**: Given two child sets, find optimal pairing. Returns `MatchResult` with `totalCost`.
+- `**computePairCost(session, virtualId, studioInstance, virtualInstances, bestSoFar)`**: Given one virtual and one studio instance, compute total cost including subtree. Calls `matchChildren` for children cost.
+
+`_computeChildrenCost` and `_computeChangeCount` are eliminated. Their logic is absorbed into these two functions.
+
+### MatchingSession
+
+```lua
+type MatchingSession = {
+    matchCache: { [string]: { [Instance]: MatchResult } },
+    costCache: { [string]: { [Instance]: number } },
+}
+```
+
+- **matchCache**: keyed by `(parentVirtualId, parentStudioInstance)`. Parent uniquely determines children set.
+- **costCache**: keyed by `(virtualId, studioInstance)`. Only stored when `cost < bestSoFar` (exact, not early-exited).
+
+Caching replaces the need for a depth limit: each unique pair/children-set is computed at most once.
+
+### bestSoFar Interaction
+
+- **costCache**: only store when computation was complete (`cost < bestSoFar`). If cached, return immediately regardless of caller's `bestSoFar` (the cached value IS the exact cost).
+- **matchCache**: `matchChildren` always completes (no `bestSoFar` on it), so results are always cacheable.
+
+---
+
+## Lua Plugin Rewrite: [matching.lua](plugin/src/Reconciler/matching.lua)
+
+### Module Header
+
+```lua
+--!strict
+--!native
+--!optimize 2
+```
+
+### Type Definitions (all concrete, minimal `any`)
+
+```lua
+type MatchPair = {
+    virtualId: string,
+    studioInstance: Instance,
+}
+
+type MatchResult = {
+    matched: { MatchPair },
+    unmatchedVirtual: { string },
+    unmatchedStudio: { Instance },
+    totalCost: number,
+}
+
+type MatchingSession = {
+    matchCache: { [string]: { [Instance]: MatchResult } },
+    costCache: { [string]: { [Instance]: number } },
+}
+
+type VCache = {
+    props: { [string]: any },
+    extraProps: { string }?,
+    tags: { [string]: boolean }?,
+    attrs: { [string]: any }?,
+    childCount: number,
+}
+
+type SCache = {
+    instance: Instance,
+    props: { [string]: any },
+    tags: { [string]: boolean }?,
+    attrs: { [string]: any }?,
+    children: { Instance },
+    childCount: number,
+}
+```
+
+### All Functions as `local function` in Dependency Order
+
+Every function is a `local function` defined in dependency order. No `Matching.foo()` table lookups. One forward declaration for the mutual recursion pair.
+
+```lua
+local UNMATCHED_PENALTY: number = 10000
+
+local function newSession(): MatchingSession
+    return { matchCache = {}, costCache = {} }
+end
+
+local function cacheVirtual(vInst: any, classKeys: any): VCache
+    -- same logic as current _cacheVirtual, with return type annotation
+end
+
+local function cacheStudio(studioInstance: Instance, classKeys: any, extraProps: { string }?): SCache
+    -- same logic as current _cacheStudio
+    -- ADDITION: store sCache.instance = studioInstance
+end
+
+local function countOwnDiffs(vCache: VCache, sCache: SCache, classKeys: any): number
+    -- same logic as current _countOwnDiffsCached
+end
+
+local function removeMatched(arr: { any }, matchedIndices: { [number]: boolean }): ()
+    -- O(n) in-place compaction instead of O(n^2) table.remove loop
+    local write = 1
+    for read = 1, #arr do
+        if not matchedIndices[read] then
+            arr[write] = arr[read]
+            write += 1
+        end
+    end
+    for i = write, #arr do
+        arr[i] = nil
+    end
+end
+```
+
+### Single Forward Declaration for Mutual Recursion
+
+`computePairCost` calls `matchChildren` and vice versa. Define `matchChildren` first (forward-declared), then `computePairCost` as a `local function`, then assign `matchChildren`:
+
+```lua
+local matchChildren  -- forward declare (matchChildren is defined after computePairCost)
+
+local function computePairCost(...)
+    -- can reference matchChildren (captured as upvalue, assigned before first call)
+end
+
+matchChildren = function(...)
+    -- can reference computePairCost (already defined above)
+end
+```
+
+### computePairCost (replaces `_computeChangeCount` + `_computeChildrenCost`)
+
+```lua
+computePairCost = function(
+    session: MatchingSession,
+    virtualId: string,
+    studioInstance: Instance,
+    virtualInstances: { [string]: any },
+    bestSoFar: number
+): number
+    -- 1. Cache lookup
+    local vc = session.costCache[virtualId]
+    if vc then
+        local cached = vc[studioInstance]
+        if cached ~= nil then return cached end
+    end
+
+    local vInst = virtualInstances[virtualId]
+    if not vInst then return UNMATCHED_PENALTY end
+
+    -- 2. Own diffs
+    local classKeys = RbxDom.getClassComparisonKeys(vInst.ClassName)
+    local vCache = cacheVirtual(vInst, classKeys)
+    local sCache = cacheStudio(studioInstance, classKeys, vCache.extraProps)
+    local cost: number = countOwnDiffs(vCache, sCache, classKeys)
+    if cost >= bestSoFar then return cost end
+
+    -- 3. Children cost (direct matchChildren call, no intermediate function)
+    local vChildren = vInst.Children
+    local studioKids = sCache.children
+
+    if (not vChildren or #vChildren == 0) and #studioKids == 0 then
+        -- leaf: no children on either side
+    elseif not vChildren or #vChildren == 0 then
+        cost += #studioKids * UNMATCHED_PENALTY
+    elseif #studioKids == 0 then
+        for _, childId in vChildren do
+            if virtualInstances[childId] then cost += UNMATCHED_PENALTY end
+        end
+    else
+        local validVChildren: { string } = {}
+        for _, childId in vChildren do
+            if virtualInstances[childId] then
+                table.insert(validVChildren, childId)
+            end
+        end
+        local childResult = matchChildren(
+            session, validVChildren, studioKids, virtualInstances,
+            virtualId, studioInstance
+        )
+        cost += childResult.totalCost
+    end
+
+    -- 4. Cache if exact (no early exit)
+    if cost < bestSoFar then
+        if not session.costCache[virtualId] then
+            session.costCache[virtualId] = {}
+        end
+        session.costCache[virtualId][studioInstance] = cost
+    end
+
+    return cost
+end
+```
+
+### matchChildren (restructured, returns totalCost)
+
+```lua
+matchChildren = function(
+    session: MatchingSession,
+    virtualChildren: { string },
+    studioChildren: { Instance },
+    virtualInstances: { [string]: any },
+    parentVirtualId: string?,
+    parentStudioInstance: Instance?
+): MatchResult
+    -- 1. Cache lookup
+    if parentVirtualId and parentStudioInstance then
+        local pc = session.matchCache[parentVirtualId]
+        if pc then
+            local cached = pc[parentStudioInstance]
+            if cached then return cached end
+        end
+    end
+
+    local matched: { MatchPair } = {}
+    local remainingVirtual: { string } = table.clone(virtualChildren)
+    local remainingStudio: { Instance } = table.clone(studioChildren)
+
+    -- 2. Group by (Name, ClassName) -- NO pcall, direct property access
+    local vByKey: { [string]: { number } } = {}
+    for i, id in remainingVirtual do
+        local vInst = virtualInstances[id]
+        if vInst then
+            local key = vInst.Name .. "\0" .. vInst.ClassName
+            local group = vByKey[key]
+            if not group then
+                group = {}
+                vByKey[key] = group
+            end
+            table.insert(group, i)
+        end
+    end
+
+    local sByKey: { [string]: { number } } = {}
+    for i, inst in remainingStudio do
+        local key = inst.Name .. "\0" .. inst.ClassName
+        local group = sByKey[key]
+        if not group then
+            group = {}
+            sByKey[key] = group
+        end
+        table.insert(group, i)
+    end
+
+    -- 3. 1:1 instant match
+    local matchedV: { [number]: boolean } = {}
+    local matchedS: { [number]: boolean } = {}
+
+    for key, vIndices in vByKey do
+        local sIndices = sByKey[key]
+        if sIndices and #vIndices == 1 and #sIndices == 1 then
+            local vi, si = vIndices[1], sIndices[1]
+            if not matchedV[vi] and not matchedS[si] then
+                table.insert(matched, {
+                    virtualId = remainingVirtual[vi],
+                    studioInstance = remainingStudio[si],
+                })
+                matchedV[vi] = true
+                matchedS[si] = true
+            end
+        end
+    end
+
+    removeMatched(remainingVirtual, matchedV)
+    removeMatched(remainingStudio, matchedS)
+
+    -- 4. Ambiguous groups (same structure as before but using computePairCost)
+    if #remainingVirtual > 0 and #remainingStudio > 0 then
+        -- Rebuild groups, pre-compute caches, N*M scoring...
+        -- Score via: ownDiffs (from pre-computed caches) + childrenCost
+        -- For childrenCost: call computePairCost (which calls matchChildren recursively)
+        -- BUT: the ambiguous loop uses pre-computed caches for ownDiffs directly
+        -- (cacheVirtual/cacheStudio/countOwnDiffs), then adds children cost via
+        -- a direct matchChildren call (same as computePairCost but with caches already built)
+        --
+        -- The inner scoring for an ambiguous (vi, si) pair:
+        --   cost = countOwnDiffs(vCaches[vi], sCaches[si], classKeys)
+        --   if cost < bestSoFar then
+        --       local vInst = virtualInstances[remainingVirtual[vi]]
+        --       local studioKids = sCaches[si].children
+        --       -- filter valid virtual children, call matchChildren, add totalCost
+        --   end
+        --
+        -- Greedy assign sorted by cost ascending (tie-break by insertion index)
+    end
+
+    -- 5. Compute totalCost for ALL matched pairs (uses session cache)
+    local totalCost: number = 0
+    for _, pair in matched do
+        totalCost += computePairCost(
+            session, pair.virtualId, pair.studioInstance, virtualInstances, math.huge
+        )
+    end
+    totalCost += (#remainingVirtual + #remainingStudio) * UNMATCHED_PENALTY
+
+    local result: MatchResult = {
+        matched = matched,
+        unmatchedVirtual = remainingVirtual,
+        unmatchedStudio = remainingStudio,
+        totalCost = totalCost,
+    }
+
+    -- 6. Cache result
+    if parentVirtualId and parentStudioInstance then
+        if not session.matchCache[parentVirtualId] then
+            session.matchCache[parentVirtualId] = {}
+        end
+        session.matchCache[parentVirtualId][parentStudioInstance] = result
+    end
+
+    return result
+end
+```
+
+### pcall Removal in Studio Grouping
+
+The current code wraps `inst.Name, inst.ClassName` in a `pcall(function() ... end)`. This allocates a closure per studio child and prevents inlining. During matching, studio children come from `:GetChildren()` on a live instance within a single-threaded sync frame -- they cannot be destroyed mid-operation. Replace with direct property access.
+
+### O(n) removeMatched
+
+Current `_removeMatched` uses `table.remove` in a reverse loop -- O(n^2) due to element shifting. Replace with single-pass in-place compaction (shown above).
+
+### Module Export
+
+```lua
+return {
+    newSession = newSession,
+    matchChildren = matchChildren,
+}
+```
+
+Only two symbols exported. All helpers are local (invisible outside module, inlineable by compiler).
+
+---
+
+## Lua Callers
+
+### [hydrate.lua](plugin/src/Reconciler/hydrate.lua)
+
+```lua
+local Matching = require(script.Parent.matching)
+
+local function hydrate(instanceMap, virtualInstances, rootId, rootInstance, session)
+    -- ...
+    local result = Matching.matchChildren(
+        session, validVirtualIds, existingChildren, virtualInstances,
+        rootId, rootInstance
+    )
+    for _, pair in result.matched do
+        hydrate(instanceMap, virtualInstances, pair.virtualId, pair.studioInstance, session)
+    end
+    -- ...
+end
+```
+
+### [Reconciler/init.lua](plugin/src/Reconciler/init.lua)
+
+```lua
+function Reconciler:hydrate(virtualInstances, rootId, rootInstance, session)
+    Timer.start("Reconciler:hydrate")
+    local result = hydrate(self.__instanceMap, virtualInstances, rootId, rootInstance, session)
+    Timer.stop()
+    return result
+end
+```
+
+### [ServeSession.lua](plugin/src/ServeSession.lua)
+
+Create session before hydrate at line 628:
+
+```lua
+local Matching = require(script.Parent.Reconciler.matching)
+-- ...
+local session = Matching.newSession()
+self.__reconciler:hydrate(readResponseBody.instances, serverInfo.rootInstanceId, game, session)
+```
+
+---
+
+## trueEquals Rewrite: [trueEquals.lua](plugin/src/Reconciler/trueEquals.lua)
+
+### Module Header
+
+```lua
+--!strict
+--!native
+--!optimize 2
+```
+
+### Problems in Current Implementation
+
+1. **Redundant type dispatch**: Every branch checks both `typeA` and `typeB` (`typeA == "CFrame" and typeB == "CFrame"`). Check type once, early-out if they differ.
+2. **Heap allocations in hot path**: `{ a:GetComponents() }`, `{ a.X, a.Y, a.Z }`, `{ a.X, a.Y }` build temporary arrays for every CFrame/Vector3/Vector2 comparison.
+3. `**checkedKeys` set**: Allocates a table for every table-vs-table comparison. Unnecessary -- second loop only needs to check for extra keys in `b`.
+4. **No type annotations**, no `--!strict`, no `--!native`, no `--!optimize 2`.
+
+### Structure: Single typeof Dispatch + Dedicated Helpers
+
+Adopt the [DeepEqualsPureUnsafe](escape-tsunami reference) pattern: `rawequal` identity check, `rawlen` for quick array length, `rawget`/`next` for table traversal, no temporary allocations. Bake in fuzzy equality for the matching use case.
+
+```lua
+local EPSILON: number = 0.0001
+
+local function fuzzyEq(a: number, b: number): boolean
+    local diff = math.abs(a - b)
+    local maxVal = math.max(math.abs(a), math.abs(b), 1)
+    return diff < EPSILON or diff < maxVal * EPSILON
+end
+
+local function color3Eq(a: Color3, b: Color3): boolean
+    return math.floor(a.R * 255) == math.floor(b.R * 255)
+        and math.floor(a.G * 255) == math.floor(b.G * 255)
+        and math.floor(a.B * 255) == math.floor(b.B * 255)
+end
+
+local function vector2Eq(a: Vector2, b: Vector2): boolean
+    return fuzzyEq(a.X, b.X) and fuzzyEq(a.Y, b.Y)
+end
+
+local function vector3Eq(a: Vector3, b: Vector3): boolean
+    return fuzzyEq(a.X, b.X) and fuzzyEq(a.Y, b.Y) and fuzzyEq(a.Z, b.Z)
+end
+
+local function cframeEq(a: CFrame, b: CFrame): boolean
+    local ax, ay, az, aR00, aR01, aR02, aR10, aR11, aR12, aR20, aR21, aR22 = a:GetComponents()
+    local bx, by, bz, bR00, bR01, bR02, bR10, bR11, bR12, bR20, bR21, bR22 = b:GetComponents()
+    return fuzzyEq(ax, bx) and fuzzyEq(ay, by) and fuzzyEq(az, bz)
+        and fuzzyEq(aR00, bR00) and fuzzyEq(aR01, bR01) and fuzzyEq(aR02, bR02)
+        and fuzzyEq(aR10, bR10) and fuzzyEq(aR11, bR11) and fuzzyEq(aR12, bR12)
+        and fuzzyEq(aR20, bR20) and fuzzyEq(aR21, bR21) and fuzzyEq(aR22, bR22)
+end
+```
+
+All helpers are small `local function`s -- the compiler can inline them into `trueEquals`.
+
+### Main Function: Single typeof Dispatch
+
+```lua
+local function trueEquals(a: any, b: any): boolean
+    if rawequal(a, b) then return true end
+
+    -- Null-ref equivalence: nil == {Ref = "000...0"}
+    if a == nil then
+        return type(b) == "table" and rawget(b, "Ref") == "00000000000000000000000000000000"
+    end
+    if b == nil then
+        return type(a) == "table" and rawget(a, "Ref") == "00000000000000000000000000000000"
+    end
+
+    local t = typeof(a)
+    if t ~= typeof(b) then
+        -- EnumItem/number cross-type
+        if t == "number" and typeof(b) == "EnumItem" then return a == (b :: EnumItem).Value end
+        if t == "EnumItem" and typeof(b) == "number" then return (a :: EnumItem).Value == b end
+        return false
+    end
+
+    -- Same-type dispatch (no redundant second type check)
+    if t == "table" then
+        -- DeepEqualsPureUnsafe pattern with fuzzy recursive equality
+        if rawlen(a) ~= rawlen(b) then return false end
+        for k, v in next, a do
+            local ov = rawget(b, k)
+            if ov == nil or not trueEquals(v, ov) then return false end
+        end
+        for k in next, b do
+            if rawget(a, k) == nil then return false end
+        end
+        return true
+    end
+
+    if t == "number" then return fuzzyEq(a, b) end
+    if t == "Color3" then return color3Eq(a, b) end
+    if t == "Vector3" then return vector3Eq(a, b) end
+    if t == "Vector2" then return vector2Eq(a, b) end
+    if t == "CFrame" then return cframeEq(a, b) end
+
+    -- NaN check (number case already handled above, this catches any remaining NaN-like values)
+    if a ~= a and b ~= b then return true end
+
+    return false
+end
+```
+
+**Key improvements:**
+
+- **Zero heap allocations**: No temporary arrays for CFrame/Vector3/Vector2. CFrame uses 12 direct `fuzzyEq` calls. Vector3/Vector2 use direct component access.
+- **No `checkedKeys` set**: Second table loop only checks existence via `rawget`, doesn't compare values (already done in first loop).
+- **Single `typeof` call path**: Check type of `a` once, verify `b` matches, then dispatch. No redundant `typeA == X and typeB == X` at every branch.
+- `**rawequal` identity check first**: Catches same-reference case before any type checking.
+- `**rawlen` fast-path for tables**: Different array lengths = not equal, no iteration needed.
+
+---
+
+## Codegen Verification
+
+After each rewrite, run:
+
+```bash
+luau-compile --remarks plugin/src/Reconciler/trueEquals.lua
+luau-compile --remarks plugin/src/Reconciler/matching.lua
+```
+
+**trueEquals.lua** -- check that:
+
+- `fuzzyEq`, `color3Eq`, `vector2Eq`, `vector3Eq`, `cframeEq` are inlined into `trueEquals`
+- No "can't inline" remarks on any helper
+
+**matching.lua** -- check that:
+
+- `countOwnDiffs`, `cacheVirtual`, `cacheStudio`, `removeMatched` are inlined at call sites
+- `computePairCost` inlines or at minimum shows no optimization barriers
+- No generic dispatch warnings for typed parameters
+
+Iterate on any functions that fail to inline (split if too large, simplify control flow).
+
+---
+
+## Rust Forward Sync: [src/snapshot/matching.rs](src/snapshot/matching.rs)
+
+Add session struct and thread through. Rust already has the clean 2-function structure (`match_forward` + `compute_change_count` / `match_children_for_scoring`).
+
+```rust
+pub struct MatchingSession {
+    cost_cache: RefCell<HashMap<(Ref, Ref), u32>>,
+}
+```
+
+- Add `total_cost: u32` to `ForwardMatchResult`
+- Add `session: &MatchingSession` param to `match_forward`, `compute_change_count`, `match_children_for_scoring`
+- Cache `compute_change_count` results by `(snap.snapshot_id, tree_ref)`, only when `cost < best_so_far`
+- Create session in [patch_compute.rs](src/snapshot/patch_compute.rs) `compute_patch_set`, thread through `compute_patch_set_internal` and `compute_children_patches`
+
+## Rust Syncback: [src/syncback/matching.rs](src/syncback/matching.rs)
+
+Same pattern. Session struct, `total_cost` on `MatchResult`, session param on all functions.
+
+- Create session in [dir.rs](src/snapshot_middleware/dir.rs) before `match_children` call
+
+## Test Updates: [matching_fixtures.rs](tests/tests/matching_fixtures.rs)
+
+Pass `&MatchingSession::new()` to `match_children` and `match_forward` calls.
+
+---
+
+## Summary of Optimizations
+
+
+| Change                                                  | Impact                                                            |
+| ------------------------------------------------------- | ----------------------------------------------------------------- |
+| **trueEquals**: single typeof dispatch                  | Eliminates redundant type checks at every branch                  |
+| **trueEquals**: dedicated type helpers                  | Zero heap allocations (no temp arrays for CFrame/Vector3/Vector2) |
+| **trueEquals**: DeepEqualsPureUnsafe table pattern      | No checkedKeys set; rawlen/rawget/next                            |
+| **trueEquals**: `--!strict`/`--!native`/`--!optimize 2` | Native codegen for the most-called function in matching           |
+| **matching**: local functions (not table methods)       | Enables compiler inlining                                         |
+| **matching**: `--!strict`/`--!native`/`--!optimize 2`   | Native codegen, type-guided optimization                          |
+| **matching**: concrete types on all params/returns      | Specialized native code paths                                     |
+| **matching**: remove pcall in grouping                  | Eliminates closure + pcall overhead per child                     |
+| **matching**: 2-function recursion (from 4)             | Simpler for optimizer, easier to reason about                     |
+| **matching**: session caching (matchCache + costCache)  | Eliminates redundant subtree scoring across recursion levels      |
+| **matching**: totalCost on matchChildren                | Eliminates _computeChildrenCost re-iteration of matched pairs     |
+| **matching**: O(n) removeMatched                        | Linear vs quadratic element removal                               |
+
+
--- a/.cursor/plans/syncback_via_rbxm_ad0231a7.plan.md
+++ b/.cursor/plans/syncback_via_rbxm_ad0231a7.plan.md
@ -0,0 +1,394 @@
+---
+name: Syncback via RBXM
+overview: Add a "Syncback" button to the plugin that serializes the DataModel per-service via SerializationService, sends the rbxm chunks to a totally segregated server endpoint, which tears down the entire serve system, runs CLI-equivalent clean-mode syncback, and restarts serve -- all within the same process.
+todos:
+  - id: server-exit-reason
+    content: Modify LiveServer::start() to return ServerExitReason via tokio select + shutdown channel (src/web/mod.rs)
+    status: completed
+  - id: api-types
+    content: Add SyncbackPayload/ServiceChunk types to src/web/interface.rs
+    status: completed
+  - id: api-endpoint
+    content: Add POST /api/syncback endpoint handler in src/web/api.rs (fully segregated, no session validation)
+    status: completed
+  - id: serve-loop
+    content: Wrap ServeCommand::run() in a loop that handles SyncbackRequested (src/cli/serve.rs)
+    status: completed
+  - id: reconstruct-dom
+    content: "Implement run_live_syncback: reconstruct WeakDom from service chunks, run syncback_loop (clean mode, all CLI defaults), write files"
+    status: completed
+  - id: plugin-button
+    content: Add Syncback button (Danger style, layoutOrder 2) to NotConnected.lua
+    status: completed
+  - id: plugin-confirm
+    content: Add confirmation dialog (StudioPluginGui, danger-themed) in App/init.lua
+    status: completed
+  - id: plugin-serialize
+    content: Implement DataModel serialization per SYNCBACK_SERVICES list using SerializationService
+    status: completed
+  - id: plugin-api
+    content: Add standalone syncback HTTP post function (no ApiContext/session needed)
+    status: completed
+  - id: plugin-flow
+    content: "Wire up full flow in App/init.lua: confirm -> serialize -> POST /api/syncback -> done (fire and forget)"
+    status: completed
+  - id: test-helpers
+    content: Add post_api_syncback, wait_to_come_back_online, make_service_chunk, make_rbxl_from_chunks, run_cli_syncback_on_chunks to serve_util.rs
+    status: completed
+  - id: test-fixtures
+    content: Create live_syncback, live_syncback_place_ids, and live_syncback_complex serve-test fixtures
+    status: completed
+  - id: test-parity
+    content: "CLI parity tests (byte-for-byte): scripts, models+properties, refs, special names, duplicates, deep hierarchy, mixed types, comprehensive properties, multi-service"
+    status: completed
+  - id: test-validation
+    content: "Rejection tests: bad protocol, bad version, blocked place, unlisted place, whitelisted place, unrestricted"
+    status: completed
+  - id: test-lifecycle
+    content: "Lifecycle tests: server comes back functional (WebSocket + file watch), syncback twice with different data, replaces all old files"
+    status: completed
+  - id: test-roundtrip
+    content: "Round-trip identity test: build -> decompose -> live syncback -> rebuild -> compare instance trees"
+    status: completed
+isProject: false
+---
+
+# Syncback via RBXM
+
+## Architecture
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant Plugin
+    participant Server
+    participant Filesystem
+
+    User->>Plugin: Click "Syncback" button
+    Plugin->>Plugin: Show danger confirmation dialog
+    User->>Plugin: Confirm
+    Plugin->>Plugin: SerializeInstancesAsync per service
+    Plugin->>Server: POST /api/syncback (msgpack, segregated endpoint)
+    Server->>Plugin: 200 OK
+    Note over Server: Store payload, signal shutdown
+    Note over Server: Full teardown: drop ServeSession, VFS watcher, ChangeProcessor, WebSocket connections
+    Note over Server: Reconstruct WeakDom from service chunks
+    Note over Server: Run syncback_loop (clean mode, all CLI defaults)
+    Server->>Filesystem: Write files
+    Note over Server: Create fresh ServeSession + LiveServer
+    Note over Server: Bind to same address:port
+    Plugin->>Plugin: Poll GET /api/rojo until server is back
+    Plugin->>Server: Normal connect flow
+```
+
+
+
+## Server Changes
+
+### 1. Make `LiveServer::start()` return instead of blocking forever
+
+`[src/web/mod.rs](src/web/mod.rs)`: Change `start()` to return a `ServerExitReason` enum. Add a shutdown mechanism via `Arc<SyncbackSignal>` (wrapping a `Mutex<Option<SyncbackPayload>>` + `tokio::sync::Notify`). The accept loop uses `tokio::select!` to watch for the notify signal alongside `listener.accept()`. When signaled, it breaks and returns.
+
+```rust
+pub enum ServerExitReason {
+    SyncbackRequested(SyncbackPayload),
+}
+
+pub struct SyncbackSignal {
+    payload: Mutex<Option<SyncbackPayload>>,
+    notify: tokio::sync::Notify,
+}
+```
+
+The `SyncbackSignal` is shared between `LiveServer` (which watches it) and the API handler (which deposits data into it). When the accept loop exits, the Tokio runtime drops, all spawned tasks (including active WebSocket connections) are cancelled, and the `Arc<ServeSession>` drops -- tearing down the ChangeProcessor thread, VFS watcher, and all in-memory state.
+
+### 2. New segregated API endpoint: `POST /api/syncback`
+
+`[src/web/api.rs](src/web/api.rs)`: Totally separate from the connect/sync flow. No session ID needed -- the plugin doesn't need to be "connected" to use this. But it does validate version and place ID.
+
+Request body is msgpack-encoded:
+
+```rust
+pub struct SyncbackRequest {
+    pub protocol_version: u64,
+    pub server_version: String,
+    pub place_id: Option<u64>,
+    pub services: Vec<ServiceChunk>,
+}
+pub struct ServiceChunk {
+    pub class_name: String,
+    pub data: Vec<u8>, // raw rbxm bytes
+}
+// Payload passed to the serve loop (version info stripped after validation)
+pub struct SyncbackPayload {
+    pub services: Vec<ServiceChunk>,
+}
+```
+
+Types live in `[src/web/interface.rs](src/web/interface.rs)`.
+
+Handler flow:
+
+1. Deserialize msgpack body into `SyncbackRequest`
+2. **Validate protocol version** -- reject with 400 if `request.protocol_version != PROTOCOL_VERSION`
+3. **Validate server version** -- reject with 400 if major.minor doesn't match `SERVER_VERSION`
+4. **Validate place ID** against the project's `servePlaceIds` (whitelist) and `blockedPlaceIds` (blocklist). Reject with 403 if not allowed.
+5. Extract `SyncbackPayload` (just the services), store in `SyncbackSignal` -> notify -> return 200 `{"status": "syncback_initiated"}`
+
+The response is sent before any teardown begins (the API handler is a separate tokio task from the accept loop).
+
+### 3. Wrap `ServeCommand::run()` in a loop
+
+`[src/cli/serve.rs](src/cli/serve.rs)`: The `run()` method becomes a loop. Each iteration creates a fresh VFS, ServeSession, and LiveServer. When `start()` returns `SyncbackRequested`, the old session is fully dropped (VFS watcher stops, ChangeProcessor thread joins, tree is freed), syncback runs, and the loop continues to create everything fresh:
+
+```rust
+loop {
+    let (vfs, errors) = Vfs::new_default_with_errors();
+    let session = Arc::new(ServeSession::new(vfs, project_path.clone(), Some(errors))?);
+    let server = LiveServer::new(session);
+
+    match server.start(addr) {
+        ServerExitReason::SyncbackRequested(payload) => {
+            log::info!("Live syncback requested, running...");
+            run_live_syncback(&project_path, payload)?;
+            log::info!("Syncback complete, restarting serve...");
+            continue;
+        }
+    }
+}
+```
+
+IP and port are resolved once before the loop (they don't change).
+
+### 4. Reconstruct WeakDom + run syncback (CLI-equivalent)
+
+New function `run_live_syncback(project_path: &Path, payload: SyncbackPayload) -> anyhow::Result<()>` in `src/cli/serve.rs`:
+
+1. **Build WeakDom** -- Create `DataModel` root, for each `ServiceChunk` parse rbxm via `rbx_binary::from_reader`, create a service instance of the correct class, deep-clone all children from the parsed dom into the target dom. Maintain a `HashMap<Ref, Ref>` ref-map to fix up Ref properties across chunks.
+2. **Run syncback** -- Identical to what CLI `atlas syncback` does:
+  - Create oneshot VFS: `Vfs::new_oneshot()`
+  - Load old tree: `ServeSession::new_oneshot(vfs, project_path)`
+  - Call `syncback_loop(vfs, &mut old_tree, new_dom, project, incremental: false)` -- clean mode
+  - Write: `fs_snapshot.write_to_vfs_parallel(base_path, vfs)`
+  - `refresh_git_index(base_path)`
+
+Same code path as CLI syncback, same defaults, same clean mode. The only difference is the WeakDom source.
+
+## Plugin Changes
+
+### 5. Syncback button on NotConnected page
+
+`[plugin/src/App/StatusPages/NotConnected.lua](plugin/src/App/StatusPages/NotConnected.lua)`: Add "Syncback" button with `style = "Danger"` and `layoutOrder = 2`, shift "Sync" to `layoutOrder = 3`:
+
+```
+[Settings]  [Syncback]  [Sync]
+```
+
+Fires `self.props.onSyncback`.
+
+### 6. Confirmation dialog
+
+`[plugin/src/App/init.lua](plugin/src/App/init.lua)`: Floating `StudioPluginGui` dialog (same pattern as `AcceptConfirm` in Confirming.lua):
+
+- Title: "Full Syncback"
+- Message: "This will overwrite your project files with the current Studio state. This cannot be undone."
+- Buttons: "Cancel" (Bordered) | "Syncback" (Danger)
+- State: `self.state.showingSyncbackConfirm`
+
+### 7. DataModel serialization
+
+After confirmation, serialize per-service. Hardcode the same `VISIBLE_SERVICES` list from the Rust side:
+
+```lua
+local SYNCBACK_SERVICES = {
+    "Lighting", "MaterialService", "ReplicatedFirst", "ReplicatedStorage",
+    "ServerScriptService", "ServerStorage", "SoundService", "StarterGui",
+    "StarterPack", "StarterPlayer", "Teams", "TextChatService",
+    "VoiceChatService", "Workspace",
+}
+
+local services = {}
+for _, className in SYNCBACK_SERVICES do
+    local service = game:FindService(className)
+    if service and #service:GetChildren() > 0 then
+        local buf = SerializationService:SerializeInstancesAsync(service:GetChildren())
+        table.insert(services, {
+            className = className,
+            data = buf, -- msgpack-luau natively encodes buffer as bin format (zero-copy)
+        })
+    end
+end
+```
+
+The msgpack-luau library handles `buffer` types natively -- when `type(data) == "buffer"`, it encodes as msgpack bin 8/16/32 using `buffer.copy` directly. No string conversion needed. On the Rust side, `rmp_serde` deserializes msgpack bin into `Vec<u8>` automatically.
+
+### 8. Direct HTTP post (no ApiContext needed)
+
+Since this endpoint is fully segregated and doesn't need a session, the plugin just needs host:port from the address entry fields. A standalone function posts directly:
+
+```lua
+local function postSyncback(host, port, services)
+    local url = ("http://%s:%s/api/syncback"):format(host, port)
+    local body = Http.msgpackEncode({
+        protocolVersion = Config.protocolVersion,
+        serverVersion = Config.expectedServerVersionString,
+        placeId = game.PlaceId,
+        services = services,
+    })
+    return Http.post(url, body):andThen(rejectFailedRequests)
+end
+```
+
+No `ApiContext` instance needed. This keeps it completely independent from the connect flow.
+
+### 9. Full flow in App
+
+`[plugin/src/App/init.lua](plugin/src/App/init.lua)`: The `onSyncback` handler:
+
+1. Show confirmation dialog
+2. On confirm: serialize DataModel per-service
+3. POST to `/api/syncback` with the chunks
+4. On 200: show a notification ("Syncback data sent. Server is processing.")
+5. Done. Stay on NotConnected page. User clicks "Sync" manually whenever they're ready.
+
+Fire and forget -- the plugin's job ends after the POST succeeds. No polling, no auto-reconnect, no special state.
+
+## Key Design Decisions
+
+- **Fully segregated endpoint**: `/api/syncback` has zero coupling to the session/connect system. No session ID, no auth. Plugin fires and forgets.
+- **Complete teardown**: When the signal fires, the entire serve system is destroyed -- Tokio runtime drops, ServeSession drops (joining the ChangeProcessor thread, stopping VFS watcher), all WebSocket connections are cancelled. Nothing survives from the previous serve.
+- **CLI-equivalent syncback**: Uses `syncback_loop` with `incremental: false` (clean mode), non-interactive. Identical behavior to `atlas syncback --input file.rbxl` with all defaults.
+- **Same process, same port**: The `atlas serve` process never exits. After syncback completes, a fresh serve starts on the same address:port. The user sees log output in their terminal throughout.
+- **Danger styling**: Uses existing `Danger` theme (`#E74C3C` red) defined in `[plugin/src/App/Theme.lua](plugin/src/App/Theme.lua)`.
+- **Per-service serialization**: 14 visible services from `VISIBLE_SERVICES` constant, matching both Rust and Lua sides. Cross-service Ref properties are lost in transit but syncback's `Rojo_Ref_`* system resolves refs from filesystem structure.
+
+## Tests
+
+The core invariant: **live syncback must produce byte-identical filesystem output to CLI syncback given the same input data.** If it doesn't, something is broken in the serialize -> transmit -> reconstruct -> syncback pipeline. Every test that verifies data accuracy runs BOTH live syncback and CLI syncback on the same input, then compares output with `assert_dirs_equal` (byte-for-byte file comparison from `[tests/rojo_test/roundtrip_util.rs](tests/rojo_test/roundtrip_util.rs)`).
+
+Tests go in a dedicated `tests/tests/live_syncback.rs`. They use `run_serve_test` from `[tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs)` plus the round-trip utilities.
+
+### Test helpers (add to `serve_util.rs`)
+
+`**post_api_syncback_raw`** -- Sends a raw msgpack body to `POST /api/syncback`. Returns the full `reqwest::blocking::Response` so tests can assert on status codes and error bodies.
+
+`**post_api_syncback`** -- Convenience wrapper that fills in correct protocol version + server version. Takes `place_id: Option<u64>` and `services: Vec<ServiceChunk>`. Panics if response is not 200.
+
+`**wait_to_come_back_online`** -- Polls `GET /api/rojo` with exponential backoff (longer timeout than initial startup since syncback runs between teardown and restart). Returns the new `ServerInfoResponse`. Panics if server doesn't come back.
+
+`**make_service_chunk`** -- Builds a `ServiceChunk` from `InstanceBuilder` children:
+
+```rust
+fn make_service_chunk(class_name: &str, children: Vec<InstanceBuilder>) -> ServiceChunk {
+    let mut dom = WeakDom::new(InstanceBuilder::new("DataModel"));
+    let root = dom.root_ref();
+    for child in children {
+        dom.insert(root, child);
+    }
+    let refs: Vec<Ref> = dom.root().children().to_vec();
+    let mut buf = Vec::new();
+    rbx_binary::to_writer(&mut buf, &dom, &refs).unwrap();
+    ServiceChunk { class_name: class_name.to_string(), data: buf }
+}
+```
+
+`**make_rbxl_from_chunks**` -- Builds a complete rbxl file from the same service chunks, for CLI parity testing. Creates a WeakDom with DataModel root + properly-classed service instances + children, serializes to rbxl. This gives CLI syncback the exact same input data as live syncback received.
+
+`**run_cli_syncback_on_chunks**` -- End-to-end helper: copies a fixture to a fresh temp dir, calls `make_rbxl_from_chunks`, runs `run_rojo_syncback_clean`, returns the temp dir path. Used by every parity test as the CLI reference.
+
+### Fixtures
+
+`**rojo-test/serve-tests/live_syncback/**` -- Multi-service project:
+
+```
+default.project.json5    # RS -> src/shared, SSS -> src/server, Workspace -> src/workspace
+src/shared/OldModule.luau
+src/server/OldScript.server.luau
+src/workspace/OldPart.model.json5
+```
+
+`**rojo-test/serve-tests/live_syncback_place_ids/**` -- Same layout + `servePlaceIds: [123]`, `blockedPlaceIds: [999]`.
+
+`**rojo-test/serve-tests/live_syncback_complex/**` -- All 14 visible services mapped to separate `$path` dirs. Used for comprehensive parity tests.
+
+### CLI parity tests (data accuracy)
+
+Every parity test follows this pattern:
+
+```rust
+fn assert_live_matches_cli(
+    fixture_name: &str,
+    chunks: Vec<ServiceChunk>,
+    place_id: Option<u64>,
+) {
+    run_serve_test(fixture_name, |session, _| {
+        // 1. Live syncback
+        session.post_api_syncback(place_id, chunks.clone());
+        session.wait_to_come_back_online();
+
+        // 2. CLI syncback on fresh copy of same fixture with same data
+        let cli_dir = run_cli_syncback_on_chunks(fixture_name, &chunks);
+
+        // 3. Byte-for-byte comparison of every file
+        assert_dirs_equal(session.path(), &cli_dir);
+    });
+}
+```
+
+`**parity_scripts**` -- ModuleScript (`return {}`), Script/Server (`print("server")`), Script/Client (`print("client")`), LocalScript. Verifies Source content preserved exactly, correct file extensions (`.luau`, `.server.luau`, `.client.luau`, `.local.luau`), RunContext encoded in extension.
+
+`**parity_models_with_properties**` -- Model containing Parts with: Anchored (Bool), Size (Vector3), Position (Vector3), Color (Color3), BrickColor, Shape (Enum), Transparency (Float64). Verifies every property value round-trips through rbxm -> reconstruct -> syncback -> meta/model files identically.
+
+`**parity_ref_properties**` -- Model with PrimaryPart -> child Part. ObjectValue with Value -> sibling Part. Verifies `Rojo_Ref_*` attribute paths in meta/model files match CLI output exactly.
+
+`**parity_special_names**` -- Instances named `"What?"`, `"Key:Value"`, `"A/B"`, `"<Angle>"`, `'He said "hi"'`. Verifies slugified filenames (`What_`, `Key_Value`, `A_B`, etc.) and meta `name` fields match CLI.
+
+`**parity_duplicate_names**` -- 3x Folder named `"Data"`, 2x Script named `"Handler"`. Verifies dedup suffixes (`Data/`, `Data~2/`, `Data~3/`, `Handler.server.luau`, `Handler~2.server.luau`) and meta files match CLI.
+
+`**parity_deep_hierarchy**` -- 6 levels: Folder > Folder > Folder > Folder > Model > Part. Verifies nested directory structure is identical.
+
+`**parity_mixed_file_types**` -- StringValue (-> `.txt`), LocalizationTable (-> `.csv`), JSON data module (-> `.json5`), binary model (-> `.rbxm`), Folder (-> directory). All under one service. Byte-for-byte match with CLI.
+
+`**parity_comprehensive_properties**` -- Single Part with every testable property type set to non-default values: Bool, Int32, Float32, Float64, String, Vector2, Vector3, Vector2int16, Vector3int16, UDim, UDim2, Color3, Color3uint8, BrickColor, NumberRange, Rect, Enum, Font, Tags, Attributes (with nested types). Verifies meta.json5 property serialization matches CLI exactly.
+
+`**parity_multi_service**` -- Data under all 14 visible services simultaneously. Each service gets a unique set of instances (scripts, models, folders). Verifies every service's `$path` directory matches CLI output.
+
+`**parity_empty_services_mixed**` -- Some services have children, some don't, some are absent from the payload entirely. Verifies populated services are correct and empty/absent services result in clean directories.
+
+### Validation / rejection tests
+
+`**rejects_bad_protocol**` -- Send `protocol_version: 999`. Expect non-200. GET /api/rojo still returns same session ID (server unaffected).
+
+`**rejects_bad_version**` -- Send `server_version: "0.0.0"`. Expect non-200. Server unaffected.
+
+`**rejects_blocked_place**` -- Fixture has `blockedPlaceIds: [999]`. Send `placeId: 999`. Expect 403. Server unaffected.
+
+`**rejects_unlisted_place**` -- Fixture has `servePlaceIds: [123]`. Send `placeId: 456`. Expect 403. Server unaffected.
+
+`**allows_whitelisted_place**` -- `servePlaceIds: [123]`, send `placeId: 123`. Expect 200. Syncback runs, server restarts.
+
+`**allows_any_place_when_unrestricted**` -- No servePlaceIds/blockedPlaceIds. Send `placeId: 99999`. Expect 200.
+
+### Lifecycle / edge case tests
+
+`**server_comes_back_functional**` -- After live syncback, verify the restarted server is fully operational: GET /api/rojo works, can read instances via /api/read, WebSocket connects and receives patches when a file is modified on disk.
+
+`**syncback_twice_different_data**` -- Run live syncback with data A. Wait for restart. Run live syncback AGAIN with data B. Wait for restart. Verify filesystem matches data B (not A). Verify server is functional. Tests the serve loop handles multiple iterations.
+
+`**syncback_replaces_all_old_files**` -- Pre-populate the fixture with files that should NOT survive clean mode (orphan scripts, old models). After live syncback, verify none of the old files remain. Only files from the new data exist.
+
+### Round-trip identity test
+
+The ultimate accuracy test:
+
+`**roundtrip_build_syncback_rebuild**` -- Using the `live_syncback_complex` fixture:
+
+1. `atlas build` -> rbxl_A
+2. Decompose rbxl_A into per-service chunks (parse with `rbx_binary::from_reader`, extract children of each service, re-serialize as rbxm chunks)
+3. POST chunks to live syncback, wait for restart
+4. `atlas build` -> rbxl_B
+5. Parse both rbxl_A and rbxl_B into WeakDoms
+6. Walk both trees recursively: compare instance names, classes, property values (using the same fuzzy equality from `variant_eq` for floats/CFrames), and child counts
+7. Any divergence = test failure with detailed diff
+
+This tests the full pipeline: build -> decompose into service chunks -> transmit -> reconstruct -> syncback -> build again. If the output matches, the live syncback system preserves 100% data fidelity.