diff --git a/.cursor/commands/audit.md b/.cursor/commands/audit.md index 252cb1f5..ce03bb1c 100644 --- a/.cursor/commands/audit.md +++ b/.cursor/commands/audit.md @@ -1,8 +1,10 @@ # /audit - Production-Grade Sync Feature Audit +**IMMEDIATE ACTION:** Before reading any files, running any commands, or performing any analysis, you MUST call `SwitchMode` with `target_mode_id: "plan"`. Do not proceed with any other step until you are in Plan mode. This is non-negotiable. + **Mode:** This command runs in **Plan mode** (read-only). It does NOT directly apply code changes. The deliverable is a `.cursor/plans/*.plan.md` file containing every approved fix, ready to be executed in a subsequent Agent-mode session. -**Workflow:** Analyze -> Report -> Quiz user on each fix -> Write plan file. +**Workflow:** Switch to Plan mode -> Analyze -> Report -> Quiz user on each fix -> Write plan file. ## Prerequisites @@ -31,7 +33,7 @@ Any divergence between the two-way sync path and the CLI syncback path is a bug ## Sync System Layout -The auditor MUST read the relevant files from each pipeline when tracing round-trip identity. This section maps every file involved in both sync systems. Read these files -- do not skip any that are in the scope of the feature being audited. +The auditor MUST read the relevant files from each pipeline when tracing round-trip identity. This section maps every file involved in all three sync systems. Read these files -- do not skip any that are in the scope of the feature being audited. ### CLI Syncback Pipeline (`atlas syncback`) @@ -52,17 +54,18 @@ Syncback Core │ middleware context through the pipeline. Methods: │ with_joined_path(), get_path_filtered_properties(). ├── src/syncback/file_names.rs Filename generation. slugify_name() replaces forbidden -│ chars with '_'. deduplicate_name() appends ~1, ~2 on -│ collision. name_for_inst() is the main entry point. -│ adjacent_meta_path() computes .meta.json5 paths for -│ scripts (strips .server/.client suffixes from stem). +│ chars (incl. ~) with '_'. deduplicate_name_with_ext() +│ appends ~2, ~3 on collision. name_for_inst() is the +│ main entry point. adjacent_meta_path() computes +│ .meta.json5 paths for scripts (strips .server/.client +│ suffixes from stem). ├── src/syncback/property_filter.rs Property filtering. filter_properties() removes Name, │ Parent, Source (scripts), Value (StringValue), Contents │ (LocalizationTable), Ref/UniqueId, unscriptable props. ├── src/syncback/ref_properties.rs Ref property resolution. collect_referents() finds all -│ Ref properties. link_referents() writes Rojo_Ref_* or -│ Rojo_Target_* attributes. Path-based preferred, falls -│ back to ID-based for ambiguous paths. +│ Ref properties. link_referents() writes Rojo_Ref_* +│ (path-based) and Rojo_Id attributes to instances. +│ Rojo_Target_* is legacy (read-only during forward sync). ├── src/syncback/hash/mod.rs Content hashing. hash_tree() hashes entire tree bottom- │ └── src/syncback/hash/variant.rs up with blake3. hash_instance() hashes class + filtered │ properties + children hashes. Ref properties hashed as @@ -80,9 +83,9 @@ Syncback Core Snapshot Middleware (syncback direction: instance → filesystem) ├── src/snapshot_middleware/mod.rs Dispatcher. Middleware enum routes to handler. -│ get_best_middleware() selects middleware for an instance. │ Each middleware has syncback() producing SyncbackReturn -│ { fs_snapshot, children }. +│ { fs_snapshot, children }. get_best_middleware() lives +│ in src/syncback/mod.rs (selects middleware for an instance). ├── src/snapshot_middleware/dir.rs Directory handler. syncback_dir() processes children, │ creates init.meta.json5, handles .gitkeep for empty dirs. │ syncback_dir_no_meta() handles child dedup and recursion. @@ -146,12 +149,28 @@ Change Batching (Plugin) │ Encodes a single property value using RbxDom.EncodedValue. │ encode(). Maps Roblox values to wire format. ├── plugin/src/ChangeBatcher/encodeInstance.lua -│ Full instance encoding for additions. Duplicate detection -│ (skips instances with duplicate-named siblings). -│ Recursive children encoding. Attributes/Tags support. +│ Full instance encoding for additions. Uses encodeHelpers +│ for property iteration. All children encoded including +│ duplicates -- server handles dedup. Duplicate detection +│ helpers: findDuplicateNames(), hasDuplicateSiblings(), +│ isPathUnique(). Skips Ref properties during addition. +├── plugin/src/ChangeBatcher/encodeHelpers.lua +│ Shared encoding helpers used by encodeInstance and +│ encodeService. Exports: encodeAttributes(), +│ encodeTags(), forEachEncodableProperty(). Defines +│ SKIP_PROPERTIES (Parent, Name, Archivable). Filters +│ by scriptability, serialization, and data type. +├── plugin/src/ChangeBatcher/encodeService.lua +│ Service-level property encoding for live syncback. +│ Encodes properties via RbxDom.findClassDescriptor(). +│ Skips: Parent, Name, Archivable, Attributes, Tags. +│ Ref properties: creates temporary ObjectValue carriers +│ (Name=propertyName, Value=target). Returns (chunk, +│ refTargets). Class-specific excludes (Lighting.ClockTime). └── plugin/src/ChangeBatcher/propertyFilter.lua - Filters which properties are synced. Controls which data - types and property names are included/excluded. + Filters which properties are synced. Only UniqueId is + blocked. Ref properties are explicitly allowed (handled + by encodePatchUpdate via InstanceMap resolution). Session & Transport (Plugin) ├── plugin/src/ServeSession.lua Orchestrates the session. onInstanceChanged → guards @@ -174,12 +193,29 @@ Server Write Processing (receives plugin writes) │ model files. Sends PatchSet to tree_mutation_sender. │ CRITICAL: file writes and PatchSet must stay consistent. ├── src/change_processor.rs handle_tree_event() receives PatchSet from api.rs. -│ Processes removals (tree update), updates (Source writes, -│ renames with slugify/dedup/meta lifecycle, ClassName -│ transitions), additions (tree insert). Applies patch via -│ apply_patch_set(). Broadcasts via message_queue. +│ Processes removals (tree update), dedup cleanup (with +│ removed_set to exclude co-removed siblings), updates +│ (Source writes, renames with slugify/dedup/meta lifecycle, +│ ClassName transitions, ref path updates via RefPathIndex), +│ additions (tree insert). Applies patch via apply_patch_set(). +│ Broadcasts via message_queue. +├── src/git.rs Git integration. compute_git_metadata() identifies changed +│ files relative to HEAD. compute_blob_sha1() hashes content +│ using git blob format. git_add() stages files after writes. +│ Used by api.rs for stageIds and by serve_session for +│ GitMetadata in server info response. +├── src/rojo_ref.rs Ref path system. ref_target_path_from_tree() builds +│ filesystem-name paths for Rojo_Ref_* attributes. RefPathIndex +│ tracks which files contain which ref paths for efficient +│ updates on rename. Constants: REF_PATH_ATTRIBUTE_PREFIX, +│ REF_POINTER_ATTRIBUTE_PREFIX, REF_ID_ATTRIBUTE_NAME. +├── src/variant_eq.rs Property value comparison. variant_eq() compares Variant +│ values with fuzzy float matching (custom fuzzy_eq_f32/ +│ fuzzy_eq_f64 with EPSILON 0.0001, absolute+relative). +│ Used by matching algorithms for property diff scoring. └── src/serve_session.rs Server-side session. Owns RojoTree, ChangeProcessor, - MessageQueue. Coordinates serve lifecycle. + MessageQueue, RefPathIndex, git_repo_root. Coordinates + serve lifecycle. SERVER → PLUGIN (Filesystem changes pushed to Studio) ===================================================== @@ -209,7 +245,12 @@ Snapshot Generation (Server, forward-sync direction: filesystem → instance) │ DirectoryMetadata types used as overlays. ├── src/snapshot_middleware/project.rs snapshot_project() reads .project.json5. Resolves nodes, │ infers class names for services, merges $properties. -└── (other middleware: csv.rs, txt.rs, json.rs, toml.rs, yaml.rs, rbxm.rs, rbxmx.rs) +├── (other middleware: csv.rs, txt.rs, json.rs, toml.rs, yaml.rs, rbxm.rs, rbxmx.rs) +│ +├── src/snapshot/matching.rs Forward sync matching algorithm. match_forward() pairs +│ InstanceSnapshots to existing RojoTree children using +│ recursive change-count scoring + greedy assignment. +│ Constants: UNMATCHED_PENALTY=10000, MAX_SCORING_DEPTH=3. Patch & Delivery (Server) ├── src/snapshot/patch_compute.rs compute_patch_set() diffs old snapshot vs tree → @@ -246,6 +287,16 @@ Patch Application (Plugin) │ via instanceMap). ├── plugin/src/Reconciler/diff.lua diff() compares virtual instance data against Studio. │ Used for confirmation dialog to show what will change. +├── plugin/src/Reconciler/matching.lua Recursive change-count scoring algorithm. matchChildren() +│ pairs virtual instances with Studio instances during +│ hydration. Signature: (virtualChildren, studioChildren, +│ virtualInstances). One of 3 parallel matching impls that +│ must produce identical pairings (see also src/snapshot/ +│ matching.rs and src/syncback/matching.rs). +├── plugin/src/Reconciler/trueEquals.lua Shared value equality. Fuzzy floats (epsilon 0.0001), +│ Color3 via RGB ints, CFrame/Vector3 component-wise, NaN +│ handling, nil/null-ref equivalence. Used by matching.lua +│ and diff.lua. ├── plugin/src/Reconciler/setProperty.lua setProperty() applies a single property to an instance. │ Handles special cases and error recovery. ├── plugin/src/Reconciler/getProperty.lua getProperty() reads a property from an instance. @@ -253,7 +304,93 @@ Patch Application (Plugin) └── plugin/src/Reconciler/Error.lua Error types for reconciliation failures. ``` -### Shared Infrastructure (used by both pipelines) +### Live Syncback Pipeline (`POST /api/syncback`) + +One-click full sync from Studio to filesystem. Does NOT require an active WebSocket connection -- the plugin POSTs directly from the NotConnected page. The server restarts after completion. + +``` +PLUGIN SIDE (encoding and transport) +==================================== + +UI & Orchestration +├── plugin/src/App/init.lua SyncbackConfirm floating dialog (StudioPluginGui with +│ isEphemeral=true). Rendered when showingSyncbackConfirm +│ is true. SYNCBACK_SERVICES constant (14 services): +│ Lighting, MaterialService, ReplicatedFirst, +│ ReplicatedStorage, ServerScriptService, ServerStorage, +│ SoundService, StarterGui, StarterPack, StarterPlayer, +│ Teams, TextChatService, VoiceChatService, Workspace. +│ performSyncback() iterates services, encodes each via +│ encodeService(), serializes all children via +│ SerializationService:SerializeInstancesAsync() into a +│ single buffer, destroys temporary carriers, POSTs to +│ /api/syncback. Triggered from NotConnected page. +└── plugin/src/App/StatusPages/NotConnected.lua + "Syncback" button (Danger style) opens SyncbackConfirm. + +Service Encoding +├── plugin/src/ChangeBatcher/encodeService.lua +│ Encodes service-level properties for a single service. +│ Returns (chunk, refTargets). chunk = { className, +│ childCount, refTargetCount, properties?, refs? }. +│ Ref properties use temporary ObjectValue carriers +│ (carrier.Name = propertyName, carrier.Value = target). +│ refs map stores 1-based carrier indices. +│ Class-specific excludes: Lighting.ClockTime. +│ Skips: Parent, Name, Archivable, Attributes, Tags. +└── plugin/src/ChangeBatcher/encodeHelpers.lua + Shared helpers used by encodeService. encodeAttributes(), + encodeTags(), forEachEncodableProperty(). + +Transport: HTTP POST /api/syncback (MessagePack body) + Request: { protocolVersion, serverVersion, placeId, data, services } + +SERVER SIDE (validation, DOM reconstruction, syncback) +====================================================== + +API Handler +├── src/web/api.rs handle_api_syncback() validates: +│ (1) protocol version == PROTOCOL_VERSION (6) +│ (2) server version MAJOR.MINOR matches +│ (3) place ID passes servePlaceIds/blockedPlaceIds +│ Fires SyncbackSignal with payload. +│ Returns 400 (protocol/version mismatch), 403 (place ID), +│ or 409 (syncback already in progress). + +Signal & Server Lifecycle +├── src/web/mod.rs SyncbackSignal: Mutex> + Notify. +│ fire() deposits payload and notifies. +│ take_payload() retrieves and clears. +│ LiveServer::start() uses tokio::select! to wait for +│ new connections OR syncback signal. Exits accept loop +│ when signal fires. + +Wire Types +├── src/web/interface.rs SyncbackRequest { protocol_version: f64, +│ server_version: String, place_id: Option, +│ data: Vec, services: Vec } +│ ServiceChunk { class_name, child_count, ref_target_count, +│ properties: HashMap, +│ refs: HashMap } +│ SyncbackPayload { data: Vec, +│ services: Vec } +│ ServerExitReason::SyncbackRequested(SyncbackPayload) + +DOM Reconstruction & Syncback +└── src/cli/serve.rs run_live_syncback() is the main handler. + build_dom_from_chunks() reconstructs WeakDom: + (1) parses rbxm blob with rbx_binary::from_reader() + (2) creates services from ServiceChunk metadata + (3) distributes children using child_count + + ref_target_count ranges per service + (4) resolves Ref properties via ObjectValue carriers + (5) ensures all VISIBLE_SERVICES exist + Runs syncback_loop(incremental=false) -- always clean + mode. write_to_vfs_parallel() writes files. + refresh_git_index() updates git. Serve loop restarts. +``` + +### Shared Infrastructure (used by all three pipelines) ``` Snapshot System @@ -264,11 +401,20 @@ Snapshot System │ InstigatingSource (Path vs ProjectNode), SyncRule. ├── src/snapshot/tree.rs RojoTree: enhanced WeakDom with metadata. path_to_ids │ mapping, specified_id_to_refs tracking, get_instance_ -│ by_path() for path-based lookups. +│ by_path() for path-based lookups, filesystem_name_for() +│ for ref path resolution. +├── src/snapshot/matching.rs Forward sync matching: match_forward() pairs snapshot +│ children with existing tree children. Uses recursive +│ change-count scoring (same algorithm as syncback and +│ plugin matching). UNMATCHED_PENALTY=10000. ├── src/snapshot/patch.rs PatchSet, AppliedPatchSet, PatchAdd, PatchUpdate, │ AppliedPatchUpdate data structures. -├── src/snapshot/patch_compute.rs compute_patch_set(): diffs snapshot vs tree. -└── src/snapshot/patch_apply.rs apply_patch_set(): applies patch to tree. +├── src/snapshot/patch_compute.rs compute_patch_set(): diffs snapshot vs tree. Uses +│ match_forward() for child pairing. +└── src/snapshot/patch_apply.rs apply_patch_set(): applies patch to tree. Deferred ref + resolution (path-based Rojo_Ref_* and ID-based + Rojo_Target_*). Cleans internal attributes before + sending to plugin. Project System └── src/project.rs Project loading. Project, ProjectNode, PathNode types. @@ -291,11 +437,45 @@ rbx-dom (Roblox format libraries, git submodule) Plugin Shared Modules ├── plugin/src/InstanceMap.lua Instance ID ↔ Studio instance mapping. Signal management. +│ Also has onInstanceInserted callback for deferred Ref +│ resolution in ChangeBatcher. ├── plugin/src/PatchSet.lua PatchSet data structure and utilities. -├── plugin/src/Config.lua Protocol version, server version compatibility. +├── plugin/src/PatchTree.lua Builds tree for patch visualization. Accepts optional +│ gitMetadata to compute smart default selections (push/ +│ pull/nil) per instance based on git change status and +│ committed script hashes. +├── plugin/src/Config.lua Protocol version (6), server version compatibility, +│ defaultPort ("34873"). +├── plugin/src/Settings.lua Persistent settings (prefixed Atlas_ in Studio store). +│ Key defaults: oneShotSync=true, twoWaySync=true. ├── plugin/src/Types.lua Shared type definitions. +├── plugin/src/SHA1.luau SHA1 hashing. Computes git blob format hash ("blob +│ \0") for comparing script Source against +│ committed versions. Used by PatchTree for defaults. +├── plugin/src/XXH32.luau XXH32 hash function for plugin use. +├── plugin/src/ChangeMetadata.lua Change metadata tracking for sync operations. ├── plugin/src/strict.lua Module export wrapper (strict mode enforcement). └── plugin/src/DiffUtil.lua Diff utilities for comparing instance data. + +Integration Tests (tests/tests/) +├── tests/tests/build.rs Build tests (gen_build_tests! macro). +├── tests/tests/serve.rs Forward-sync serve tests. +├── tests/tests/connected_mode.rs Connected mode tests. +├── tests/tests/two_way_sync.rs Two-way sync via /api/write tests. +├── tests/tests/syncback.rs CLI syncback tests. +├── tests/tests/syncback_roundtrip.rs Full round-trip (syncback → build → compare). +├── tests/tests/syncback_format_transitions.rs File-to-folder and folder-to-file. +├── tests/tests/clean_mode.rs Clean mode syncback tests. +├── tests/tests/clean_mode_stress.rs Clean mode stress tests. +├── tests/tests/live_syncback.rs Live syncback parity tests (live output matches CLI), +│ validation tests (protocol/version/placeId), lifecycle +│ tests (server restart, sequential syncbacks), edge cases +│ (attributes, tags, cross-service refs, rapid sequential). +├── tests/tests/git_sync_defaults.rs Git-based sync default selection tests. +│ Hash computation parity, default selection logic, +│ auto-staging flow, no-git-repo fallback. +└── tests/tests/matching_fixtures.rs Matching algorithm fixture tests across all three + implementations (Rust syncback, Rust forward, Lua plugin). ``` ### Audit Reading Order @@ -307,8 +487,11 @@ When auditing a feature, read files in this order to build understanding before 3. **Syncback path:** `src/syncback/mod.rs`, then the relevant middleware's `syncback()` function -- understand how instances become filesystem 4. **Two-way sync server:** `src/web/api.rs` (`handle_api_write`, `syncback_*` functions), then `src/change_processor.rs` (`handle_tree_event`) -- understand how plugin writes are processed 5. **Two-way sync plugin:** `plugin/src/ChangeBatcher/` (encoding path), then `plugin/src/Reconciler/` (decoding path) -- understand both directions in the plugin -6. **File naming:** `src/syncback/file_names.rs` -- understand slugification, dedup, meta path computation -7. **Ref properties:** `src/syncback/ref_properties.rs`, `src/snapshot/patch_apply.rs` (deferred ref section) -- understand cross-instance references +6. **File naming:** `src/syncback/file_names.rs` -- understand slugification, dedup, meta path computation (`adjacent_meta_path` strips script suffixes: `Foo.server.luau` pairs with `Foo.meta.json5`) +7. **Ref properties:** `src/rojo_ref.rs` (constants, `RefPathIndex`, `ref_target_path_from_tree`), `src/syncback/ref_properties.rs` (syncback ref linking), `src/snapshot/patch_apply.rs` (deferred ref resolution) -- understand cross-instance references +8. **Matching algorithms:** `src/snapshot/matching.rs` (forward sync), `src/syncback/matching.rs` (CLI syncback), `plugin/src/Reconciler/matching.lua` (plugin) -- three parallel implementations that must produce identical pairings. Also `src/variant_eq.rs` (Rust) and `plugin/src/Reconciler/trueEquals.lua` (Lua) for property comparison +9. **Git integration:** `src/git.rs` (server-side git metadata, blob hashing, auto-staging), `plugin/src/PatchTree.lua` (default selection logic using gitMetadata), `plugin/src/SHA1.luau` (plugin-side hash computation) +10. **Live syncback:** `plugin/src/ChangeBatcher/encodeService.lua` (service encoding), `plugin/src/App/init.lua` (`performSyncback`, `SyncbackConfirm`, `SYNCBACK_SERVICES`), `src/web/api.rs` (`handle_api_syncback`), `src/web/mod.rs` (`SyncbackSignal`, `LiveServer`), `src/cli/serve.rs` (`build_dom_from_chunks`, `run_live_syncback`), `src/web/interface.rs` (`SyncbackRequest`, `ServiceChunk`) --- @@ -559,7 +742,20 @@ Given the same instance tree, does the feature ALWAYS produce the same filesyste Running the operation twice should produce zero changes on the second run. If re-running writes the same files again (even with identical content), something is unstable. -#### 11j. Edge Cases +#### 11j. Review Dialogue Invariant + +**No two-way sync change may bypass the review dialogue.** In Always and Initial sync modes, every incoming change must be cataloged in the confirmation UI. The user must explicitly choose a resolution (Atlas / Skip / Studio) for each entry. There is no "fast path" or silent application. + +**Verify:** +- Are there any code paths in `ServeSession`, `ApiContext`, or the Reconciler that apply patches without routing through the confirmation UI? +- Does the ChangeBatcher or any batch-processing logic silently apply changes that should appear in the review dialogue? +- When multiple changes arrive in a single WebSocket message or batch, does every individual change appear as a separate reviewable entry in the UI? +- Are there error/fallback paths that skip the dialogue and apply changes directly (e.g., "if dialogue fails, apply anyway")? +- Does the plugin correctly block patch application until the user has resolved every entry in the dialogue? + +Any code path that writes to Studio instances without the user having explicitly approved it through the review UI is a **critical** bug. This applies to property updates, additions, removals, renames, and any other mutation — nothing is exempt. + +#### 11k. Edge Cases - **ProjectNode instances**: Can the feature interact with instances defined in project files? Is there a guard? - **Nested projects**: Does data span project boundaries correctly? @@ -569,6 +765,75 @@ Running the operation twice should produce zero changes on the second run. If re - **Script type transitions**: ClassName changes (ModuleScript -> Script) cause file extension changes. Does the feature handle this? - **One-shot mode**: Does `Settings:get("oneShotSync")` correctly block the feature's outgoing writes? +#### 11l. Matching Algorithm Parity + +The matching algorithm exists in **3 parallel implementations** that must produce identical pairings for the same input: + +1. **Rust syncback** -- `src/syncback/matching.rs` (`match_children`, operates on `WeakDom` instances) +2. **Rust forward sync** -- `src/snapshot/matching.rs` (`match_forward`, operates on `InstanceSnapshot` vs `RojoTree`) +3. **Lua plugin** -- `plugin/src/Reconciler/matching.lua` (`Matching.matchChildren`, operates on virtual vs Studio instances) + +**Verify:** +- If the feature modifies matching logic in one implementation, was the same change applied to all three? +- Do all three use the same constants? (`UNMATCHED_PENALTY=10000`, `MAX_SCORING_DEPTH=3`) +- Do all three use the same grouping strategy? (fast-path by `(Name, ClassName)`, then recursive scoring for ambiguous groups) +- Is sort stability preserved? (Rust `sort_by` is stable; Lua `table.sort` is NOT -- ties must be broken by insertion index) +- Are the property comparison functions consistent? (`variant_eq` in Rust, `trueEquals.lua` in Lua -- both use fuzzy float equality, both treat nil and null-ref as equal) + +#### 11m. Git-Based Sync Defaults and Auto-Staging + +The confirmation dialog uses git metadata for smart default selections (`PatchTree.lua` + `src/git.rs`). Verify: + +- **Server-side `compute_git_metadata()`**: Does it correctly identify changed files vs HEAD? Does the two-phase tree lock (snapshot paths briefly, release before git subprocesses) prevent deadlocks? +- **Hash computation parity**: Server uses `compute_blob_sha1()` in `git.rs` and plugin uses `SHA1.luau` -- both must produce identical hashes for the same content using `SHA1("blob \0")` git blob format. +- **Default selection logic** (`PatchTree.lua`): File has no git changes → default "pull". File has git changes AND is script AND Studio Source matches committed hash → default "push". Otherwise → `nil` (user must decide). Verify these rules are applied correctly. +- **Auto-staging (`stageIds`)**: After confirmation, `api.rs` receives `stageIds` and calls `git_add`. Push-accepted items are always staged. Pull-accepted items are only staged if auto-selected (`defaultSelection ~= nil`). Manually-chosen pulls are left unstaged. Is this flow correct? +- **Staging split**: `api.rs` stages additions/removals/push files directly. `change_processor` stages Source writes after they complete (via `stage_ids` on PatchSet). Verify both paths handle the `stage_ids` list correctly. +- **No git repo**: When `git_repo_root` is `None`, all defaults should be `nil` and no staging should be attempted. + +#### 11n. Live Syncback Parity + +Live syncback (`POST /api/syncback`) is a third sync path that must produce filesystem output identical to CLI syncback. It runs in clean mode (`incremental=false`) and restarts the server after completion. + +**Verify plugin-side encoding:** +- Does `encodeService.lua` encode all service-level properties that CLI syncback captures? Compare against `filter_properties()` in `property_filter.rs`. +- Are `Attributes` and `Tags` handled correctly? (`encodeService` skips them from the property loop but `encodeHelpers.encodeAttributes/encodeTags` handle them separately.) +- Are class-specific excludes (`Lighting.ClockTime`) consistent with CLI syncback behavior? +- Is the `SKIP_PROPERTIES` list in `encodeHelpers.lua` (Parent, Name, Archivable) consistent with server-side filtering? + +**Verify ObjectValue carrier lifecycle:** +- Are carriers created correctly? (`carrier.Name = propertyName`, `carrier.Value = targetInstance`) +- Are carriers included in the serialization buffer? (appended to service's children range) +- Are carriers destroyed after serialization? (cleanup in `performSyncback()`) +- Does `build_dom_from_chunks()` correctly parse the `ref_target_count` range and resolve carriers? +- Does the server destroy carriers after resolving Ref properties? (they must not appear in the synced tree) + +**Verify rbxm blob reconstruction:** +- Does `build_dom_from_chunks()` distribute children correctly using `child_count` + `ref_target_count` ranges? +- Are services with zero children handled? (empty services still get created) +- Are `VISIBLE_SERVICES` in Rust consistent with `SYNCBACK_SERVICES` in Lua? (must list the same 14 services) +- Does the server handle out-of-range or zero-index carrier references gracefully? + +**Verify clean mode semantics:** +- Live syncback always uses `incremental=false`. Does this match CLI syncback behavior for the same input? +- Are old files properly removed before new files are written? +- Is `refresh_git_index()` called after writes? + +**Verify server restart behavior:** +- Does the server exit the accept loop cleanly when `SyncbackSignal` fires? +- Is the serve loop restarted with a new session after syncback completes? +- Can a new plugin connection be established after restart? +- Does `SyncbackSignal.fire()` return false if syncback is already in progress? (409 Conflict) + +**Verify validation:** +- Protocol version mismatch → 400 +- Server version MAJOR.MINOR mismatch → 400 +- Place ID not in `servePlaceIds` whitelist → 403 +- Place ID in `blockedPlaceIds` → 403 +- No restrictions configured → any place ID allowed + +**Verify round-trip:** Build → live syncback → rebuild → compare. The rebuilt instance tree must be bit-identical to the original (same as CLI syncback round-trip test). + ### 12. Run Static Analysis Run `cargo clippy` on the modified code to catch Rust issues the audit might miss: diff --git a/.cursor/commands/format.md b/.cursor/commands/format.md new file mode 100644 index 00000000..3c4f7203 --- /dev/null +++ b/.cursor/commands/format.md @@ -0,0 +1,33 @@ +# /format - Auto-fix Formatting & Static Analysis + +Run formatting and static analysis (linting) across the entire project. + +## Instructions + +### 1. Run the Format Script + +**Windows (PowerShell):** +```powershell +.\scripts\format.ps1 +``` + +**macOS/Linux (Bash):** +```bash +bash scripts/format.sh +``` + +**Monitor the script output as it runs.** + +### 2. Fix ALL Failures + +If any step fails, you MUST fix the issues and re-run until everything passes. This applies to **every** failure -- Clippy, Selene, Prettier, all of them. + +**Clippy is not optional.** Fix every Clippy warning, even if the warning is in code you did not write or that existed before your changes. There is no "pre-existing" or "before my time" exception. If Clippy reports it, fix it. The CI gate runs `cargo clippy` and will reject the PR regardless of who introduced the warning. + +The same rule applies to Selene warnings on Lua code. If the linter flags it, fix it. + +Keep re-running the format script until the overall result is **PASS** with zero failures. + +### 3. Report + +Once formatting passes clean, provide a brief summary and confirm the final result. diff --git a/.cursor/commands/luau-optimize.md b/.cursor/commands/luau-optimize.md new file mode 100644 index 00000000..7eb677fe --- /dev/null +++ b/.cursor/commands/luau-optimize.md @@ -0,0 +1,578 @@ +# /luau-optimize - Roblox Luau Performance Optimization + +Optimize Roblox Luau files for maximum performance, type safety, and native codegen quality using `luau-compile`, `luau-analyze`, `luau-ast`, and the `luau` CLI. + +All scripts are **Roblox Luau**. Roblox APIs are always available. The vector type is `Vector3` (Roblox userdata). We are NOT using the new type solver. Studio testing uses `--!optimize 1` by default; live experiences use `--!optimize 2`. All `luau-compile` invocations must include `--vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3`. Prefer `.luau` extension over `.lua` everywhere. + +**Workspace hygiene:** CLI tools generate residual files (`stats.json`, `profile.out`, `coverage.out`, `trace.json`). At the start of execution, create a temp working directory and use it for all tool output: + +``` +# Windows +$LUAU_TMP = New-Item -ItemType Directory -Path "$env:TEMP\luau-optimize-$(Get-Random)" +# Linux/Mac +LUAU_TMP=$(mktemp -d) +``` + +Route all output there: `--stats-file=$LUAU_TMP\stats.json`, write benchmark files to `$LUAU_TMP\bench.luau`, run profiling/coverage tools with `$LUAU_TMP` as working directory. The OS handles cleanup -- no manual deletion needed. + +--- + +## Phase 0 -- Scope Resolution and Configuration + +### Resolve target + +- If user specifies a file: optimize that file. +- If user specifies a directory: discover all `.luau`/`.lua` files, enter Phase 6 (multi-file mode). +- If no argument: optimize the currently open file. + +### Infer script context + +Check in this order: + +1. **Filename suffix** (highest priority): + - `*.server.luau` / `*.server.lua` -> **server** (`--!native` is high-value) + - `*.client.luau` / `*.client.lua` -> **client** (`--!native` less beneficial due to device diversity) + - `*.legacy.luau` / `*.legacy.lua` -> **ambiguous** (could be cloned/moved at runtime). Ask the user. + - `*.luau` / `*.lua` (no suffix) -> ModuleScript, fall through to path check. +2. **Path-based inference** (for ModuleScripts and ambiguous cases): + - `ServerScriptService`/`ServerStorage` in path -> server-side module + - `StarterPlayer`/`StarterGui`/`StarterCharacterScripts`/`ReplicatedFirst` in path -> client-side module + - `ReplicatedStorage`/`shared` in path -> shared module (conservative `--!native`) + - Ambiguous or outside Roblox project -> ask the user + +### Quiz the user + +Use the AskQuestion tool with these three questions: + +**Question 1: "Compilation optimization intensity"** +- `minimal` -- Headers (`--!strict`/`--!native`/`--!optimize 2`), type annotations on all function signatures, deprecated pattern replacements only +- `moderate` -- + function restructuring for inlining, import hoisting, compound operators, fastcall enablement, allocation reduction +- `insane` -- + every micro-optimization from the pattern catalog, full bytecode analysis, register pressure optimization, closure caching analysis + +**Question 2: "Algorithm optimization"** +- `none` -- Don't touch logic or data flow +- `low hanging fruit` -- Obvious O(n^2)->dict, redundant iterations, missing caches +- `moderate` -- + data structure changes, caching strategies, event-driven refactors, loop fusion +- `insane` -- + full algorithmic redesign where beneficial, dynamic programming, architectural restructuring + +**Question 3: "Benchmarking"** +- `yes` -- Benchmark all non-trivial changes with `luau` CLI (when function is standalone) +- `no` -- Trust bytecode/codegen metrics only +- `+EV only` -- Benchmark only when algorithmic changes or ambiguous bytecode results warrant it + +### Derived behavior + +| Setting | minimal | moderate | insane | +|---|---|---|---| +| Headers | Always | Always | Always | +| Type annotations | Function signatures | + hot path locals | + all locals | +| Pattern replacements | Deprecated only | Full Priority 2 | + Priority 4 | +| Function restructuring | No | Yes (Priority 3) | Aggressive splitting | +| Bytecode verification | After all changes | After each priority | After each change | +| Algorithmic changes | Per algo quiz | Per algo quiz | Per algo quiz | +| `@native` vs `--!native` | `--!native` unless big file | Selective `@native` on hot functions | Full per-function analysis | + +**Restructuring scope** scales with compilation intensity: +- **minimal**: No restructuring. Only additive changes (headers, annotations, pattern swaps). +- **moderate**: Break up monoliths, reorder locals for inlining, hoist imports. Preserve exports/API. +- **insane**: Aggressive restructuring permitted. May change internal function boundaries, rewrite data flow. + +--- + +## Phase 1 -- Deep Code Read and Algorithmic Analysis + +This phase comes FIRST. Algorithmic improvements produce order-of-magnitude speedups that dwarf any bytecode optimization. This is also the largest code change, so it must happen before any bytecode-level work. + +Read the entire source file thoroughly before touching any CLI tools. + +### Step 1 -- Structural decomposition + +Monolithic functions hide their algorithmic structure behind interleaved concerns. Before analyzing complexity, understand what the code is *actually doing*: + +- **Identify responsibilities**: A 200-line function usually contains 3-5 logical steps that should be separate functions. +- **Trace data flow**: What goes in, what comes out, what is mutated along the way. +- **Separate concerns**: Validation, transformation, I/O, state management, error handling are often tangled. Separating them reveals the core algorithm. +- **Name the steps**: If you can't name what a code block does in 3-5 words, it's doing too much. +- **Look for hidden abstractions**: Repeated "iterate, filter, transform, collect" patterns that could be a single helper. + +When the quiz allows restructuring (moderate/insane): +- Physically decompose monoliths into focused local functions. Each does one thing. +- This decomposition often *reveals* optimization opportunities invisible in the monolith: step 2 recomputes what step 1 already knew, or an inner loop could be a lookup table built in the outer loop. +- Smaller functions are easier for the compiler to inline at `-O2` -- clearer code AND better bytecode. +- Preserve the original function as a thin orchestrator calling the decomposed pieces. + +Even at "minimal" intensity, still perform this mental decomposition to inform the analysis below. + +### Step 2 -- Algorithmic analysis + +With the structure understood (or cleaned up), analyze: + +**Complexity analysis** -- Identify time complexity of every significant function: +- O(n^2) or worse: nested loops, repeated linear searches, quadratic string building +- O(n) where O(1) is possible: linear search replaceable with dictionary/set lookup +- Repeated work: same computation done multiple times, cacheable/memoizable +- Unnecessary copies: cloning data that could be referenced or mutated in place +- Hidden quadratics: O(n) function that calls another O(n) function inside a loop + +**Data structure fitness** -- Is the right data structure being used? +- Array for membership testing -> set (dictionary with `true` values) +- Linear scan for lookup by key -> dictionary +- Repeated sort of mostly-sorted data -> maintain sorted order on insert +- Flat list with frequent removal from middle -> swap-and-pop +- Large string built incrementally -> `table.concat` pattern or `buffer` +- Multiple parallel arrays -> single array of structs (table of tables) +- Unbounded growth without cleanup -> size limits, LRU eviction, periodic pruning + +**Caching and memoization:** +- Pure functions called repeatedly with same args -> memoize +- Expensive property reads in loops -> cache in local before loop +- Computed values unchanged within a scope -> hoist out +- Derived state recomputed from scratch on every access -> maintain incrementally + +**Redundant work elimination:** +- Multiple passes over same data -> fuse into one +- Recomputing derived state -> maintain incrementally +- Sort/filter that could be done once and reused +- Deep clone where shallow clone or reference suffices +- Intermediate result immediately discarded after extracting one field + +**Architectural patterns:** +- Per-frame polling where event-driven would work (RunService events are expensive) +- Synchronous work that could be chunked across frames with `task.wait()` +- Undisconnected event connections (memory leak + wasted computation) +- Rebuilding entire state on small change -> incremental update +- God-object owning too much state -> split responsibilities + +**Hot path identification:** +- Event handlers on RunService (PreAnimation, PreRender, PreSimulation, PostSimulation, Heartbeat) +- Functions called inside loops +- Recursive functions +- Functions called from other hot functions + +### Step 3 -- Present findings and quiz the user + +Before applying any changes, present a summary of all findings organized by impact. For each finding: + +- **What**: One-line description of the issue +- **Where**: Function name / line range +- **Why**: What makes it slow (e.g., "O(n^2) nested scan on every frame") +- **Fix**: Proposed change in plain language +- **Impact**: Estimated improvement (order-of-magnitude, constant-factor, or clarity-only) + +Then quiz the user using the AskQuestion tool: + +- For fewer than 5 findings: one question per finding with options `apply` / `skip` / `modify` +- For 5+ findings: group by category with options `apply all` / `cherry-pick` / `skip all`: + - Structural decomposition + - Data structure changes + - Algorithm changes + - Architectural changes + +If the algorithm quiz from Phase 0 was "none", present findings as informational only (no apply options). The user still sees what was found. + +If the user selects "modify", wait for their input before proceeding with that specific change. + +### Step 4 -- Apply approved changes + +Apply only user-approved changes, in this order (dependencies flow downward): + +1. **Structural decomposition** -- break monoliths into focused functions (foundation for everything else) +2. **Data structure changes** -- swap arrays for dicts, add indexes, etc. +3. **Algorithm changes** -- replace O(n^2) with O(n), add caching/memoization +4. **Architectural changes** -- event-driven refactors, incremental updates + +Benchmark algorithmic changes if the benchmarking quiz warrants it (see Phase 4.5). + +--- + +## Phase 2 -- Baseline Capture + +Run all diagnostic tools on the code (post-algorithmic changes if any were applied in Phase 1). These metrics are the baseline for bytecode optimization phases. + +### Tools to run + +Detect OS: use `--target=x64_ms` on Windows, `--target=x64` on Linux/Mac. Route stats output to temp directory. + +``` +luau-analyze --mode=strict +luau-analyze --annotate +luau-compile --remarks -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 +luau-compile --text -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 +luau-compile --codegen --target=x64_ms --record-stats=function --stats-file=$LUAU_TMP\stats.json -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 +``` + +### Metrics to capture + +- Total bytecode instruction count (from `--text`) +- Allocation count: NEWTABLE, NEWCLOSURE occurrences (from `--remarks`) +- Inlining success/failure count (from `--remarks`: "inlining succeeded"/"inlining failed") +- Type coverage: count of `any` types in `--annotate` output +- Register spills and skipped functions (from `--record-stats` JSON `lowerStats`) +- Lint warning count (from `--mode=strict`) + +--- + +## Phase 3 -- Compilation-Level Code Analysis + +Using both the source (already read in Phase 1) and tool output from Phase 2, identify bytecode-level optimization opportunities: + +- **Missing types**: `any` inferences hurting native codegen (especially `Vector3`, `CFrame`, `buffer` params). JIT uses annotations directly -- no runtime type analysis. Unannotated params assumed to be tables. +- **Import patterns**: `math.max` resolved at load time via GETIMPORT. Broken by `getfenv`/`setfenv`/`loadstring` (marks env "impure"). +- **Allocation in loops**: NEWTABLE, NEWCLOSURE in hot loops. High allocation rate = more GC assist work. +- **Closure caching**: Repeated function expressions cached when: no upvalues, or all upvalues immutable and module-scope. Mutable captures prevent caching. +- **Upvalue mutability**: ~90% immutable in typical code. Immutable = no allocation, no closing, faster access. Mutable = extra object. +- **Method call patterns**: `obj:Method()` uses fast method call instruction. Avoid `obj.Method(obj)`. `__index` should point at a table directly (not function or deep chain). +- **Inline caching**: Best when field name known at compile time, no metatables, uniform shapes. +- **pcall in hot paths**: Prevents native codegen optimization. +- **String concatenation**: `..` in loops -> `table.concat` or string interpolation. +- **Deoptimizing APIs**: `getfenv`/`setfenv` (even read-only!), `loadstring` disable ALL import optimization and fastcalls. +- **Builtin global writes**: `math = ...` disables fastcall. Lint `BuiltinGlobalWrite` catches this. +- **Metamethod cost**: `__eq` always called on `==`/`~=` even for rawequal values. +- **GC pressure**: Incremental GC with "GC assists" -- allocating code pays proportional GC work. + +--- + +## Phase 4 -- Compilation Optimization + +Apply changes in priority order. Re-run `luau-compile --remarks -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 ` after structural changes to verify the compiler benefits. + +### Priority 1 -- Headers and type safety (always apply) + +**Add headers:** +- `--!strict` / `--!native` / `--!optimize 2`. (`--!optimize 2` is default in live but not Studio testing.) + +**Remove deoptimizers:** +- Replace `getfenv`/`setfenv` -- disables builtins, imports, and optimizations globally. +- Replace `table.getn` -> `#t`, `table.foreach`/`table.foreachi` -> `for..in`. + +**Type accuracy -- the first big battle:** + +Getting the file to pass `--!strict` cleanly is the single most impactful compilation change. Every `any` type is a function the native codegen cannot specialize. But the goal is *accurate* types, not silencing errors with casts: + +- Run `luau-analyze --annotate` and identify every `any` inference. These are the targets. +- Add real type annotations to function signatures first (parameters and return types). This has the highest leverage -- it propagates type info to every caller and every local inside the function. +- Then annotate locals in hot paths where inference still falls to `any`. +- Annotate `Vector3`, `CFrame`, `buffer` params explicitly -- native codegen generates specialized vector code. Unannotated params are assumed to be generic tables with extra type checks. +- **Do NOT paper over type errors with `:: any` casts.** A cast to `any` is worse than no annotation -- it actively tells the compiler to give up. If a type error is hard to fix, use the narrowest cast possible (`:: SpecificType`), or restructure the code so the type flows naturally. +- **Narrow `any` in dictionaries.** `{[string]: any}` is common but hurts codegen. Ask: what values actually go in there? If it's a known set of types, use a union: `{[string]: string | number | boolean}`. If the dictionary has known keys, use a typed table: `{name: string, score: number}`. Only fall back to `any` when the value type is genuinely unbounded (e.g., serialized data from an external source). +- Use type refinements (`type()`, `typeof()`, `:IsA()`, `assert()`) to provide type info from runtime checks instead of casts. +- For OOP patterns: use `typeof(setmetatable(...))` with explicit `self: ClassName` on methods (old typechecker compatible). +- For generic code: use proper generics (``) instead of `any`. If the function truly accepts anything, use `unknown` and narrow explicitly. +- Track type coverage: count `any` in `--annotate` output. The goal is zero (or as close as practical). + +### Priority 2 -- Low-hanging structural wins + +- Hoist library functions: `local floor = math.floor` enables GETIMPORT fastcall. Fastcall builtins: `assert`, `type`, `typeof`, `rawget`/`rawset`/`rawequal`, `getmetatable`/`setmetatable`, `tonumber`/`tostring`, most `math.*` (not `noise`, `random`/`randomseed`), `bit32.*`, some `string.*`/`table.*`. Partial specializations: `assert` (unused return + truthy), `bit32.extract` (constant field/width), `select(n, ...)` O(1). +- Replace `pairs(t)`/`ipairs(t)` -> `for k, v in t do`. Generalized iteration skips the `pairs()` call. `for i=1,#t` is slightly slower. +- `math.floor(a / b)` -> `a // b` (dedicated VM opcode, `//=` compound form). +- Use compound assignment (`+=`, `-=`, `*=`, `//=`, `..=`) -- LHS evaluated once. +- String concat in loops -> `table.concat` or backtick interpolation (lowers to optimized `string.format`). +- `table.create(n)` for known-size arrays. Sequential fill: `local t = table.create(N); for i=1,N do t[i] = ... end`. +- `table.insert(t, v)` for unknown-size append -- `#t` is O(1) cached, worst case O(log N). +- `a + (b - a) * t` -> `math.lerp(a, b, t)` (exact at endpoints, monotonic). +- Manual byte swap -> `bit32.byteswap(n)` (CPU bswap). +- Manual log2 -> `bit32.countlz(n)` (CPU instruction, ~8x faster). +- Manual linear search -> `table.find(t, v)`. +- Manual `pairs` clone -> `table.clone(t)`. +- Manual `string.byte`/`string.char` -> `string.pack`/`string.unpack`. +- `rawlen(t)` when metamethods not needed. +- Explicit `./`/`../`/`@` prefixes in `require()`. + +### Priority 3 -- Function structure for inlining + +- Break monoliths into small local functions (compiler inlines at `-O2`). +- Inlining requirements: local, non-mutated, non-recursive, not OOP (`:` syntax), not metamethod. Disabled by `getfenv`/`setfenv`. +- Inlining + constant folding cascade: `local function double(x) return x*2 end; local y = double(5)` folds to `y = 10`. +- Create local function wrappers for frequently-called imported module methods. +- Move `pcall` out of hot loops. +- Use `obj:Method()` not `obj.Method(obj)` -- fast method call instruction. +- `__index` should point at a table directly for inline caching. +- Reduce mutable upvalue captures in loop closures. Immutable upvalues = no allocation, no closing. +- Set all table fields at once (compiler infers hash capacity from subsequent assignments). +- Loop unrolling: constant bounds only, simple body. +- Native codegen size limits: 64K instructions/block, 32K blocks/function, 1M instructions/module. Split if exceeded. + +### Priority 4 -- Micro-optimizations (hot paths only) + +- Table shape consistency (same fields, same order) -- inline caching predicts hash slots. +- `buffer` for binary data (fixed-size, offset-based, efficient native lowering). +- `buffer.readu32` + `bit32` over `buffer.readbits` when schema known. +- Strength reduction: `* 2^n` -> `bit32.lshift`, `/ 2^n` -> `bit32.rshift`. +- Minimize `tostring`/`tonumber` in tight loops. +- `table.freeze` for readonly config (avoids proxy `__index` overhead). +- Keep `__eq` cheap -- fires on every `==`/`~=` and `table.find`. +- `math.isfinite`/`math.isnan`/`math.isinf` over `x ~= x`. +- Annotate `Vector3` params for native specialization. +- `if expr then A else B` over `cond and A or B` (one branch, falsy-safe). + +--- + +## Phase 4.5 -- Benchmark with `luau` CLI + +When a non-trivial optimization is applied to an isolated, pure-logic function (no Roblox API dependencies), write a small benchmark harness: + +```luau +local function original(...) + -- paste original implementation +end + +local function optimized(...) + -- paste optimized implementation +end + +local ITERATIONS = 1_000_000 +local clock = os.clock + +for _ = 1, 1000 do original(...) end +for _ = 1, 1000 do optimized(...) end + +local t0 = clock() +for _ = 1, ITERATIONS do original(...) end +local t1 = clock() +for _ = 1, ITERATIONS do optimized(...) end +local t2 = clock() + +print(`Original: {t1 - t0:.4f}s`) +print(`Optimized: {t2 - t1:.4f}s`) +print(`Speedup: {(t1 - t0) / (t2 - t1):.2f}x`) +``` + +Run with both paths: + +``` +luau -O2 bench.luau +luau -O2 --codegen bench.luau +``` + +**When to benchmark:** Self-contained function, algorithmic/data structure change, ambiguous bytecode diff, tiebreak between approaches. + +**When NOT to benchmark:** Roblox API dependencies (use Studio Script Profiler instead), purely additive changes, cold code. + +Write the benchmark file to `$LUAU_TMP\bench.luau`, not the project directory. Run `luau` from `$LUAU_TMP` as working directory so `profile.out` and other residuals land there too. + +--- + +## Phase 5 -- Verification + +Re-run all Phase 2 tools and present a before/after comparison: + +- Bytecode instruction count delta +- Allocation count delta +- Inlining success rate delta +- Type coverage improvement (fewer `any` types) +- Register spill delta +- Lint warning delta +- Benchmark results (if Phase 4.5 was used) + +If any metric regressed, investigate and explain why (or revert). + +--- + +## Phase 6 -- Multi-file Mode + +When optimizing a directory: + +1. Run Phase 1 (code read) on all files to identify algorithmic improvement potential. +2. Run Phase 2 (baseline) on all files to identify bytecode optimization potential. +3. Prioritize by: worst algorithmic complexity, most lint warnings, most `any` types, most allocations, most failed inlines. +4. Process files one at a time through Phases 1-5. +5. Present aggregate metrics at the end. + +--- + +## Key Principles + +These principles govern every optimization decision: + +- **Same implementation, better performance** -- behavioral equivalence is non-negotiable. Never change what the code does, only how fast it does it. +- **Algorithms first, bytecode second** -- an O(n) algorithm with unoptimized bytecode beats an O(n^2) algorithm with perfect bytecode every time. +- **Low-hanging fruit first** -- headers and types before restructuring, restructuring before micro-optimizations. +- **Compiler feedback loop** -- verify each structural change with `luau-compile --remarks`. If the compiler didn't benefit, the change wasn't worth it. +- **Small functions over monoliths** -- easier for the compiler to inline, easier for humans to maintain, easier to reason about algorithmically. +- **Local functions for inlining** -- even at the cost of duplicating imported behavior in high-traffic code. +- **Quantify everything** -- before/after metrics for every optimization pass. No "I think this is faster" -- prove it. +- **Type annotations drive native codegen quality** -- every `any` type is a missed specialization opportunity. The JIT uses annotations directly with no runtime analysis. +- **Event-driven over polling** -- avoid per-frame calculations when events suffice. + +--- + +## Reference: Type System (Old Typechecker Only) + +**Available NOW:** +- `--!strict` / `--!nonstrict` / `--!nocheck` +- Basic annotations: `local x: number`, `function f(a: string): boolean` +- Optional: `string?`, Union: `number | string`, Intersection: `T1 & T2` +- Generics: `function id(x: T): T`, explicit instantiation: `f<>()` +- Cast: `x :: string` +- Array: `{number}`, Dict: `{[string]: number}`, Table: `type T = { field: type }` +- `typeof()`, `export type`, typed variadics, alias defaults, singletons +- `never`, `unknown`, `read`/`write` modifiers +- Type refinements: `type()`, `typeof()`, `:IsA()`, truthiness, equality, `assert()` + +**DO NOT recommend (requires new typechecker):** +- `keyof`, `index`, `rawkeyof`, `rawget` +- `getmetatable`, `setmetatable` +- User-defined type functions, negation types (`~T`) +- Relaxed recursive type restrictions + +## Reference: Type Refinement Patterns + +The compiler narrows types after certain checks, improving both type safety and native codegen. Use refinements to provide type information without explicit annotations where the type is known from a runtime check: + +- **Truthiness:** `if x then` narrows `x` from falsy (`nil`/`false`) +- **Type guards:** `if type(x) == "number" then` narrows `x` to `number` +- **Typeof guards (Roblox):** `if typeof(x) == "Vector3" then` narrows to `Vector3`; `x:IsA("TextLabel")` narrows `Instance` to subclass +- **Equality:** `if x == "hello" then` narrows `x` to singleton `"hello"` +- **Assert:** `assert(type(x) == "string")` narrows `x` to `string` after the call +- **Composition:** Supports `and`, `or`, `not` for compound refinements + +## Reference: Deprecated Patterns + +| Deprecated | Replacement | Impact | +|---|---|---| +| `getfenv()` / `setfenv()` | `debug.info(i, "snl")` | Disables ALL optimizations. Even read-only `getfenv()` deoptimizes. | +| `loadstring()` | Restructure with `require` | Marks env "impure", disables imports | +| `table.getn(t)` | `#t` | Slower | +| `table.foreach(t, f)` | `for k, v in t do` | Slower | +| `table.foreachi(t, f)` | `for i, v in t do` | Slower | +| `wait()` | `task.wait()` | Deprecated Roblox API | +| `obj.Method(obj, ...)` | `obj:Method(...)` | Misses fast method call instruction | +| `string:method()` | `string.method(s)` | Fastcall: `string.byte(s)` faster than `s:byte()` | + +## Reference: Lint Rules + +| Lint | Name | Optimization impact | +|---|---|---| +| 10 | BuiltinGlobalWrite | Overwriting builtins disables fastcall | +| 22 | DeprecatedApi | Deprecated APIs: perf/correctness issues | +| 23 | TableOperations | Wrong index, redundant insert, `#` on non-arrays | +| 3 | GlobalUsedAsLocal | Global in one function -> should be local | +| 7 | LocalUnused | Dead locals | +| 12 | UnreachableCode | Dead code | +| 25 | MisleadingAndOr | `a and false or c` bugs | + +## Reference: Bytecode Pattern Catalog + +| Slow | Fast | Why | +|---|---|---| +| `math.floor(a / b)` | `a // b` | Dedicated VM opcode | +| `data[i].x = data[i].x + 1` | `data[i].x += 1` | LHS evaluated once | +| `a + (b - a) * t` | `math.lerp(a, b, t)` | Exact at endpoints, monotonic | +| `"pre" .. v .. "suf"` | `` `pre{v}suf` `` | Optimized `string.format` | +| Manual byte swap | `bit32.byteswap(n)` | CPU bswap | +| Manual log2 loop | `bit32.countlz(n)` | CPU instruction, ~8x faster | +| `cond and A or B` | `if cond then A else B` | Safe for falsy, one branch | + +## Reference: Allocation Reduction + +| Slow | Fast | Why | +|---|---|---| +| `local t = {}` in loop | `table.clear(t)` + reuse | Avoids GC pressure | +| Repeated `table.insert` | `table.create(n)` + indexed writes | Preallocated | +| Manual `pairs` clone | `table.clone(t)` | Faster, copies layout | +| `string.byte`/`char` loops | `string.pack`/`unpack` | Native implementation | +| String binary data | `buffer` type | Fixed-size, efficient | + +## Reference: Inlining Rules + +| Blocks inlining | Enables inlining | Why | +|---|---|---| +| `function Module.foo()` | `local function foo()` | Mutable table prevents | +| `function Obj:method()` | `local function method(self)` | OOP syntax not inlineable | +| Recursive function | Split base/recursive | Recursion prevents | +| Deep upvalue captures | Pass as parameters | Reduces closure cost | + +## Reference: Native Codegen + +**`@native` vs `--!native`:** +- `--!native`: entire script. Good for math-heavy utility modules. +- `@native`: per-function. Better for selective hot functions or near the 1M instruction limit. +- Inner functions do NOT inherit `@native`. +- Top-level code runs once; `@native` has minimal benefit there. + +**Hurts native:** `getfenv`/`setfenv`, wrong types to typed functions, non-numeric math args, breakpoints, size limits (64K instructions/block, 32K blocks/function, 1M/module). + +**Helps native:** Type annotations (especially `Vector3`), small functions, consistent table shapes, `buffer` ops, `bit32` ops. + +## Reference: CLI Tools + +### `luau-compile` -- Static bytecode/codegen analysis (primary tool) + +Modes (mutually exclusive): `binary` (raw bytecode), `text` (human-readable opcodes with source annotations), `remarks` (source with inlining/allocation comments -- **most useful**), `codegen` (native assembly, requires `--target`). + +Options: `-O` (0-2, default 1; `-O2` enables inlining), `-g` (debug level 0-2), `--target=` (`x64`, `x64_ms`, `a64`, `a64_nf`; use `x64_ms` on Windows), `--record-stats=` (JSON stats: bytecodeInstructionCount, lowerStats with spills/skipped/blocks/errors), `--bytecode-summary` (opcode distribution, requires `--record-stats=function`), `--stats-file=` (default `stats.json`), `--timetrace` (trace.json), `--vector-lib`/`--vector-ctor`/`--vector-type` (always set for Roblox). + +### `luau-analyze` -- Type checking and linting + +Modes: omitted (typecheck + lint), `--annotate` (source with all inferred types inline -- **critical for finding `any`**). + +Options: `--mode=strict` (force strict even without directive), `--formatter=plain` (machine-parseable), `--formatter=gnu` (grep-friendly), `--timetrace`. + +### `luau` -- Runtime execution, profiling, and coverage + +**Limited for Roblox scripts** (most depend on engine APIs). Only usable for pure-logic modules. For Roblox scripts, use Studio Script Profiler and `debug.dumpcodesize()`. + +Options: `-O`, `-g`, `--codegen` (native execution), `--profile[=N]` (sampling at N Hz, default 10000, outputs `profile.out`), `--coverage` (outputs `coverage.out`), `--timetrace`, `-a` (program args). + +### `luau-ast` -- AST dump + +No flags. Outputs full JSON AST with node types, source locations, variable scopes, type annotations. Useful for programmatic analysis: function nesting depth, function body size (monolith detection), closure captures (upvalue analysis), table construction patterns, loop-invariant expression detection. + +## Reference: VM Architecture + +**Fastcall builtins:** `assert`, `type`, `typeof`, `rawget`/`rawset`/`rawequal`/`rawlen`, `getmetatable`/`setmetatable`, `tonumber`/`tostring`, most `math.*` (not `noise`/`random`/`randomseed`), `bit32.*`, some `string.*`/`table.*`. Partial: `assert` (unused return + truthy), `bit32.extract` (constant field/width), `select(n, ...)` O(1). With `-O2`: constant-arg builtins folded; `math.pi`, `math.huge` folded. + +**Inline caching:** HREF-style for table fields. Compiler predicts hash slot; VM corrects. Best: compile-time field name, no metatables, uniform shapes. + +**Import resolution:** `math.max` resolved at load time. Invalidated by `getfenv`/`setfenv`/`loadstring`. + +**GC:** Incremental mark-sweep with "GC assists" (allocation pays proportional GC). Paged sweeper (16 KB pages). No `__gc` metamethod. + +**Compiler:** Multi-pass. Constant folding across functions/locals. Upvalue optimization. `-O2`: inlining, loop unrolling (constant bounds), aggressive constant folding. Interprocedural limited to single modules. + +**Table length:** `#t` is O(1) cached, worst case O(log N) branch-free binary search. `table.insert`/`table.remove` update the cache. + +**Upvalues:** ~90% immutable. Immutable = no allocation, no closing, faster access. Mutable = extra object. + +**Closures:** Cached when no upvalues, or all upvalues immutable and module-scope. + +## Reference: Documentation to Consult + +Read these docs during execution if needed for specific detail. Two doc trees: +- `.cursor/rules/luau/` -- Official Luau site docs (guides, types, reference, getting-started) +- `.cursor/rules/luau-rfcs/` -- Luau RFCs (language proposals, library additions) + +**Luau site docs (primary):** +- `.cursor/rules/luau/guides/performance.md` -- **The official Luau performance guide.** Covers interpreter, compiler, fastcalls, inline caching, imports, table creation, GC, inlining, loop unrolling, upvalues, closures. +- `.cursor/rules/luau/guides/profile.md` -- Profiling guide (flame graphs, sampling) +- `.cursor/rules/luau/types/` -- Full type system docs (basic-types, tables, generics, refinements, unions-and-intersections, object-oriented-programs, roblox-types, considerations) +- `.cursor/rules/luau/getting-started/lint.md` -- All 28 lint rules +- `.cursor/rules/luau/reference/library.md` -- Complete library reference +- `.cursor/rules/luau/getting-started/compatibility.md` -- Luau vs Lua differences + +**Luau RFCs (specific features):** +- `.cursor/rules/luau-rfcs/function-inlining.md` -- Inlining RFC (no user `@inline`, automatic at `-O2`) +- `.cursor/rules/luau-rfcs/syntax-attribute-functions-native.md` -- `@native` per-function attribute +- `.cursor/rules/luau-rfcs/function-table-create-find.md`, `function-table-clear.md`, `function-table-clone.md`, `function-table-freeze.md` -- Table operations +- `.cursor/rules/luau-rfcs/function-math-lerp.md` -- `math.lerp` +- `.cursor/rules/luau-rfcs/function-bit32-byteswap.md`, `function-bit32-countlz-countrz.md` -- CPU-level bit ops +- `.cursor/rules/luau-rfcs/function-buffer-bits.md`, `type-byte-buffer.md` -- Buffer type and ops +- `.cursor/rules/luau-rfcs/vector-library.md` -- Vector library and fastcalls +- `.cursor/rules/luau-rfcs/function-string-pack-unpack.md` -- Binary string operations +- `.cursor/rules/luau-rfcs/syntax-floor-division-operator.md` -- `//` dedicated opcode +- `.cursor/rules/luau-rfcs/syntax-compound-assignment.md` -- `+=` single LHS evaluation +- `.cursor/rules/luau-rfcs/deprecate-getfenv-setfenv.md` -- Why fenv disables optimizations +- `.cursor/rules/luau-rfcs/deprecate-table-getn-foreach.md` -- Deprecated table functions +- `.cursor/rules/luau-rfcs/generalized-iteration.md` -- Modern iteration patterns +- `.cursor/rules/luau-rfcs/generic-functions.md`, `never-and-unknown-types.md`, `syntax-type-ascription.md` -- Type system RFCs + +**Cursor rules:** +- `.cursor/rules/luau.mdc` -- Luau language rules and optimization tips + +**Roblox docs:** +- `.cursor/rules/roblox/en-us/luau/native-code-gen.md` -- Native codegen (size limits, Vector3 annotation impact) +- `.cursor/rules/roblox/en-us/luau/type-checking.md` -- Type annotation syntax and modes +- `.cursor/rules/roblox/en-us/performance-optimization/improve.md` -- Script computation, memory, physics, rendering +- `.cursor/rules/roblox/en-us/performance-optimization/design.md` -- Event-driven patterns, frame budgets +- `.cursor/rules/roblox/en-us/luau/variables.md` -- Local vs global performance +- `.cursor/rules/roblox/en-us/luau/scope.md` -- Scope performance implications diff --git a/.cursor/plans/ambiguous_path_handling_f2eaef11.plan.md b/.cursor/plans/ambiguous_path_handling_f2eaef11.plan.md new file mode 100644 index 00000000..1b0bc3b4 --- /dev/null +++ b/.cursor/plans/ambiguous_path_handling_f2eaef11.plan.md @@ -0,0 +1,425 @@ +--- +name: Ambiguous Path Handling +overview: "Replace the current duplicate-name blacklist/skip system with a full ambiguous path handling system: a 3-pass matching algorithm (3 implementations: 2 Rust, 1 Lua), filesystem-collision-based dedup (~N suffixes), and filesystem-name-based ref paths. Implementation starts with the Rust syncback matching algorithm for testability, then expands outward." +todos: + - id: phase1a-remove-blacklist-rust + content: Remove duplicate-skip blacklist from Rust side (dir.rs, project.rs, syncback/mod.rs) + status: completed + - id: phase1b-remove-blacklist-plugin + content: Cherry-pick plugin relaxations from rbxm-ambiguous (diff.lua, encodeInstance.lua) + status: completed + - id: phase1c-two-pass-syncback + content: Restructure syncback_dir_no_meta to two-pass child processing (tentative names, then collision grouping) + status: completed + - id: phase1d-dedup-key-semantics + content: Change dedup key in file_names.rs from bare slug to full filesystem name component + status: completed + - id: phase1e-suffix-cleanup + content: Implement suffix cleanup rules (gap-tolerant, base-name promotion, group-to-1) + status: completed + - id: phase2a-matching-syncback + content: Implement 3-pass matching algorithm for syncback (src/syncback/matching.rs) + status: completed + - id: phase2b-integrate-syncback + content: Integrate matching into syncback_dir_no_meta (dir.rs) + status: completed + - id: phase2c-matching-forward + content: Implement 3-pass matching algorithm for forward sync (src/snapshot/matching.rs) + status: completed + - id: phase2d-integrate-forward + content: Integrate matching into compute_children_patches (patch_compute.rs) + status: completed + - id: phase2e-matching-cache + content: Add optional matching cache (parent_ref -> match_assignments) with invalidation + status: pending + - id: phase3a-fs-name-ref-paths + content: Switch ref_target_path() to filesystem-name-based paths in rojo_ref.rs + status: completed + - id: phase3b-tree-resolution + content: Update get_instance_by_path() in tree.rs to resolve filesystem-name segments + status: completed + - id: phase3c-ref-properties + content: Update ref_properties.rs for filesystem-name-based linking + status: completed + - id: phase3d-patch-apply + content: Update patch_apply.rs ref resolution for new path format + status: completed + - id: phase3e-ref-update-infra + content: Extend RefPathIndex and ref update infrastructure for all rename/dedup cases + status: completed + - id: phase4a-change-processor + content: Update change_processor.rs for VFS echo prevention on dedup renames and suffix cleanup + status: completed + - id: phase4b-api-write + content: Update api.rs for two-pass write, dedup in all 4 branches, dual write consistency + status: completed + - id: phase5a-lua-matching + content: Implement 3-pass matching algorithm in Lua (hydrate.lua + matching.lua + XXH32.luau) + status: completed + - id: phase5b-plugin-integration + content: Update ChangeBatcher and confirmation dialog for duplicate-named instances + status: completed + - id: phase6-cherry-picks + content: Cherry-pick roundtrip test improvements and macOS test ignore from rbxm-ambiguous + status: completed + - id: phase7-testing + content: "Implement comprehensive test suite: shared test vectors, all spec scenarios, roundtrip tests" + status: completed +isProject: false +--- + +# Ambiguous Path Handling + +## Architecture + +The feature introduces three interlocking subsystems that replace the current "skip duplicates" approach: + +```mermaid +flowchart TD + subgraph matching [Matching Algorithm] + MA1[Pass 1: Unique Names] + MA2[Pass 2: Ref Discriminators] + MA3[Pass 3: Similarity Scoring] + MA1 --> MA2 --> MA3 + end + + subgraph dedup [Dedup System] + D1[Compute tentative FS name per child] + D2[Group by lowercased FS collision] + D3[Assign ~N suffixes within groups] + end + + subgraph refs [Ref Path System] + R1[Paths use full FS names incl extensions] + R2[RefPathIndex tracks all refs] + R3[Batch update on rename/dedup change] + end + + matching -->|"pairing determines which get new names"| dedup + dedup -->|"final FS names feed into"| refs +``` + + + +**Execution order per sync operation:** Matching runs first (pairs instances), dedup runs second (assigns filesystem names), ref paths are computed last (using final filesystem names). + +**Performance target:** System must handle large games (10K+ instances) with <100ms latency per operation. Architecture must not have O(n^2) bottlenecks on the full tree. The matching algorithm should only run on the affected subtree when possible. Optimization can be deferred but the structure must support it. + +--- + +## Phase 1: Foundation - Remove Blacklist and Restructure Dedup + +### 1a. Remove duplicate-skip blacklist (Rust) + +`**[src/snapshot_middleware/dir.rs](src/snapshot_middleware/dir.rs)`** -- Remove the duplicate child detection and skip logic in `syncback_dir_no_meta()`: + +- Lines 151-181: `child_name_counts` / `duplicate_names` HashSet construction +- Lines 235-239, 321-324: `if duplicate_names.contains(...) { continue; }` guards +- Related stats recording (`record_duplicate_names_batch`) + +`**[src/snapshot_middleware/project.rs](src/snapshot_middleware/project.rs)`** -- Replace `bail!` on duplicate names with dedup suffix handling. Adapt the rbxm-ambiguous branch's case-insensitive detection (skip + warn), but instead of skipping, route through the new dedup system. **Important:** project KEYS are unique by JSON definition and do NOT participate in dedup themselves. Only filesystem children under project nodes need dedup when they collide. Evaluate whether to remove the warn-and-skip guard from the rbxm branch or keep it as a safety net. + +`**[src/syncback/mod.rs](src/syncback/mod.rs)`** -- Remove `has_duplicate_children` detection used for skipping. Keep the infrastructure for detecting duplicates since it feeds into the dedup system. + +### 1b. Remove duplicate-skip blacklist (Plugin) + +Take the rbxm-ambiguous branch changes directly: + +`**[plugin/src/Reconciler/diff.lua](plugin/src/Reconciler/diff.lua)`** -- Remove `findDuplicateNames()`, `ambiguousIds`, `scanForDuplicates()`, all `if ambiguousIds[id] then` guards, and the `skippedDuplicateCount` tracking. Duplicate-named instances now flow through diff normally. + +`**[plugin/src/ChangeBatcher/encodeInstance.lua](plugin/src/ChangeBatcher/encodeInstance.lua)`** -- Remove `skippedDuplicateCount`, remove child duplicate skip loop, keep `isPathUnique` as a debug log (not a skip). Return single function instead of wrapper. + +### 1c. Restructure syncback to two-pass child processing + +`**[src/snapshot_middleware/dir.rs](src/snapshot_middleware/dir.rs)`** `syncback_dir_no_meta()` -- Replace the current one-at-a-time `with_joined_path()` loop with: + +**Pass 1 -- Tentative names:** Iterate ALL children, call `get_best_middleware()` for each, compute tentative filesystem name (slug + extension for files, directory name for dirs). Also determine whether each instance would be file or directory (for init-style collision detection). + +**Pass 2 -- Dedup grouping:** Group children by lowercased tentative filesystem name. Within each collision group (size > 1), assign ~N suffixes. The matching algorithm's pairing (Phase 2) determines which instances preserve existing filenames and which get new suffixes. + +### 1d. Change dedup key semantics in file_names.rs + +`**[src/syncback/file_names.rs](src/syncback/file_names.rs)`** -- The current `dedup_key` is a bare slug (without extension). Change it to be the **full filesystem name component** (slug + extension for files, directory name for dirs), lowercased. + +Current dedup key for `Script "Foo"` = `"foo"` (bare slug). +New dedup key for `Script "Foo"` = `"foo.server.luau"` (full filesystem name). + +This means: + +- `Folder "Foo"` (key: `"foo"`) and `Script "Foo"` (key: `"foo.server.luau"`) do NOT collide -- correct. +- `Script "Foo" (server)` and `Script "Foo" (server)` both key: `"foo.server.luau"` -- collision, dedup applied. +- `Folder "Foo"` and init-style `Script "Foo"` both key: `"foo"` (both directories) -- collision, dedup applied. + +Update `deduplicate_name()` to accept the full filesystem name as input and apply the suffix to the stem before the compound extension: `Foo~1.server.luau`, not `Foo.server~1.luau`. + +### 1e. Suffix cleanup rules + +Implement in a shared helper (used by syncback, change_processor, and api.rs): + +- **Gap-tolerant deletion:** Deleting `~1` from `{Foo, Foo~1, Foo~2}` leaves `{Foo, Foo~2}`. No renumbering. +- **Base-name promotion:** Deleting `Foo` (base) from `{Foo, Foo~1, Foo~2}` promotes `Foo~1` to `Foo`. `Foo~2` stays. +- **Group-to-1 cleanup:** Deleting from `{Foo, Foo~1}` leaves one instance -- remove its suffix entirely (rename to clean name). + +--- + +## Phase 2: Matching Algorithm + +### 2a. Rust syncback matching (`src/syncback/matching.rs` -- new file) + +Core 3-pass algorithm operating on `WeakDom` instances (syncback context): + +``` +Input: new_children: Vec (from Roblox file WeakDom) + old_children: Vec (from filesystem WeakDom) + new_dom: &WeakDom, old_dom: &WeakDom + hashes: &HashMap (precomputed by hash_tree) +Output: MatchResult { matched: Vec<(new_ref, old_ref)>, + unmatched_new: Vec, + unmatched_old: Vec } +``` + +**Pass 1 -- Unique names (O(n)):** Build name-count maps for both sides. Any name that appears exactly once on both sides is an instant match. **Within same-Name groups, ClassName is a fast first discriminator:** if two siblings share name "Part" but one is Part and one is MeshPart, and each (Name, ClassName) pair is unique on both sides, match them in Pass 1 without needing Pass 2/3. **Fast path:** if all names are unique (or all resolved by name+class), return immediately (skip P2+P3). + +**Pass 2 -- Ref discriminators:** For remaining unmatched groups (same true Name), use pre-resolved Ref properties to differentiate. If instance A has a Ref pointing to a target already matched in P1, and only one candidate on the other side has the equivalent Ref, that's a match. Also consider incoming refs (other matched instances pointing TO a candidate). **Unmatched ref targets:** if a Ref target is itself in an unmatched ambiguous group (not resolved in Pass 1), treat it as a weaker signal -- compare target names/classes without requiring a confirmed match. Do NOT recursively match the target's group. If insufficient, the instance falls through to Pass 3. + +**Pass 3 -- Similarity scoring:** For still-unmatched groups, narrow by ClassName first, then compute pairwise similarity using precomputed blake3 subtree hashes. Greedy best-match-first assignment: pick highest similarity pair, remove both, repeat. Tiebreaker: DOM child order (index in parent's children list). + +**Ref pre-resolution (prerequisite):** Before any pass, resolve Ref properties on both sides. On the new side (Roblox file), Refs are WeakDom referents -- build a `Ref -> instance_path` map. On the old side (filesystem), Refs may be stored as `Rojo_Ref_`* attributes -- parse and resolve. + +### 2b. Integrate into syncback_dir_no_meta (dir.rs) + +In `syncback_dir_no_meta()`: + +1. Call matching algorithm to pair `new_inst.children()` with old children +2. For matched pairs: preserve existing filesystem assignment (old instance keeps its filename) +3. For unmatched new: generate new filename via dedup system (Phase 1c) +4. For unmatched old: mark for deletion + +### 2c. Forward sync matching (`src/snapshot/matching.rs` -- new file) + +Same 3-pass algorithm but operating on `InstanceSnapshot` vs `RojoTree` instances: + +``` +Input: snapshot_children: Vec + tree_children: Vec (RojoTree instance IDs) + tree: &RojoTree +Output: MatchResult { matched, unmatched_snapshot, unmatched_tree } +``` + +Key differences from syncback version: + +- **True Name resolution from filesystem:** On the filesystem side, resolving the true Instance.Name is non-trivial. Resolution chain: (1) `name` field in `.meta.json5` if present (overrides filename), (2) `name` field in `.model.json5`, (3) filename stem after stripping extensions and script suffixes (if no meta override). The matching algorithm must resolve all true Names before building groups. On the tree/plugin side, `Instance.Name` / `instance.name()` is readily available. +- **Ref pre-resolution fallback order:** Try the old RojoTree FIRST (most refs target existing instances). If the path can't be resolved in the old tree (target is a new instance), fall back to walking the snapshot tree. The snapshot is only a subtree (the re-snapshotted directory), so this fallback only works for refs within the same subtree. Refs to new instances outside the current snapshot remain unresolved and are skipped during matching (treated as nil). +- Snapshot children have `Rojo_Ref_`* as string paths in attributes -- resolve against tree (then snapshot) before matching +- Tree children have Refs as UIDs -- already resolved +- Hash infrastructure: snapshot side doesn't have precomputed hashes yet -- compute structural fingerprint on demand + +### 2d. Integrate into patch_compute.rs + +Replace the greedy name+class matching in `compute_children_patches()` with the forward sync matching algorithm. The current code: + +```6:17:src/snapshot/patch_compute.rs +// Current: greedy first-match by name+class +.find(|(instance_index, instance_child_id)| { + if paired_instances[*instance_index] { return false; } + snapshot_child.name == instance_child.name() + && snapshot_child.class_name == instance_child.class_name() +}) +``` + +Becomes: call to matching algorithm, then iterate matched pairs for recursive `compute_patch_set_internal()`, add unmatched snapshots as `PatchAdd`, add unmatched tree children as `removed_instances`. + +### 2e. Matching cache + +Add an optional cache mapping `parent_ref -> (children_list_hash, match_assignments)` to avoid re-running the full algorithm when children haven't changed. Invalidate per-parent when a child is added, removed, or renamed under that parent. During two-way sync, each `/api/write` mutation invalidates the affected parent's cache. + +--- + +## Phase 3: Ref Path System + +### 3a. Switch to filesystem-name-based paths + +`**[src/rojo_ref.rs](src/rojo_ref.rs)**` -- `ref_target_path()`: + +Current: uses `escape_ref_path_segment(&inst.name)` producing instance-name paths like `"Workspace/A\/B"`. + +New: uses the **full filesystem name** of each instance (slug + extension + dedup suffix for files; directory name + dedup suffix for dirs). Path from DataModel's children downward. + +Examples: + +- `Folder "Foo"` -> segment `"Foo"` +- `ModuleScript "Foo"` -> segment `"Foo.luau"` +- `Script "Foo"` (server) -> segment `"Foo.server.luau"` +- `Script "Foo"` with children (init-style) -> segment `"Foo"` (directory) +- Dedup'd `ModuleScript "Foo"` -> segment `"Foo~1.luau"` + +Remove `escape_ref_path_segment()`, `unescape_ref_path_segment()`, `split_ref_path()`. Path splitting becomes a simple `path.split('/')` since filesystem names can't contain `/` (slugified to `_`). + +### 3b. Filesystem-name resolution in tree.rs + +`**[src/snapshot/tree.rs](src/snapshot/tree.rs)`** -- `get_instance_by_path()`: + +Current: walks tree matching each segment against `child.name` (instance name). + +New: walks tree matching each segment against the **filesystem name** of each child. Derive from: + +1. `instigating_source` filename/dirname (for file-backed instances) +2. Fallback: compute what the filesystem name WOULD be (slugify name + middleware extension) for instances without instigating_source (e.g., inside .rbxm files, newly added) + +### 3c. Update ref_properties.rs + +`**[src/syncback/ref_properties.rs](src/syncback/ref_properties.rs)`:** + +`link_referents()` currently writes path-based (`Rojo_Ref`_*) using instance-name paths. Update to use filesystem-name paths (the dedup'd filenames assigned during syncback). + +`collect_referents()` currently uses `is_path_unique` to decide between path-based and ID-based linking. With the dedup system, ALL paths are unique (filesystem names are unique by definition). Simplify: always use path-based linking for file-backed instances. Fall back to `$id`/`Rojo_Target`_* only for instances without filesystem backing (inside .rbxm files). + +### 3d. Update patch_apply.rs + +`**[src/snapshot/patch_apply.rs](src/snapshot/patch_apply.rs)`:** + +Update the ref resolution in `finalize_patch_application()` to use the new filesystem-name-based `get_instance_by_path()`. + +### 3e. Ref path update infrastructure + +Extend `RefPathIndex` and `update_ref_paths_after_rename()` in [change_processor.rs](src/change_processor.rs) to handle all cases: + +- Instance rename -> slug change -> all refs to it and descendants update +- Dedup reassignment -> suffix change -> all refs update +- Reparenting -> entire ancestor path changes -> all refs update +- Script type transition -> extension change -> all refs through this instance update +- File-to-folder transition -> segment format change -> all refs update +- Batch writes: collect ALL affected ref updates, write each file once + +--- + +## Phase 4: Two-Way Sync Integration + +### 4a. change_processor.rs + +`**[src/change_processor.rs](src/change_processor.rs)`:** + +- **VFS echo prevention for dedup renames:** When suffix cleanup renames files (e.g., `Attachment~1` -> `Attachment`), call `suppress_path()` / `suppress_path_any()` for both old and new paths before the rename. **New instance additions must NOT be suppressed** -- the VFS watcher needs to pick up new files for forward sync. +- **Dedup suffix cleanup on deletion:** When a child deletion triggers suffix cleanup (base-name promotion or group-to-1), perform the rename and ref path updates. +- **Self-initiated rename tagging:** Tag dedup-triggered renames so `handle_vfs_event()` recognizes them as self-initiated and skips re-snapshotting. + +### 4b. api.rs + +`**[src/web/api.rs](src/web/api.rs)`:** + +- **Two-pass /api/write:** First pass: apply ALL mutations (renames, property changes, adds, removes) to the tree. Second pass: recompute all affected ref slug paths based on final tree state. This ensures refs always use newest slugs. +- **4 file format branches in `syncback_updated_properties()`:** Each branch must compute the correct dedup'd filename for the meta/model file. When an instance has a dedup suffix, the meta path includes it: `Foo~1.meta.json5`, `Foo~1/init.meta.json5`, etc. +- **Added instances in dedup groups:** `syncback_added_instance()` must check for filesystem-name collisions with existing siblings and assign appropriate ~N suffix. +- **Dual write path consistency:** File writes and PatchSet sent to ChangeProcessor must use the same dedup'd names. + +--- + +## Phase 5: Plugin Updates + +### 5a. Lua matching algorithm (plugin/src/Reconciler/matching.lua -- new file) + +`**[plugin/src/Reconciler/hydrate.lua](plugin/src/Reconciler/hydrate.lua)`:** + +Replace the current name+class greedy matching with the 3-pass algorithm: + +- **Pass 1:** Unique name matching (same as Rust) +- **Pass 2:** Ref discriminators. Server sends virtual instances with UIDs. Plugin resolves UIDs via instanceMap. If a Ref property on a virtual instance points to UID-7 (already matched in P1), and only one Studio candidate has the equivalent Ref, match. +- **Pass 3:** Structural fingerprinting with XXH32. Fingerprint = hash of (child count + sorted child name+class pairs). Do NOT hash full property values (too expensive). For actual similarity scoring, compare properties directly with early-exit ordering: ClassName -> Tags -> Attributes -> scalar properties -> Source (last, most expensive). + +`**[plugin/src/XXH32.luau](plugin/src/XXH32.luau)`** -- New file containing the XXH32 implementation from the spec (lines 239-350 of the prompt). + +**Caching:** Compute structural fingerprints LAZILY (only for instances entering Pass 3). Cache for duration of the matching call, not across calls. + +### 5b. ChangeBatcher updates + +`**[plugin/src/ChangeBatcher/init.lua](plugin/src/ChangeBatcher/init.lua)`:** Handle duplicate-named instances flowing through the batcher without skipping. + +### 5c. Confirmation dialog + +Dedup suffix renames, ref path updates, and meta file changes triggered by the matching algorithm must appear in the review UI in Always and Initial sync modes. No silent application. + +--- + +## Phase 6: Cherry-Picks from rbxm-ambiguous + +Selectively take these from the `rbxm-ambiguous` branch: + +- **Plugin relaxations** (diff.lua, encodeInstance.lua) -- already described in Phase 1b +- **Roundtrip test improvements** (`[tests/tests/syncback_roundtrip.rs](tests/tests/syncback_roundtrip.rs)`) -- Vec-based `compare_children` (handles duplicates correctly), Tags comparison in `compare_properties` +- **project.rs case-insensitive detection** -- case-insensitive name counting for duplicate detection (adapt for dedup instead of skip) +- **macOS ref_properties test ignore** -- platform-specific test attribute + +**Do NOT take:** `ambiguous_container` metadata, `find_rbxm_container`, `reserialize_rbxm_container`, rbxm fallback block, ambiguous test fixtures that output .rbxm files. + +**Non-requirement: no .rbxm transition logic.** Syncback rewrites the filesystem from scratch (clean) or matches instances to existing files (incremental) -- either way, old .rbxm containers simply don't get recreated. User-intentional .rbxm files placed manually in their project still load fine via `snapshot_rbxm` during forward sync -- that's unchanged. Do not build migration code for converting old .rbxm files. + +--- + +## Phase 7: Testing + +### Shared test vectors + +Create structured test data (JSON or Rust structs) defining: + +- Two lists of instance descriptions (name, class, properties, refs, children) +- Expected match pairing output + +Both Rust implementations and the Lua implementation must produce identical pairings for all vectors. + +### Test scenarios (minimum required from spec) + +**Dedup:** Two identical Folders, Folders with different children, Folders with identical content (tiebreaker test), Folder + Script same name (no dedup), Folder + init-style Script (dedup), natural tilde name ("Foo~1" slugifies differently), third duplicate added, suffix cleanup (group-to-1), gap tolerance, base-name promotion, idempotency, roundtrip. + +**Ref paths:** Basic ref survives syncback, ref to dedup'd instance, ref survives gap-tolerant deletion, ref survives base-name promotion, ref survives group-to-1, ref survives rename, ref survives reparent, deeply nested through dedup layers, cross-file ref update, ref to deleted instance, tilde-named instance ref, two-way sync ref update, concurrent ref+rename, forward sync manual edit, fallback to $id, mass ref stress test (50+ instances). + +**Matching:** ClassName discriminator, Ref discriminator, property similarity, matching stability, preserves existing pairings, fast path verification. + +**Dedup collision:** Folder + ModuleScript (no collision), Folder + Folder (collision), Server + Client script (no collision), Server + Server (collision), ModuleScript + ModuleScript, slug collision ("Foo?" + "Foo*"), case-insensitive collision ("Foo" + "foo"). + +**Cross-system integration:** Full roundtrip with duplicates, incremental syncback after rename, two-way sync add/delete, forward sync edit dedup'd file, slugify+dedup interaction, project file + filesystem duplicates. + +**Sync invariants:** CLI syncback parity, 4 file format branches, compound extension dedup, InstigatingSource after dedup, file-to-folder transition with dedup, script type transition in dedup group, dual write path consistency, review dialogue with dedup, atomicity, determinism, idempotency, pre-seeded dedup. + +### Test infrastructure + +- Syncback test fixtures in `[rojo-test/syncback-tests/](rojo-test/syncback-tests/)` +- Build test fixtures in `[rojo-test/build-tests/](rojo-test/build-tests/)` +- Integration tests in `[tests/tests/](tests/tests/)` +- Generate .rbxm fixtures programmatically (see `tests/generate_ambiguous_fixtures.rs` from rbxm-ambiguous branch) + +--- + +## Key Invariants to Preserve + +1. **Round-trip identity:** syncback -> build -> compare = bit-identical trees +2. **CLI syncback parity:** two-way sync produces byte-identical filesystem output to CLI syncback +3. **Determinism:** same input always produces same filesystem output (sorted child ordering, deterministic suffix assignment) +4. **INSTIGATING SOURCE consistency:** dedup'd paths reflected in InstigatingSource::Path +5. **Compound extensions:** suffix before compound extension (`Foo~1.server.luau`, not `Foo.server~1.luau`) +6. **Meta pairing:** `Foo~1.server.luau` pairs with `Foo~1.meta.json5` (strip script suffix, keep dedup suffix) + +--- + +## Key Files Changed + + +| File | Change | +| --------------------------------------------- | ------------------------------------------------------------ | +| `src/syncback/file_names.rs` | Dedup key = full FS name; suffix on stem before compound ext | +| `src/syncback/dir.rs` (snapshot_middleware) | Two-pass child processing; remove blacklist | +| `src/syncback/matching.rs` (NEW) | 3-pass matching algorithm for syncback | +| `src/snapshot/matching.rs` (NEW) | 3-pass matching algorithm for forward sync | +| `src/snapshot/patch_compute.rs` | Replace greedy matching with algorithm | +| `src/snapshot/patch_apply.rs` | Update ref resolution for FS-name paths | +| `src/snapshot/tree.rs` | `get_instance_by_path()` uses FS names | +| `src/rojo_ref.rs` | FS-name paths; remove escape/split; update `ref_target_path` | +| `src/syncback/ref_properties.rs` | FS-name-based linking; simplify path uniqueness | +| `src/change_processor.rs` | VFS echo for dedup renames; suffix cleanup | +| `src/web/api.rs` | Two-pass write; dedup in all 4 branches | +| `src/snapshot_middleware/project.rs` | Replace bail with dedup; case-insensitive | +| `plugin/src/Reconciler/hydrate.lua` | 3-pass matching algorithm (Lua) | +| `plugin/src/Reconciler/diff.lua` | Remove blacklist | +| `plugin/src/ChangeBatcher/encodeInstance.lua` | Remove blacklist | +| `plugin/src/XXH32.luau` (NEW) | Hash function for Lua-side fingerprinting | + + diff --git a/.cursor/plans/ambiguous_paths_audit_fixes_9d3ed6da.plan.md b/.cursor/plans/ambiguous_paths_audit_fixes_9d3ed6da.plan.md new file mode 100644 index 00000000..a26e32c7 --- /dev/null +++ b/.cursor/plans/ambiguous_paths_audit_fixes_9d3ed6da.plan.md @@ -0,0 +1,310 @@ +--- +name: Ambiguous paths audit fixes +overview: "Fix 6 issues found during the ambiguous-paths branch audit: make Rust matching recursive, fix ref path format in api.rs added_paths, restructure syncback ref collection for dedup-aware paths, improve dedup cleanup grouping precision, extract shared trueEquals module, and correct a misleading comment." +todos: + - id: fix1-recursive-matching + content: "Fix 1: Make Rust matching implementations recursive (both forward sync and syncback) to match the Lua plugin behavior. Add depth limit and best_so_far pruning." + status: completed + - id: fix2-added-paths + content: "Fix 2: Replace escape_ref_path_segment with slugify_name in api.rs added_paths to fix ref path format mismatch." + status: completed + - id: fix3-dedup-refs + content: "Fix 3: Restructure syncback ref collection to use final dedup'd filesystem names instead of tentative paths. Fix misleading comment." + status: completed + - id: fix4-dedup-cleanup + content: "Fix 4: Change dedup cleanup in change_processor to match by full filesystem name (stem + extension) instead of stem only." + status: completed + - id: fix5-shared-trueequals + content: "Fix 5: Extract trueEquals to a shared Lua module to eliminate duplication between matching.lua and diff.lua." + status: completed + - id: run-tests + content: Write all required tests and run CI to verify everything passes. + status: completed +isProject: false +--- + +# Ambiguous Path Handling Audit Fixes + +> This plan was generated by `/audit` from a Plan-mode session. +> Refer to `.cursor/rules/atlas.mdc` for full project standards. + +## Standards + +These standards govern every fix in this plan. An implementer MUST read `.cursor/rules/atlas.mdc` before starting. + +### Round-Trip Identity (from `atlas.mdc`) + +Syncback (or two-way sync) writes a directory tree. Building an rbxl from that directory tree and forward-syncing it back must produce a **bit-identical instance tree** -- same names, same classes, same properties, same hierarchy, same ref targets. Any deviation is a bug. + +### CLI Syncback Parity (from `atlas.mdc`) + +Plugin-based sync must produce **exactly the same filesystem output** as `atlas syncback`. Any divergence is a bug until proven otherwise. + +### Code Quality (from `atlas.mdc`) + +DRY code that is easy to reason about and hard to get wrong. The same slugify/dedup/meta-update pattern should not be copy-pasted across files. Small refactors (2-3 call sites) are included as fixes. Major rewrites (10+ call sites) are deferred. + +## Context + +Auditing all changes on `ambiguous-paths` branch (10 commits, 41 files, +4275/-1020 lines) against `origin/master`. The feature replaces the duplicate-skip blacklist with a matching algorithm (3 implementations), filesystem-collision-based dedup, filesystem-name-based ref paths, dedup suffix cleanup, and plugin updates. + +The audit traced round-trip identity through all sync paths and found 3 critical issues, 3 correctness concerns, and 2 code quality items. + +--- + +## Fixes + +Each fix below was approved by the user during the audit quiz. Implement them in order. Each fix includes its test requirements -- a fix is not complete until its tests pass. + +### Fix 1: Make Rust matching implementations recursive + +- **Status:** Approved +- **Finding:** C1/CC3 -- Rust matching is flat (non-recursive) while Lua is recursive, causing pairing divergence +- **Files:** + - [src/snapshot/matching.rs](src/snapshot/matching.rs) -- forward sync matching + - [src/syncback/matching.rs](src/syncback/matching.rs) -- syncback matching +- **Problem:** Both Rust `count_own_diffs` functions only compare top-level properties and add +1 for children count difference. They do NOT recursively score subtrees. The `UNMATCHED_PENALTY` constant (10,000) is defined but never used. For ambiguous groups where instances have identical top-level properties but different children, Rust scores them identically (0 cost), falling back to child order. The Lua implementation correctly recurses into subtrees and applies `UNMATCHED_PENALTY` for unmatched children. +- **Solution:** +**For `src/snapshot/matching.rs` (forward sync):** + 1. Rename `count_own_diffs` to `count_own_diffs_only` (internal helper). + 2. Create new `compute_change_count(snap: &InstanceSnapshot, inst: &InstanceWithMeta, tree: &RojoTree, best_so_far: u32) -> u32`: + - Call `count_own_diffs_only` for the flat property cost. + - If cost >= best_so_far, early return. + - Recursively call `match_forward_internal` on children (a new variant that returns matched pairs + unmatched counts without consuming snapshots -- use `&[InstanceSnapshot]` and `&[Ref]` refs). + - For each matched pair, recursively call `compute_change_count`. If cost >= best_so_far, early return. + - Add `UNMATCHED_PENALTY * unmatched_count` to cost. + 3. In the ambiguous group scoring loop, replace `count_own_diffs(snap, &inst)` with `compute_change_count(snap, &inst, tree, best_so_far)`. + 4. The `match_forward` function needs a variant that doesn't consume snapshots (borrows instead). Create `match_forward_scoring(snap_children: &[InstanceSnapshot], tree_children: &[Ref], tree: &RojoTree) -> ScoringResult` that returns `(matched_count, unmatched_snap_count, unmatched_tree_count, Vec<(usize, usize)>)` -- index pairs for recursive scoring. + 5. **Depth limit:** Add a `depth: u32` parameter to `compute_change_count`. At depth > 3 (or configurable), fall back to flat comparison to prevent O(n^k) explosion on deeply nested identical trees. + **For `src/syncback/matching.rs` (syncback):** + Same transformation: create `compute_change_count` that recursively matches children and scores subtrees. The hash fast-path remains: if new_hash == old_hash, return 0 immediately (skips recursion). When hashes differ, recurse into children. + **Key constraint:** The recursive scoring functions must NOT allocate heavily in the inner loops. Use stack-based recursion with early exit on `best_so_far`. The hash fast-path in syncback makes this efficient -- most subtrees are identical and short-circuit at depth 0. +- **Risk:** Medium. Recursive matching is more expensive than flat matching. The `best_so_far` pruning and depth limit mitigate this. Hash fast-path in syncback means most comparisons never recurse. Forward sync doesn't have hashes but the (Name, ClassName) grouping limits the cross-product size. +- **Verify round-trip:** Create 10 same-named Parts with different subtrees, syncback them, rebuild. The pairings should be stable across syncback + forward sync. +- **Verify syncback parity:** N/A (matching algorithm is internal, output is the same files) +- **Tests required:** + - Rust unit test in `src/snapshot/matching.rs`: 10 Parts named "Line" with different Transparency values, reversed order in tree. Verify each pair has matching Transparency (proves recursive scoring produces correct pairing). + - Rust unit test in `src/syncback/matching.rs`: Same test with WeakDom instances. + - Rust unit test: Identical instances (all properties same, same children) -- verify greedy falls back to child order (stable). + - Rust unit test: Depth limit test -- deeply nested tree (depth > 5) completes in reasonable time. + +### Fix 2: Replace `escape_ref_path_segment` with filesystem-compatible names in `added_paths` + +- **Status:** Approved +- **Finding:** C2 -- `added_paths` mixes filesystem-name parent path with escape-encoded child name +- **Files:** + - [src/web/api.rs](src/web/api.rs) lines 685-686 +- **Problem:** `crate::escape_ref_path_segment(&added.name)` produces escaped instance names (e.g., `"Hey\/Bro"`) but `get_instance_by_path` splits on `/` without unescaping. An instance named `"Hey/Bro"` gets path segment `"Hey\/Bro"` which splits into `["Hey\", "Bro"]`, failing resolution. Should use the same filesystem name format as `ref_target_path_from_tree`. +- **Solution:** Replace lines 685-686 with: + +```rust + // Use slugified name (matching filesystem conventions) instead of + // escape_ref_path_segment (which uses legacy \/ escaping). + let fs_name = if crate::syncback::name_needs_slugify(&added.name) { + crate::syncback::slugify_name(&added.name) + } else { + added.name.clone() + }; + let path = if parent_path.is_empty() { + fs_name + } else { + format!("{}/{}", parent_path, fs_name) + }; + + +``` + + Note: This doesn't include the file extension (e.g., `.server.luau`) because added instances don't have middleware info at this point. The `get_instance_by_path` fallback uses case-insensitive instance name matching, which will handle the resolution. For a more complete fix, compute the tentative extension based on `added.class_name` and `added.children.is_empty()`, similar to `tentative_fs_name` in `ref_properties.rs`. + +- **Risk:** Low. Only affects ref properties referencing just-added instances with `/` or `\` in their names. +- **Verify round-trip:** Add an instance named `"Hey/Bro"` via two-way sync, then verify a ref property pointing to it resolves correctly after a server restart. +- **Verify syncback parity:** N/A (two-way sync only, CLI syncback doesn't use added_paths) +- **Tests required:** + - Rust integration test in `tests/tests/two_way_sync.rs`: Send a `/api/write` with an added instance named `"A/B"` and an update that references it via Ref. Verify the Ref resolves to the correct tree instance after patch application. + +### Fix 3: Restructure syncback ref collection for dedup-aware paths + +- **Status:** Approved +- **Finding:** C3 -- `tentative_fs_path` doesn't include dedup suffixes, producing ambiguous paths for duplicate-named instances +- **Files:** + - [src/syncback/ref_properties.rs](src/syncback/ref_properties.rs) -- `tentative_fs_path`, `collect_referents`, comment + - [src/syncback/mod.rs](src/syncback/mod.rs) -- `syncback_loop` orchestration + - [src/syncback/file_names.rs](src/syncback/file_names.rs) -- `name_for_inst` (produces dedup'd names) + - [src/snapshot_middleware/dir.rs](src/snapshot_middleware/dir.rs) -- `syncback_dir_no_meta` (calls `name_for_inst`) +- **Problem:** `collect_referents()` calls `tentative_fs_path()` which computes paths BEFORE dedup suffix assignment. Two instances with the same name/class get identical paths (e.g., both `"Workspace/Data"`). The `Rojo_Ref`_* attribute stores an ambiguous path that resolves to the first matching child, potentially the wrong one. +The comment at line ~147 says "ALL filesystem paths are unique" which is misleading -- tentative paths aren't unique for duplicate-named instances. +- **Solution:** +**Phase 1: Correct the comment (immediate)** +Replace the misleading comment in `collect_referents` with: + +```rust + // NOTE: tentative_fs_path() does not include dedup suffixes (~N). + // For instances with duplicate names under the same parent, this + // produces ambiguous paths. Resolution via get_instance_by_path() + // returns the first matching child, which may be wrong. + // + // To fully fix this, collect_referents must run AFTER dedup suffix + // assignment, with access to the final filesystem names. See Phase 2. + + +``` + + **Phase 2: Pass dedup assignments to ref collection** + +1. In `syncback_loop()` (`src/syncback/mod.rs`), after the recursive `syncback_dir_no_meta` / middleware dispatch completes (which assigns filenames via `name_for_inst`), build a `HashMap` mapping each instance's Ref to its FINAL dedup'd filesystem path. +2. Pass this map to `collect_referents()` as a new parameter `final_paths: &HashMap`. In `collect_referents`, when building the target path, look up the target Ref in `final_paths` first. If found, use that path. If not found (external ref, pruned instance), fall back to `tentative_fs_path`. +3. The `final_paths` map is built bottom-up during the syncback walk. Each middleware's `syncback()` call returns a `SyncbackReturn` which contains the `fs_snapshot` with file paths. Extract the Ref→path mapping from the `SyncbackSnapshot` path + the child assignments. + **Implementation detail:** The simplest approach is to add a `ref_path_map: &mut HashMap` parameter to `SyncbackSnapshot::with_joined_path` and record the mapping there. Each time a child gets a filesystem name assigned, record `(child_ref, parent_path + "/" + filename)` in the map. + +- **Risk:** Medium. Changes the syncback flow to thread a map through the recursive walk. Must be careful not to break incremental mode. The `final_paths` map is only populated during syncback, so forward sync is unaffected. +- **Verify round-trip:** Create two Folders named "Data" under Workspace, add a Script with PrimaryPart pointing to the second "Data". Syncback, rebuild, verify PrimaryPart points to the correct "Data" (the one with dedup suffix `~1`). +- **Verify syncback parity:** The CLI syncback output must include the correct `Rojo_Ref_`* attribute with the dedup'd path (e.g., `"Workspace/Data~1"`). Two-way sync should produce the same path. +- **Tests required:** + - Rust integration test in `tests/tests/syncback_roundtrip.rs`: Two same-named Folders with a ref property pointing to the second one. Verify the `Rojo_Ref_`* attribute contains the dedup'd path (e.g., `"Workspace/Data~1"`). + - Rust unit test: `collect_referents` with `final_paths` containing dedup'd paths. Verify the output uses dedup'd paths instead of tentative paths. + +### Fix 4: Dedup cleanup should match by full filesystem name, not stem alone + +- **Status:** Approved +- **Finding:** CC1 -- Change processor groups dedup siblings by stem only, ignoring extension +- **Files:** + - [src/change_processor.rs](src/change_processor.rs) lines 1098-1125 +- **Problem:** The dedup cleanup code extracts the stem by splitting on `'.'` and groups siblings by base stem. `Foo.server.luau` (stem `Foo`) and `Foo.luau` (stem `Foo`) are grouped together even though they have different dedup keys (`foo.server.luau` vs `foo.luau`) and are NOT in the same dedup group. This inflates `remaining_stems.len()` and can change the cleanup action. +- **Solution:** When finding siblings in the dedup group, also compare extensions. Extract the extension from the removed instance's filesystem name, and only include siblings whose extension matches. Specifically: + +```rust + // Extract the compound extension (everything after the stem) + let removed_extension = removed_fs_name + .find('.') + .map(|i| &removed_fs_name[i + 1..]); + + for &sibling_ref in parent_inst.children() { + if sibling_ref == removed_id { continue; } + let sibling_fs = tree.filesystem_name_for(sibling_ref); + let sibling_stem = sibling_fs.split('.').next().unwrap_or(&sibling_fs); + let sibling_ext = sibling_fs.find('.').map(|i| &sibling_fs[i + 1..]); + + // Only group siblings with the same extension + if sibling_ext != removed_extension { continue; } + + let sibling_base = match parse_dedup_suffix(sibling_stem) { + Some((b, _)) => b, + None => sibling_stem, + }; + if sibling_base.eq_ignore_ascii_case(&base_stem) { + remaining_stems.push(sibling_stem.to_string()); + } + } + + +``` + +- **Risk:** Very low. Makes grouping more precise, no behavior change for the common case (all siblings in a dedup group have the same extension by definition). +- **Verify round-trip:** Create siblings `Foo.server.luau` (Script) and `Foo.luau` (ModuleScript) and `Foo~1.server.luau` (Script). Delete `Foo.server.luau` via two-way sync. Verify `Foo~1.server.luau` is renamed to `Foo.server.luau` and `Foo.luau` is untouched. +- **Verify syncback parity:** N/A (dedup cleanup is a two-way sync operation) +- **Tests required:** + - Rust unit test in `src/syncback/dedup_suffix.rs`: Verify `compute_cleanup_action` with mixed-extension input (though this tests the helper, the real fix is in change_processor). + - Rust integration test or manual verification: Mixed-extension siblings deletion scenario as described above. + +### Fix 5: Extract `trueEquals` to a shared module + +- **Status:** Approved +- **Finding:** CC2 -- `trueEquals` (and `fuzzyEq`) duplicated between `matching.lua` and `diff.lua` +- **Files:** + - New: [plugin/src/Reconciler/trueEquals.lua](plugin/src/Reconciler/trueEquals.lua) + - [plugin/src/Reconciler/matching.lua](plugin/src/Reconciler/matching.lua) -- remove local copy, require shared + - [plugin/src/Reconciler/diff.lua](plugin/src/Reconciler/diff.lua) -- remove local copy, require shared +- **Problem:** `trueEquals` is copy-pasted between two files. If either is updated without the other, matching and diff will disagree on value equality, causing phantom changes (matching says "equal" but diff says "different"). +- **Solution:** + 1. Create `plugin/src/Reconciler/trueEquals.lua` containing `fuzzyEq` and `trueEquals`. Export `trueEquals` as the module return value. + 2. In `matching.lua`, remove the local `fuzzyEq` and `trueEquals` definitions (lines 27-111). Add `local trueEquals = require(script.Parent.trueEquals)`. + 3. In `diff.lua`, remove the local `fuzzyEq` and `trueEquals` definitions (lines 19-115). Add `local trueEquals = require(script.Parent.trueEquals)`. +- **Risk:** Very low. Pure refactor, no behavior change. +- **Verify round-trip:** Existing tests cover both callers. +- **Verify syncback parity:** N/A (plugin-only change) +- **Tests required:** + - Existing diff.spec.lua and diff.stress.spec.lua tests verify equality behavior. No new tests needed. + +--- + +## Skipped Fixes + +### Skipped: Remove unused `UNMATCHED_PENALTY` from Rust matching files + +- **Finding:** Q1 -- dead constant in both Rust matching files +- **Problem:** `UNMATCHED_PENALTY` defined but never used +- **Reason skipped:** Fix 1 (recursive matching) will start using this constant, making removal unnecessary. + +--- + +## Accepted Limitations + +None -- all limitations flagged were approved for fixing. + +--- + +## Deferred Refactors + +- **Matching cache (Phase 2e from original plan):** Add optional cache mapping `parent_ref -> (children_list_hash, match_assignments)` to avoid re-running the full algorithm when children haven't changed. Not needed for correctness, optimization only. +- **Duplicated name-to-index grouping pattern:** All three matching implementations repeat the same "build HashMap of name to indices" pattern. Could be factored into a shared helper, but the three implementations operate on different types (InstanceSnapshot, WeakDom, Lua tables) making a shared abstraction non-trivial. Flagged for consideration after the matching implementations stabilize. + +--- + +## Test Plan + +Tests are not optional. Every approved fix must have test coverage. A fix is not complete until its tests pass (or intentionally fail, for known limitations). + +### Rust Unit Tests (`#[cfg(test)]` blocks) + +- **10 same-named Parts with different properties, reversed order:** Verify recursive matching pairs correctly by property content, not position. (Fix 1) +- **Identical instances (all same):** Verify stable child-order tiebreaker. (Fix 1) +- **Depth limit test:** Deeply nested tree (depth > 5) completes without hanging. (Fix 1) +- `**collect_referents` with `final_paths`:** Verify dedup'd paths used instead of tentative. (Fix 3) + +### Rust Integration Tests (`tests/tests/`) + +- **Two-way sync with added instance named `"A/B"`:** Verify ref resolution works with slugified names. (Fix 2) +- **Syncback roundtrip with duplicate-named Folders + ref:** Verify `Rojo_Ref`_* contains dedup'd path. (Fix 3) + +### Lua Spec Tests (`.spec.lua` files) + +- Existing diff.spec.lua and diff.stress.spec.lua cover `trueEquals` behavior. (Fix 5) + +### Snapshot Tests (`insta` crate) + +- Existing snapshot tests should be re-verified after recursive matching change. Run `cargo insta review` after Fix 1. (Fix 1) + +--- + +## Test Rules + +The implementer MUST follow these rules when writing tests for this plan: + +### For bugs found and fixed + +Every bug fix MUST have a test that: + +1. Would have **FAILED** before the fix +2. **PASSES** after the fix +3. Prevents the bug from regressing + +### For missing coverage identified + +For each "missing test coverage" gap: + +1. Write the test +2. Verify it passes against the current implementation +3. If it fails, investigate -- it may have found another bug + +### Scope: test every affected layer + +For each finding, determine which test layers are affected and write tests in ALL of them: + +- Does the bug manifest at the plugin level? -> Lua spec test +- Does it manifest at the server level? -> Rust unit test or integration test +- Does it affect the round-trip? -> Integration test with filesystem verification +- Could it regress from a different code path? -> Test that code path too + +## Final Step: Run CI + +After ALL fixes and tests are implemented, run the `/ci` command (see `.cursor/commands/ci.md`) to execute the full CI pipeline. Every fix must pass CI before the plan is considered complete. Do not skip this step. \ No newline at end of file diff --git a/.cursor/plans/audit_ambiguous_path_handling_30e99d9a.plan.md b/.cursor/plans/audit_ambiguous_path_handling_30e99d9a.plan.md new file mode 100644 index 00000000..e6726a3f --- /dev/null +++ b/.cursor/plans/audit_ambiguous_path_handling_30e99d9a.plan.md @@ -0,0 +1,212 @@ +--- +name: Audit Ambiguous Path Handling +overview: Production audit of the Ambiguous Path Handling feature (matching algorithm, dedup system, ref path system). Identifies 7 critical/correctness issues, 3 code quality items, and extensive missing test coverage." +todos: + - id: c1-pass3-double-match + content: "Fix: Forward sync pass3 can re-match pass1-matched items (snap_available not consumed)" + status: completed + - id: c2-case-sensitivity + content: "Fix: get_instance_by_path inconsistent case sensitivity between primary and fallback" + status: completed + - id: c3-projectnode-path + content: "Fix: ref_target_path_from_tree uses project filename instead of instance name for ProjectNode" + status: completed + - id: c5-return-value + content: "Verify: encodeInstance return value change doesn't break callers" + status: completed + - id: c6-dead-code + content: "Fix: Remove dead code in ref_properties.rs (compute_refs_with_duplicate_siblings, is_path_unique_with_cache)" + status: completed + - id: c7-ref-path-format + content: "Critical: ref_properties.rs writes instance-name paths but tree.rs expects filesystem-name paths" + status: completed + - id: q2-hashset-perf + content: "Fix: Use HashSet instead of Vec for matched tracking in syncback matching" + status: completed + - id: c8-matching-lua-require-crash + content: "Critical: matching.lua has wrong require path for Log module - crashes on load, breaks hydration entirely" + status: completed + - id: q3-unused-import + content: "Fix: Remove unused Log import in Lua matching module (after fixing c8, Log is still unused)" + status: completed + - id: t1-syncback-integration + content: "Test: Add syncback integration tests for duplicate-named instances" + status: completed + - id: t3-new-fn-tests + content: "Test: Add tests for ref_target_path_from_tree and filesystem_name_for" + status: completed +isProject: false +--- + +# Audit: Ambiguous Path Handling Feature + +> This plan was generated by `/audit` from a Plan-mode session. +> Refer to `.cursor/rules/atlas.mdc` for full project standards. + +## Standards + +### Round-Trip Identity (from `atlas.mdc`) + +Syncback writes a directory tree. Building an rbxl from that tree and forward-syncing it back must produce a **bit-identical instance tree**. Any deviation is a bug. + +### CLI Syncback Parity (from `atlas.mdc`) + +Plugin-based sync must produce **exactly the same filesystem output** as `atlas syncback`. Any divergence is a bug. + +### Code Quality (from `atlas.mdc`) + +DRY code, no duplicated slugify/dedup/meta patterns across files. + +## Context + +Auditing all changes on `ambiguous-paths` branch (4 commits, 24 files, +3560/-825 lines) against `origin/master`. The feature replaces the duplicate-skip blacklist with a 3-pass matching algorithm, filesystem-collision-based dedup, and filesystem-name-based ref paths. + +--- + +## Critical Issues + +### C1. Forward sync matching does not filter already-matched items in Pass 3 + +**File:** [src/snapshot/matching.rs](src/snapshot/matching.rs), `pass3_similarity()` line 199-262 + +**Problem:** `pass3_similarity` builds its own `snap_by_name` / `tree_by_name` maps from ALL items in `snap_available` / `tree_children`, including items already matched in Pass 1 (which are marked only via `snap_matched[si]=true` / `tree_available[ti]=false` in the caller). However, `snap_by_name` iterates `snap_available` checking `if let Some(snap) = snap_opt` -- but snap_available entries are never set to `None` after Pass 1 matching. This means Pass 3 **re-groups items already matched in Pass 1** into its name maps. The `snap_matched` guard inside the pair-building loop catches this for snap items, but the `tree_by_name` check is `if !tree_available[i]` which IS correctly applied. The snap side relies on `snap_matched[si]` which is initialized to `vec![false; ...]` at the top of pass3 -- it does NOT inherit the pass1 match state. + +**Impact:** Pass 3 will attempt to re-match snapshots already matched in Pass 1, potentially creating duplicate matched pairs in the output. The `build_result()` function does `.take()` on snap entries, so the second `.take()` would return `None` and the pair would be silently dropped. This means the unmatched counts would be wrong (items appear in neither matched nor unmatched). + +**Fix:** Either set `snap_available[si] = None` for pass1-matched items before calling pass3, or initialize pass3's `snap_matched` from the caller's `snap_matched` state. The simplest fix: filter `snap_by_name` to only include indices where `snap_available[i].is_some()` AND the index was not already matched. Since we use a separate `snap_matched` bool vec, we can check: the `snap_available` entries for pass1-matched items were never consumed (they're still `Some`). We need to either mark them `None` or pass the matched state. + +### C2. `get_instance_by_path` uses case-insensitive comparison for filesystem names but case-SENSITIVE for fallback + +**File:** [src/snapshot/tree.rs](src/snapshot/tree.rs), `get_instance_by_path()` line ~248 + +**Problem:** The primary lookup uses `fs_name.eq_ignore_ascii_case(segment)` but the fallback name lookup uses `child.name == *segment` (case-sensitive). This inconsistency means: + +- A ref path `"workspace/Part"` would match filesystem name `"Workspace"` (case-insensitive) in the primary check +- But if the instance had no `instigating_source`, the fallback would fail because `"Workspace" != "workspace"` + +More critically, this creates an asymmetry: `ref_target_path_from_tree()` builds paths from filesystem names preserving their original case, but resolution uses case-insensitive matching. This means two distinct instances `"Foo.luau"` and `"foo.luau"` (which should be dedup'd for cross-platform safety) would both match the same path segment. + +**Fix:** Use case-insensitive comparison in BOTH the primary and fallback paths. Or use exact case matching in both (since filesystem names are deterministic and paths should round-trip exactly). + +### C3. `ref_target_path_from_tree` falls back to instance name for ProjectNode instances, creating inconsistent paths + +**File:** [src/rojo_ref.rs](src/rojo_ref.rs), `ref_target_path_from_tree()` line ~157-173 + +**Problem:** When an instance has `InstigatingSource::ProjectNode`, the code falls through to the `else` branch (no instigating_source) and uses `inst.name.clone()`. But `InstigatingSource::ProjectNode` DOES have a path -- `source.path()` returns the project file path, whose `file_name()` would be something like `default.project.json5`. This is wrong -- for project nodes, the segment should be the instance name (not the project filename). + +The current code handles this correctly by accident: the `if let Some(source) = &meta.instigating_source` check matches `ProjectNode` (it has a source), so it uses `source.path().file_name()` which returns the project filename like `"default.project.json5"`. This produces WRONG ref path segments for project-sourced instances -- a service like "Workspace" would get segment `"default.project.json5"` instead of `"Workspace"`. + +**Fix:** Check for `InstigatingSource::ProjectNode` specifically and use the instance name, not the project file path. The `instigating_source.path()` for ProjectNode returns the project file itself, not the instance's filesystem representation. + +### C4. Lua matching `_pass1NameAndClass` has index invalidation bug when removing from arrays + +**File:** [plugin/src/Reconciler/matching.lua](plugin/src/Reconciler/matching.lua), lines 168-190 + +**Problem:** After collecting matched indices, the code sorts them in reverse order and calls `table.remove(remainingVirtual, idx)` / `table.remove(remainingStudio, idx)` for each. However, if Pass 1 matched both a unique-name pair AND a class-narrowed pair from the same name group, the same index could theoretically appear twice in `matchedVirtualIndices`. The deduplication check (`seenV[idx]`) handles this. + +But the more fundamental issue: indices collected during the matching loop refer to positions in `remainingVirtual` at the START of the function. As `table.remove` shifts elements, subsequent indices become invalid. The reverse-order sort mitigates this for sequential removals, but if indices from DIFFERENT name groups interleave (e.g., index 3 from name "A" and index 2 from name "B"), removing index 3 first then index 2 works. However, if the same index appears for both a unique match and a class match (shouldn't happen since unique match `continue`s), the dedup prevents double-removal. + +**Real bug:** The virtual and studio index lists are independent. `matchedVirtualIndices` contains indices into `remainingVirtual`, `matchedStudioIndices` contains indices into `remainingStudio`. Removing from one array doesn't affect the other. But removing from `remainingVirtual` in reverse order IS correct for that array alone. The issue is that indices 1, 3, 5 removed in order 5, 3, 1 correctly removes the right elements. + +Actually, this is correct IF no duplicates. But there's a subtle issue: the class narrowing loop can match items whose indices overlap with the unique-name match. Since the unique-name match does `continue` (skips class narrowing for that name), this shouldn't happen. **Low risk but fragile.** + +### C5. `encodeInstance.lua` return value change breaks callers expecting 2 return values + +**File:** [plugin/src/ChangeBatcher/encodeInstance.lua](plugin/src/ChangeBatcher/encodeInstance.lua), bottom + +**Problem:** The old function returned `(result, skippedDuplicateCount)` -- two values. The new code returns just `encodeInstance(instance, parentId, false)` -- one value. Per the Lua silent argument mismatch rule, callers that destructured the second return value now silently get `nil` instead of a count. Need to verify all callers. + +**Verification needed:** Search for all callers of `encodeInstance` / `encodeInstanceWithDuplicateTracking` to confirm none destructure the second return value. + +### C6. Dead code left behind in `ref_properties.rs` + +**File:** [src/syncback/ref_properties.rs](src/syncback/ref_properties.rs), lines 54-131 + +**Problem:** `compute_refs_with_duplicate_siblings()` and `is_path_unique_with_cache()` are now dead code (clippy confirms: "function is never used"). The `collect_referents()` function was simplified to always use path-based linking, but the old helper functions were not removed. This is dead code that will confuse future maintainers. + +### C7. `ref_properties.rs` always uses instance-name paths, not filesystem-name paths + +**File:** [src/syncback/ref_properties.rs](src/syncback/ref_properties.rs), `collect_referents()` line ~170 + +**Problem:** The simplified `collect_referents()` calls `inst_path(dom, *target_ref)` which uses the legacy `ref_target_path()` (instance-name-based with escaping). This means Rojo_Ref_* attributes written during syncback still use the OLD path format (instance names), not the new filesystem-name format. But `get_instance_by_path()` in tree.rs now expects filesystem-name segments. This creates a **format mismatch**: syncback writes instance-name paths, forward sync tries to resolve them as filesystem-name paths. + +The fallback in `get_instance_by_path()` (case-sensitive instance name match) partially mitigates this, but it's fragile and won't work for instances whose name differs from their filesystem name (slugified names, dedup'd names). + +**Impact:** Critical round-trip violation for any instance with a slugified or dedup'd name that is referenced by a Ref property. The ref path written during syncback won't resolve during forward sync. + +**Fix:** `collect_referents()` needs to build filesystem-name paths. This requires access to the filesystem names assigned during syncback (via instigating_source or a Ref-to-filesystem-name map built during the syncback pass). + +--- + +### C8. Lua matching module has wrong `require` path -- crashes on load, breaks hydration + +**File:** [plugin/src/Reconciler/matching.lua](plugin/src/Reconciler/matching.lua), line 16 + +**Problem:** `local Log = require(script.Parent.Parent.Log)` resolves to `plugin/src/Log` which does not exist. Every other file in `plugin/src/Reconciler/` uses `local Packages = script.Parent.Parent.Parent.Packages` followed by `require(Packages.Log)`. Since Luau's `require()` executes immediately at module load time, this incorrect path causes a runtime error when `matching.lua` loads, which prevents the 3-pass matching algorithm from functioning and breaks hydration entirely (`hydrate.lua` line 15 requires `matching`). + +**Impact:** The plugin cannot connect to the server at all. Hydration is the first thing that runs after the initial `/api/read` -- if matching.lua fails to load, the entire sync session crashes. + +**Fix:** Replace line 16 with: + +```lua +local Packages = script.Parent.Parent.Parent.Packages +local Log = require(Packages.Log) +``` + +--- + +## Code Quality Issues + +### Q1. Duplicated name→index grouping pattern across 3 matching modules + +All three matching implementations (syncback, forward sync, Lua plugin) repeat the same pattern: build `HashMap>` for both sides, iterate names, check counts. This is ~40 lines repeated 6+ times across the codebase. Should be factored into a shared helper (or at minimum documented as intentional duplication per the spec requirement of separate implementations). + +### Q2. `matched_new.contains(r)` uses O(n) linear scan in syncback matching + +**File:** [src/syncback/matching.rs](src/syncback/matching.rs), lines 196, 262, 358, 400 + +The `matched_new.contains()` / `matched_old.contains()` checks are O(n) linear scans. For large ambiguous groups (10+ duplicates), this creates O(n^2) behavior within Pass 2 and Pass 3. Should use `HashSet` instead of `Vec` for the matched tracking sets. + +### Q3. Unused `Log` import in Lua matching module + +**File:** [plugin/src/Reconciler/matching.lua](plugin/src/Reconciler/matching.lua), line 16 + +`local Log = require(script.Parent.Parent.Log)` is imported but never used. Will trigger selene lint warning. + +--- + +## Missing Test Coverage + +### T1. No integration tests for duplicate-named instances through syncback + +The feature's core value proposition -- syncing duplicate-named instances -- has zero integration tests. The only tests are unit tests for the matching algorithm and dedup suffix cleanup. Need syncback test fixtures with: + +- Two Folders with same name +- Folder + Script same name (no collision) +- Init-style collision (Folder + Script with children) +- Roundtrip test with duplicates + +### T2. No tests for forward sync with duplicate-named instances + +`compute_children_patches` now uses the matching algorithm, but there are no tests verifying that editing one of two same-named files correctly pairs to the right tree instance. + +### T3. No tests for `ref_target_path_from_tree` or `filesystem_name_for` + +New functions with zero test coverage. Critical for ref path correctness. + +### T4. No tests for dedup key semantics change + +The `deduplicate_name_with_ext` function is tested via updated existing tests, but there are no tests specifically for the Folder+Script no-collision scenario or init-style collision scenario. + +### T5. No Lua spec tests for the matching module + +The Lua matching module has zero tests. The `hydrate.lua` changes are untested. + +--- + +## Deferred Refactors + +- **Syncback ref path format migration**: `collect_referents()` needs to build filesystem-name paths instead of instance-name paths. This requires restructuring the syncback flow to make filesystem name assignments available during ref collection. (Blocked on: syncback flow restructure) +- **Matching cache (Phase 2e)**: Deferred optimization, not needed for correctness. +- **change_processor dedup cleanup integration**: The `dedup_suffix.rs` module provides cleanup rules but they aren't integrated into the change processor's deletion handling yet. + diff --git a/.cursor/plans/audit_ambiguous_paths_d9495fcc.plan.md b/.cursor/plans/audit_ambiguous_paths_d9495fcc.plan.md new file mode 100644 index 00000000..85196078 --- /dev/null +++ b/.cursor/plans/audit_ambiguous_paths_d9495fcc.plan.md @@ -0,0 +1,280 @@ +--- +name: Audit Ambiguous Paths +overview: "Audit of the \"Ambiguous Path Handling\" feature: three parallel matching algorithm implementations (2 Rust, 1 Lua), Reconciler integration, and encoding changes." +todos: + - id: fix-stale-comment + content: "RESOLVED: encodeInstance.lua header comment already updated in prior session" + status: pending + - id: align-tags-attrs-scoring + content: "RESOLVED by matching_algorithm_parity plan (Fix 2): Rust countOwnDiffs now has granular Tags/Attributes scoring" + status: pending + - id: ref-identity-rust + content: "RESOLVED by matching_algorithm_parity plan (Fix 1): Rust countOwnDiffs now has Ref-target-aware scoring" + status: pending + - id: double-cost-perf + content: "Fix: eliminate double cost computation in Lua matchChildren totalCost loop by tracking scored costs from ambiguous loop and reusing them" + status: completed + - id: trueequals-types + content: "Fix: add UDim, UDim2, NumberRange, NumberSequence, ColorSequence, Rect fuzzy equality to trueEquals.lua" + status: completed +isProject: false +--- + +# Audit: Ambiguous Path Handling (Reconciler Changes) + +## Scope + +Files audited: + +- [plugin/src/Reconciler/matching.lua](plugin/src/Reconciler/matching.lua) -- Lua matching algorithm (631 lines) +- [plugin/src/Reconciler/matching.spec.lua](plugin/src/Reconciler/matching.spec.lua) -- Lua matching tests +- [plugin/src/Reconciler/hydrate.lua](plugin/src/Reconciler/hydrate.lua) -- Integration with matching +- [plugin/src/Reconciler/reify.lua](plugin/src/Reconciler/reify.lua) -- Nil-instance guards for ambiguous exclusions +- [plugin/src/Reconciler/trueEquals.lua](plugin/src/Reconciler/trueEquals.lua) -- Shared value equality +- [plugin/src/Reconciler/diff.lua](plugin/src/Reconciler/diff.lua) -- Uses shared trueEquals +- [plugin/src/ChangeBatcher/encodeInstance.lua](plugin/src/ChangeBatcher/encodeInstance.lua) -- Duplicate detection helpers +- [src/snapshot/matching.rs](src/snapshot/matching.rs) -- Rust forward sync matching +- [src/syncback/matching.rs](src/syncback/matching.rs) -- Rust syncback matching +- [src/variant_eq.rs](src/variant_eq.rs) -- Rust property comparison + +--- + +## Architecture Summary + +```mermaid +flowchart TD + subgraph luaPlugin ["Lua Plugin (Studio)"] + Hydrate["hydrate.lua"] + LuaMatch["matching.lua"] + TrueEquals["trueEquals.lua"] + Hydrate -->|"calls"| LuaMatch + LuaMatch -->|"uses"| TrueEquals + end + + subgraph rustForward ["Rust Forward Sync"] + PatchCompute["patch_compute.rs"] + FwdMatch["snapshot/matching.rs"] + VariantEq["variant_eq.rs"] + PatchCompute -->|"calls"| FwdMatch + FwdMatch -->|"uses"| VariantEq + end + + subgraph rustSyncback ["Rust Syncback"] + SbEntry["syncback entry"] + SbMatch["syncback/matching.rs"] + SbEntry -->|"calls"| SbMatch + SbMatch -->|"uses"| VariantEq + end +``` + + + +All three implementations use the same algorithm: + +1. **Fast path**: Group by `(Name, ClassName)` -- 1:1 groups instant-match +2. **Ambiguous groups**: Pairwise recursive `computeChangeCount` scoring +3. **Greedy assignment**: Sort by cost ascending, pick lowest-cost unmatched pair +4. **Depth limit**: `MAX_SCORING_DEPTH = 3` prevents O(n^k) explosion +5. **Session caching**: `MatchingSession` avoids redundant recursive scoring + +--- + +## Findings + +### FINDING 1: Stale header comment in encodeInstance.lua (Low -- misleading docs) + +The file header at lines 12-16 states: + +```1:17:plugin/src/ChangeBatcher/encodeInstance.lua +--[[ + // ... + DUPLICATE HANDLING: + - Instances with duplicate-named siblings at ANY level of the path are skipped + - This matches the behavior in ref_properties.rs::is_path_unique + - A path is ambiguous if duplicates exist anywhere from the instance to the root + - A warning is logged when duplicates are encountered +]] +``` + +But the actual code at lines 159-168 does **not** skip duplicates -- it logs a debug message and encodes normally: + +```159:168:plugin/src/ChangeBatcher/encodeInstance.lua +encodeInstance = function(instance, parentId, _skipPathCheck) + -- Log duplicate-named siblings as a debug message (server handles them via + -- dedup suffixes) but do NOT skip encoding. + if not _skipPathCheck and not isPathUnique(instance) then + Log.debug( + "Instance '{}' ({}) has duplicate-named siblings in path (server will handle via dedup)", + instance:GetFullName(), + instance.ClassName + ) + end +``` + +The inline comment (line 161) is correct; the file-level docblock is stale from a prior implementation. **Fix: update the header comment.** + +--- + +### FINDING 2: Tags/Attributes granularity divergence across implementations (Medium -- scoring asymmetry) -- RESOLVED + +> **Resolved by [matching_algorithm_parity](matching_algorithm_parity_01524d4c.plan.md) Fix 2** (status: completed). Rust `countOwnDiffs` in both forward sync and syncback now extracts Tags/Attributes and scores them granularly (per-tag, per-attribute), matching the Lua behavior. Parity tests added (Fix 6). + +~~The three `countOwnDiffs` implementations score Tags and Attributes differently.~~ + +Original analysis preserved below for reference: + +The Lua implementation (`matching.lua` lines 282-307) counts each individual tag/attribute diff as +1. The Rust implementations previously counted the entire `Tags`/`Attributes` Variant as a single +1 diff. This meant 3 of 10 tags differing scored +1 in Rust but +3 in Lua, potentially causing different pairings. + +--- + +### FINDING 3: Lua Ref scoring uses identity heuristic not present in Rust (Medium -- divergence) -- RESOLVED + +> **Resolved by [matching_algorithm_parity](matching_algorithm_parity_01524d4c.plan.md) Fix 1** (status: completed). Both Rust `countOwnDiffs` functions now resolve Ref targets to `(Name, ClassName)` and score mismatches at `UNMATCHED_PENALTY`, matching the Lua behavior. Rust unit tests `ref_scoring_forward_sync` and `ref_scoring_syncback` added. + +~~The Rust implementations had no equivalent of Lua's Ref identity heuristic.~~ + +Original analysis preserved below for reference: + +The Lua `countOwnDiffs` scored Ref property mismatches using `RefIdentity` (name + className) at a cost of `UNMATCHED_PENALTY` (10,000). The Rust implementations compared `Variant::Ref` values as opaque IDs (+1 per mismatch), which were always different between contexts, making Refs useless for disambiguation. + +--- + +### FINDING 4: Performance -- double cost computation in Lua matchChildren (Low -- perf only) + +In `matchChildren`, ambiguous group costs are computed inline (lines 534-557) for assignment decisions but never stored in `session.costCache`. Then lines 604-607 recompute total cost by calling `computePairCost` for every matched pair: + +```604:608:plugin/src/Reconciler/matching.lua + local totalCost = 0 + for _, pair in ipairs(matched) do + totalCost += computePairCost(session, pair.virtualId, pair.studioInstance, virtualInstances, math.huge, depth) + end + totalCost += (#unmatchedVirtual + #unmatchedStudio) * UNMATCHED_PENALTY +``` + +This means ambiguous group pairs have their full subtree cost computed **twice**: once during the scoring loop, once during totalCost. For 1:1 fast-path pairs, cost is computed **once** (only in totalCost), which is correct. + +**Impact**: Wasted computation proportional to ambiguous group size squared. Since ambiguous groups are typically small (2-10 instances) and the depth limit caps recursion, this is acceptable. The `computePairCost` call does populate `session.costCache`, so subsequent recursive calls at deeper levels benefit. + +**Fix (deferred)**: Store the computed cost from the ambiguous scoring loop and reuse it in the totalCost sum instead of calling `computePairCost` again. + +--- + +### FINDING 5: `trueEquals` lacks fuzzy equality for compound float types (Low -- edge case) + +`trueEquals.lua` provides fuzzy float comparison for: `number`, `Color3`, `Vector3`, `Vector2`, `CFrame`. It falls through to `return false` for all other types (line 111). + +The Rust `variant_eq.rs` provides fuzzy comparison for additional float-containing types: `UDim`, `UDim2`, `NumberRange`, `NumberSequence`, `ColorSequence`, `Rect`, `PhysicalProperties`, `Ray`, `Region3`, `Float32`, `Float64`, `Font`. + +In Luau, `UDim2 == UDim2` uses `__eq` which may or may not apply float tolerance. If Roblox's `__eq` does exact comparison, two UDim2 values that differ by float precision would fail `==` and then fail `trueEquals` (which returns false for unrecognized types). This would inflate match costs slightly. + +**Impact**: Unlikely to cause wrong pairings since the inflation would be symmetric across all candidates. But it could trigger unnecessary reconciler updates (diff sees a change that isn't really a change). + +**Fix (deferred)**: Add UDim2, NumberRange, NumberSequence, Rect handlers to `trueEquals`. These are uncommon in ambiguous matching scenarios but would improve diff accuracy. + +--- + +### FINDING 6: Reconciler integration is clean (Positive) + +**hydrate.lua** (lines 33-45) correctly filters out missing virtual children before calling `matchChildren`, and passes a shared `MatchingSession` through recursive hydration: + +```32:48:plugin/src/Reconciler/hydrate.lua + -- Filter out virtual children missing from virtualInstances + local validVirtualIds: { string } = {} + for _, childId in ipairs(virtualChildIds) do + if virtualInstances[childId] then + table.insert(validVirtualIds, childId) + else + Log.warn( + "Hydration: virtualInstances missing child ID {} (parent: {} '{}')", + childId, + rootId, + virtualInstance.Name + ) + end + end + + local result = + Matching.matchChildren(session, validVirtualIds, existingChildren, virtualInstances, rootId, rootInstance) +``` + +**reify.lua** (lines 52-57) defensively handles nil virtual instances (excluded from patch): + +```52:57:plugin/src/Reconciler/reify.lua + if virtualInstance == nil then + -- This child was intentionally excluded from the patch (e.g., due to + -- ambiguous paths with duplicate-named siblings). Skip it gracefully. + return + end +``` + +**diff.lua** (line 133) uses the shared `trueEquals` module, ensuring matching and diff agree on equality: + +```133:133:plugin/src/Reconciler/diff.lua + if not trueEquals(existingValue, decodedValue) then +``` + +Session sharing through recursive hydration is correct. The `--!native` and `--!optimize 2` directives on `matching.lua` and `trueEquals.lua` ensure native codegen for hot paths. + +--- + +### FINDING 7: Test coverage is solid but lacks cross-implementation parity tests (Low) -- RESOLVED + +> **Resolved by [matching_algorithm_parity](matching_algorithm_parity_01524d4c.plan.md) Fix 6** (status: completed). Parity tests added: `parity_forward_vs_syncback` (Rust), `round_trip_matching_identity` (Rust integration), `parity_with_rust_fixture` (Lua spec). + +**Lua tests** (`matching.spec.lua`): 15+ test cases covering session caching, totalCost computation, Ref scoring, 2x2 and 3x3 ambiguous groups, recursive matching, edge cases, and now cross-implementation parity fixtures. + +**Rust forward sync tests** (`src/snapshot/matching.rs`): 12+ test cases including property-based matching, depth limit, session cache, default-omission tests, and Ref scoring. + +**Rust syncback tests** (`src/syncback/matching.rs`): 6+ test cases covering basics, class grouping, unmatched, duplicates, stability, session cache, and Ref scoring. + +**Integration test**: `ref_ambiguous_path_no_crash` and `ref_ambiguous_target_no_crash` in `tests/tests/two_way_sync.rs` verify that Ref properties with ambiguous paths don't crash. + +--- + +## Summary Table + + +| # | Finding | Severity | Status | Fix Effort | +| --- | -------------------------------------------- | -------- | -------- | ------------------ | +| 1 | Stale header comment in encodeInstance.lua | Low | RESOLVED | -- (prior session) | +| 2 | Tags/Attributes granularity divergence | Medium | RESOLVED | -- (parity plan) | +| 3 | Lua Ref identity heuristic missing in Rust | Medium | RESOLVED | -- (parity plan) | +| 4 | Double cost computation in Lua matchChildren | Low | **Fix** | Small | +| 5 | trueEquals lacks compound float types | Low | **Fix** | Small | +| 6 | Reconciler integration is clean | Positive | N/A | N/A | +| 7 | No cross-implementation parity tests | Low | RESOLVED | -- (parity plan) | + + +Additionally, the parity plan also resolved a float epsilon divergence (Fix 3: Rust `variant_eq.rs` now uses a 0.0001 epsilon matching Lua's `trueEquals`), and a VCache session-level caching optimization (Fix 5), partially mitigating Finding 4. + +**Overall assessment**: The ambiguous path handling feature is well-implemented and correct. The matching algorithm is sound, the three implementations are now fully aligned (via the parity plan fixes), and the integration points (hydrate, reify, diff) are clean. No round-trip identity violations found. + +--- + +## Actionable Fixes + +### Fix A: Eliminate double cost computation in Lua matchChildren (Finding 4) + +- **File:** `plugin/src/Reconciler/matching.lua` +- **Problem:** In `matchChildren`, the ambiguous scoring loop (lines 528-575) computes full subtree costs for each pair, but these costs are only stored in `scoredPairs` for sorting. Then the totalCost loop (lines 615-618) calls `computePairCost` for **every** matched pair (including ambiguous ones), recomputing the same costs from scratch. The `computePairCost` function does check `session.costCache`, but the ambiguous scoring loop never writes to it, so nothing is cached. +- **Solution:** Track the winning cost for each matched pair during greedy assignment and sum them directly, instead of calling `computePairCost` again. For 1:1 fast-path matches (which have no pre-computed cost), call `computePairCost` only for those. + +Concretely in `matchChildren`: + +1. Add a `matchedCosts: { [number]: number }` table (keyed by match index in `matched` array) +2. In the ambiguous greedy assignment loop (lines 585-595), store the pair's cost from `scoredPairs` alongside each match +3. In the totalCost loop, use the stored cost for ambiguous pairs and only call `computePairCost` for 1:1 fast-path pairs (which don't have a pre-computed cost) + +### Fix B: Add compound float type fuzzy equality to trueEquals (Finding 5) + +- **File:** `plugin/src/Reconciler/trueEquals.lua` +- **Problem:** `trueEquals` handles `number`, `Color3`, `Vector3`, `Vector2`, `CFrame` with fuzzy float comparison, but falls through to `return false` for `UDim`, `UDim2`, `NumberRange`, `NumberSequence`, `ColorSequence`, `Rect`. Since these types contain floats, Luau's `==` operator may fail on precision differences, causing phantom diffs in both matching and diff. +- **Solution:** Add handlers before the final `return false` (line 111): + - `UDim2`: decompose via `.X.Scale`, `.X.Offset`, `.Y.Scale`, `.Y.Offset` -- fuzzyEq on Scale, exact on Offset (integer) + - `UDim`: decompose via `.Scale`, `.Offset` -- fuzzyEq on Scale, exact on Offset + - `NumberRange`: fuzzyEq on `.Min`, `.Max` + - `Rect`: fuzzyEq on `.Min.X`, `.Min.Y`, `.Max.X`, `.Max.Y` + - `NumberSequence`: compare `.Keypoints` length, then per-keypoint fuzzyEq on `.Time`, `.Value`, `.Envelope` + - `ColorSequence`: compare `.Keypoints` length, then per-keypoint fuzzyEq on `.Time` + color3Eq on `.Value` + +These match the Rust `variant_eq.rs` coverage exactly. \ No newline at end of file diff --git a/.cursor/plans/audit_ambiguous_paths_followup_00831072.plan.md b/.cursor/plans/audit_ambiguous_paths_followup_00831072.plan.md new file mode 100644 index 00000000..f83f38e2 --- /dev/null +++ b/.cursor/plans/audit_ambiguous_paths_followup_00831072.plan.md @@ -0,0 +1,231 @@ +--- +name: audit ambiguous paths followup +overview: Production audit of the ambiguous-paths branch against origin/master. Documents critical round-trip issues, approved fixes, accepted limitations, and required regression tests for implementation in a follow-up Agent-mode session. +todos: + - id: fix-metadata-drift + content: "Fix 1: Update dedup cleanup rename flow to keep tree metadata/path indexes in sync" + status: completed + - id: fix-batch-removal-dedup + content: "Fix 2: Exclude co-removed siblings from dedup cleanup sibling enumeration" + status: completed + - id: fix-added-paths + content: "Fix 3: Extend added_paths in api.rs to include middleware extension and dedup suffix" + status: completed + - id: fix-ref-path-final + content: "Fix 4: Complete Phase 2 of previous audit Fix 3 -- pass final_paths to collect_referents instead of global text rewrite" + status: completed + - id: fix-regression-tests + content: "Fix 5: Add regression tests for Fixes 1-4" + status: completed +isProject: false +--- + +# Ambiguous Paths Audit Fix Plan + +> This plan was generated by `/audit` from a Plan-mode session. +> Refer to `.cursor/rules/atlas.mdc` for full project standards. + +## Standards + +These standards govern every fix in this plan. An implementer MUST read `.cursor/rules/atlas.mdc` before starting. + +### Round-Trip Identity (from `atlas.mdc` §Quality Standard) + +Syncback (or two-way sync) writes a directory tree. Building an rbxl from that directory tree and forward-syncing it back must produce a bit-identical instance tree -- same names, same classes, same properties, same hierarchy, same ref targets. Any deviation is a bug. + +### CLI Syncback Parity (from `atlas.mdc` §Two-Way Sync Strategy) + +Plugin-based sync must produce exactly the same filesystem output that `atlas syncback` would give for the same input. Byte-for-byte identical files, identical directory structures, identical naming. Any divergence is a bug until proven otherwise. + +### Code Quality (from `atlas.mdc` §Code Quality Standard) + +DRY code that is easy to reason about and hard to get wrong. Small helper refactors are in scope; major pipeline redesign is deferred. + +## Context + +Audited branch `ambiguous-paths` against `origin/master` using commit range `origin/master..HEAD`. + +Primary findings: + +- Critical path drift in dedup cleanup rename flow (`[src/change_processor.rs](d:/UserGenerated/rojo/src/change_processor.rs)`). +- Critical batch-removal dedup cleanup counts co-removed siblings as "remaining", defeating group-to-1 cleanup (`[src/change_processor.rs](d:/UserGenerated/rojo/src/change_processor.rs)`). +- Critical same-batch Ref target path synthesis mismatch for added instances (`[src/web/api.rs](d:/UserGenerated/rojo/src/web/api.rs)`). +- Critical ref-path post-processing strategy relies on global text replacement (`[src/syncback/mod.rs](d:/UserGenerated/rojo/src/syncback/mod.rs)`, `[src/syncback/fs_snapshot.rs](d:/UserGenerated/rojo/src/syncback/fs_snapshot.rs)`). +- Known limitation accepted for this release: duplicate-named children under ProjectNode currently skipped in `[src/snapshot_middleware/project.rs](d:/UserGenerated/rojo/src/snapshot_middleware/project.rs)`. +- Known limitation accepted for this release: greedy matcher is not globally optimal (`[src/snapshot/matching.rs](d:/UserGenerated/rojo/src/snapshot/matching.rs)`, `[src/syncback/matching.rs](d:/UserGenerated/rojo/src/syncback/matching.rs)`, `[plugin/src/Reconciler/matching.lua](d:/UserGenerated/rojo/plugin/src/Reconciler/matching.lua)`). + +## Fixes + +Each fix below was approved by the user during the audit quiz. Implement in order. + +### Fix 1: Keep tree metadata consistent after dedup cleanup rename + +- **Status:** Approved +- **Finding:** Step 2d / Step 8 (rename/delete interactions + VFS suppression) +- **Files:** `[src/change_processor.rs](d:/UserGenerated/rojo/src/change_processor.rs)` +- **Problem:** In dedup cleanup (`~1073-1203`), the code renames files (`fs::rename`) and suppresses both old/new paths, but does not update the renamed sibling's `InstigatingSource::Path` or metadata path indexes. Subsequent writes can target stale paths. +- **Solution:** After successful `DedupCleanupAction::RemoveSuffix` / `PromoteLowest`, locate the sibling instance being renamed, update its metadata source path and relevant path indexes in-memory, and keep suppression counters consistent with the updated path. +- **Risk:** Medium; touches tree mutation bookkeeping and rename lifecycle. +- **Verify round-trip:** Create duplicate siblings (`Foo`, `Foo~1`), delete `Foo`, ensure `Foo~1 -> Foo` rename occurs, then issue a property update to the survivor in same session and verify disk write targets new path and survives rebuild. +- **Verify syncback parity:** Compare output of two-way deletion flow to CLI syncback on equivalent tree state; both must produce the same remaining filename/meta layout. +- **Tests required:** + - Rust integration test in `[tests/tests/two_way_sync.rs](d:/UserGenerated/rojo/tests/tests/two_way_sync.rs)` covering delete-triggered dedup cleanup followed by update. + - Optional focused unit test around cleanup metadata path update helper in `[src/change_processor.rs](d:/UserGenerated/rojo/src/change_processor.rs)`. + +### Fix 2: Exclude co-removed siblings from dedup cleanup sibling enumeration + +- **Status:** Approved +- **Files:** `[src/change_processor.rs](d:/UserGenerated/rojo/src/change_processor.rs)` +- **Problem:** In the dedup cleanup loop (lines 1077-1203), the sibling enumeration (line 1120) only skips `sibling_ref == removed_id` -- the single instance being processed in the current iteration. It does NOT skip other instances that are also in `patch_set.removed_instances`. When multiple siblings from the same dedup group are removed in a single patch, the co-removed instances are incorrectly counted as "remaining", inflating the sibling count and defeating cleanup rules. +- **Failure path:** Remove `Foo` (A) and `Foo~1` (B) from a group of 3, leaving `Foo~2` (C) as the sole survivor. api.rs deletes all files first. In the dedup cleanup loop, iteration for A finds B and C as remaining (count=2), so group-to-1 does not fire; instead `PromoteLowest` tries to rename the already-deleted `Foo~1` → `Foo` (fails). Iteration for B finds A and C as remaining (count=2), group-to-1 again does not fire. Result: C keeps its `~2` suffix permanently when it should have been renamed to `Foo`. +- **Solution:** Before the dedup cleanup loop, build a `HashSet` from `patch_set.removed_instances`. In the sibling enumeration (line 1121), change the skip condition from `sibling_ref == removed_id` to `removed_set.contains(&sibling_ref)`: + +```rust +let removed_set: HashSet = patch_set.removed_instances.iter().copied().collect(); + +// ... inside sibling loop: +for &sibling_ref in parent_inst.children() { + if removed_set.contains(&sibling_ref) { + continue; + } + // ... rest of sibling processing +} +``` + +- **Risk:** Low; narrow, mechanical change. The HashSet construction is O(n) and the lookup is O(1). +- **Verify round-trip:** Remove 2 of 3 duplicate siblings via two-way sync. Verify the sole survivor has its dedup suffix removed and survives rebuild with the clean name. +- **Verify syncback parity:** N/A (dedup cleanup is a two-way sync operation only). +- **Tests required:** + - Rust integration test in `[tests/tests/two_way_sync.rs](d:/UserGenerated/rojo/tests/tests/two_way_sync.rs)`: batch-remove 2 of 3 same-named siblings, verify survivor is renamed from `Foo~2` to `Foo`. + - Edge test: batch-remove ALL members of a dedup group (no survivor); verify no spurious rename attempts. + +### Fix 3: Build correct same-batch added Ref paths + +- **Status:** Approved +- **Lineage:** Extends `ambiguous_paths_audit_fixes_9d3ed6da` Fix 2 (completed), which replaced `escape_ref_path_segment` with `slugify_name` in `added_paths` but explicitly acknowledged the missing extension as a limitation. The current code at lines 637-641 of `api.rs` uses `slugify_name`; this fix adds the remaining segments. +- **Finding:** Step 2a / Step 10 (plugin->server wire format and write processing) +- **Files:** `[src/web/api.rs](d:/UserGenerated/rojo/src/web/api.rs)` +- **Problem:** `added_paths` map (`~622-649`) uses only slugified instance name (from the previous fix); it still omits middleware extension and dedup suffix. `syncback_updated_properties` then writes incorrect `Rojo_Ref_`* when target is newly added in same request (`~2694-2706`). +- **Solution:** Extend the existing `added_paths` builder (lines 637-649) to derive the full filesystem name: slugified name + middleware extension based on `added.class_name` and `added.children.is_empty()` + dedup suffix if colliding with siblings. Reuse `tentative_fs_name` logic from `ref_properties.rs` or the shared helper from the Deferred Refactors section. +- **Risk:** Medium; affects same-request Ref persistence edge path. +- **Verify round-trip:** Send `/api/write` with add(target script/model) + update(ref property pointing to target) in one batch; verify persisted path includes correct segment (`.server.luau`, `.model.json5`, or dedup suffix) and resolves after rebuild. +- **Verify syncback parity:** Run equivalent scenario via CLI syncback fixture and compare written `Rojo_Ref_`* attribute string. +- **Tests required:** + - Rust integration test in `[tests/tests/two_way_sync.rs](d:/UserGenerated/rojo/tests/tests/two_way_sync.rs)` for same-batch add+ref on file-backed and directory-backed targets. + - Edge test for dedup suffix target (e.g., `Foo~1`). + +### Fix 4: Complete attribute-scoped final paths (previously incomplete) + +- **Status:** Approved +- **Finding:** Step 2a / Step 9 / Step 11h (round-trip identity, parity, determinism) +- **Lineage:** `ambiguous_paths_audit_fixes_9d3ed6da` Fix 3 defined a two-phase fix. Phase 1 (correct misleading comment) was completed -- visible at lines 194-199 of `ref_properties.rs`. Phase 2 (pass `final_paths` to `collect_referents`) was NOT completed despite the todo being marked "completed". The API was prepared (`final_paths: Option<&HashMap>` parameter exists at line 188 of `ref_properties.rs`) but the call site at line 177 of `mod.rs` still passes `None`. The `ref_path_map` infrastructure was built by `ref_path_and_dedup_integration_81a47dfe` and is populated during the syncback walk (used at lines 614-637 of `mod.rs` for the text replacement). This fix completes that unfinished Phase 2. +- **Files:** `[src/syncback/mod.rs](d:/UserGenerated/rojo/src/syncback/mod.rs)`, `[src/syncback/ref_properties.rs](d:/UserGenerated/rojo/src/syncback/ref_properties.rs)`, `[src/syncback/fs_snapshot.rs](d:/UserGenerated/rojo/src/syncback/fs_snapshot.rs)` +- **Problem:** `collect_referents(&new_tree, &pre_prune_paths, None)` is called at line 177 of `mod.rs` BEFORE the syncback walk that populates `ref_path_map`. The `None` means tentative paths (without dedup suffixes) are written into `Rojo_Ref_`* attributes. Then the post-hoc `fix_ref_paths` (lines 614-637) performs global text replacement across entire meta/model file contents to substitute tentative paths with final dedup'd paths. This is not attribute-scoped and can alter unrelated strings that happen to contain the same substring as a tentative path. +- **Existing infrastructure to build on:** + 1. `collect_referents` already accepts `final_paths: Option<&HashMap>` (line 188 of `ref_properties.rs`). + 2. `ref_path_map` is already populated during the syncback walk with `(Ref, final_path)` entries. + 3. `tentative_fs_path_public` is already used at line 623 of `mod.rs` to compute tentative paths for the substitution diff. +- **Solution:** Move the `collect_referents` call from line 177 (before the syncback walk) to after the walk completes (after line 612). Pass `Some(&ref_path_map.borrow())` instead of `None`. Then remove or guard the `fix_ref_paths` block (lines 614-637) -- it should no longer be needed for any path that has a `final_paths` entry. +- **Ordering constraint:** `collect_referents` is currently called before property filtering (lines 179-198). Moving it after the syncback walk means it runs after filtering. Verify that `collect_referents` does not depend on properties that are removed by filtering (it should not -- it only reads Ref-type properties, which are not in the filtered set unless `ignore_referents` is true, in which case refs are skipped entirely). +- **Risk:** Medium; changes the ordering of the syncback pipeline. The infrastructure is already in place, reducing implementation risk. +- **Verify round-trip:** Scenario with duplicate-named ref target and non-ref string property containing same substring as tentative path; ensure only `Rojo_Ref_`* changes and non-ref string is untouched. +- **Verify syncback parity:** Compare CLI syncback and two-way outputs for refs targeting dedup-suffixed instances; paths must be byte-identical. +- **Tests required:** + - Rust integration test in `[tests/tests/syncback_roundtrip.rs](d:/UserGenerated/rojo/tests/tests/syncback_roundtrip.rs)` validating non-ref string preservation while refs are rewritten correctly. + - Rust integration test in `[tests/tests/two_way_sync.rs](d:/UserGenerated/rojo/tests/tests/two_way_sync.rs)` for dedup-aware ref persistence without text rewrite side effects. + +### Fix 5: Add regression coverage for approved fixes + +- **Status:** Approved +- **Finding:** Step 13 (missing coverage) +- **Files:** `[tests/tests/two_way_sync.rs](d:/UserGenerated/rojo/tests/tests/two_way_sync.rs)`, `[tests/tests/syncback_roundtrip.rs](d:/UserGenerated/rojo/tests/tests/syncback_roundtrip.rs)`, optionally helper fixtures under `[rojo-test/syncback-tests](d:/UserGenerated/rojo/rojo-test/syncback-tests)`. +- **Problem:** Current tests do not fully guard the four approved bug classes. +- **Solution:** Add targeted regression tests tied to Fixes 1-4 and ensure they fail pre-fix and pass post-fix. +- **Risk:** Low. +- **Verify round-trip:** Covered by added integration scenarios. +- **Verify syncback parity:** Add assertions comparing resulting file paths/attribute values against expected CLI-format output. +- **Tests required:** + - All tests listed under Fixes 1-4. + - One deterministic re-run assertion (second run no changes) for affected scenarios. + +## Skipped Fixes + +### Skipped: ProjectNode duplicate-child handling rewrite + +- **Finding:** Step 2d / Step 11k +- **Problem:** Duplicate-named children under ProjectNode are currently skipped with warnings (`[src/snapshot_middleware/project.rs](d:/UserGenerated/rojo/src/snapshot_middleware/project.rs)` around `~587-683`). +- **Reason skipped:** User accepted as release limitation for now. + +### Skipped: Globally optimal matcher replacement + +- **Finding:** Step 11h (determinism/assignment quality) +- **Problem:** Greedy assignment may produce suboptimal pairing in rare ambiguous groups. +- **Reason skipped:** User accepted greedy behavior for this release. + +## Accepted Limitations + +- **Limitation:** ProjectNode duplicate-named children are skipped during syncback. +- **Scenario:** Under project-backed nodes, duplicate names are detected and omitted from child maps; warning logged. +- **User decision:** Accepted for this release. +- **Mitigation:** Keep explicit warning logs; add failing acceptance test (below) to track until fixed. +- **Limitation:** Greedy matching is not globally optimal. +- **Scenario:** Ambiguous duplicate groups can choose a local minimum causing rename churn. +- **User decision:** Accepted for this release. +- **Mitigation:** Add stress/failing test for known counterexample and keep deterministic tie behavior. + +## Deferred Refactors + +- Consolidate filesystem-segment derivation into one shared helper used by `[src/rojo_ref.rs](d:/UserGenerated/rojo/src/rojo_ref.rs)`, `[src/snapshot/tree.rs](d:/UserGenerated/rojo/src/snapshot/tree.rs)`, and `[src/web/api.rs](d:/UserGenerated/rojo/src/web/api.rs)` to eliminate duplicate naming logic. *(Continuing from `ambiguous_paths_audit_fixes_9d3ed6da` Deferred Refactors -- still open.)* +- Revisit optimal assignment algorithm for large ambiguous groups (major rewrite; out of current scope). *(Continuing from `ambiguous_paths_audit_fixes_9d3ed6da` Deferred Refactors -- still open.)* +- Phase 2e matching cache from the original plan (`ambiguous_path_handling_f2eaef11`): add optional cache mapping `parent_ref -> (children_list_hash, match_assignments)` to avoid re-running the full algorithm when children haven't changed. Optimization only, not needed for correctness. *(Never completed or cancelled across any subsequent plan -- explicitly deferred here.)* + +## Test Plan + +### Rust Unit Tests (`#[cfg(test)]`) + +- Dedup cleanup metadata update helper behavior after rename (Fix 1). +- Added-path segment generation helper returns correct extension/suffix across middleware types (Fix 3). + +### Rust Integration Tests (`tests/tests/`) + +- Delete base dedup name then update promoted sibling in same serve session; verify writes hit new path and persist after rebuild (Fix 1). +- Batch-remove 2 of 3 same-named siblings; verify sole survivor is renamed from `Foo~2` to `Foo` (Fix 2). +- Batch-remove all members of a dedup group; verify no spurious rename attempts (Fix 2). +- Same `/api/write` request: add target + set Ref to target, verify persisted path includes expected extension/suffix (Fix 3). +- Ref rewrite safety: non-ref string fields are unchanged while `Rojo_Ref_`* gets final dedup path (Fix 4). + +### Lua Spec Tests (`.spec.lua`) + +- No mandatory Lua-side changes for approved fixes; optional guard test for same-batch add+ref behavior if plugin-side helper touched. + +### Snapshot Tests (`insta`) + +- Add or update syncback snapshot fixture showing final dedup-aware `Rojo_Ref_`* output without incidental string changes (Fix 4). + +### Failing Tests for Known Limitations + +- ProjectNode duplicate-named child round-trip identity test asserting correct behavior (expected to fail until limitation removed). +- Ambiguous-group matching counterexample test asserting globally optimal pairing (expected to fail with greedy algorithm). + +## Test Rules + +### For bugs found and fixed + +- Each approved bug fix must have a regression test that fails pre-fix and passes post-fix. + +### For missing coverage identified + +- Add coverage in affected layer(s); investigate any unexpected failure as a potential additional bug. + +### For known limitations (Accepted Limitations section) + +- Write correctness-asserting tests even if they fail now. +- Leave them failing (do not ignore), to serve as future acceptance criteria. + +### Scope + +- Cover both persistence correctness (filesystem outputs) and in-memory tree consistency where applicable. + +## Final Step: Run CI + +After implementing all approved fixes and tests, run `/ci` and require a clean pass before considering this plan complete. \ No newline at end of file diff --git a/.cursor/plans/audit_ref_path_system_c21fc130.plan.md b/.cursor/plans/audit_ref_path_system_c21fc130.plan.md new file mode 100644 index 00000000..ed4c8235 --- /dev/null +++ b/.cursor/plans/audit_ref_path_system_c21fc130.plan.md @@ -0,0 +1,185 @@ +--- +name: Audit Ref Path System +overview: Audit the Luau-style ref path system (rojo_ref.rs, tree.rs, patch_apply.rs, change_processor.rs, ref_properties.rs, api.rs, meta.rs) for round-trip correctness, resolution consistency, and edge-case safety. +todos: + - id: roundtrip + content: Verify compute_relative_ref_path <-> resolve_ref_path_to_absolute round-trip for all prefix classes + status: completed + - id: dual-resolution + content: Verify resolve_ref_path (tree.rs) and resolve_ref_path_to_absolute (rojo_ref.rs) produce identical results + status: completed + - id: priority + content: Audit Rojo_Ref_ vs Rojo_Target_ priority handling in patch_apply.rs + status: completed + - id: cleanup + content: Verify attribute cleanup completeness after ref resolution + status: completed + - id: fs-name-consistency + content: Audit filesystem_name_for vs ref_target_path_from_tree consistency + status: completed + - id: index-correctness + content: Audit RefPathIndex operations (populate, find_by_prefix, update_prefix, rename_file) + status: completed + - id: rename-fixup + content: Audit update_ref_paths_after_rename path reconstruction logic + status: completed + - id: placeholder + content: Audit placeholder system in CLI syncback (uniqueness, substitution order, fallback) + status: completed + - id: added-paths + content: Audit added_paths same-batch ref handling in api.rs (extensions, dedup suffixes) + status: completed + - id: test-gaps + content: Identify test coverage gaps and flag missing scenarios + status: completed +isProject: false +--- + +# Audit: Ref Path System (Luau-Style Ref Paths) + +## Scope + +The ref path system converts Roblox `Ref` properties (like `Model.PrimaryPart`) into filesystem-path-based `Rojo_Ref_*` attributes using Luau require-by-string syntax (`@self`, `./`, `../`, `@game/`). This audit covers: + +- Path generation: `[src/rojo_ref.rs](src/rojo_ref.rs)` (`compute_relative_ref_path`, `ref_target_path_from_tree`) +- Path resolution: `[src/rojo_ref.rs](src/rojo_ref.rs)` (`resolve_ref_path_to_absolute`), `[src/snapshot/tree.rs](src/snapshot/tree.rs)` (`resolve_ref_path`, `walk_segments`, `get_instance_by_path`, `filesystem_name_for`) +- Forward sync application: `[src/snapshot/patch_apply.rs](src/snapshot/patch_apply.rs)` (`finalize_patch_application`, `defer_ref_properties`) +- Syncback generation: `[src/syncback/ref_properties.rs](src/syncback/ref_properties.rs)` (`collect_referents`, `tentative_fs_path`) +- Two-way sync: `[src/web/api.rs](src/web/api.rs)` (`syncback_updated_properties`, `added_paths`) +- Rename updates: `[src/change_processor.rs](src/change_processor.rs)` (`update_ref_paths_after_rename`), `[src/syncback/meta.rs](src/syncback/meta.rs)` (`update_ref_paths_in_file`) +- Index: `[src/rojo_ref.rs](src/rojo_ref.rs)` (`RefPathIndex`) + +## Audit Methodology + +Evaluate each area against the **round-trip identity invariant**: syncback writes `Rojo_Ref_`* paths, forward sync resolves them back to the same `Ref` targets. Any code path where a ref target can be lost, silently changed, or fail to resolve is a bug. + +## Audit Areas + +### 1. Round-Trip: `compute_relative_ref_path` <-> `resolve_ref_path_to_absolute` + +**Files:** `src/rojo_ref.rs` lines 205-289 + +Verify the compute/resolve pair is bijective for all prefix classes: + +- `@self` / `@self/...` -- descendant paths +- `./...` -- sibling paths (1 up) +- `../...` chains -- 2+ levels up within same service +- `@game/...` -- cross-service / ancestor refs +- Bare paths (no prefix) -- legacy fallback + +**Specific checks:** + +- `compute_relative_ref_path` uses **case-sensitive** common-prefix detection (line 222: `a == b`), but `resolve_ref_path` / `walk_segments` uses **case-insensitive** matching (line 365: `eq_ignore_ascii_case`). This asymmetry is safe only if path generation always uses canonical filesystem case. Verify this invariant holds across all callers of `ref_target_path_from_tree`. +- When `remaining_parts.is_empty()` (target is ancestor of source), falls through to `@game/` (line 231-233). Verify `resolve_ref_path_to_absolute` handles this correctly. +- Empty `source_abs` or `target_abs` edge cases. + +### 2. Dual Resolution: `resolve_ref_path` (tree.rs) vs `resolve_ref_path_to_absolute` (rojo_ref.rs) + +**Files:** `src/snapshot/tree.rs` lines 300-389, `src/rojo_ref.rs` lines 254-289 + +Two independent implementations resolve relative paths: + +- `resolve_ref_path` (tree.rs) -- walks the live `RojoTree`, used during forward sync +- `resolve_ref_path_to_absolute` (rojo_ref.rs) -- pure string manipulation, used by `RefPathIndex` and `update_ref_paths_in_file` + +Verify they handle **all prefix variants identically**: + +- `../` prefix: tree.rs pops to grandparent THEN walks rest. rojo_ref.rs pops source + parent from parts THEN processes rest. Both handle subsequent `..` in rest via loop. +- Edge case: `../` on a root-level instance (1 segment in `source_abs`). `resolve_ref_path_to_absolute` does `p.pop(); p.pop()?` -- the second pop returns `None`. `resolve_ref_path` checks `grandparent.is_none()`. Both return None. Consistent. +- Empty rest after prefix strip: e.g., `"@self/"` → empty rest. tree.rs: `walk_segments(source, "")` returns `Some(source)`. rojo_ref.rs: no segments to iterate, returns `source_abs`. Consistent (equivalent to `@self`). + +### 3. Priority: `Rojo_Ref_`* vs `Rojo_Target_`* in Forward Sync + +**File:** `src/snapshot/patch_apply.rs` lines 110-205, 321-360 + +In `defer_ref_properties`: both `Rojo_Ref_`* and `Rojo_Target_`* are collected into **separate** maps without deduplication. In `finalize_patch_application`: `attribute_refs_to_rewrite` (Target) runs first (line 131), then `path_refs_to_rewrite` (Ref) runs second (line 144), **overwriting** the Target result. + +**Finding:** Priority is correct (`Rojo_Ref_`* wins) but implemented via **execution order**, not explicit logic. If the blocks are reordered, priority silently flips. Consider whether this warrants a code comment or explicit dedup in `defer_ref_properties`. + +### 4. Attribute Cleanup Completeness + +**File:** `src/snapshot/patch_apply.rs` lines 171-202 + +After resolution, `instances_needing_attr_cleanup` triggers removal of all `Rojo_Id`, `Rojo_Target_`*, and `Rojo_Ref_`* attributes. Verify: + +- Both `defer_ref_properties` code paths (Rojo_Target_ and Rojo_Ref_) add to `instances_needing_attr_cleanup`. +- Rojo_Id is also added (line 357). +- Cleanup iterates ALL attributes and removes matches by prefix -- catches attributes even if they weren't in the rewrite maps (e.g., orphaned attributes). +- Check: does an update patch also trigger defer_ref_properties? Yes (line 310). + +### 5. `filesystem_name_for` Consistency + +**File:** `src/snapshot/tree.rs` lines 394-418 + +This function is the **ground truth** for path segment names during forward sync. Verify: + +- `InstigatingSource::Path` -- returns filename from disk. Correct: includes extension, dedup suffix, slugified name. +- `InstigatingSource::ProjectNode` -- returns instance name. Correct: project nodes are services or named containers, not files. +- No `instigating_source` -- falls back to instance name. This happens for newly-added instances (two-way sync). Verify this fallback doesn't break ref path resolution for refs pointing at just-added instances. +- `ref_target_path_from_tree` (rojo_ref.rs lines 140-191) independently reimplements the same logic. Check for consistency: both use `meta.instigating_source`, both have the same ProjectNode vs Path branching. + +### 6. `RefPathIndex` Correctness + +**File:** `src/rojo_ref.rs` lines 339-511 + +- `populate_from_dir` (line 400): scans filesystem directly (bypassing VFS), reads JSON5 files, resolves relative paths to absolute. The `source_abs` comes from `tree.get_ids_at_path`. **Edge case:** orphaned meta file (no tree entry) gets `source_abs = ""`, which corrupts relative path resolution (e.g., `"./Sibling"` resolves to `"Sibling"` instead of `"Service/Parent/Sibling"`). Low severity since orphaned meta files shouldn't exist during normal serve sessions. +- `find_by_prefix` (line 464): checks exact match OR prefix-with-slash. Correct. No false positives from segment-prefix overlap (e.g., `"Workspace/Foo"` won't match `"Workspace/FooBar"`). +- `update_prefix` (line 491): merges file sets when new key already exists. Correct for convergent renames. +- `rename_file` (line 481): updates filesystem paths in all entries. O(n) scan -- acceptable since called rarely. + +### 7. `update_ref_paths_after_rename` Path Fixup + +**File:** `src/change_processor.rs` lines 429-540 + +When the indexed file path no longer exists (directory was renamed), this code tries to reconstruct the new path by replacing the old directory segment with the new one in the PathBuf components (lines 462-497). + +**Checks:** + +- Only replaces the **first** occurrence. If the old segment appears in multiple path components (e.g., `src/Foo/Foo/file.meta.json5`), only the first gets replaced. This is correct for single-level renames but could fail for deeply nested same-named directories. **Low severity** since dedup suffixes prevent truly identical sibling names. +- Handles both plain name and slugified name variants (lines 448-484). +- After updating files, updates the index with both `update_prefix` and `rename_file` (lines 532-540). The zip iteration assumes `original_paths` and `files_to_check` have the same length and order -- they do since `files_to_check` is a 1:1 map of the originals. + +### 8. Placeholder System (CLI Syncback) + +**Files:** `src/syncback/ref_properties.rs`, `src/syncback/mod.rs`, `src/syncback/fs_snapshot.rs` + +Placeholders (`__ROJO_REF__TO_`__) are written during the syncback walk before dedup suffixes are known, then substituted with final relative paths. + +**Checks:** + +- Placeholder uniqueness: encodes both source and target Refs. Unique per (source, target) pair. +- Substitution in `fix_ref_paths`: verify it handles placeholders appearing in both meta and model files. +- Length-descending sort prevents partial prefix matches (e.g., `__ROJO_REF_A_TO_B`__ won't partially match `__ROJO_REF_A_TO_BC_`_ if the longer one is processed first). +- Fallback: if final path not in `ref_path_map`, falls back to `tentative_fs_path_public()`. Verify this fallback produces a valid path. + +### 9. Two-Way Sync: `added_paths` for Same-Batch Refs + +**File:** `src/web/api.rs` + +When the plugin adds instances and sets Ref properties targeting them in the same `/api/write` request, `added_paths` pre-computes filesystem paths. Verify: + +- `added_instance_fs_segment` mirrors `tentative_fs_name` logic from ref_properties.rs. +- Paths include correct extensions (`.server.luau`, `.luau`, `.model.json5`, no extension for folders). +- Dedup suffixes are NOT included in `added_paths` (the instance hasn't been written yet), which could cause the ref path to mismatch if the actual write adds a dedup suffix. + +### 10. Test Coverage Assessment + +**Files:** `src/rojo_ref.rs` (tests), `tests/tests/two_way_sync.rs`, `tests/tests/syncback.rs` + +Existing tests cover: basic ref write, nil ref, target change, rename update, same-batch add, dedup cleanup, RefPathIndex population, relative path round-trips (100+ cases). + +**Gaps to check:** + +- Multi-level ancestor renames (rename grandparent, verify ref 3 levels down updates) +- Cross-nested-project refs +- Refs where source and target are in the same dedup group +- `../` chains exceeding 2 levels +- Unicode instance names in ref paths + +## Severity Scale + +- **Critical** -- Round-trip identity violated; ref targets silently lost or changed +- **High** -- Refs fail to resolve in specific but reproducible scenarios +- **Medium** -- Code fragility or missing error handling that could cause future regressions +- **Low** -- Minor edge cases, code quality, or documentation gaps + diff --git a/.cursor/plans/changebatcher_encoding_audit_bade61c8.plan.md b/.cursor/plans/changebatcher_encoding_audit_bade61c8.plan.md new file mode 100644 index 00000000..3194ecbd --- /dev/null +++ b/.cursor/plans/changebatcher_encoding_audit_bade61c8.plan.md @@ -0,0 +1,211 @@ +--- +name: ChangeBatcher Encoding Audit +overview: Audit of the ChangeBatcher encoding refactors covering encodeService.lua (new), encodeInstance.lua (modified), and related test changes across 4 files (132 insertions, 63 deletions). +todos: + - id: fix-header-comment + content: "Fix stale header comment in encodeInstance.lua (Finding 1): update DUPLICATE HANDLING section to say duplicates are encoded and delegated to server" + status: completed + - id: extract-shared-filter + content: Extract shared property encoding filter from encodeService.lua and encodeInstance.lua into a common helper to eliminate duplication (Finding 2) + status: completed + - id: add-encodeService-tests + content: Create encodeService.spec.lua with tests for property encoding, Ref carrier logic, Attributes/Tags, EXCLUDE_PROPERTIES, and empty services (Finding 3) + status: completed + - id: document-clocktime-exclude + content: Add comment to EXCLUDE_PROPERTIES in encodeService.lua explaining why Lighting.ClockTime is excluded (Finding 4) + status: completed + - id: fix-servicechunk-doc + content: Fix misleading doc comment on ServiceChunk in src/web/interface.rs to correctly describe refs as 1-based carrier indices (Finding 5) + status: completed +isProject: false +--- + +# ChangeBatcher Encoding Refactors -- Audit Report + +## Scope + +4 files changed: +132 / -63 across 13 commits. + + +| File | Change Type | Lines | +| -------------------------------- | --------------- | -------------------------------- | +| `encodeService.lua` | New (108 lines) | +108 | +| `encodeInstance.lua` | Modified | +24 / -53 | +| `encodeInstance.spec.lua` | Modified | +3 / -4 | +| `encodeInstance.stress.spec.lua` | Modified | -6 (removed skippedCount checks) | + + +--- + +## Architecture Summary + +```mermaid +flowchart TD + subgraph TwoWaySync [Two-Way Sync Path] + InstanceMap["InstanceMap signals"] --> ChangeBatcher + ChangeBatcher --> createPatchSet + createPatchSet --> encodePatchUpdate + encodePatchUpdate --> encodeProperty + end + + subgraph LiveSyncback [Live Syncback Path] + performSyncback["App:performSyncback()"] --> encodeService + encodeService --> encodeProperty + performSyncback --> encodeInstance + encodeInstance --> encodeProperty + end + + encodeProperty --> RbxDom + encodeService --> propertyFilter + encodeInstance --> propertyFilter + encodePatchUpdate --> propertyFilter +``` + + + +`encodeService` is new and serves the live syncback feature. `encodeInstance` was refactored to delegate duplicate handling to the server. + +--- + +## Finding 1: Stale Header Comment (Bug) + +**Severity: Moderate** | **File:** `[encodeInstance.lua](plugin/src/ChangeBatcher/encodeInstance.lua)` lines 1-17 + +The module header comment claims: + +``` +DUPLICATE HANDLING: +- Instances with duplicate-named siblings at ANY level of the path are skipped +``` + +But the code was changed to do the opposite -- duplicates are now **encoded** (not skipped), with a debug log, and the server handles dedup. The header directly contradicts the code behavior. + +**Fix:** Update the header to reflect the new behavior. + +--- + +## Finding 2: Duplicated Encoding Filter Logic (Code Quality) + +**Severity: Moderate (small refactor candidate)** | **Files:** `[encodeService.lua](plugin/src/ChangeBatcher/encodeService.lua)`, `[encodeInstance.lua](plugin/src/ChangeBatcher/encodeInstance.lua)` + +The property encoding filter is copy-pasted across both files with only minor differences: + + +| Logic | `encodeService.lua` | `encodeInstance.lua` | +| --------------------------------------------- | ------------------- | -------------------- | +| SKIP_PROPERTIES (Parent, Name, Archivable) | Lines 7-11 | Lines 34-38 | +| Skip Attributes/Tags (handled separately) | Line 33 | Lines 217-219 | +| Scriptability check (ReadWrite, Read, Custom) | Lines 37-39 | Lines 224-226 | +| DoesNotSerialize check | Line 40 | Line 227 | +| UNENCODABLE_DATA_TYPES filter | Line 45 | Line 233 | +| Attributes encoding (pcall + descriptor) | Lines 70-81 | Lines 121-138 | +| Tags encoding (pcall + descriptor) | Lines 83-94 | Lines 141-157 | + + +If either filter is updated (e.g., adding a new skip property, changing the scriptability check), the other must be updated identically or a divergence bug is introduced. + +**Fix:** Extract a shared `iterEncodableProperties(className, callback)` helper that handles the filter logic once. Both `encodeService` and `encodeInstance` would call it. The Attributes/Tags encoding blocks could also be extracted to a shared `encodeAttributesAndTags(instance, properties)` function (the `encodeAttributes`/`encodeTags` functions in `encodeInstance.lua` already do this but aren't exported). + +--- + +## Finding 3: No Tests for `encodeService.lua` (Test Gap) + +**Severity: Moderate** | **File:** `encodeService.lua` (new, 108 lines, no spec file) + +Every other encoding module has a `.spec.lua` test: + +- `encodeInstance.spec.lua` (exists) +- `encodeInstance.stress.spec.lua` (exists) +- `encodePatchUpdate.spec.lua` (exists) +- `encodeProperty.spec.lua` (exists) +- `**encodeService.spec.lua` -- MISSING** + +`encodeService` handles Ref property encoding via ObjectValue carriers (a tricky serialization path), service-level property filtering, and the carrier index calculation. These all warrant test coverage. + +**Fix:** Add `encodeService.spec.lua` covering: + +- Basic property encoding for a service (e.g., Lighting) +- Ref property carrier creation and index correctness +- Attributes/Tags encoding on services +- EXCLUDE_PROPERTIES filtering (Lighting.ClockTime) +- Empty service (no properties, no children) + +--- + +## Finding 4: `EXCLUDE_PROPERTIES` Undocumented (Documentation) + +**Severity: Low** | **File:** `[encodeService.lua](plugin/src/ChangeBatcher/encodeService.lua)` lines 13-15 + +```lua +local EXCLUDE_PROPERTIES = { + Lighting = { ClockTime = true }, +} +``` + +`ClockTime` (Float32) and `TimeOfDay` (String) are linked properties on Lighting -- they represent the same value in different formats, and both have `Serialization: "Serializes"`. Excluding `ClockTime` avoids sending a redundant/conflicting representation alongside `TimeOfDay`. This is a reasonable design choice, but there is no comment explaining the reasoning. If someone sees `TimeOfDay` behaving oddly, they won't know `ClockTime` was intentionally excluded or why. + +**Fix:** Add a comment explaining the linked-property reason. + +--- + +## Finding 5: Misleading `ServiceChunk` Doc Comment (Documentation) + +**Severity: Low** | **File:** `[src/web/interface.rs](src/web/interface.rs)` lines 30-42 + +The doc comment on `ServiceChunk` states: + +> `refs` maps property names to 1-based indices into this combined range (0 = nil). Indices <= child_count point directly to a child instance. Indices > child_count point to an ObjectValue carrier. + +But the server code in `[serve.rs](src/cli/serve.rs)` lines 198-204 treats `refs` values as **1-based indices into the carrier slice only** (not the combined range): + +```rust +let carriers = &service_range[carrier_start..carrier_end]; +for (prop_name, &idx) in &chunk.refs { + if idx == 0 || (idx as usize) > carriers.len() { continue; } + ref_entries.push((prop_name.clone(), carriers[idx as usize - 1])); +} +``` + +And the plugin sets `refs[propertyName] = #refTargets` (1-based into refTargets only). The "indices into combined range" and "indices <= child_count point to a child" parts of the comment are incorrect -- carriers never point to children. + +**Fix:** Update the doc comment to say `refs` maps property names to 1-based indices into the carrier slice. + +--- + +## Positive Changes (No Action Needed) + +### P1: Module-Level Mutable State Removed (encodeInstance.lua) + +The old code had `local skippedDuplicateCount = 0` at module scope, mutated across calls and reset inside a wrapper function. This was a shared mutable state pattern that risked stale counts if the wrapper wasn't used. The new code removes this entirely -- clean improvement. + +### P2: Duplicate Handling Delegated to Server + +Previously, `encodeInstance` skipped any instance whose path contained duplicate-named siblings, silently losing data. Now all instances are encoded and sent to the server, which handles filesystem name collisions via the dedup suffix system (`~2`, `~3`, etc.). This is the correct design -- the server has the filesystem context needed to resolve collisions, while the plugin should send complete data. + +### P3: `Custom` Scriptability Added + +The filter now includes `propertyMeta.scriptability == "Custom"` alongside `ReadWrite` and `Read`. The reflection database has 9 `Custom` scriptability properties. After accounting for those handled separately (Attributes, Tags, Source, Contents), this adds coverage for `Scale` (NumberValue), `WorldPivotData` (Model), `PropertiesSerialize` (StyleSheet), and `MaterialColors` (Workspace) -- all of which serialize and were previously silently dropped. + +### P4: Simplified Wrapper Function + +The `encodeInstanceWithDuplicateTracking` wrapper was replaced with a simple `return function(instance, parentId)` that calls `encodeInstance(instance, parentId, false)`. Tests updated to match (no more second `skippedCount` return value). + +### P5: Tests Updated Correctly + +Both `encodeInstance.spec.lua` and `encodeInstance.stress.spec.lua` are updated to assert that duplicates ARE included (e.g., `expect(#encoded.children).to.equal(3)` instead of `1`). All `skippedCount` assertions removed. + +--- + +## Summary + + +| # | Finding | Severity | Type | +| --- | ------------------------------------------ | -------- | --------------------- | +| 1 | Stale header comment in encodeInstance.lua | Moderate | Bug (misleading docs) | +| 2 | Duplicated encoding filter logic | Moderate | Code quality | +| 3 | No tests for encodeService.lua | Moderate | Test gap | +| 4 | EXCLUDE_PROPERTIES undocumented | Low | Documentation | +| 5 | Misleading ServiceChunk doc comment | Low | Documentation | + + +No critical or round-trip-identity issues found. The encoding logic is correct, the server-side consumption matches, and the refactors improve the codebase. The main action items are updating stale comments, adding a test file, and extracting the duplicated filter logic. \ No newline at end of file diff --git a/.cursor/plans/compact_json5_arrays_36a8cfab.plan.md b/.cursor/plans/compact_json5_arrays_36a8cfab.plan.md new file mode 100644 index 00000000..d2ae36ba --- /dev/null +++ b/.cursor/plans/compact_json5_arrays_36a8cfab.plan.md @@ -0,0 +1,328 @@ +--- +name: Compact JSON5 arrays +overview: Arrays with <12 non-compound elements render on one line. Recursive -- inner arrays compact independently. +todos: + - id: modify-array-arm + content: "Modify the Array arm of write_to() in src/json.rs: inline if <12 elements and no element is Array/Object" + status: completed + - id: snapshot-tests + content: Run cargo test, then cargo insta review to update affected snapshots + status: completed +isProject: false +--- + +# Compact JSON5 Array Formatting + +## Rule + +An array renders inline if: + +1. Fewer than 12 direct elements +2. No direct element is an `Array` or `Object` + +Format: `[ elem, elem, elem ]` (spaces inside brackets). Recursive -- each nested array independently decides. + +## Change + +One file: [src/json.rs](src/json.rs), the Array arm of `Json5Value::write_to()` (lines 154-169). ~10 lines added. + +--- + +## Expected Output Examples + +### Color3 (3 floats) + +Before: + +```json5 +Color3: [ + 0.41568630933761597, + 0, + 1 +] +``` + +After: + +```json5 +Color3: [ 0.41568630933761597, 0, 1 ] +``` + +### NumberRange (2 floats) + +Before: + +```json5 +Lifetime: { + NumberRange: [ + 0.25, + 0.6000000238418579 + ] +} +``` + +After: + +```json5 +Lifetime: { + NumberRange: [ 0.25, 0.6000000238418579 ] +} +``` + +### Tags (string array) + +Before: + +```json5 +Tags: [ + "SoundGroupEffects" +] +``` + +After: + +```json5 +Tags: [ "SoundGroupEffects" ] +``` + +Multiple tags: + +```json5 +Tags: [ "_VFX", "Collectible", "PowerUp" ] +``` + +### CFrame (object with position + orientation) + +Before: + +```json5 +CFrame: { + position: [ + -0.09659165, + 1.517803, + -0.83597946 + ], + orientation: [ + [ + 0.10630996, + -0.0024746126, + -0.9943304 + ], + [ + 0.021991052, + 0.999758, + -0.0021534874 + ], + [ + 0.9941102, + -0.021623673, + 0.1063388 + ] + ] +} +``` + +After: + +```json5 +CFrame: { + position: [ -0.09659165, 1.517803, -0.83597946 ], + orientation: [ + [ 0.10630996, -0.0024746126, -0.9943304 ], + [ 0.021991052, 0.999758, -0.0021534874 ], + [ 0.9941102, -0.021623673, 0.1063388 ] + ] +} +``` + +- `position` (3 numbers) -> inline +- `orientation` (3 arrays) -> multi-line because elements are arrays +- Each inner orientation row (3 numbers) -> inline + +### UDim2 (2 arrays of 2 numbers) + +Before: + +```json5 +Size: [ + [ + 1, + 0 + ], + [ + 0, + 100 + ] +] +``` + +After: + +```json5 +Size: [ + [ 1, 0 ], + [ 0, 100 ] +] +``` + +Outer has array elements -> multi-line. Inner arrays (2 numbers each) -> inline. + +### Rect (2 Vector2s) + +Before: + +```json5 +ImageRectOffset: [ + [ + 100, + 100 + ], + [ + 200, + 200 + ] +] +``` + +After: + +```json5 +ImageRectOffset: [ + [ 100, 100 ], + [ 200, 200 ] +] +``` + +### ColorSequence keypoints (array of objects -- NO CHANGE) + +Before and after (objects always expand): + +```json5 +Color: { + ColorSequence: { + keypoints: [ + { + color: [ 0.364705890417099, 0.11372549086809158, 1 ], + time: 0 + }, + { + color: [ 0.364705890417099, 0.11372549086809158, 1 ], + time: 1 + } + ] + } +} +``` + +- `keypoints` array has objects -> stays multi-line +- `color` arrays inside each keypoint (3 numbers) -> inline + +### NumberSequence with 20 keypoints (array of objects -- NO CHANGE) + +Stays fully expanded. Both because elements are objects AND count >= 12. + +### Full file: Texture.model.json5 + +Before: + +```json5 +{ + className: "Texture", + properties: { + Color3: [ + 0.41568630933761597, + 0, + 1 + ], + Face: "Top", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10025668350", + Transparency: 0.6000000238418579 + } +} +``` + +After: + +```json5 +{ + className: "Texture", + properties: { + Color3: [ 0.41568630933761597, 0, 1 ], + Face: "Top", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10025668350", + Transparency: 0.6000000238418579 + } +} +``` + +### Full file: Water.model.json5 + +Before: + +```json5 +{ + attributes: { + Rojo_Ref_SoundGroup: "ServerStorage/__server_muted.model.json5" + }, + className: "Sound", + properties: { + AudioContent: "rbxassetid://9120549564", + Looped: true, + Playing: true, + SourceAssetId: 9120549564, + Tags: [ + "SoundGroupEffects" + ], + Volume: 3 + } +} +``` + +After: + +```json5 +{ + attributes: { + Rojo_Ref_SoundGroup: "ServerStorage/__server_muted.model.json5" + }, + className: "Sound", + properties: { + AudioContent: "rbxassetid://9120549564", + Looped: true, + Playing: true, + SourceAssetId: 9120549564, + Tags: [ "SoundGroupEffects" ], + Volume: 3 + } +} +``` + +### Edge: empty array (NO CHANGE) + +```json5 +Tags: [] +``` + +### Edge: 12+ primitive elements (stays multi-line) + +```json5 +SomeList: [ + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 11, + 12 +] +``` + diff --git a/.cursor/plans/consolidate_filesystem_name_for_f2bdef69.plan.md b/.cursor/plans/consolidate_filesystem_name_for_f2bdef69.plan.md new file mode 100644 index 00000000..47e61382 --- /dev/null +++ b/.cursor/plans/consolidate_filesystem_name_for_f2bdef69.plan.md @@ -0,0 +1,96 @@ +--- +name: Consolidate filesystem_name_for +overview: Eliminate the duplicated InstigatingSource branching logic by having `ref_target_path_from_tree` call `tree.filesystem_name_for()` instead of reimplementing the same logic inline. +todos: + - id: consolidate + content: Replace inline InstigatingSource branching in ref_target_path_from_tree with tree.filesystem_name_for() call + status: completed + - id: verify + content: Run cargo test and cargo clippy to confirm no regressions + status: completed +isProject: false +--- + +# Consolidate `filesystem_name_for` Duplication + +## Problem + +`ref_target_path_from_tree` (`[src/rojo_ref.rs](src/rojo_ref.rs)` lines 140-191) reimplements the same `InstigatingSource` branching logic that `filesystem_name_for` (`[src/snapshot/tree.rs](src/snapshot/tree.rs)` lines 394-418) already provides. The [ref path audit](c:/Users/Joe/.cursor/plans/audit_ref_path_system_c21fc130.plan.md) confirmed these produce identical results for all branches. + +## Approach + +`ref_target_path_from_tree` already receives `&RojoTree` and `filesystem_name_for` is already `pub` on `RojoTree`. The fix is to replace the inline branching with a call to the existing method. No new functions, no signature changes, no new files. + +## Change + +**Single file: `[src/rojo_ref.rs](src/rojo_ref.rs)`** -- Replace the 25-line `let fs_name = if let Some(meta) ...` block (lines 158-183) inside `ref_target_path_from_tree` with a single call to `tree.filesystem_name_for(current)`: + +Before (lines 140-191): + +```rust +pub fn ref_target_path_from_tree(tree: &crate::snapshot::RojoTree, target_ref: Ref) -> String { + let dom = tree.inner(); + let root_ref = dom.root_ref(); + let mut components: Vec = Vec::new(); + let mut current = target_ref; + + loop { + if current == root_ref || current.is_none() { break; } + let inst = match dom.get_by_ref(current) { + Some(i) => i, + None => break, + }; + + // 25 lines of InstigatingSource branching (duplicated from tree.rs) + let fs_name = if let Some(meta) = tree.get_metadata(current) { + // ... Path branch ... + // ... ProjectNode branch ... + // ... no source fallback ... + } else { + inst.name.clone() + }; + + components.push(fs_name); + current = inst.parent(); + } + components.reverse(); + components.join("/") +} +``` + +After: + +```rust +pub fn ref_target_path_from_tree(tree: &crate::snapshot::RojoTree, target_ref: Ref) -> String { + let dom = tree.inner(); + let root_ref = dom.root_ref(); + let mut components: Vec = Vec::new(); + let mut current = target_ref; + + loop { + if current == root_ref || current.is_none() { break; } + let inst = match dom.get_by_ref(current) { + Some(i) => i, + None => break, + }; + components.push(tree.filesystem_name_for(current)); + current = inst.parent(); + } + components.reverse(); + components.join("/") +} +``` + +## Scope + +- **1 file** changed +- **~25 lines** removed (duplicated branching logic) +- **1 line** added (call to existing method) +- **0 call sites** affected (function signature unchanged) +- **0 test changes** needed (behavior is identical per the audit) + +## Verification + +- `cargo test` -- all existing ref path tests (100+ unit, 30+ integration) exercise this function +- `cargo clippy` -- no new warnings expected + diff --git a/.cursor/plans/dedup_suffix_start_at_2_89e66bf2.plan.md b/.cursor/plans/dedup_suffix_start_at_2_89e66bf2.plan.md new file mode 100644 index 00000000..002d8b29 --- /dev/null +++ b/.cursor/plans/dedup_suffix_start_at_2_89e66bf2.plan.md @@ -0,0 +1,89 @@ +--- +name: Dedup suffix start at 2 +overview: Change the dedup suffix numbering to start at `~2` instead of `~1`, so the sequence is `Something.luau`, `Something~2.luau`, `Something~3.luau` -- consistent with Windows duplicate naming conventions. +todos: + - id: core-loops + content: Change `for i in 1..` to `for i in 2..` in both `deduplicate_name_with_ext` and `deduplicate_name` in `src/syncback/file_names.rs` + status: completed + - id: unit-tests + content: Update unit test expectations in `file_names.rs` for generated suffix assertions (~1 -> ~2, ~2 -> ~3) + status: completed + - id: integration-tests + content: Update integration test assertions in `two_way_sync.rs` and `connected_mode.rs` + status: completed + - id: fixtures-rename + content: "Rename test fixture files: connected_slugify, ref_ambiguous_path, UFOWave_matching expected outputs" + status: completed + - id: snapshot-tests + content: Run cargo test + cargo insta review to update snapshot expectations + status: completed + - id: docs-update + content: Update atlas-dedup.mdc examples to reflect ~2 starting suffix + status: completed +isProject: false +--- + +# Dedup Suffix: Start at ~2 + +## Core Change + +The only production logic change is in `[src/syncback/file_names.rs](src/syncback/file_names.rs)` -- two `for` loops that generate dedup suffixes: + +`**deduplicate_name_with_ext` (line ~297):** + +```rust +// Before: +for i in 1.. { +// After: +for i in 2.. { +``` + +`**deduplicate_name` (line ~315):** + +```rust +// Before: +for i in 1.. { +// After: +for i in 2.. { +``` + +These are the only two places where dedup suffix numbers are generated. All other code (`parse_dedup_suffix`, `compute_cleanup_action`, `build_dedup_name`, cleanup logic in `change_processor.rs`, `api.rs`) operates on whatever numbers exist on disk and needs no changes. + +## Backward Compatibility + +- `**parse_dedup_suffix**` in `[src/syncback/dedup_suffix.rs](src/syncback/dedup_suffix.rs)` (line ~~53): keep `if n > 0` unchanged. Old filesystems may have `~~1` files; forward sync must still read them correctly. +- `**build_dedup_name**`: takes an explicit number, no change needed. +- **Cleanup rules**: operate on parsed suffix numbers dynamically, work with any starting value. + +## Test Updates + +### Unit Tests + +- `[src/syncback/file_names.rs](src/syncback/file_names.rs)` test module (~~line 389+): update assertions that expect generated `~~1`to`~~2`,`~~ 2`to`~3`, etc. Key tests:` dedup_single_collision`,` dedup_multiple_collisions`,` dedup_skips_taken_suffix`,` dedup_gap_in_suffixes`. +- `[src/syncback/dedup_suffix.rs](src/syncback/dedup_suffix.rs)` test module: `parse_suffix_basic` still tests `~1` parsing (backward compat) -- no change. `build_name_file` references `~1` as explicit input to `build_dedup_name` -- no change (function takes arbitrary numbers). Cleanup tests (`cleanup_group_to_one`, `cleanup_base_deleted_promote_lowest`, etc.) use `~1`/`~2` as existing filesystem state -- no change needed since they test cleanup of pre-existing suffixes. + +### Integration Tests + +- `[tests/tests/two_way_sync.rs](tests/tests/two_way_sync.rs)`: update all generation assertions: `X_Y~1` to `X_Y~2`, `X_Y~2` to `X_Y~3`. Affected tests: `add_two_colliding_instances_deduplicates`, `add_three_colliding_instances_deduplicates`, `delete_deduped_instance_group_to_1_cleanup`, `delete_base_deduped_instance_promotes_lowest`, `delete_middle_deduped_instance_tolerates_gap`, `add_same_named_folders_deduplicates`, `ref_through_deduped_instance`, `ref_path_updated_after_dedup_cleanup`. +- `[tests/tests/connected_mode.rs](tests/tests/connected_mode.rs)`: update tests that interact with generated dedup files. `add_third_collision` expects `Hey_Bro~2` -- should become `Hey_Bro~3`. Tests that read existing fixture files (`edit_dedup_file_with_meta`, `remove_one_of_two_colliding_files`) need fixture updates (see below). + +### Test Fixtures (rename files) + +- `rojo-test/serve-tests/connected_slugify/src/Hey_Bro~1.luau` -> `Hey_Bro~2.luau` +- `rojo-test/serve-tests/connected_slugify/src/Hey_Bro~1.meta.json5` -> `Hey_Bro~2.meta.json5` +- `rojo-test/serve-tests/ref_ambiguous_path/src/Workspace/DupParent/Child~1.model.json5` -> `Child~2.model.json5` (and update any ref paths inside) +- `rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/`: rename all `~N` files to `~(N+1)` (e.g., `Texture~1` -> `Texture~2`, ..., `Texture~11` -> `Texture~12`) + +**Keep unchanged (backward compat reading):** + +- `rojo-test/build-tests/dedup_suffix_with_meta/src/Foo~1.`* -- tests that forward sync reads old `~1` files correctly +- `rojo-test/build-tests/tilde_no_meta/src/Foo~1.luau` -- tests literal tilde name (no meta) + +### Snapshot Tests + +- Run `cargo test` then `cargo insta review` to accept updated snapshot expectations. The `.snap` files in `rojo-test/build-test-snapshots/` and `rojo-test/serve-test-snapshots/` will auto-update. + +## Documentation Update + +- `[.cursor/rules/atlas-dedup.mdc](.cursor/rules/atlas-dedup.mdc)`: update examples that reference `~1` as generated output (e.g., `"Foo~1.server.luau"` -> `"Foo~2.server.luau"` where it represents generated names; keep `~1` where it represents parseable/legacy names). + diff --git a/.cursor/plans/extension-free_ref_paths_b9a68637.plan.md b/.cursor/plans/extension-free_ref_paths_b9a68637.plan.md new file mode 100644 index 00000000..5fbce765 --- /dev/null +++ b/.cursor/plans/extension-free_ref_paths_b9a68637.plan.md @@ -0,0 +1,284 @@ +--- +name: extension-free ref paths +overview: Remove file extensions from ref path segments and enforce dedup suffixes for ALL instances sharing an Instance.Name, regardless of middleware/file type. Ref paths change from `Workspace/Foo.server.luau` to `Workspace/Foo`, and siblings like `LocalScript.local.luau` + `LocalScript.client.luau` now become `LocalScript.local.luau` + `LocalScript~2.client.luau`. +todos: + - id: strip-ext + content: Add `strip_known_extensions()` utility to `file_names.rs` with unit tests + status: pending + - id: dedup-bare-slug + content: Change `deduplicate_name_with_ext()` and `name_for_inst()` to use bare slug for collision detection; update dedup_key semantics + status: pending + - id: taken-names-sites + content: Update ALL taken_names seeding sites (dir.rs, project.rs, api.rs) to use bare slugs via strip_known_extensions + status: pending + - id: ref-segment + content: Add `ref_segment_for()` to tree.rs; update `walk_segments()` and `get_instance_by_path()` to match bare stems + status: pending + - id: ref-construction + content: Update `ref_target_path_from_tree()`, `tentative_fs_name()`, `added_instance_fs_segment()`, `record_ref_path()` to use bare stems + status: pending + - id: cleanup-grouping + content: Update change_processor.rs cleanup to group by bare stem (remove extension filter), refactor compute_cleanup_action for mixed-extension groups + status: pending + - id: unit-tests-dedup + content: Update all file_names.rs unit tests for new bare-slug taken_names semantics; add same-name-different-type tests + status: pending + - id: unit-tests-ref + content: Update rojo_ref.rs unit tests for extension-free ref path format + status: pending + - id: forward-sync-constraint + content: "Add forward sync handling for bare-stem uniqueness constraint: walk_segments backward compat fallback for old-format refs, and warning/auto-dedup for legacy same-bare-stem files" + status: pending + - id: integration-tests + content: Add build test fixture for same-name-different-type dedup; add connected_mode cleanup test; update snapshot expectations with `cargo insta review` + status: pending +isProject: false +--- + +# Extension-Free Ref Paths + +## Core Semantic Change + +**Old behavior:** Dedup key = full filesystem name (slug + extension). `Foo.server.luau` and `Foo.luau` do NOT collide. Ref paths include extensions: `Workspace/Foo.server.luau`. + +**New behavior:** Dedup key = bare slug only (no extension). ALL instances with the same slug collide and get `~2`, `~3`, etc. Ref paths use bare stems: `Workspace/Foo`, `Workspace/Foo~2`. + +`~1` is never produced (already the case, maintained). + +## Hard Constraint: Bare Stem Uniqueness + +**Files in the same directory can NEVER share a primary name (bare stem), even if they have different types/extensions.** This is the foundational invariant of this change. Previously, `Foo.server.luau` and `Foo.luau` could coexist because their full filenames differed. Now they CANNOT -- one must get a dedup suffix (`Foo~2`). + +This constraint must be enforced at multiple levels: + +1. **Writeout (syncback/two-way sync):** The dedup system uses bare slugs in `taken_names`, so same-named instances of different types always get `~2`, `~3`, etc. This is the primary enforcement mechanism. +2. **Forward sync (reading files):** When reading a directory, if two files share a bare stem (legacy state from before this change), forward sync needs to handle it. The `walk_segments` resolver must support a **backward compatibility fallback** that matches old-format refs (with extensions) so existing projects don't break. Additionally, we should log a warning when same-bare-stem siblings are detected, guiding users to re-run syncback to fix the state. +3. **Ref path resolution backward compat:** `walk_segments()` should have a 3-tier fallback: + - **Primary:** Match against `ref_segment_for()` (bare stem) -- new format + - **Fallback 1:** Match against `filesystem_name_for()` (full filename) -- old format backward compat + - **Fallback 2:** Match against instance name -- existing fallback + +This ensures old ref paths like `./Foo.server.luau` still resolve correctly until the project is re-synced. + +## Phase 1: Core Dedup Change + +### 1a. Add `strip_known_extensions()` to [file_names.rs](src/syncback/file_names.rs) + +New utility function that strips known middleware extensions from a filename to recover the bare stem (slug + dedup suffix). Must check compound extensions before simple ones: + +```rust +pub fn strip_known_extensions(filename: &str) -> &str { + const COMPOUND: &[&str] = &[ + ".server.luau", ".client.luau", ".plugin.luau", ".local.luau", ".legacy.luau", + ".server.lua", ".client.lua", ".plugin.lua", + ".model.json5", ".model.json", + ".project.json5", ".project.json", + ".meta.json5", ".meta.json", + ]; + const SIMPLE: &[&str] = &[ + ".luau", ".lua", ".json5", ".json", ".csv", ".txt", + ".rbxm", ".rbxmx", ".toml", ".yml", ".yaml", + ]; + for ext in COMPOUND { if let Some(s) = filename.strip_suffix(ext) { return s; } } + for ext in SIMPLE { if let Some(s) = filename.strip_suffix(ext) { return s; } } + filename +} +``` + +### 1b. Modify `deduplicate_name_with_ext()` in [file_names.rs](src/syncback/file_names.rs) + +Change collision check from full filesystem name to bare slug: + +```rust +// OLD: checks "foo.server.luau" against taken_names +// NEW: checks "foo" (bare slug) against taken_names +if !taken_names.contains(&base.to_lowercase()) { ... } +``` + +`taken_names` now stores bare slugs (lowercased, no extensions). + +### 1c. Modify `name_for_inst()` in [file_names.rs](src/syncback/file_names.rs) + +- **New instance branch:** `dedup_key` = bare slug (with dedup suffix if applied), NOT full filesystem name. +- **Old instance branch:** `dedup_key` = `strip_known_extensions(filename)` (bare stem from existing file). +- Doc comments updated to reflect new semantics. + +### 1d. Update all `taken_names` seeding sites + +These files seed `taken_names` from existing filesystem entries and must switch to bare slugs: + +- [dir.rs](src/snapshot_middleware/dir.rs) lines 224-242: `taken_names.insert(filename.to_lowercase())` must become `taken_names.insert(strip_known_extensions(filename).to_lowercase())` +- [project.rs](src/snapshot_middleware/project.rs) lines 736-744: same change +- [api.rs](src/web/api.rs): any taken_names seeding from tree siblings + +## Phase 2: Ref Path Segment Change + +### 2a. Add `ref_segment_for()` to [tree.rs](src/snapshot/tree.rs) + +New method on `RojoTree` that returns the bare stem for use in ref paths. Keep `filesystem_name_for()` unchanged for any non-ref-path callers (verify first). + +```rust +pub fn ref_segment_for(&self, id: Ref) -> String { + if let Some(meta) = self.metadata_map.get(&id) { + if let Some(source) = &meta.instigating_source { + match source { + InstigatingSource::Path(_) => { + if let Some(name) = source.path().file_name().and_then(|f| f.to_str()) { + return strip_known_extensions(name).to_string(); + } + } + InstigatingSource::ProjectNode { .. } => { + if let Some(inst) = self.inner.get_by_ref(id) { + return inst.name.clone(); + } + } + } + } + } + self.inner.get_by_ref(id).map(|i| i.name.clone()).unwrap_or_default() +} +``` + +For `Foo~2.server.luau` this returns `Foo~2`. For `Foo/` (dir) this returns `Foo`. For project nodes, returns instance name. + +### 2b. Update `walk_segments()` and `get_instance_by_path()` in [tree.rs](src/snapshot/tree.rs) + +Replace `filesystem_name_for()` with `ref_segment_for()` in the segment matching loop. The fallback to instance name matching remains. + +### 2c. Update `ref_target_path_from_tree()` in [rojo_ref.rs](src/rojo_ref.rs) + +Use `tree.ref_segment_for()` instead of `tree.filesystem_name_for()` when building path segments. + +### 2d. Update `tentative_fs_name()` in [ref_properties.rs](src/syncback/ref_properties.rs) + +Return just the bare slug (slugified instance name, no extension). This produces tentative ref-path segments like `Foo` instead of `Foo.server.luau`. + +### 2e. Update `added_instance_fs_segment()` in [api.rs](src/web/api.rs) + +Return bare slug (slugified instance name, no extension) for same-batch ref target paths. + +### 2f. Update `record_ref_path()` in [snapshot.rs](src/syncback/snapshot.rs) + +Currently passes the full filename (e.g., `"Foo~2.server.luau"`) as the child segment. Must pass the bare stem instead (e.g., `"Foo~2"`). The `name` variable from `name_for_inst()` is the full filename, so we need to derive the bare stem -- use `strip_known_extensions(&name)`. + +## Phase 2.5: Forward Sync Constraint Enforcement + +### 2.5a. Update `walk_segments()` with 3-tier fallback in [tree.rs](src/snapshot/tree.rs) + +The current 2-tier fallback (filesystem_name_for -> instance name) becomes 3-tier: + +```rust +// Tier 1: match ref segment (bare stem) -- new format +for &child_ref in children { + let ref_seg = self.ref_segment_for(child_ref); + if ref_seg.eq_ignore_ascii_case(segment) { ... break; } +} +// Tier 2: match full filesystem name -- backward compat for old refs +if !found { + for &child_ref in children { + let fs_name = self.filesystem_name_for(child_ref); + if fs_name.eq_ignore_ascii_case(segment) { ... break; } + } +} +// Tier 3: match instance name -- existing fallback +if !found { + for &child_ref in children { + if child.name.eq_ignore_ascii_case(segment) { ... break; } + } +} +``` + +Same structure for `get_instance_by_path()`. + +### 2.5b. Add bare-stem collision warning in forward sync + +In [dir.rs](src/snapshot_middleware/dir.rs) `snapshot_dir_no_meta()`, after collecting all child snapshots, check for bare-stem collisions among siblings. If found, log a warning like: + +``` +Warning: Files "Foo.server.luau" and "Foo.luau" share bare stem "Foo" -- +this is ambiguous for ref paths. Run syncback to fix. +``` + +This is a non-fatal warning. The forward sync still works (instances get their correct names/classes), but ref resolution may be ambiguous until syncback adds dedup suffixes. + +## Phase 3: Dedup Cleanup Update + +### 3a. Update cleanup grouping in [change_processor.rs](src/change_processor.rs) (lines 1090-1156) + +Currently groups siblings by matching extension (`sibling_ext != removed_extension`). Remove this extension filter -- all siblings with the same bare stem should be in the same dedup group regardless of extension. + +- `removed_stem` derivation: use `strip_known_extensions(removed_fs_name)` instead of splitting on first `.` +- `sibling_stem` derivation: use `strip_known_extensions(sibling_fs)` instead of splitting on first `.` +- Remove the `sibling_ext != removed_extension` guard entirely +- `compute_cleanup_action()` `extension` parameter: still needed for building filesystem paths. Pass `None` for the extension since cleanup now operates on bare stems. The renamed file keeps its original extension -- we just need to rename `Foo~2.server.luau` to `Foo.server.luau`. The extension is derived from the actual file's path, not from the dedup group. + +Wait -- `compute_cleanup_action()` builds rename paths using `build_dedup_name(base_stem, suffix, extension)`. It needs the extension to know the full filename. Since siblings now have DIFFERENT extensions within the same group, the survivor's extension must be read from its actual filesystem name. This requires a small refactor of `compute_cleanup_action()` or the caller: instead of passing a single `extension`, the cleanup should track each remaining sibling's full filename so renames produce correct paths. + +**Approach:** In the caller loop in `change_processor.rs`, track `remaining_entries: Vec<(stem, full_fs_name)>` instead of just `remaining_stems: Vec`. The cleanup action can then derive the correct filesystem path from the survivor's actual full name. + +### 3b. Update `compute_cleanup_action()` in [dedup_suffix.rs](src/syncback/dedup_suffix.rs) + +Modify to accept `remaining_entries` with both stem and full filename info, so it can build correct rename paths for mixed-extension groups. + +## Phase 4: Tests + +### 4a. Update unit tests in [file_names.rs](src/syncback/file_names.rs) + +**All tests using full filesystem names in `taken_names`** must switch to bare slugs: + +- `nfi_dedup_collision`: `"foo.luau"` -> `"foo"` +- `nfi_slug_collision_file_middleware`: `"hey_bro.luau"` -> `"hey_bro"` +- `disk_seed_file_middleware_*`: entries like `"mymodule.luau"` -> `"mymodule"` +- `nightmare_mixed_middleware_*`: These tests will fundamentally change. `Folder "Shared"` and `ModuleScript "Shared"` now DO collide (same bare slug "shared") where they previously didn't. + +**New tests to add:** + +- `same_name_different_type_dedup`: Two instances "Foo" (ModuleScript + ServerScript) get dedup: `Foo.luau`, `Foo~2.server.luau` +- `same_name_dir_and_file_dedup`: Folder "Foo" and Script "Foo" get dedup: `Foo/`, `Foo~2.server.luau` +- `three_way_mixed_type_dedup`: Three instances "X" (ModuleScript, ServerScript, Folder) all dedup +- `dedup_key_is_bare_slug`: Verify dedup_key returned by `name_for_inst()` is the bare slug + +### 4b. Add `strip_known_extensions` unit tests in [file_names.rs](src/syncback/file_names.rs) + +Test all compound and simple extension stripping, including edge cases (dots in instance names, no extension, etc.). + +### 4c. Update ref path unit tests in [rojo_ref.rs](src/rojo_ref.rs) + +Update expected ref path formats: `"Workspace/Foo.luau"` -> `"Workspace/Foo"`, etc. + +### 4d. Add build test fixture + +New test in [build-tests/](rojo-test/build-tests/) for same-name-different-type instances. Verify the build output snapshot shows correct dedup suffixes. + +### 4e. Add connected_mode test + +Test dedup cleanup when a mixed-type group member is deleted. Verify the survivor's suffix is cleaned correctly even though siblings have different extensions. + +### 4f. Update ALL existing snapshot test expectations + +Run `cargo test` and `cargo insta review` to update snapshot files affected by the new ref path format and dedup behavior. + +## Key Files Summary + + +| File | Change Type | +| ------------------------------------------------------------------------ | ------------------------------------------------------------------------- | +| [src/syncback/file_names.rs](src/syncback/file_names.rs) | Add `strip_known_extensions`, modify dedup logic, update tests | +| [src/snapshot/tree.rs](src/snapshot/tree.rs) | Add `ref_segment_for`, update `walk_segments`, `get_instance_by_path` | +| [src/rojo_ref.rs](src/rojo_ref.rs) | Update `ref_target_path_from_tree`, update tests | +| [src/syncback/ref_properties.rs](src/syncback/ref_properties.rs) | Update `tentative_fs_name` to return bare slug | +| [src/web/api.rs](src/web/api.rs) | Update `added_instance_fs_segment`, taken_names seeding | +| [src/change_processor.rs](src/change_processor.rs) | Remove extension-based grouping, update ref paths | +| [src/syncback/dedup_suffix.rs](src/syncback/dedup_suffix.rs) | Update `compute_cleanup_action` for mixed-extension groups | +| [src/syncback/snapshot.rs](src/syncback/snapshot.rs) | Update `record_ref_path` to use bare stems | +| [src/snapshot_middleware/dir.rs](src/snapshot_middleware/dir.rs) | Update taken_names seeding to bare slugs; add bare-stem collision warning | +| [src/snapshot_middleware/project.rs](src/snapshot_middleware/project.rs) | Update taken_names seeding to bare slugs | + + +## Risk Areas + +1. `**compute_cleanup_action` refactor** -- The most complex change. Currently assumes all group members share an extension. Must now handle mixed extensions and still produce correct filesystem rename paths. +2. **Custom sync rules** -- `strip_known_extensions()` only handles built-in extensions. Custom `syncRules` patterns won't be stripped. Acceptable for initial implementation; the ref segment will include the custom extension, and resolution will match correctly since both sides use the same derivation. +3. **Snapshot test churn** -- Many snapshot tests will have new expected output. Use `cargo insta review` to batch-accept. +4. **Legacy project migration** -- Projects with same-bare-stem files (e.g., `Foo.server.luau` + `Foo.luau` in same dir) are now in an invalid state. The 3-tier `walk_segments` fallback ensures old ref paths still resolve, but the ambiguity warning encourages users to re-run syncback. The first syncback after this change will automatically fix the filesystem by applying dedup suffixes. + diff --git a/.cursor/plans/fix_infinity_comparison_71fd9057.plan.md b/.cursor/plans/fix_infinity_comparison_71fd9057.plan.md new file mode 100644 index 00000000..5d8ea86c --- /dev/null +++ b/.cursor/plans/fix_infinity_comparison_71fd9057.plan.md @@ -0,0 +1,68 @@ +--- +name: Fix infinity comparison +overview: Make `fuzzyEq` self-contained for all IEEE 754 special values (infinity and NaN), eliminating phantom diffs on properties like `UISizeConstraint.MaxSize` and fixing NaN component comparison in compound types. +todos: + - id: fix-lua-fuzzyeq + content: Add exact-equality and NaN checks to `fuzzyEq` in trueEquals.lua + status: completed + - id: add-tests + content: Add infinity and NaN test cases to trueEquals.spec.lua + status: completed +isProject: false +--- + +# Fix Infinity and NaN Comparison in fuzzyEq + +## Root Cause + +Two bugs in `fuzzyEq`: + +**1. Infinity:** `fuzzyEq(inf, inf)` returns `false` because `inf - inf = NaN`, and `NaN < x` is always `false`. Affects `UISizeConstraint.MaxSize` which defaults to `Vector2(inf, inf)`. + +**2. NaN in compound types:** NaN equality is handled in the `trueEquals` number branch (line 93), but `fuzzyEq` is called directly from `vector2Eq`, `vector3Eq`, `cframeEq`, etc. A Vector2/Vector3/CFrame with a NaN component silently returns `false` instead of `true` because `fuzzyEq` doesn't check for NaN. + +## Fix + +Make `fuzzyEq` self-contained by adding exact-equality and NaN checks at the top: + +```lua +local function fuzzyEq(a: number, b: number): boolean + if a == b then + return true + end + if a ~= a then + return b ~= b + end + local diff = math.abs(a - b) + local maxVal = math.max(math.abs(a), math.abs(b), 1) + return diff < EPSILON or diff < maxVal * EPSILON +end +``` + +Behavior: + +- `inf == inf` -> `true` (caught by `a == b`) +- `-inf == -inf` -> `true` (caught by `a == b`) +- `inf == -inf` -> `false` (falls through, `inf - (-inf) = inf`, fails comparison) +- `NaN == NaN` -> `true` (`a == b` is false for NaN, then `a ~= a` catches it) +- `NaN == 5` -> `false` (`a ~= a` is true but `b ~= b` is false) + +Remove the now-redundant NaN check in `trueEquals` line 93 (the number branch `if a ~= a then return b ~= b end`) since `fuzzyEq` handles it directly. + +## Files + +### 1. [plugin/src/Reconciler/trueEquals.lua](plugin/src/Reconciler/trueEquals.lua) (line 15) + +Replace `fuzzyEq` with the version above. + +### 2. [plugin/src/Reconciler/trueEquals.spec.lua](plugin/src/Reconciler/trueEquals.spec.lua) + +Add test cases for: + +- `trueEquals(math.huge, math.huge)` -> true +- `trueEquals(-math.huge, -math.huge)` -> true +- `trueEquals(math.huge, -math.huge)` -> false +- `trueEquals(Vector2.new(math.huge, math.huge), Vector2.new(math.huge, math.huge))` -> true +- `trueEquals(Vector3.new(0/0, 1, 2), Vector3.new(0/0, 1, 2))` -> true (NaN component) +- `trueEquals(Vector2.new(0/0, 0/0), Vector2.new(0/0, 0/0))` -> true (NaN components) + diff --git a/.cursor/plans/fix_live_syncback_service_properties_d7c2c927.plan.md b/.cursor/plans/fix_live_syncback_service_properties_d7c2c927.plan.md new file mode 100644 index 00000000..43dce05b --- /dev/null +++ b/.cursor/plans/fix_live_syncback_service_properties_d7c2c927.plan.md @@ -0,0 +1,235 @@ +--- +name: Fix Live Syncback Service Properties +overview: Fix live syncback to preserve service-level properties and respect syncCurrentCamera by encoding service properties/attributes/tags/refs alongside the existing rbxm children data. +todos: + - id: plugin-encode-service + content: Create encodeService helper in plugin that returns className, data, properties, attributes, tags, refs + status: completed + - id: plugin-use-helper + content: Update performSyncback to use encodeService, remove children-only guard + status: completed + - id: server-service-chunk + content: Extend ServiceChunk struct with properties, attributes, tags, refs fields + status: completed + - id: server-apply-props + content: Update build_dom_from_chunks to apply service properties/attributes/tags and resolve refs + status: completed + - id: test-helper + content: Update make_service_chunk helpers and make_rbxl_from_chunks + status: completed + - id: fixture-update + content: Update live_syncback fixture project with service $path entries + status: completed + - id: roundtrip-update + content: Update roundtrip test to include service properties + status: completed + - id: new-tests + content: Add parity tests for service properties, childless services, and Camera removal + status: completed +isProject: false +--- + +# Fix Live Syncback Service Properties + +## Root Cause + +1. **Service properties lost** -- Plugin serializes `service:GetChildren()` only. Service-level properties never sent. +2. **Childless services skipped** -- `#service:GetChildren() > 0` guard skips services like VoiceChatService. +3. **Camera not removed** -- `syncback_loop` reads `Workspace.CurrentCamera` to find/remove Camera, but Workspace has no properties in the DOM. + +## Constraint + +`SerializationService:SerializeInstancesAsync` throws if services are in the top-level list. Children-only rbxm stays; service metadata must be encoded separately. + +## Changes + +### 1. Plugin -- New `encodeService` helper + +Create `[plugin/src/ChangeBatcher/encodeService.lua](plugin/src/ChangeBatcher/encodeService.lua)`. Returns a full `ServiceChunk` table: + +```lua +-- Returns: +-- { +-- className = "Lighting", +-- data = buffer, -- rbxm of children (empty buffer if no children) +-- properties = {}, -- non-Ref encoded properties (via RbxDom) +-- attributes = {}, -- encoded attributes map +-- tags = {}, -- tags array +-- refs = {}, -- Ref properties: { PropName = { name, className } } +-- } +``` + +Implementation reuses existing `encodeProperty`, `RbxDom.findClassDescriptor`, `RbxDom.findCanonicalPropertyDescriptor`, and the `UNENCODABLE_DATA_TYPES` filter from `[plugin/src/ChangeBatcher/propertyFilter.lua](plugin/src/ChangeBatcher/propertyFilter.lua)`. + +**Property encoding** -- same loop as `[encodeInstance.lua](plugin/src/ChangeBatcher/encodeInstance.lua)` lines 200-249 but without script-specific skips: + +```lua +for propertyName, propertyMeta in classDescriptor.properties do + -- skip Parent, Name, Archivable, Attributes, Tags + if isReadable and doesSerialize then + if descriptor.dataType == "Ref" then + -- Read ref target; if it's a direct child, record name+className + local ok, target = descriptor:read(service) + if ok and target and target.Parent == service then + refs[propertyName] = { name = target.Name, className = target.ClassName } + end + elseif not UNENCODABLE_DATA_TYPES[descriptor.dataType] then + local ok, encoded = encodeProperty(service, propertyName, descriptor) + if ok and encoded ~= nil then + properties[propertyName] = encoded + end + end + end +end +``` + +**Attributes** -- encoded via the Attributes property descriptor (same as `encodeAttributes` in `encodeInstance.lua` lines 121-137), but returned as a separate field. + +**Tags** -- read via `service:GetTags()`, returned as a plain array. + +**Refs** -- Ref properties where the target is a direct child of the service. Encoded as `{ name = target.Name, className = target.ClassName }` so the server can resolve them by finding the matching child after cloning from rbxm. + +**Children rbxm** -- `SerializationService:SerializeInstancesAsync(service:GetChildren())` if children exist, otherwise empty buffer. + +### 2. Plugin -- Update `performSyncback` in `[plugin/src/App/init.lua](plugin/src/App/init.lua)` (lines 641-651) + +```lua +local encodeService = require(Plugin.ChangeBatcher.encodeService) + +local services = {} +for _, className in SYNCBACK_SERVICES do + local ok, service = pcall(game.FindService, game, className) + if ok and service then + table.insert(services, encodeService(service)) + end +end +``` + +No children guard. Every found service is encoded. + +### 3. Server -- Extend `ServiceChunk` in `[src/web/interface.rs](src/web/interface.rs)` (lines 30-37) + +```rust +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct ServiceChunk { + pub class_name: String, + #[serde(with = "serde_bytes")] + pub data: Vec, + #[serde(default)] + pub properties: HashMap, + #[serde(default)] + pub attributes: HashMap, + #[serde(default)] + pub tags: Vec, + #[serde(default)] + pub refs: HashMap, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct ServiceRef { + pub name: String, + pub class_name: String, +} +``` + +All new fields use `#[serde(default)]` for backwards compatibility. + +### 4. Server -- Update `build_dom_from_chunks` in `[src/cli/serve.rs](src/cli/serve.rs)` (lines 140-179) + +After creating the service instance, apply properties/attributes/tags: + +```rust +for chunk in &payload.services { + let mut builder = InstanceBuilder::new(&chunk.class_name); + + // Apply encoded properties + for (key, value) in &chunk.properties { + builder = builder.with_property(key.as_str(), value.clone()); + } + + // Apply attributes + if !chunk.attributes.is_empty() { + let attrs = rbx_dom_weak::types::Attributes::new(); + // ... convert HashMap to Attributes ... + builder = builder.with_property("Attributes", Variant::Attributes(attrs)); + } + + // Apply tags + if !chunk.tags.is_empty() { + builder = builder.with_property("Tags", Variant::Tags(chunk.tags.clone().into())); + } + + let service_ref = dom.insert(root_ref, builder); + created_services.insert(chunk.class_name.clone()); + + // Clone children from rbxm + if !chunk.data.is_empty() { + let chunk_dom = rbx_binary::from_reader(Cursor::new(&chunk.data))?; + for &child_ref in chunk_dom.root().children() { + deep_clone_into(&chunk_dom, &mut dom, child_ref, service_ref, &mut global_ref_map); + } + } + + // Resolve Ref properties by finding matching children + if !chunk.refs.is_empty() { + let children: Vec = dom.get_by_ref(service_ref).unwrap().children().to_vec(); + for (prop_name, target) in &chunk.refs { + let found = children.iter().find(|&&child_ref| { + let child = dom.get_by_ref(child_ref).unwrap(); + child.name.as_str() == target.name && child.class.as_str() == target.class_name + }); + if let Some(&child_ref) = found { + let service = dom.get_by_ref_mut(service_ref).unwrap(); + service.properties.insert(prop_name.as_str().into(), Variant::Ref(child_ref)); + } + } + } +} +``` + +This handles `Workspace.CurrentCamera` naturally -- the Camera is cloned from the rbxm, then the Ref is resolved by matching `name="Camera", className="Camera"` among children. `syncback_loop` can then find it and destroy it when `syncCurrentCamera: false`. + +### 5. Test helpers -- `[tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs)` + +Update `make_service_chunk` to accept optional properties/attributes/tags/refs. Add companion: + +```rust +pub fn make_service_chunk_full( + class_name: &str, + properties: Vec<(&str, Variant)>, + attributes: Vec<(&str, Variant)>, + tags: Vec, + refs: Vec<(&str, &str, &str)>, // (prop_name, target_name, target_class) + children: Vec, +) -> ServiceChunk +``` + +Update `make_rbxl_from_chunks` to apply properties/attributes/tags/refs to the service in the DOM (mirrors server logic). + +Update `roundtrip_build_syncback_rebuild` to extract service properties from the built rbxl and populate `ServiceChunk` fields. + +### 6. Fixture -- `[rojo-test/serve-tests/live_syncback/default.project.json5](rojo-test/serve-tests/live_syncback/default.project.json5)` + +Add service entries with `$path` for Lighting, SoundService, StarterPlayer, TextChatService, VoiceChatService. Create empty `src/` subdirectories. + +### 7. New tests -- `[tests/tests/live_syncback.rs](tests/tests/live_syncback.rs)` + +**Service property parity tests** (use `assert_live_matches_cli`): + +- `**parity_lighting_properties`** -- Lighting with `Ambient`, `ClockTime`, `Brightness` + child +- `**parity_soundservice_properties`** -- SoundService with `AmbientReverb`, `DistanceFactor` +- `**parity_starterplayer_properties`** -- StarterPlayer with `CameraMaxZoomDistance` +- `**parity_textchatservice_properties**` -- TextChatService with `ChatVersion` +- `**parity_childless_service_with_properties**` -- Service with properties but zero children +- `**parity_service_properties_with_children**` -- Service with both properties AND children +- `**parity_multiple_services_with_properties**` -- Multiple services in one request + +**Camera removal test:** + +- `**parity_camera_not_synced`** -- Workspace with Camera child + `CurrentCamera` ref. Verify Camera does NOT appear on disk (default `syncCurrentCamera: false`). + +**Round-trip test:** + +- `**roundtrip_service_properties`** -- Build -> live syncback -> rebuild. Compare service properties survive the cycle. + diff --git a/.cursor/plans/fix_test_port_toctou_86726cb4.plan.md b/.cursor/plans/fix_test_port_toctou_86726cb4.plan.md new file mode 100644 index 00000000..55859e75 --- /dev/null +++ b/.cursor/plans/fix_test_port_toctou_86726cb4.plan.md @@ -0,0 +1,104 @@ +--- +name: Fix test port TOCTOU +overview: Two-layer fix for port-in-use race conditions. (1) Server-side -- retry bind with backoff when a known port is busy. (2) Test-side -- retry with a new random port when the serve process dies from AddrInUse. +todos: + - id: server-bind-retry + content: Add bind retry with backoff in `LiveServer::start()` for known ports + status: completed + - id: respawn-method + content: Add `respawn_with_new_port()` method to `TestServeSession` + status: completed + - id: wait-retry + content: Modify `wait_to_come_online()` to detect AddrInUse and retry via respawn + status: completed + - id: fresh-rebuild-retry + content: Add port-retry logic to `fresh_rebuild_read()` + status: completed +isProject: false +--- + +# Fix Port-in-Use Race Conditions + +## Problem + +Two failure modes: + +1. **Tests (random ports):** `get_port_number()` in `[tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs)` binds port 0, reads the assigned port, drops the listener, then passes the port to the spawned serve process. Another test can grab the same port in that TOCTOU window. +2. **Server (known ports):** `TcpListener::bind(address).await.unwrap()` in `[src/web/mod.rs:80](src/web/mod.rs)` panics immediately if the port is in use, with no retry. A previous server instance shutting down or a transient conflict gives no grace period. + +## Layer 1: Server-side bind retry (`src/web/mod.rs`) + +Replace the `.unwrap()` on `TcpListener::bind()` at line 80 with a retry loop: + +- On `AddrInUse`: retry up to 5 times with exponential backoff (200ms, 400ms, 800ms, 1600ms, 3200ms -- ~6s total) +- On any other bind error, or after retries exhausted: return `anyhow::Error` (propagates up cleanly instead of panicking) + +```rust +let listener = { + const MAX_BIND_RETRIES: u32 = 5; + const BASE_BACKOFF_MS: u64 = 200; + let mut last_err = None; + for attempt in 0..MAX_BIND_RETRIES { + match TcpListener::bind(address).await { + Ok(listener) => { last_err = None; break; } + Err(err) if err.kind() == std::io::ErrorKind::AddrInUse => { + let delay = BASE_BACKOFF_MS * 2u64.pow(attempt); + log::warn!( + "Port {} in use, retrying in {}ms (attempt {}/{})", + address.port(), delay, attempt + 1, MAX_BIND_RETRIES + ); + tokio::time::sleep(Duration::from_millis(delay)).await; + last_err = Some(err); + } + Err(err) => { last_err = Some(err); break; } + } + } + // unwrap or propagate +}; +``` + +This helps both tests (brief TOCTOU window) and production (previous process releasing a port). + +## Layer 2: Test-side retry with new random port (`tests/rojo_test/serve_util.rs`) + +If the serve process still dies after the server-side retries (e.g., the port is held long-term by another test), the test infrastructure retries with a completely new port. + +### 2a. Add `respawn_with_new_port()` to `TestServeSession` + +Kills the current process, picks a new ephemeral port, and respawns: + +```rust +fn respawn_with_new_port(&mut self) { + let port = get_port_number(); + let port_string = port.to_string(); + let working_dir = get_working_dir_path(); + + let rojo_process = Command::new(ROJO_PATH) + .args(["serve", self.project_path.to_str().unwrap(), "--port", &port_string]) + .current_dir(working_dir) + .stderr(Stdio::piped()) + .spawn() + .expect("Couldn't start Rojo"); + + self.rojo_process = KillOnDrop(rojo_process); + self.port = port; +} +``` + +### 2b. Modify `wait_to_come_online()` + +In the `Ok(Some(status))` branch (line ~266-276), detect `AddrInUse` in stderr and retry: + +- Add a `port_retries` counter (max 3 attempts) +- On `AddrInUse`: call `self.respawn_with_new_port()`, reset the backoff counter, continue +- On other errors: panic as before + +### 2c. Apply same pattern to `fresh_rebuild_read()` + +`fresh_rebuild_read()` (line ~929) has its own inline wait loop. Wrap the spawn + wait in a retry loop that detects `AddrInUse` and retries with a new port. + +## Files changed + +- `[src/web/mod.rs](src/web/mod.rs)` -- bind retry with backoff (Layer 1) +- `[tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs)` -- respawn with new port (Layer 2) + diff --git a/.cursor/plans/git-based_sync_defaults_be6371b6.plan.md b/.cursor/plans/git-based_sync_defaults_be6371b6.plan.md new file mode 100644 index 00000000..95d6f521 --- /dev/null +++ b/.cursor/plans/git-based_sync_defaults_be6371b6.plan.md @@ -0,0 +1,329 @@ +--- +name: Git-Based Sync Defaults +overview: Add git-aware default sync direction selection to the two-way sync confirmation UI. Server sends changed-instance IDs + SHA1 hashes (both staged and HEAD) to the plugin for auto-selecting defaults. Accepted files are auto-staged via git add. +todos: + - id: sha1-dep + content: Add sha1 crate to Cargo.toml + status: completed + - id: sha1-luau + content: Add SHA1.luau to plugin/src/ from user-provided file + status: completed + - id: git-module + content: Create src/git.rs with git CLI helpers (repo root, changed files, committed content, SHA1 computation, git add) + status: completed + - id: interface-rs + content: Add GitMetadata struct and field to ServerInfoResponse, add stage_ids to WriteRequest and PatchSet + status: completed + - id: api-rs + content: Call compute_git_metadata in handle_api_rojo (two-phase lock), wire up project root, handle stage_ids in handle_api_write + status: completed + - id: patchtree-lua + content: Update PatchTree.build to accept gitMetadata and set defaultSelection based on git status + SHA1 hash comparison + status: completed + - id: app-init-lua + content: Pass serverInfo.gitMetadata to PatchTree.build call sites in App/init.lua, build stage_ids in confirmation flow + status: completed + - id: auto-stage + content: "Auto git-add: all push-accepted files + auto-selected pull-accepted files. Split staging between api.rs and change_processor." + status: completed + - id: test-verify + content: "Build and verify: cargo build, cargo test, cargo clippy" + status: completed +isProject: false +--- + +# Git-Based Sync Direction Defaults + +## Summary + +When the plugin shows the initial sync confirmation dialog, currently all items default to `nil` (unselected). This feature adds smart defaults based on git status: + +- **File has no git changes** (same as HEAD): default to "Studio" (pull) -- any diff is because Studio was changed +- **File has git changes AND is a script AND Studio Source matches committed version**: default to "Atlas" (push) -- developer changed the file, Studio hasn't diverged from the last commit +- **Otherwise**: `nil` -- user must decide (both sides changed from committed, or non-script with changes) +- **No git repo**: all defaults remain `nil` (feature is no-op) + +These defaults only apply to **Updated** items (where both Studio and filesystem have the instance). Added/Removed items remain `nil` to avoid destructive auto-selections. + +**Hash comparison is scripts-only for now.** Non-script property/misc changes that are git-changed remain `nil` (deferred feature -- would require reproducing a bit-for-bit state on the Roblox side to compare). + +Recomputed fresh on **each connect** (not cached). + +## User Stories + +1. **Scripts changed locally, hashes match Studio Source**: Developer changed scripts in git. Plugin receives committed SHA1 hashes. Studio's `script.Source` (non-draft version) matches the committed hashes. Result: **"Atlas" auto-selected** for those scripts. +2. **Scripts changed by someone else too**: Developer pulled commits that changed files, AND someone edited in Studio. Committed SHA1 won't match Studio Source. Result: **No bubble selected** -- user must decide. +3. **Property/misc changes in git**: Non-script files flagged as changed. Result: **No auto-selection** (deferred feature). +4. **No git changes on a file**: File is identical to HEAD. Any diff is because Studio was changed. Result: **"Studio" auto-selected**. + +## Architecture + +```mermaid +sequenceDiagram + participant Git as Git CLI + participant Server as Rust Server + participant Tree as RojoTree + participant Plugin as Lua Plugin + participant UI as PatchTree / UI + + Note over Server: On each GET /api/rojo + Server->>Tree: snapshot path_to_ids + class names (brief lock) + Server->>Git: git rev-parse --show-toplevel + Git-->>Server: repo root (or fail = no git) + Server->>Git: git diff HEAD --name-only + Git-->>Server: list of changed files + Server->>Server: map paths to Refs using snapshot + loop For each changed script with committed version + Server->>Git: git show HEAD:path + git show :0:path + Git-->>Server: committed + staged content + Server->>Server: SHA1 in blob format (no normalization) + end + Server->>Plugin: ServerInfoResponse + gitMetadata + + Note over Plugin: Building PatchTree + Plugin->>UI: For each Updated node + alt Not in changedIds + UI->>UI: defaultSelection = "pull" + else Script with committed hash + Plugin->>Plugin: SHA1 blob format of script.Source + alt Any hash matches + UI->>UI: defaultSelection = "push" + else No hash matches + UI->>UI: defaultSelection = nil + end + else Non-script or no committed hash + UI->>UI: defaultSelection = nil + end +``` + + + +## Server Side (Rust) + +### 1. Add `sha1` crate dependency + +In [Cargo.toml](Cargo.toml), add under `[dependencies]`: + +```toml +sha1 = "0.10" +``` + +We compute hashes ourselves using the git blob format scheme (`"blob \0"`) so both sides produce identical hashes from identical content. + +### 2. New module: `src/git.rs` + +All git interaction via `std::process::Command`. + +- `**git_repo_root(project_root: &Path) -> Option**`: Runs `git -C rev-parse --show-toplevel`. Returns `None` if not a git repo or git not installed. +- `**git_changed_files(repo_root: &Path) -> Option>**`: Runs `git -C diff HEAD --name-only` to get files differing from HEAD (covers staged + unstaged). Also runs `git -C ls-files --others --exclude-standard` for untracked files. Returns paths relative to repo root. **Edge case**: if HEAD doesn't exist (no commits yet), treat all tracked files as changed or return None gracefully. +- `**git_show_head(repo_root: &Path, rel_path: &Path) -> Option`**: Runs `git -C show HEAD:`. Returns committed content at HEAD. Returns `None` for new files. +- `**git_show_staged(repo_root: &Path, rel_path: &Path) -> Option`**: Runs `git -C show :0:`. Returns staged (index) content. Returns `None` if not staged or file doesn't exist in index. +- `**compute_blob_sha1(content: &str) -> String`**: Computes `SHA1("blob \0")`. Returns 40-char lowercase hex string. No normalization. +- `**git_add(repo_root: &Path, paths: &[PathBuf])`**: Runs `git -C add -- `. Stages files for commit. No-op if paths is empty. +- `**compute_git_metadata(path_snapshot: &PathSnapshot, project_root: &Path) -> Option`**: Takes a pre-extracted `PathSnapshot` (see section 4). Orchestrates git commands: + 1. Find repo root + 2. Get changed files set + 3. Map changed file paths to instance Refs using the path snapshot + 4. For each changed script: collect up to 2 hashes -- HEAD version and staged version (skip staged if identical to HEAD) + 5. Returns `None` if no git repo + +### 3. Update data structures in [src/web/interface.rs](src/web/interface.rs) + +**GitMetadata** (new struct for ServerInfoResponse): + +```rust +#[derive(Debug, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct GitMetadata { + /// Instance IDs whose backing files have changes relative to HEAD + pub changed_ids: Vec, + /// For changed script instances: SHA1 hashes of prior versions (git blob format). + /// Contains 1-2 hashes per script: HEAD version and staged version (if different). + /// Plugin checks if Studio Source matches ANY hash to default "Atlas". + pub script_committed_hashes: HashMap>, +} +``` + +Add to `ServerInfoResponse`: + +```rust +#[serde(default, skip_serializing_if = "Option::is_none")] +pub git_metadata: Option, +``` + +**WriteRequest** (add `stage_ids` field): + +```rust +pub struct WriteRequest { + pub session_id: SessionId, + pub removed: Vec, + #[serde(default)] + pub added: HashMap, + pub updated: Vec, + /// Instance IDs whose backing files should be staged via git add. + /// Includes: all push-accepted IDs + auto-selected pull-accepted IDs. + #[serde(default)] + pub stage_ids: Vec, +} +``` + +**PatchSet** (add `stage_ids` field for change_processor): + +Add to the existing PatchSet struct in [src/snapshot/patch.rs](src/snapshot/patch.rs): + +```rust +/// Instance IDs whose backing files should be staged after writing. +/// Populated from WriteRequest.stage_ids, consumed by change_processor. +#[serde(default, skip)] +pub stage_ids: HashSet, +``` + +### 4. Update `handle_api_rojo()` in [src/web/api.rs](src/web/api.rs) -- two-phase lock + +**Critical**: Do NOT hold the tree lock while running git subprocesses. + +**Phase 1 (with lock)**: Snapshot the data needed from the tree: + +- `path_to_ids` entries (clone the relevant mappings) +- Instance class names for each Ref +- `instigating_source` paths + +**Phase 2 (no lock)**: Run git commands and compute hashes using the snapshot. + +```rust +// Phase 1: brief lock +let path_snapshot = { + let tree = self.serve_session.tree(); + extract_path_snapshot(&tree) // Clone path_to_ids, class names, instigating sources +}; +// Lock released here + +// Phase 2: slow git work, no lock held +let git_metadata = compute_git_metadata(&path_snapshot, project_root); +``` + +### 5. Wire up in [src/serve_session.rs](src/serve_session.rs) + +Expose `project_root()` from `ServeSession` (the project file's parent directory). Compute and store `git_repo_root` as `Option` at session start for use by auto-staging in `handle_api_write` and change_processor. + +### 5b. Auto-stage accepted files + +After the user confirms sync selections, automatically `git add` affected files based on these rules: + +- **Push (Atlas) accepted**: Always stage the backing file. The filesystem changes are being finalized into Studio, so they're ready to commit. Applies to ALL push items (auto-selected and manually-selected), including non-scripts. +- **Pull (Studio) accepted**: Only stage if the item was auto-selected (had a non-nil `defaultSelection`). Manually-chosen pulls are left unstaged so the user can diff them. + +**Staging is split between two components** to handle the async Source write timing: + +1. `**handle_api_write()` in [src/web/api.rs](src/web/api.rs)**: Stages files that api.rs writes directly -- additions (new files), removals (deleted files), and push-accepted files (already on disk, just need staging). Resolves Ref IDs to filesystem paths via `instigating_source` in the tree. Calls `git_add` in one batch. Also copies `stage_ids` into the PatchSet before sending to change_processor. +2. `**handle_tree_event()` in [src/change_processor.rs](src/change_processor.rs)**: After writing Source to disk (line ~1807), checks if the instance's Ref is in `patch.stage_ids`. If so, calls `git_add` on the written path. The change_processor receives `repo_root: Option` during construction (computed once at session start). + +**Why split?** Source writes are async -- api.rs sends the PatchSet to change_processor via a channel and returns immediately. If api.rs staged those files, it would stage the OLD content before change_processor writes the new content. Staging must happen AFTER the write. + +**Plugin side** ([plugin/src/ServeSession.lua](plugin/src/ServeSession.lua)): In `__confirmAndApplyInitialPatch`, build the `stage_ids` list: + +- For each push-selected item: always include in `stage_ids` +- For each pull-selected item: include only if the PatchTree node had `defaultSelection ~= nil` +- Always send the write request if there are either pull changes OR stage IDs (currently only sent when pullPatch is non-empty -- expand the condition to also fire when `stage_ids` is non-empty) + +If not in a git repo (`repo_root` is `None`), all stage calls are no-ops. + +## Plugin Side (Lua) + +### 6. Add SHA1.luau to plugin + +Copy the user-provided [SHA1.luau](c:\Users\Joe\Downloads\SHA1.luau) to `plugin/src/SHA1.luau`. This is a native-optimized SHA1 implementation that takes a `buffer` and returns a 40-char hex string. + +### 7. Update `PatchTree.build()` in [plugin/src/PatchTree.lua](plugin/src/PatchTree.lua) + +Add a 4th parameter `gitMetadata` (the `gitMetadata` table from `serverInfo`, may be nil). + +At the start of `build()`, convert `changedIds` array to a Set for O(1) lookup: + +```lua +local changedIdSet = nil +if gitMetadata and gitMetadata.changedIds then + changedIdSet = {} + for _, id in gitMetadata.changedIds do + changedIdSet[id] = true + end +end +``` + +For **Updated** nodes (around line 308-318), replace `defaultSelection = nil` with: + +```lua +local defaultSelection = nil +if changedIdSet then + if not changedIdSet[change.id] then + defaultSelection = "pull" + elseif instance:IsA("LuaSourceContainer") + and gitMetadata.scriptCommittedHashes then + local hashes = gitMetadata.scriptCommittedHashes[change.id] + if hashes then + local source = instance.Source + local gitBlob = "blob " .. tostring(#source) .. "\0" .. source + local studioHash = SHA1(buffer.fromstring(gitBlob)) + for _, hash in hashes do + if studioHash == hash then + defaultSelection = "push" + break + end + end + end + end +end +``` + +**Key**: `script.Source` returns the non-draft (published) version when Script Drafts are enabled. Both sides use the git blob format `"blob \0"` as SHA1 input. No normalization -- committed content is LF (repo standard), Studio Source is LF. + +**Added** and **Removed** nodes: keep `defaultSelection = nil` (unchanged). + +### 8. Update `PatchTree.build()` call sites in [plugin/src/App/init.lua](plugin/src/App/init.lua) + +Must support both **"Initial"** and **"Always"** confirmation modes. In "Always" mode, every incoming WebSocket patch triggers the confirmation dialog via `__onWebSocketMessage` -> `__userConfirmCallback(instanceMap, patch, self.__serverInfo)`, so `serverInfo.gitMetadata` is available for ongoing patches too. + +**Note**: Per-item push/pull/ignore selections for ongoing WebSocket patches (Always mode) are a pre-existing limitation (see `ServeSession.lua` line 311-314). The git defaults will be set in the PatchTree but the existing "Accept"/"Abort" handling for ongoing patches doesn't yet support them. This is not introduced by this feature. + +Three call sites in `App:startSession()` and one approach to wire them: + +**Cache `serverInfo` in a closure-scoped local** accessible to all callbacks: + +```lua +local cachedServerInfo = nil +``` + +- **Line 867** (`setConfirmCallback`, initial + ongoing confirms): Has `serverInfo` directly. Cache it: `cachedServerInfo = serverInfo`. Pass `serverInfo.gitMetadata` to `PatchTree.build`. Works for both Initial and Always modes since `__onWebSocketMessage` passes `self.__serverInfo`. +- **Line 918** (`setPatchUpdateCallback`, patches merging during active confirmation): Doesn't receive `serverInfo`. Use cached: `cachedServerInfo and cachedServerInfo.gitMetadata`. This is safe because the confirm callback always fires first (sets `cachedServerInfo`) before any patch updates can arrive. +- **Line 659** (precommit hook, auto-accepted patches): For patches that bypass confirmation (e.g., "Never" mode), this builds a tree for the Connected page display. Use `cachedServerInfo and cachedServerInfo.gitMetadata`. Will be `nil` if no confirmation has happened yet (first connect stores it). + +### 9. Import SHA1 in PatchTree.lua + +```lua +local SHA1 = require(Plugin.SHA1) +``` + +## Key Design Decisions + +- **Self-computed SHA1 with git blob format scheme**: Read content via `git show HEAD:` and `git show :0:` (staged), compute SHA1 using `"blob \0"`. Send both hashes (HEAD + staged if different) so the plugin can match against either version. +- **Two-phase tree lock**: Snapshot path mappings under brief lock, then release before running git subprocesses. Prevents blocking all API requests during I/O-bound git operations. +- **Auto-staging with asymmetric rules**: Push-accepted items are always staged (filesystem changes finalized). Pull-accepted items are only staged if auto-selected (manually-chosen pulls left unstaged for manual diff). Plugin sends `stage_ids` in the WriteRequest; server resolves to paths and runs `git add`. +- **Split staging responsibility**: api.rs stages additions, removals, and push files directly. change_processor stages Source writes after they complete (via `stage_ids` on PatchSet). This avoids staging old content before async writes finish. +- **Only Updated items get defaults**: Added (would destructively delete files if "pull") and Removed (IDs aren't in server tree) are left as `nil` +- **Scripts-only hash comparison**: Non-script property changes are deferred (see note below) +- **Git via subprocess**: No git2/libgit2 dependency, just `std::process::Command`. Graceful fallback if git not installed or not a repo. Per-file `git show` calls are acceptable for typical changed file counts; `git cat-file --batch` is a future optimization for large changesets. +- **Draft handling**: `script.Source` returns the non-draft version when Script Drafts are enabled, which is exactly what we compare against the committed version +- **Fresh computation per connect**: Not cached -- each `GET /api/rojo` recomputes git state +- **No normalization anywhere**: Committed content is LF (repo standard), Studio Source is LF, VFS normalizes reads. Both sides hash identical LF content. +- **Init-style scripts (init.luau) handled correctly**: `dir.rs` registers all init file paths (init.luau, init.server.luau, etc.) in `relevant_paths`, so `path_to_ids` maps `MyFolder/init.luau` to the correct Ref. `instigating_source` is the actual init file path (not the directory), so Source writes and git-add target the correct file. Git reports file-level changes (`MyFolder/init.luau`), matching the indexed path. +- **No-HEAD edge case**: If the repo has no commits (`git diff HEAD` fails), treat as "no git metadata" and return None. All defaults remain nil. + +## Deferred: Non-Script (Attribute/Property/Tag) Hash Comparison + +Hash comparison for non-script changes (properties, attributes, tags) is **not included in this feature** and would be significantly harder: + +- Would require building a **deterministic payload** of the instance's properties/attributes/tags (possibly msgpack with guaranteed key ordering, including default values) to hash against the committed file content (meta.json5, model.json5, etc.) +- Even with a deterministic serialization, there is **no "draft" concept** for non-script properties in Studio. `script.Source` has the draft/non-draft distinction, but property values are always live. There's no way to get "the version before the user started editing" for properties. +- This makes the comparison fundamentally less useful: if a user drags a Part in Studio, the property is immediately the new value with no way to recover the "original" for comparison. +- For now, non-script changes that are git-changed simply get `defaultSelection = nil` (user must decide). + diff --git a/.cursor/plans/git_sync_defaults_audit_557237e0.plan.md b/.cursor/plans/git_sync_defaults_audit_557237e0.plan.md new file mode 100644 index 00000000..4dea27b1 --- /dev/null +++ b/.cursor/plans/git_sync_defaults_audit_557237e0.plan.md @@ -0,0 +1,272 @@ +--- +name: Git Sync Defaults Audit +overview: Audit of the git-based sync defaults feature spanning git.rs (server), PatchTree.lua (plugin default selection logic), SHA1.luau (plugin hashing), and the auto-staging pipeline. +todos: [] +isProject: false +--- + +# Audit: Git-Based Sync Defaults + +## Feature Summary + +When the plugin connects during `atlas serve`, the server computes git metadata (which files changed, SHA1 hashes of committed/staged script content) and sends it to the plugin. The plugin uses this to auto-select sync directions in the confirmation dialog: + +- **File unchanged in git** -> default "pull" (Studio changed it) +- **File changed + script + Studio Source matches committed hash** -> default "push" (developer changed file, Studio still has old version) +- **Otherwise** -> nil (user must decide) + +After confirmation, accepted files are auto-staged via `git add`. + +```mermaid +sequenceDiagram + participant Plugin + participant Server + participant Git + + Plugin->>Server: GET /api/rojo + Server->>Git: git diff HEAD --name-only + Server->>Git: git ls-files --others + Note over Server: Brief tree lock: resolve paths to Refs + Server->>Git: git show HEAD:path (per changed script) + Server->>Git: git show :0:path (per changed script) + Note over Server: compute_blob_sha1 for each version + Server-->>Plugin: ServerInfoResponse with gitMetadata + + Note over Plugin: PatchTree.build() computes defaultSelection per node + Note over Plugin: User reviews/overrides defaults + + Plugin->>Server: POST /api/write with stageIds + Note over Server: api.rs resolves stageIds to paths + Note over Server: change_processor writes Source, appends paths + Server->>Git: git add (consolidated, single call) +``` + + + +## Files Audited + + +| File | Lines | Purpose | +| ---------------------------------------------------------------------------------------- | ---------- | ------------------------------------------------------------------ | +| `[src/git.rs](src/git.rs)` | 766 | Server-side git helpers: changed files, blob SHA1, auto-staging | +| `[plugin/src/PatchTree.lua](plugin/src/PatchTree.lua)` | 715 | Default selection logic, tree building | +| `[plugin/src/SHA1.luau](plugin/src/SHA1.luau)` | 112 | Native SHA1 implementation | +| `[src/web/interface.rs](src/web/interface.rs)` | ~20 lines | `GitMetadata` struct, `stage_ids` on `WriteRequest` | +| `[src/web/api.rs](src/web/api.rs)` | ~30 lines | `handle_api_rojo` metadata computation, `handle_api_write` staging | +| `[src/change_processor.rs](src/change_processor.rs)` | ~40 lines | Source write staging, rename path tracking | +| `[src/snapshot/patch.rs](src/snapshot/patch.rs)` | ~10 lines | `stage_ids`/`stage_paths` fields on `PatchSet` | +| `[plugin/src/App/init.lua](plugin/src/App/init.lua)` | ~30 lines | `autoSelectedIds` construction, `cachedServerInfo` | +| `[plugin/src/ServeSession.lua](plugin/src/ServeSession.lua)` | ~80 lines | `stageIds` construction per selection type | +| `[plugin/src/App/StatusPages/Confirming.lua](plugin/src/App/StatusPages/Confirming.lua)` | 459 | Selection state management, `didUpdate` merge logic | +| `[tests/tests/git_sync_defaults.rs](tests/tests/git_sync_defaults.rs)` | 952 | Integration tests | +| `[plugin/src/PatchTree.spec.lua](plugin/src/PatchTree.spec.lua)` | ~200 lines | Plugin unit tests for git defaults | + + +--- + +## Verdict: No Critical Issues Found + +The feature is well-designed and correctly implemented. The round-trip identity invariant is not directly affected by this feature -- git metadata is read-only advisory data that influences UI defaults, not filesystem writes. The auto-staging is purely additive (`git add`) and cannot corrupt the filesystem. + +--- + +## Findings + +### Finding 1: Performance -- O(N) Subprocess Spawning [Medium] + +`compute_git_metadata` spawns **2 + 2N** git subprocesses for N changed scripts: + +```35:35:src/git.rs +// 1 call +``` + +- `git diff HEAD --name-only` (1 process) +- `git ls-files --others --exclude-standard` (1 process) +- Per changed script: `git show HEAD:` (N processes) +- Per changed script: `git show :0:` (N processes) + +On Windows, process creation is 10-100x slower than Linux. For 50 changed scripts, this is 102 subprocess spawns on every plugin connect. + +**Recommendation**: Replace per-file `git show` calls with a single `git cat-file --batch` invocation that reads all file contents in one subprocess. This collapses 2N calls to 1. Not a correctness issue, but could cause noticeable connect latency on large repos with many changes. Defer unless users report slow connects. + +### Finding 2: Windows Path Normalization Edge Case [Low] + +Git commands return paths with `/` separators. `PathBuf::from("src/module.luau")` on Windows preserves the forward slashes. The tree lookup may fail if the tree stores paths with `\` separators. + +```188:199:src/git.rs + for rel_path in &changed_files { + let abs_path = repo_root.join(rel_path); + let canonical_path = std::fs::canonicalize(&abs_path).ok(); + + let first = tree.get_ids_at_path(&abs_path); + let ids = if !first.is_empty() { + first + } else if let Some(ref canon) = canonical_path { + tree.get_ids_at_path(canon) + } else { + first + }; +``` + +The `canonicalize` fallback mitigates this, but `canonicalize` can fail (file deleted, permissions). When it fails, the instance won't appear in `changedIds` -- it degrades to "no default" which is safe. + +**Recommendation**: No action needed. The fallback is sufficient and the failure mode is safe (user decides manually). + +### Finding 3: `refresh_git_index` Duplicates Repo Detection [Low -- Code Quality] + +`[src/git.rs:252-260](src/git.rs)` walks up the directory tree to find `.git`, but the repo root is already cached in `ServeSession.git_repo_root`. + +```251:260:src/git.rs +pub fn refresh_git_index(project_dir: &Path) { + let mut check_dir = Some(project_dir); + let mut is_git_repo = false; + while let Some(dir) = check_dir { + if dir.join(".git").exists() { + is_git_repo = true; + break; + } + check_dir = dir.parent(); + } +``` + +Both call sites (`[src/cli/serve.rs:133](src/cli/serve.rs)` and `[src/cli/syncback.rs:282](src/cli/syncback.rs)`) could pass the known repo root instead. However, `syncback.rs` doesn't have a `ServeSession` -- it runs standalone. The current approach is correct for both call sites. + +**Recommendation**: Accept as-is. The directory walk is negligible cost and keeps the function self-contained for both serve and CLI syncback contexts. + +### Finding 4: Hash Parity Verified [No Issue] + +The critical correctness requirement is that the Rust server and Luau plugin produce identical SHA1 hashes for the same content. Both implementations use the git blob format `"blob \0"`: + +**Rust** (`[src/git.rs:118-124](src/git.rs)`): + +```118:124:src/git.rs +pub fn compute_blob_sha1(content: &str) -> String { + let header = format!("blob {}\0", content.len()); + let mut hasher = Sha1::new(); + hasher.update(header.as_bytes()); + hasher.update(content.as_bytes()); + format!("{:x}", hasher.finalize()) +} +``` + +**Luau** (`[plugin/src/PatchTree.lua:363-364](plugin/src/PatchTree.lua)`): + +```363:364:plugin/src/PatchTree.lua + local gitBlob = "blob " .. tostring(#source) .. "\0" .. source + local studioHash = SHA1(buffer.fromstring(gitBlob)) +``` + +- Rust `content.len()` = byte length. Luau `#source` = byte length. Consistent. +- Both use UTF-8 encoding. `\0` is the null byte in both languages. +- The `sha1` crate (Rust) and `SHA1.luau` (native Luau implementation with `--!native` + `--!optimize 2`) are standard SHA1. +- Integration tests in `[tests/tests/git_sync_defaults.rs](tests/tests/git_sync_defaults.rs)` verify against real `git hash-object` output. + +### Finding 5: Draft Source Fallback is a Nice Touch [No Issue] + +```372:384:plugin/src/PatchTree.lua + if defaultSelection == nil then + local ok, draft = pcall(ScriptEditorService.GetEditorSource, ScriptEditorService, instance) + if ok and draft ~= source then + local draftBlob = "blob " .. tostring(#draft) .. "\0" .. draft + local draftHash = SHA1(buffer.fromstring(draftBlob)) + for _, hash in hashes do + if draftHash == hash then + defaultSelection = "push" + break + end + end + end + end +``` + +When Script Drafts is enabled, `instance.Source` may not reflect the user's current editor content until they "commit" the draft. This fallback checks the draft source against committed hashes, ensuring the default selection is still accurate even with Script Drafts. + +### Finding 6: Consolidated git_add Prevents index.lock Races [No Issue] + +The staging pipeline correctly consolidates all paths into a single `git add` call: + +```1875:1880:src/change_processor.rs + // Consolidated git staging: one git_add call for all paths + // ... + if !pending_stage_paths.is_empty() { + if let Some(ref repo_root) = self.git_repo_root { + crate::git::git_add(repo_root, &pending_stage_paths); + } + } +``` + +Pre-resolved paths from `api.rs` are merged with Source-write paths from `change_processor.rs`. Rename handling correctly updates `pending_stage_paths` (retains old path, pushes new path) at lines 1363-1368, 1531-1534, and 1713-1717. The batch staging test (`batch_stage_20_files_no_index_lock_race`) confirms this works for 20 files. + +### Finding 7: Stale Git Metadata After Connect [Low -- By Design] + +Git metadata is computed once per `GET /api/rojo` call (plugin connect/reconnect). If git state changes while the user reviews the confirmation dialog, the defaults may be stale. This is acceptable because: + +1. The defaults are advisory -- the user can always override +2. Recomputing during the dialog would require a round-trip and could confuse the user if defaults change mid-review +3. Reconnecting (which triggers a fresh `GET /api/rojo`) gets fresh metadata + +### Finding 8: Selection State Management in Confirming.lua is Correct [No Issue] + +The `didUpdate` logic in `[Confirming.lua:107-132](plugin/src/App/StatusPages/Confirming.lua)` correctly handles streaming patch updates during confirmation: + +1. Preserves existing selections for unchanged items +2. Unselects items in `changedIds` (forces re-review) +3. Merges new default selections for newly added items +4. Only overwrites if the item doesn't already have a selection + +This prevents the confusing UX of defaults overriding conscious user choices during streaming updates. + +### Finding 9: `autoSelectedIds` Staging Logic is Sound [No Issue] + +The staging rules in `[ServeSession.lua:719-886](plugin/src/ServeSession.lua)`: + +- **Push-accepted**: Always staged (lines 732, 792, 837) -- the user accepted the filesystem version +- **Pull-accepted + auto-selected**: Staged (lines 759-761, 806-808) -- high confidence automatic selection +- **Pull-accepted + manually selected**: NOT staged -- deliberate design choice: manually chosen pulls may need review before staging +- **Ignored**: Never staged + +This is a thoughtful design that avoids staging files the user chose manually (they might want to `git diff` first). + +--- + +## Test Coverage Assessment + +**Excellent coverage across all three layers:** + + +| Layer | File | Tests | Coverage | +| ------------------ | ---------------------------------- | --------- | -------------------------------------------------------- | +| Unit (Rust) | `src/git.rs` | 20+ tests | SHA1 parity, changed files, staging, edge cases | +| Integration (Rust) | `tests/tests/git_sync_defaults.rs` | 20 tests | Full pipeline: metadata, staging, renames, batch 20-file | +| Unit (Luau) | `plugin/src/PatchTree.spec.lua` | ~15 tests | Default selections, hash matching, edge cases | + + +**Notable test scenarios covered:** + +- No git repo (feature disabled) +- No commits yet (empty HEAD) +- Untracked, staged, unstaged, deleted files +- Init-style scripts (`init.luau` in directories) +- All script types (Module, Server, Client) +- Non-script changes (model files) +- Batch staging of 20 files (index.lock race prevention) +- Source write + staging interaction +- Rename + staging (new path staged, not old) +- Reconnect gets fresh metadata +- Draft source fallback +- Mixed push/pull batch operations + +--- + +## Summary + +The git-based sync defaults feature is **well-architected and correctly implemented**. Key design strengths: + +1. **Three-phase tree lock** in `compute_git_metadata` minimizes lock contention +2. **Consolidated `git add`** prevents index.lock races +3. **Safe degradation** at every level (no git = no defaults, hash mismatch = user decides) +4. **Draft source fallback** handles Script Drafts edge case +5. **Thorough test coverage** across all three layers +6. **Clear separation of concerns**: server computes data, plugin makes UI decisions, staging is consolidated + +The only actionable recommendation is the subprocess spawning optimization (Finding 1), which is a performance concern, not a correctness issue. It should be deferred unless users report slow plugin connects on repos with many changed files. \ No newline at end of file diff --git a/.cursor/plans/live_syncback_audit_fixes_561d17c7.plan.md b/.cursor/plans/live_syncback_audit_fixes_561d17c7.plan.md new file mode 100644 index 00000000..509cf17a --- /dev/null +++ b/.cursor/plans/live_syncback_audit_fixes_561d17c7.plan.md @@ -0,0 +1,373 @@ +--- +name: Live Syncback Audit Fixes +overview: "Fix 6 issues found during the live syncback audit: resource leaks from mem::forget, crash-on-error instead of recovery, concurrent syncback races, silent malformed input corruption, misleading doc comments, and missing test coverage." +todos: + - id: fix1-mem-forget + content: "Fix 1: Replace `mem::forget` with `drop()` in serve.rs (lines 61, 135)" + status: completed + - id: fix2-error-recovery + content: "Fix 2: Catch syncback errors and log+recover instead of crashing (serve.rs lines 84-88)" + status: completed + - id: fix3-concurrent-guard + content: "Fix 3: Reject concurrent syncback requests (SyncbackSignal::fire returns bool, api.rs returns 409)" + status: completed + - id: fix4-validate-counts + content: "Fix 4: Validate chunk counts match rbxm blob size (build_dom_from_chunks)" + status: completed + - id: fix5-doc-comment + content: "Fix 5: Correct misleading ServiceChunk doc comment (interface.rs lines 30-42)" + status: completed + - id: fix6-tests + content: "Fix 6: Add 5 new integration tests (attributes/tags, multiple refs, concurrent, malformed, properties-only)" + status: completed + - id: run-ci + content: Run CI after all fixes to verify everything passes + status: completed +isProject: false +--- + +# Live Syncback Audit Fixes + +> This plan was generated by `/audit` from a Plan-mode session. +> Refer to `.cursor/rules/atlas.mdc` for full project standards. + +## Standards + +These standards govern every fix in this plan. An implementer MUST read `.cursor/rules/atlas.mdc` before starting. + +### Round-Trip Identity (from `atlas.mdc` Quality Standard) + +Syncback (or two-way sync) writes a directory tree. Building an rbxl from that directory tree and forward-syncing it back must produce a **bit-identical instance tree** -- same names, same classes, same properties, same hierarchy, same ref targets. Any deviation is a bug. + +### CLI Syncback Parity (from `atlas.mdc` Two-Way Sync Strategy) + +Plugin-based sync must produce **exactly the same filesystem output** that `atlas syncback` would give for the same input. Byte-for-byte identical files, identical directory structures, identical naming. Any divergence is a bug until proven otherwise. + +### Code Quality (from `atlas.mdc` Code Quality Standard) + +DRY code that is easy to reason about and hard to get wrong. The same slugify/dedup/meta-update pattern should not be copy-pasted across files. Small refactors (2-3 call sites) are included as fixes. Major rewrites (10+ call sites) are deferred. + +## Context + +Audited the "Live syncback" feature on the `ambiguous-paths` branch. This feature adds a one-click full sync from Studio to filesystem via `POST /api/syncback`. The plugin serializes all services (properties via msgpack, children via a single rbxm blob), sends them to the server, which reconstructs a `WeakDom`, runs `syncback_loop(incremental=false)`, and writes the result. The server shuts down and restarts after each syncback. + +**Key files audited:** + +- `src/cli/serve.rs` -- serve loop, `run_live_syncback()`, `build_dom_from_chunks()` +- `src/web/api.rs` -- `handle_api_syncback()` endpoint +- `src/web/mod.rs` -- `SyncbackSignal`, `LiveServer` +- `src/web/interface.rs` -- `ServiceChunk`, `SyncbackRequest`, `SyncbackPayload` +- `plugin/src/App/init.lua` -- `performSyncback()`, `SYNCBACK_SERVICES` +- `plugin/src/ChangeBatcher/encodeService.lua` -- service property/ref encoding + +**No critical round-trip identity violations found.** The CLI parity tests (`assert_live_matches_cli`) provide strong guarantees. The round-trip test (`roundtrip_build_syncback_rebuild`) covers the full lifecycle. All 6 fixes below are correctness hardening, resilience, and test coverage. + +## Fixes + +Each fix below was approved by the user during the audit quiz. Implement them in order. Each fix includes its test requirements -- a fix is not complete until its tests pass. + +--- + +### Fix 1: Replace `mem::forget` with proper `drop` + +- **Status:** Approved +- **Finding:** Step 14 (Challenged Limitations) -- resource leak accumulating per syncback invocation +- **Files:** `[src/cli/serve.rs](src/cli/serve.rs)` (lines 61, 135) +- **Problem:** `forget(session_old)` at line 135 leaks the entire oneshot `ServeSession` (tree, VFS, project) after every live syncback. `forget(session)` at line 61 leaks the initial config-reading session at startup. The oneshot session has `change_processor: None`, `suppressed_paths: None`, `ref_path_index: None`, `git_repo_root: None` -- all `Option` types that drop cleanly as `None`. The `Vfs` destructor will stop its watcher thread. There is no reason to use `forget()`. +- **Solution:** + - Line 135: Replace `forget(session_old);` with `drop(session_old);` + - Line 61: Replace `forget(session);` with `drop(session);` + - Remove `use std::mem::forget;` from the imports (line 4) if no other uses remain +- **Risk:** Very low. The oneshot session's fields are all `Option::None` or standard types. The VFS watcher thread is dormant (no watches registered). Existing live syncback tests exercise this path. +- **Verify round-trip:** Run `parity_scripts` and `roundtrip_build_syncback_rebuild` -- both must pass. +- **Verify syncback parity:** Run `assert_live_matches_cli` tests -- filesystem output unchanged. +- **Tests required:** + - No new tests needed. Existing tests `syncback_twice_different_data` and `server_comes_back_functional` already exercise repeated syncback and verify the server recovers correctly. + +--- + +### Fix 2: Recover from syncback errors instead of crashing + +- **Status:** Approved +- **Finding:** Step 14 (Challenged Limitations) -- server crashes permanently on syncback failure +- **Files:** `[src/cli/serve.rs](src/cli/serve.rs)` (lines 84-88) +- **Problem:** `run_live_syncback(&project_path, payload)?` propagates errors via `?`, crashing the serve process. The plugin shows "Syncback data sent" but the server never comes back. +- **Solution:** Catch the error and log it, then continue to restart the server: + +```rust +ServerExitReason::SyncbackRequested(payload) => { + log::info!("Live syncback requested, running..."); + match run_live_syncback(&project_path, payload) { + Ok(()) => log::info!("Syncback complete, restarting serve..."), + Err(err) => log::error!("Live syncback failed: {err:#}. Restarting serve..."), + } + continue; +} +``` + +- **Risk:** Very low. The server restarts fresh regardless (new VFS, new session). An error just means the filesystem wasn't updated -- the old state persists. +- **Verify round-trip:** Existing tests still pass. The error path is tested by Fix 6 test 4. +- **Verify syncback parity:** N/A -- this fix doesn't change the syncback logic, only error handling. +- **Tests required:** + - **Rust integration test** (Fix 6, test 4): Send malformed chunk counts that trigger an error in `build_dom_from_chunks`. Verify the server recovers and comes back online. + +--- + +### Fix 3: Reject concurrent syncback requests + +- **Status:** Approved +- **Finding:** Step 14 (Challenged Limitations) -- second syncback silently overwrites first payload +- **Files:** `[src/web/mod.rs](src/web/mod.rs)` (`SyncbackSignal::fire`, lines 43-46), `[src/web/api.rs](src/web/api.rs)` (`handle_api_syncback`, line 259) +- **Problem:** `fire()` unconditionally overwrites any existing payload. If two requests arrive before the accept loop processes the first, the first payload is silently lost. +- **Solution:** + 1. Change `SyncbackSignal::fire()` to return `bool`: + +```rust +pub fn fire(&self, payload: SyncbackPayload) -> bool { + let mut guard = self.payload.lock().unwrap_or_else(|e| e.into_inner()); + if guard.is_some() { + return false; + } + *guard = Some(payload); + self.notify.notify_one(); + true +} +``` + +1. In `handle_api_syncback`, check the return value: + +```rust +if !syncback_signal.fire(payload) { + return msgpack( + ErrorResponse::bad_request("A syncback is already in progress"), + StatusCode::CONFLICT, + ); +} +``` + +- **Risk:** Very low. Happy path unchanged. Only adds rejection for the concurrent case. +- **Verify round-trip:** Existing tests pass (single syncback). New test verifies rejection. +- **Verify syncback parity:** N/A -- this fix doesn't change syncback output. +- **Tests required:** + - **Rust integration test** (Fix 6, test 3): Fire two syncback requests rapidly, verify the second returns 409 Conflict and the server still processes the first successfully. + +--- + +### Fix 4: Validate chunk counts match rbxm blob size + +- **Status:** Approved +- **Finding:** Step 11g (Audit Step 4, server-side processing) -- silent cursor clamping corrupts subsequent services +- **Files:** `[src/cli/serve.rs](src/cli/serve.rs)` (`build_dom_from_chunks`, lines 184-210) +- **Problem:** Line 187 `let end = (cursor + total).min(cloned_children.len());` silently clamps when counts exceed blob size. This corrupts cursor position for all subsequent services. +- **Solution:** + 1. Inside the service loop, replace the `.min()` clamp with an early error: + +```rust +let end = cursor + total; +anyhow::ensure!( + end <= cloned_children.len(), + "Service '{}' claims {} children + {} carriers but only {} instances remain in blob", + chunk.class_name, child_count, ref_count, + cloned_children.len().saturating_sub(cursor) +); +``` + +1. After the service loop, validate all instances were consumed: + +```rust +anyhow::ensure!( + cursor == cloned_children.len(), + "Service chunks account for {} instances but rbxm blob contains {}", + cursor, cloned_children.len() +); +``` + +- **Risk:** Very low. Only triggers on malformed input. Normal requests are unaffected. With Fix 2, the error is caught and the server recovers. +- **Verify round-trip:** Existing tests pass (all have correct counts). New test verifies rejection. +- **Verify syncback parity:** N/A -- this fix doesn't change output for valid input. +- **Tests required:** + - **Rust integration test** (Fix 6, test 4): Send a request where service chunks claim more children than the blob contains. Verify the server logs an error and recovers. + +--- + +### Fix 5: Correct misleading `ServiceChunk` doc comment + +- **Status:** Approved +- **Finding:** Step 4 (server-side processing audit) -- doc describes behavior that doesn't exist +- **Files:** `[src/web/interface.rs](src/web/interface.rs)` (lines 30-42) +- **Problem:** Doc says "Indices <= child_count point directly to a child instance" but the implementation only indexes into the carrier subrange. This misleads future maintainers. +- **Solution:** Replace the doc comment block (lines 30-42) with: + +```rust +/// Metadata for a single service in the live syncback request. +/// +/// Children are NOT stored here -- they live in the top-level `data` blob +/// (a single rbxm containing all children from all services). +/// +/// Each service's range in the blob is `child_count + ref_target_count` +/// consecutive entries. The first `child_count` are real children (parented +/// to the service). The next `ref_target_count` are ObjectValue carriers +/// whose `Value` property holds a Ref to the actual target. +/// +/// `refs` maps property names to 1-based indices into the carrier subrange +/// (NOT the full service range). Index 1 = first carrier, index 2 = second, +/// etc. Index 0 or out-of-range values are ignored (treated as nil). +``` + +- **Risk:** Zero. Documentation-only change. +- **Verify round-trip:** N/A +- **Verify syncback parity:** N/A +- **Tests required:** None. + +--- + +### Fix 6: Add missing test coverage + +- **Status:** Approved +- **Finding:** Step 13 (audit report) -- several edge cases lack test coverage +- **Files:** `[tests/tests/live_syncback.rs](tests/tests/live_syncback.rs)` +- **Problem:** Missing tests for service attributes/tags, multiple Ref properties, concurrent requests, malformed input, and properties-only services. +- **Solution:** Add the following tests to `tests/tests/live_syncback.rs`: + +**Test 1: `parity_service_attributes_and_tags`** + +- Create a Lighting service with custom Attributes (e.g., `{"CustomFloat": Float32(1.5)}`) and Tags (e.g., `["TestTag"]`) +- Use `make_service_chunk_full` with Attributes and Tags encoded as `Variant::Attributes` and `Variant::Tags` in the properties map +- Run `assert_live_matches_cli` + +**Test 2: `parity_multiple_service_refs`** + +- Create a service (Workspace) with two Ref properties: one pointing to a Part child, one pointing to a Light child +- Use `make_service_chunk_full` with two carrier ObjectValues and two `refs` entries +- Run `assert_live_matches_cli` + +**Test 3: `rejects_concurrent_syncback`** (for Fix 3) + +- Use `run_serve_test` to start a server +- Send a valid syncback request (don't wait for restart) +- Immediately send a second syncback request +- Verify the second returns 409 Conflict +- Wait for the server to come back online (first syncback completes) + +**Test 4: `recovers_from_malformed_chunks`** (for Fixes 2+4) + +- Use `run_serve_test` to start a server +- Record the initial session info +- Send a syncback request with mismatched counts (e.g., `child_count: 999` but empty data blob) +- Verify the server comes back online (new session ID) +- Verify the filesystem is unchanged (the malformed syncback was rejected before writing) + +**Test 5: `parity_properties_only_no_children`** + +- Create 3 services (Lighting, SoundService, TextChatService) all with properties but zero children +- Verify empty rbxm blob case works with properties +- Run `assert_live_matches_cli` +- **Risk:** Zero. Test-only additions. +- **Verify round-trip:** The parity tests verify round-trip by design. +- **Verify syncback parity:** The parity tests verify CLI parity by design. +- **Tests required:** Self-contained -- the fix IS the tests. + +--- + +## Skipped Fixes + +None -- all identified fixes were approved. + +## Accepted Limitations + +### rbxm Child Ordering Assumption + +- **Limitation:** Live syncback relies on `SerializationService:SerializeInstancesAsync()` preserving the array order of top-level instances in the resulting rbxm blob. If Roblox ever changes the serializer to reorder by class, the cursor-based partitioning in `build_dom_from_chunks` would break. +- **Scenario:** A future Roblox engine update changes serialization order, causing children to be assigned to wrong services. +- **User decision:** Accepted for this release. This is the standard approach used by all Rojo-family tools. The rbxm format preserves parent-child relationships and child ordering through the PRNT chunk. Roblox has never changed this behavior. +- **Mitigation:** Fix 4 adds count validation that would catch the symptom (cursor doesn't consume all instances) even if not the root cause. + +### Asymmetric ClockTime Exclusion + +- **Limitation:** `encodeService.lua` excludes `Lighting.ClockTime` (alias for `TimeOfDay`) but the server-side `build_dom_from_chunks` doesn't have this exclusion. +- **Scenario:** A crafted request with both `ClockTime` and `TimeOfDay` could produce conflicting properties. This never happens in normal usage since the plugin prevents it. +- **User decision:** Accepted. The plugin is the only intended client, and it already filters correctly. +- **Mitigation:** None needed beyond the existing plugin-side filter. + +### No Session ID on `/api/syncback` + +- **Limitation:** Unlike `/api/write`, the syncback endpoint has no session ID validation. Any client that knows the host/port can trigger a syncback. +- **Scenario:** On a shared network with `serveAddress: "0.0.0.0"`, someone could overwrite project files. +- **User decision:** Accepted by design. The syncback is intended to work from the NotConnected page (no active session). The `servePlaceIds`/`blockedPlaceIds` whitelist provides protection for team environments. +- **Mitigation:** Existing place ID validation. The serve address defaults to `127.0.0.1` (localhost only). + +## Deferred Refactors + +### Duplicate Service Lists + +The service list exists in 3 places that must stay synchronized: + +1. `SYNCBACK_SERVICES` in `plugin/src/App/init.lua` (lines 7-22) +2. `VISIBLE_SERVICES` in `src/syncback/mod.rs` (lines 73-88) +3. Implicitly in test helper code (`serve_util.rs`) + +Currently they match exactly (14 services). If these diverge, Ref properties targeting instances in a missing service would be silently lost. A cross-language shared constant is impossible (Lua vs Rust), but a CI check or a test that compares the lists (e.g., parsing the Lua source) could prevent drift. Deferred because it's a build/CI infrastructure change, not a code change. + +### `build_dom_from_chunks` Duplication with Test Helper + +`build_dom_from_chunks` in `serve.rs` and `make_rbxl_from_chunks` in `serve_util.rs` implement nearly identical DOM reconstruction logic. The test helper was written to mirror the production code for CLI parity tests. If the production code changes, the test helper must be updated in lockstep. Consider extracting the shared logic into a `librojo` function that both can call. Deferred because it requires making `build_dom_from_chunks` public and adjusting its API. + +## Test Plan + +Tests are not optional. Every approved fix must have test coverage. A fix is not complete until its tests pass. + +### Rust Integration Tests (`tests/tests/live_syncback.rs`) + +- `parity_service_attributes_and_tags` -- service with Attributes + Tags survives round-trip (Fix 6, test 1) +- `parity_multiple_service_refs` -- service with 2+ Ref properties, all carriers resolved correctly (Fix 6, test 2) +- `rejects_concurrent_syncback` -- second syncback returns 409 Conflict (Fix 6, test 3) +- `recovers_from_malformed_chunks` -- mismatched counts cause error, server recovers (Fix 6, test 4) +- `parity_properties_only_no_children` -- 3 services with properties, zero children, empty blob (Fix 6, test 5) + +### Existing Tests (Must Continue Passing) + +All existing tests in `tests/tests/live_syncback.rs` must pass after all fixes: + +- `parity_scripts`, `parity_models_with_properties`, `parity_special_names`, `parity_duplicate_names` +- `parity_deep_hierarchy`, `parity_mixed_file_types`, `parity_multi_service` +- `parity_empty_services_mixed`, `parity_lighting_properties`, `parity_soundservice_properties` +- `parity_starterplayer_properties`, `parity_textchatservice_properties` +- `parity_childless_service_with_properties`, `parity_service_properties_with_children` +- `parity_multiple_services_with_properties`, `parity_cross_service_refs` +- `parity_camera_not_synced`, `rejects_bad_protocol`, `rejects_bad_version` +- `rejects_blocked_place`, `rejects_unlisted_place`, `allows_whitelisted_place` +- `allows_any_place_when_unrestricted`, `server_comes_back_functional` +- `syncback_twice_different_data`, `syncback_replaces_all_old_files` +- `roundtrip_build_syncback_rebuild` + +## Test Rules + +The implementer MUST follow these rules when writing tests for this plan: + +### For bugs found and fixed + +Every bug fix MUST have a test that: + +1. Would have **FAILED** before the fix +2. **PASSES** after the fix +3. Prevents the bug from regressing + +### For missing coverage identified + +For each missing test coverage gap: + +1. Write the test +2. Verify it passes against the current implementation +3. If it fails, investigate -- it may have found another bug + +### Scope: test every affected layer + +For each finding, determine which test layers are affected and write tests in ALL of them: + +- Fixes 1-5 affect the Rust server -- integration tests in `tests/tests/live_syncback.rs` +- No Lua spec tests needed (plugin encoding is unchanged) +- No snapshot tests needed (syncback output format is unchanged) + +## Final Step: Run CI + +After ALL fixes and tests are implemented, run the `/ci` command (see `.cursor/commands/ci.md`) to execute the full CI pipeline. Every fix must pass CI before the plan is considered complete. Do not skip this step. \ No newline at end of file diff --git a/.cursor/plans/live_syncback_float_precision_1be79350.plan.md b/.cursor/plans/live_syncback_float_precision_1be79350.plan.md new file mode 100644 index 00000000..1aa5f16d --- /dev/null +++ b/.cursor/plans/live_syncback_float_precision_1be79350.plan.md @@ -0,0 +1,195 @@ +--- +name: Live syncback fixes +overview: "Fix five issues: lost Workspace attributes, spurious ClockTime, Rojo_Ref_CurrentCamera leak, and float precision noise via f32 truncation." +todos: + - id: fix-attributes-encoding + content: Add explicit Attributes/Tags encoding to encodeService.lua (mirroring encodeInstance.lua) + status: completed + - id: fix-clocktime + content: Filter non-serializable properties server-side in build_dom_from_chunks using the Rust reflection DB + status: completed + - id: fix-currentcamera-ref + content: "Stop Rojo_Ref_CurrentCamera from leaking: filter deferred_referents after camera destruction in syncback_loop" + status: completed + - id: fix-float-rounding + content: Truncate all float values to f32 in UnresolvedValue::from_variant for rbxl parity + status: completed + - id: fix-index-refs + content: Change service-level refs to index-based with ObjectValue carriers (childCount + refTargetCount, 1-based index, 0=nil) + status: completed +isProject: false +--- + +# Live Syncback Fixes + +## Issues 1-4: DONE + +## Issue 5: Index-Based Service Refs via ObjectValue Carriers + +**Problem:** Service Ref targets aren't always direct children. Can't include arbitrary instances directly in the rbxm without duplicating or breaking serialization. + +**Solution:** Create temporary ObjectValue instances as Ref carriers. The ObjectValue's `Value` property IS a Ref that gets preserved in the rbxm because both the ObjectValue and the target are in the same blob. + +**Layout:** + +``` +allChildren = [ + ...svc1_children, ...svc1_objectValues, + ...svc2_children, ...svc2_objectValues, + ... +] +``` + +### Plugin -- `[plugin/src/ChangeBatcher/encodeService.lua](plugin/src/ChangeBatcher/encodeService.lua)` + +Return metadata + list of ObjectValue carrier instances to append: + +```lua +local children = service:GetChildren() +local refTargets = {} -- ObjectValue carriers + +for propertyName ... do + if descriptor.dataType == "Ref" then + local readOk, target = descriptor:read(service) + if readOk and target then + -- Check if target is a direct child + for i, child in children do + if child == target then + refs[propertyName] = i -- 1-based into children range + goto found + end + end + -- Not a child: create ObjectValue carrier + local carrier = Instance.new("ObjectValue") + carrier.Name = propertyName + carrier.Value = target + table.insert(refTargets, carrier) + refs[propertyName] = #children + #refTargets + ::found:: + end + continue + end +end + +chunk.childCount = #children +chunk.refTargetCount = #refTargets +return chunk, refTargets +``` + +### Plugin -- `[plugin/src/App/init.lua](plugin/src/App/init.lua)` `performSyncback` + +Append children + ObjectValue carriers, clean up after serialization: + +```lua +local allRefTargets = {} +for _, className in SYNCBACK_SERVICES do + local ok, service = pcall(game.FindService, game, className) + if ok and service then + local chunk, refTargets = encodeService(service) + for _, child in service:GetChildren() do + table.insert(allChildren, child) + end + for _, carrier in refTargets do + table.insert(allChildren, carrier) + table.insert(allRefTargets, carrier) + end + table.insert(services, chunk) + end +end + +local data = buffer.create(0) +if #allChildren > 0 then + data = SerializationService:SerializeInstancesAsync(allChildren) +end + +-- Clean up temporary ObjectValue carriers +for _, carrier in allRefTargets do + carrier:Destroy() +end +``` + +### Server -- `[src/web/interface.rs](src/web/interface.rs)` + +```rust +pub struct ServiceChunk { + pub class_name: String, + #[serde(default)] + pub child_count: u32, + #[serde(default)] + pub ref_target_count: u32, + #[serde(default)] + pub properties: HashMap, + #[serde(default)] + pub refs: HashMap, // prop_name -> 1-based index, 0=nil +} +``` + +Remove `ServiceRef` struct. + +### Server -- `[src/cli/serve.rs](src/cli/serve.rs)` `build_dom_from_chunks` + +```rust +let child_count = chunk.child_count as usize; +let ref_count = chunk.ref_target_count as usize; +let total = child_count + ref_count; +let end = (cursor + total).min(cloned_children.len()); +let service_range = &cloned_children[cursor..end]; + +// Parent only the real children +for &child_ref in &service_range[..child_count.min(service_range.len())] { + dom.transfer_within(child_ref, service_ref); +} + +// Resolve refs using the full range +let mut carriers_to_destroy = Vec::new(); +if !chunk.refs.is_empty() { + for (prop_name, &idx) in &chunk.refs { + if idx == 0 || (idx as usize - 1) >= service_range.len() { + continue; + } + let carrier_ref = service_range[idx as usize - 1]; + // If in the ref target range, it's an ObjectValue carrier + if (idx as usize) > child_count { + let carrier = dom.get_by_ref(carrier_ref).unwrap(); + if let Some(Variant::Ref(actual_target)) = carrier.properties.get("Value") { + let service = dom.get_by_ref_mut(service_ref).unwrap(); + service.properties.insert( + prop_name.as_str().into(), + Variant::Ref(*actual_target), + ); + } + carriers_to_destroy.push(carrier_ref); + } else { + // Direct child ref + let service = dom.get_by_ref_mut(service_ref).unwrap(); + service.properties.insert( + prop_name.as_str().into(), + Variant::Ref(carrier_ref), + ); + } + } +} + +// Destroy ObjectValue carriers +for carrier_ref in carriers_to_destroy { + dom.destroy(carrier_ref); +} + +cursor = end; +``` + +### Test helpers -- `[tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs)` + +- `ServiceEntry` gets `ref_targets: Vec` for ObjectValue carriers +- `make_service_chunk_full` refs param: `Vec<(&str, u32)>` (prop_name, 1-based index) +- `build_syncback_request` appends ref_targets after children per service +- `make_rbxl_from_chunks` resolves refs: direct children by index, carriers by reading `Value` property + +### Tests + +- Update `parity_camera_not_synced` to use index-based ref with ObjectValue carrier +- Existing tests that don't use refs continue to work (empty refs map, refTargetCount=0) + +## Deferred: Matching Parity + +Revisit after these fixes land. \ No newline at end of file diff --git a/.cursor/plans/luau-optimize_cursor_command_ce1448a8.plan.md b/.cursor/plans/luau-optimize_cursor_command_ce1448a8.plan.md new file mode 100644 index 00000000..10bf0311 --- /dev/null +++ b/.cursor/plans/luau-optimize_cursor_command_ce1448a8.plan.md @@ -0,0 +1,743 @@ +--- +name: luau-optimize cursor command +overview: Create a `/luau-optimize` Cursor command at `.cursor/commands/luau-optimize.md` that uses the `luau-compile`, `luau-analyze`, and `luau-ast` CLI tools to analyze and optimize Luau files for maximum performance, type safety, and native codegen quality. +todos: + - id: create-command + content: Create `.cursor/commands/luau-optimize.md` with the full multi-phase optimization workflow + status: completed +isProject: false +--- + +# /luau-optimize Cursor Command + +## Deliverable + +Single file: `[.cursor/commands/luau-optimize.md](.cursor/commands/luau-optimize.md)` + +## Context + +**This command is always for Roblox Luau.** All scripts being optimized run in the Roblox engine (Studio / live servers / clients). This means: + +- Roblox APIs (`game`, `workspace`, `Players`, `RunService`, etc.) are always available +- The vector type is `Vector3` (Roblox userdata), not the `vector` primitive +- Native codegen (`--!native`) primarily benefits server-side Scripts; client LocalScripts benefit less due to device diversity +- The `luau` standalone runtime cannot execute most Roblox scripts (they depend on engine APIs); it is only useful for pure-logic library modules with no Roblox dependencies +- `luau-compile` should use `--vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3` for accurate Roblox vector optimization analysis +- Roblox performance patterns (event-driven design, frame budgets, RunService events, instance streaming, replication) are always relevant +- Studio testing uses `--!optimize 1` by default; live experiences use `--!optimize 2` + +## Design + +The command is a structured prompt file that instructs the AI agent through a multi-phase optimization workflow. It supports both single-file and multi-file/directory modes. + +### Phase Structure + +**Phase 0 -- Scope Resolution and Configuration** + +Resolve the target: + +- If user specifies a file: optimize that file +- If user specifies a directory: discover all `.luau`/`.lua` files, prioritize by traffic/complexity +- If no argument: optimize the currently open file + +Infer script context from filename and path (checked in this order): + +1. **Filename suffix** (highest priority, overrides path): + - `*.server.luau` / `*.server.lua` -> always server Script (`--!native` is high-value) + - `*.client.luau` / `*.client.lua` -> always client Script/LocalScript (`--!native` less beneficial due to device diversity) + - `*.legacy.luau` / `*.legacy.lua` -> **ambiguous** (could be cloned/moved at runtime). Ask the user where it runs. + - `*.luau` / `*.lua` (no suffix) -> ModuleScript, context depends on who requires it. Fall through to path check. +2. **Path-based inference** (for ModuleScripts and ambiguous cases): + - Path contains `ServerScriptService`/`ServerStorage` -> server-side module + - Path contains `StarterPlayer`/`StarterGui`/`StarterCharacterScripts`/`ReplicatedFirst` -> client-side module + - Path contains `ReplicatedStorage`/`shared` -> shared module (runs on both; conservative `--!native` recommendation) + - If path is ambiguous or outside a Roblox project structure -> ask the user + +**Quiz the user using AskQuestion tool (combined, structured):** + +``` +Title: "/luau-optimize Configuration" + +Question 1: "Compilation optimization intensity" + - minimal -- Headers (--!strict/--!native/--!optimize 2), type annotations on all function signatures, simple deprecated pattern replacements + - moderate -- + function restructuring for inlining, import hoisting, compound operators, fastcall enablement, allocation reduction + - insane -- + every micro-optimization from the pattern catalog, full bytecode analysis, register pressure optimization, closure caching analysis + - (other: freeform) + +Question 2: "Algorithm optimization" + - none -- Don't touch logic or data flow + - low hanging fruit -- Obvious O(n^2)->dict, redundant iterations, missing caches + - moderate -- + data structure changes, caching strategies, event-driven refactors, loop fusion + - insane -- + full algorithmic redesign where beneficial, dynamic programming, architectural restructuring + - (other: freeform) + +Question 3: "Benchmarking" + - yes -- Benchmark all non-trivial changes with luau CLI (when function is standalone) + - no -- Trust bytecode/codegen metrics only + - +EV only -- Benchmark only when algorithmic changes or ambiguous bytecode results warrant it +``` + +**Derived behavior from quiz answers:** + + +| Setting | minimal | moderate | insane | +| ------------------------ | --------------------------- | ------------------------------------- | ---------------------------------------- | +| Headers | Always | Always | Always | +| Type annotations | Function signatures | Function signatures + hot path locals | Function signatures + all locals | +| Pattern replacements | Deprecated only | Full Priority 2 catalog | Full Priority 2 + Priority 5 | +| Function restructuring | No | Yes (Priority 3) | Yes + aggressive splitting | +| Algorithmic changes | Per algo quiz | Per algo quiz | Per algo quiz | +| Bytecode verification | After all changes | After each priority | After each individual change | +| `@native` vs `--!native` | `--!native` unless big file | Selective `@native` on hot functions | Full analysis of which functions benefit | + + +**Restructuring scope** scales with compilation intensity: + +- minimal: No restructuring. Only additive changes (headers, annotations, pattern swaps). +- moderate: Break up monoliths, reorder locals for inlining, hoist imports. Preserve existing exports/API. +- insane: Aggressive restructuring permitted. May change internal function boundaries, introduce helper modules, rewrite data flow. + +**Phase 1 -- Deep Code Read and Algorithmic Analysis (FIRST)** + +Read the entire source file thoroughly before touching any tools. This is the highest-leverage phase -- algorithmic improvements produce order-of-magnitude speedups that dwarf any bytecode-level optimization. This is also the largest code change, so it must come first (everything else builds on top of it). + +Understand the code's purpose, data flow, and architecture. + +**Step 1 -- Structural decomposition (read and mentally refactor)** + +Monolithic functions hide their actual algorithmic structure behind walls of interleaved concerns. Before analyzing complexity, first understand what the code is *actually doing* by decomposing it mentally (and then physically if restructuring is allowed): + +- **Identify responsibilities**: What distinct jobs does each large function perform? A 200-line function usually contains 3-5 logical steps that should be separate functions. +- **Trace data flow**: What goes in, what comes out, what is mutated along the way? Map the pipeline. +- **Separate concerns**: Validation, transformation, I/O, state management, error handling -- these are often tangled together. Separating them reveals the core algorithm. +- **Name the steps**: If you can't name what a code block does in 3-5 words, it's doing too much. +- **Look for hidden abstractions**: Repeated patterns of "iterate, filter, transform, collect" that could be a single well-named helper. + +When the quiz allows restructuring (moderate/insane compilation intensity): + +- Physically decompose monoliths into focused local functions. Each function should do one thing. +- This decomposition often *reveals* optimization opportunities that were invisible in the monolith: you can now see that step 2 recomputes what step 1 already knew, or that the inner loop could be replaced with a lookup table built in the outer loop. +- Smaller functions are also easier for the compiler to inline at `-O2`, so this restructuring pays double -- clearer code AND better bytecode. +- Preserve the original function as a thin orchestrator that calls the decomposed pieces. + +Even when the quiz says "minimal" (no restructuring), still perform this mental decomposition to inform the analysis below. + +**Step 2 -- Algorithmic analysis** + +With the structure understood (or cleaned up), analyze: + +- **Complexity analysis**: Identify the time complexity of every significant function. Look for: + - O(n^2) or worse: nested loops over the same data, repeated linear searches, quadratic string building + - O(n) where O(1) is possible: linear search replaceable with dictionary/set lookup + - Repeated work: same computation done multiple times when it could be cached/memoized + - Unnecessary copies: cloning data that could be referenced or mutated in place + - Hidden quadratics: a function that looks O(n) but calls another O(n) function inside a loop +- **Data structure fitness**: Is the right data structure being used? + - Array used for membership testing -> should be a set (dictionary with `true` values) + - Linear scan for lookup by key -> should be a dictionary + - Repeated sort of mostly-sorted data -> consider maintaining sorted order on insert + - Flat list with frequent removal from middle -> consider swap-and-pop + - Large string built incrementally -> `table.concat` pattern or `buffer` + - Multiple parallel arrays tracking related data -> single array of structs (table of tables) + - Unbounded growth without cleanup -> consider size limits, LRU eviction, or periodic pruning +- **Caching and memoization**: + - Pure functions called with the same args repeatedly -> memoize + - Expensive property reads in loops -> cache in local before loop + - Computed values that don't change within a scope -> hoist out + - Derived state recomputed from scratch on every access -> maintain incrementally +- **Redundant work elimination**: + - Multiple passes over the same data that could be fused into one + - Recomputing derived state that could be maintained incrementally + - Sorting or filtering that could be done once and reused + - Deep cloning where shallow clone or reference suffices + - Building an intermediate result that is immediately discarded after extracting one field +- **Architectural patterns**: + - Per-frame polling where event-driven would work (RunService events are expensive) + - Synchronous work that could be chunked across frames with `task.wait()` + - Undisconnected event connections (memory leak + wasted computation) + - Rebuilding entire state on small change where incremental update would work + - God-object that owns too much state -> split responsibilities +- **Hot path identification**: Which functions are called most frequently? + - Event handlers connected to RunService (PreAnimation, PreRender, PreSimulation, PostSimulation, Heartbeat) + - Functions called inside loops + - Recursive functions + - Functions called from other hot functions + +**Step 3 -- Present findings and quiz the user** + +Before applying any changes, present a summary of all findings organized by impact: + +``` +For each finding, present: +- What: One-line description of the issue +- Where: Function name / line range +- Why: What makes it slow (e.g., "O(n^2) nested scan on every frame") +- Fix: Proposed change in plain language +- Impact: Estimated improvement (order-of-magnitude, constant-factor, or clarity-only) +``` + +Then quiz the user using AskQuestion tool to let them control the flow: + +``` +Title: "Phase 1 Findings -- Select which changes to apply" + +For each finding, one question with options: + - apply -- Apply this change + - skip -- Skip this change + - modify -- I want a different approach (wait for user input) + +If there are many findings (5+), group them by category and present one question per category: + - "Structural decomposition: [list of proposed decompositions]. Apply all / cherry-pick / skip all" + - "Data structure changes: [list]. Apply all / cherry-pick / skip all" + - "Algorithm changes: [list]. Apply all / cherry-pick / skip all" + - "Architectural changes: [list]. Apply all / cherry-pick / skip all" +``` + +If the algorithm quiz from Phase 0 was "none", present findings as informational only (no apply option). The user still sees what was found. + +**Step 4 -- Apply approved changes** + +Apply only the user-approved changes, in this order: + +1. Structural decomposition first (if approved) -- break monoliths into focused functions. This is the foundation; other changes may depend on it. +2. Data structure changes -- swap arrays for dicts, add indexes, etc. +3. Algorithm changes -- replace O(n^2) with O(n), add caching/memoization +4. Architectural changes -- event-driven refactors, incremental updates + +If the user selected "modify" on any finding, wait for their input before proceeding with that specific change. + +Benchmark algorithmic changes if the benchmarking quiz answer warrants it (Phase 4.5 pattern). + +**Phase 2 -- Baseline Capture (read-only)** + +Run all diagnostic tools on the code (post-algorithmic changes if any were made in Phase 1) and capture baseline metrics for the bytecode optimization phases: + +1. `luau-analyze --mode=strict ` -- type errors and lint warnings +2. `luau-analyze --annotate ` -- inferred types (count `any` occurrences for type coverage score) +3. `luau-compile --remarks -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 ` -- inlining success/failure, allocation remarks (with Roblox vector optimization) +4. `luau-compile --text -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 ` -- full bytecode listing (instruction count, opcodes) +5. `luau-compile --codegen --target=x64_ms --record-stats=function -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 ` -- native codegen stats (spills, skipped functions, block counts). Use `--target=x64` on Linux/Mac. + +Parse and summarize: + +- Total bytecode instruction count +- Allocation count (NEWTABLE, NEWCLOSURE from remarks) +- Inlining success/failure count (from remarks) +- Type coverage: count of `any` types in annotate output +- Register spills and skipped functions from codegen stats +- Lint warnings count + +**Phase 3 -- Compilation-Level Code Analysis** + +Using both the source (already read in Phase 1) and the tool output from Phase 2, identify bytecode-level optimization opportunities: + +- **Missing types**: `any` inferences that hurt native codegen (especially `Vector3`, `CFrame`, `buffer` params) +- **Import patterns**: Global chains like `math.max` are resolved at load time (not execution) via GETIMPORT. Broken by `getfenv`/`setfenv`/`loadstring` which mark environment "impure" and disable this. +- **Allocation in loops**: Table/closure creation inside hot loops (NEWTABLE, NEWCLOSURE). High allocation rate = more GC assist work. Avoid temporary tables and userdata in hot loops. +- **Closure caching**: Repeated function expressions can be cached (same closure reused) when: no upvalues, or all upvalues are immutable and declared at module scope. Mutable upvalue captures prevent caching. +- **Upvalue mutability**: ~90% of upvalues are immutable in typical code. Immutable upvalues = no extra allocation, no closing, faster access, better locality. Mutable upvalues need extra objects. +- **Method call patterns**: `obj:Method()` uses specialized fast method call instruction. Avoid `obj.Method(obj)`. For tables: `__index` should point at a table directly (not a function or deep chain). Ideal: metatable whose `__index` points to itself. +- **Inline caching**: VM uses inline caching for table field access. Best when field name is known at compile time, no metatables, and object shapes are uniform. Varying keys/shapes degrade performance. +- **pcall in hot paths**: Prevents native codegen optimization +- **String concatenation patterns**: `..` in loops vs `table.concat` +- **Deprecated/deoptimizing API usage**: `getfenv`/`setfenv` (marks env "impure", disables ALL import optimization and fastcalls), `loadstring` (same deoptimization), `table.getn`/`table.foreach`/`table.foreachi`, `wait()`. Even read-only `getfenv()` deoptimizes. +- **Builtin global writes**: Overwriting builtins (e.g., `math = ...`) disables fastcall optimization. Lint rule `BuiltinGlobalWrite` catches this. +- **Metamethod cost**: `__eq` is always called on `==`/`~=` even for rawequal values; ensure cheap implementations +- **GC pressure**: Incremental GC uses "GC assists" -- allocating code pays for collection work proportionally. Fewer allocations = less GC overhead. + +**Phase 4 -- Compilation Optimization (graduated priority)** + +Apply changes in priority order, re-running `luau-compile --remarks -O2` after structural changes to verify the compiler actually benefits: + +**Priority 1 -- Headers and type safety (always apply)** + +- Add `--!strict` / `--!native` / `--!optimize 2` header (note: `--!optimize 2` is default in live experiences but not in Studio testing) +- Fix type errors reported by `luau-analyze` +- Add missing type annotations where `--annotate` shows `any` (focus on function signatures first, then locals in hot paths) +- Annotate `Vector3`, `CFrame`, `buffer` parameters explicitly -- native codegen generates specialized vector code when types are known; unannotated params are assumed to be tables with extra checks +- Replace `getfenv`/`setfenv` usage -- these disable builtin calls, imports, and optimizations globally for the affected function +- Replace `table.getn` with `#t` or `rawlen(t)`, `table.foreach`/`table.foreachi` with `for..in` loops + +**Priority 2 -- Low-hanging structural wins** + +- Hoist frequently-used library functions: `local floor = math.floor` enables `GETIMPORT` fastcall. Fastcall builtins include: `assert`, `type`, `typeof`, `rawget`/`rawset`/`rawequal`, `getmetatable`/`setmetatable`, `tonumber`/`tostring`, most `math.`* (not `noise`, `random`/`randomseed`), `bit32.`*, and some `string.`*/`table.*`. Partial specializations: `assert` when return value unused and condition truthy; `bit32.extract` when field/width constant; `select(n, ...)` is O(1) via builtin dispatch. +- Replace `pairs(t)` / `ipairs(t)` with generalized `for k, v in t do` iteration. Generalized iteration skips the `pairs()` call, which matters for very short tables. Performance of all three is similar for longer tables. Note: `for i=1,#t` is slightly slower due to extra element access cost. +- Use `//` floor division instead of `math.floor(a / b)` -- dedicated VM opcode, supports `__idiv` metamethod, also `//=` compound form +- Use compound assignment (`+=`, `-=`, `*=`, `//=`, `..=`) -- LHS evaluated once, e.g. `data[index].cost += 1` avoids double-indexing +- Replace string concatenation in loops with `table.concat` or string interpolation (backticks lower to `string.format` with no extra overhead) +- Use `table.create(n)` for known-size arrays -- preallocates capacity, much faster than repeated `table.insert` +- For sequential fill with known index, use indexed writes with `table.create`: `local t = table.create(N); for i=1,N do t[i] = ... end` (fastest pattern) +- Use `table.insert(t, v)` for appending when size unknown -- `#t` is usually O(1) (cached, updated by `table.insert`/`table.remove`; worst case O(log N) branch-free binary search, "50%+ faster" than Lua's O(N)) +- Use `math.lerp(a, b, t)` instead of `a + (b - a) * t` -- exact for t=0/1, monotonic, bounded +- Use `bit32.byteswap(n)` instead of manual shifting/masking for endian conversion (uses CPU bswap instruction) +- Use `bit32.countlz`/`bit32.countrz` instead of manual log2 loops (uses CPU instructions, ~8x faster) +- Use `table.find(t, v)` instead of manual linear search loops +- Use `table.clone(t)` instead of manual `pairs` copy loops +- Use `string.pack`/`string.unpack` instead of manual `string.byte`/`string.char`/`bit32` for binary data +- Use `rawlen(t)` when metamethods are not needed for table length +- Use explicit `./`/`../`/`@` prefixes in `require()` to avoid extra path-resolution work + +**Priority 3 -- Function structure for inlining** + +- Break monolith functions into smaller, focused local functions (compiler inlines automatically at `-O2`; small local functions = best inlining candidates) +- Inlining requirements: function must be local, non-mutated, non-recursive, not an OOP method (`Foo:bar()`), not a metamethod. Exported functions in mutable tables CANNOT be inlined. Disabled in modules using `getfenv`/`setfenv`. +- Inlining + constant folding: inlined functions with constant args can enable further constant folding (e.g., `local function double(x) return x*2 end; local y = double(5)` folds to `y = 10`) +- For high-traffic functions that call imported module methods frequently, create local function wrappers at module top-level to enable inlining +- Move `pcall` out of hot loops where possible (wrap the loop, not the body) -- pcall prevents native codegen optimization +- Use `obj:Method()` not `obj.Method(obj)` -- colon syntax gets specialized fast method call instruction +- Keep `__index` pointing at a table directly (not a function or deep chain) for optimal inline caching +- Reduce upvalue captures in closures created inside loops. Immutable upvalues (never reassigned) are much cheaper -- no extra allocation, no closing. Mutable upvalues need an extra object. +- Prefer object-like table construction with all fields set at once (enables compiler capacity inference): `local v = {}; v.x = 1; v.y = 2; v.z = 3` -- compiler infers hash capacity from subsequent assignments +- Loop unrolling: only for compile-time constant bounds (`for i=1,4 do`); body must be simple. Unrolling enables further constant folding on the loop index. +- Watch for native codegen size limits: 64K instructions per code block, 32K blocks per function, 1M instructions per module. Split large functions if limits are hit. + +**Priority 4 -- Micro-optimizations (hot paths only)** + +- Table shape consistency (same fields, same order across constructors) -- VM uses inline caching (HREF-style) that predicts hash slots; uniform shapes = better cache hit rates +- Use `buffer` for binary data manipulation instead of string ops (fixed-size, offsets start at 0, far more space/time efficient) +- Use `buffer.readu32` + `bit32` operations instead of `buffer.readbits` when schema is known (faster due to fewer bounds checks) +- Strength reduction: multiplication by power-of-2 to `bit32.lshift`, division by power-of-2 to `bit32.rshift` +- Minimize `tostring`/`tonumber` in tight loops +- Use `table.freeze` for readonly config tables (avoids need for proxy tables and `__index` overhead) +- Keep `__eq` metamethods cheap -- they fire on every `==`/`~=` and `table.find` comparison +- Use `math.isfinite`/`math.isnan`/`math.isinf` instead of `x ~= x` patterns for clarity (no perf difference) +- For vector math: use `Vector3.new(x, y, z)` (Roblox userdata, not the `vector` primitive); annotate params as `Vector3` for native specialization +- Use `if expr then A else B` instead of `cond and A or B` ternary (only evaluates one branch; avoids falsy-value bugs) + +**Phase 4.5 -- Benchmark with `luau` CLI (when warranted)** + +When a non-trivial optimization is applied to an isolated, pure-logic function (no Roblox API dependencies), write a small benchmark harness and run it with the `luau` CLI to measure actual wall-clock impact. This is especially valuable for: + +- Algorithmic changes (Priority 4) where bytecode metrics alone can't prove the win +- Micro-optimizations (Priority 5) where the improvement is ambiguous +- Competing approaches where both look reasonable in bytecode but one may be faster at runtime + +**Benchmark harness pattern:** + +```luau +-- bench.luau (temporary file, deleted after benchmarking) +local function original(...) + -- paste original implementation +end + +local function optimized(...) + -- paste optimized implementation +end + +local ITERATIONS = 1_000_000 +local clock = os.clock + +-- Warmup +for _ = 1, 1000 do original(...) end +for _ = 1, 1000 do optimized(...) end + +local t0 = clock() +for _ = 1, ITERATIONS do original(...) end +local t1 = clock() +for _ = 1, ITERATIONS do optimized(...) end +local t2 = clock() + +print(`Original: {t1 - t0:.4f}s`) +print(`Optimized: {t2 - t1:.4f}s`) +print(`Speedup: {(t1 - t0) / (t2 - t1):.2f}x`) +``` + +**Run with both interpreter and native codegen to compare:** + +``` +luau -O2 bench.luau # Interpreter at -O2 +luau -O2 --codegen bench.luau # Native codegen at -O2 +``` + +**When to benchmark:** + +- The function is self-contained (no `game`, `workspace`, Roblox services) +- The optimization changes algorithm or data structure (not just headers/types) +- The bytecode diff is ambiguous (instruction count similar, or more instructions but fewer allocations) +- Two candidate approaches need a tiebreak + +**When NOT to benchmark:** + +- The function depends on Roblox APIs (use Studio Script Profiler instead) +- The change is purely additive (headers, type annotations) -- trust the bytecode metrics +- The function is not hot (benchmarking cold code is misleading) + +**Cleanup:** Delete the temporary benchmark file after capturing results. Do not leave bench files in the project. + +**Phase 5 -- Verification** + +Re-run all Phase 2 tools and present a before/after comparison: + +- Bytecode instruction count delta +- Allocation count delta +- Inlining success rate delta +- Type coverage improvement (fewer `any` types) +- Register spill delta +- Lint warning delta +- Benchmark results (if Phase 4.5 was used) -- wall-clock speedup for interpreter and native codegen + +If any metric regressed, investigate and explain why (or revert that change). + +**Phase 6 -- Multi-file mode (directory scope only)** + +When optimizing a directory: + +- Run Phase 1 (code read) on all files first to identify which files have the most algorithmic improvement potential +- Run Phase 2 (baseline capture) on all files to identify which have the most bytecode optimization potential +- Prioritize files by: worst algorithmic complexity, most lint warnings, most `any` types, most allocations, most failed inlines +- Process files one at a time through Phases 1-5 +- Present aggregate metrics at the end + +### Key Principles (embedded in command) + +- **Same implementation, better performance** -- behavioral equivalence is non-negotiable +- **Low-hanging fruit first** -- headers and types before restructuring +- **Compiler feedback loop** -- verify each structural change with `--remarks` +- **Small functions over monoliths** -- easier for compiler, easier to maintain +- **Local functions for inlining** -- even at cost of duplicating imported behavior in high-traffic code +- **Quantify everything** -- before/after metrics for every optimization pass +- **Type annotations drive native codegen quality** -- every `any` is a missed specialization opportunity +- **Event-driven over polling** -- avoid per-frame calculations when events suffice + +### Type System Availability (Old Typechecker) + +The command must only recommend type features available with the OLD typechecker (we are NOT using the new type solver). + +**Available NOW (old typechecker):** + +- `--!strict` / `--!nonstrict` / `--!nocheck` directives +- Basic type annotations: `local x: number`, `function f(a: string): boolean` +- Optional types: `string?` (sugar for `string | nil`) +- Union types: `number | string` +- Intersection types: `type1 & type2` +- Generic functions: `function id(x: T): T` +- Explicit generic instantiation: `f<>()` +- Type ascription (cast): `x :: string` +- Array types: `{number}`, dictionary types: `{[string]: number}` +- Typed table constructors: `type Car = { Speed: number, Drive: (Car) -> () }` +- `typeof()` for inferring types from values +- `export type` for cross-module type sharing +- Typed variadics: `function f(...: number)` +- Type alias defaults: `type Map = {[K]: V}` +- Singleton/literal types: `"hello"`, `true` +- `never` and `unknown` types (exhaustiveness checking, safe top type) +- `read` / `write` property modifiers +- Sealed/unsealed table subtyping +- `table.freeze` type narrowing + +**REQUIRES new typechecker (DO NOT recommend):** + +- `keyof`, `index`, `rawkeyof`, `rawget` type functions +- `getmetatable`, `setmetatable` type functions +- User-defined type functions (`type function ...`) +- Negation types (`~T`) +- Relaxed recursive type restrictions +- Local type inference improvements (better bounds tracking) +- `new-nonstrict` unified engine + +### Native Codegen Knowledge Base + +Embedded in the command for reference during optimization: + +**When to use `@native` vs `--!native`:** + +- `--!native`: entire script compiled natively. Good for math-heavy utility modules. +- `@native`: per-function. Better when only specific hot functions benefit, or when script is close to the 1M instruction module limit. +- Inner functions do NOT inherit `@native` -- each needs its own annotation. +- Top-level module code usually runs once; `@native` on it has minimal benefit. + +**What hurts native execution:** + +- `getfenv()` / `setfenv()` -- disables optimizations +- Passing wrong types to typed functions (triggers type guard failures) +- Non-numeric args to math builtins +- Breakpoints disable native execution for affected functions +- Functions exceeding size limits: 64K instructions/block, 32K blocks/function, 1M instructions/module + +**What helps native execution:** + +- Type annotations, especially `Vector3` parameters (specialized vector codegen). JIT uses annotations directly -- no runtime type analysis. +- Small, focused functions (fewer register spills, better block optimization) +- Consistent table shapes (predictable field access, inline caching hits) +- `buffer` operations (efficient native lowering) +- `bit32` operations (map directly to CPU instructions) +- Native 3-component 32-bit float vector type: 16-byte tagged values, VM-level 3-wide SIMD, lower GC pressure than userdata Vector3 + +### VM Architecture Knowledge + +The command should understand how the Luau VM works to give accurate advice: + +**Bytecode interpreter:** + +- Highly tuned portable bytecode interpreter in C. Core dispatch loop ~16 KB on x64 (cache-friendly). +- Can match LuaJIT interpreter on some workloads. +- Computationally intensive scripts mostly use the interpreter core + builtins. + +**Compiler pipeline:** + +- Multi-pass: frontend parses AST, backend emits bytecode. +- Without type info: deep constant folding (across functions and locals), upvalue optimization for non-mutated upvalues, builtin usage analysis, multi-assignment optimization, peephole optimizations. +- With `-O2`: function inlining, loop unrolling (constant bounds only), more aggressive constant folding after inlining. +- Type-directed: currently small peephole optimizations; potential for CSE and allocation hoisting. +- Interprocedural: limited to single modules. Local function calls optimized with known arg/return counts. + +**Inline caching:** + +- Table field access uses HREF-style inline caching. Compiler predicts hash slot; VM corrects dynamically. +- Best when: field name known at compile time, no metatables, uniform object shapes. +- `table.field` and `table["constantString"]` both get inline caching. + +**Import resolution:** + +- Global chains like `math.max` resolved at load time, not execution time. +- Invalidated by `getfenv`/`setfenv`/`loadstring` (marks environment "impure"). + +**Fastcall builtins (complete list):** + +- `assert`, `type`, `typeof`, `rawget`, `rawset`, `rawequal`, `rawlen` +- `getmetatable`, `setmetatable` +- `tonumber`, `tostring` +- Most `math.`* (NOT `math.noise`, `math.random`, `math.randomseed`) +- `bit32.`* +- Some `string.`*, `table.`* +- Partial specializations: `assert` (unused return + truthy), `bit32.extract` (constant field/width), `select(n, ...)` (O(1) via builtin dispatch) +- `math.floor` and similar use SSE4.1 when available +- With `-O2`: builtins with constant args and single return can be constant-folded; `math.pi`, `math.huge` etc. are folded + +**GC architecture:** + +- Incremental mark-sweep with "GC assists" (allocating code pays proportional GC work) +- Paged sweeper: 16 KB pages by size class, 2-3x faster than linked-list, saves 16 bytes/object on 64-bit +- No `__gc` metamethod (removed for performance) +- Weak tables with `__mode = "s"` are "shrinkable" (resized during GC) + +### Type Refinement Patterns + +The compiler narrows types after certain checks, improving both type safety and native codegen quality: + +- **Truthiness:** `if x then` narrows `x` from falsy (`nil`/`false`) +- **Type guards:** `if type(x) == "number" then` narrows `x` to `number` +- **Typeof guards (Roblox):** `if typeof(x) == "Vector3" then` narrows to `Vector3`; `x:IsA("TextLabel")` narrows `Instance` to subclass +- **Equality:** `if x == "hello" then` narrows `x` to singleton `"hello"` +- **Assert:** `assert(type(x) == "string")` narrows `x` to `string` after the call +- **Composition:** Supports `and`, `or`, `not` for compound refinements + +Use refinements to provide type information to the compiler without explicit annotations in contexts where the type is known from a runtime check. + +### Comprehensive Pattern Catalog + +The command will embed this as the reference for pattern replacement: + +**Bytecode-level improvements:** + + +| Slow pattern | Fast pattern | Why | +| ----------------------------- | ------------------------------ | ----------------------------------- | +| `math.floor(a / b)` | `a // b` | Dedicated VM opcode | +| `data[i].x = data[i].x + 1` | `data[i].x += 1` | LHS evaluated once | +| `a + (b - a) * t` | `math.lerp(a, b, t)` | Exact at endpoints, monotonic | +| `"prefix" .. var .. "suffix"` | `prefix{var}suffix` | Lowers to optimized `string.format` | +| Manual byte swap | `bit32.byteswap(n)` | CPU bswap instruction | +| Manual log2 loop | `bit32.countlz(n)` | CPU instruction, ~8x faster | +| `table.getn(t)` | `#t` or `rawlen(t)` | Deprecated, slower | +| `table.foreach(t, f)` | `for k, v in t do f(k, v) end` | Deprecated, slower | +| `cond and A or B` | `if cond then A else B` | Safe for falsy values, one branch | +| `x ~= x` (NaN check) | `math.isnan(x)` | Clearer, same performance | + + +**Allocation reduction:** + + +| Slow pattern | Fast pattern | Why | +| --------------------------------- | ---------------------------------- | ----------------------------------- | +| `local t = {}` in loop | `table.clear(t)` + reuse | Avoids GC pressure | +| `table.insert(t, v)` repeated | `table.create(n)` + indexed writes | Preallocated capacity | +| Manual `pairs` clone | `table.clone(t)` | Faster, copies layout | +| `string.byte`/`string.char` loops | `string.pack`/`string.unpack` | Native implementation | +| String-based binary data | `buffer` type | Fixed-size, offset-based, efficient | + + +**Inlining enablers:** + + +| Blocks inlining | Enables inlining | Why | +| ----------------------- | ----------------------------- | ---------------------------------------- | +| `function Module.foo()` | `local function foo()` | Exported/mutable table prevents inlining | +| `function Obj:method()` | `local function method(self)` | OOP colon syntax not inlineable | +| Recursive function | Split base/recursive cases | Recursion prevents inlining | +| Deep upvalue captures | Pass as parameters | Reduces closure cost | + + +**Deprecated patterns to detect and replace:** + + +| Deprecated | Replacement | Impact | +| ------------------------- | -------------------------- | -------------------------------------------------------------------------------------------------- | +| `getfenv()` / `setfenv()` | `debug.info(i, "snl")` | Disables ALL optimizations (imports, fastcalls, inlining). Even read-only `getfenv()` deoptimizes. | +| `loadstring()` | Restructure with `require` | Marks environment "impure", disables import optimization | +| `table.getn(t)` | `#t` | Slower | +| `table.foreach(t, f)` | `for k, v in t do` | Slower | +| `table.foreachi(t, f)` | `for i, v in t do` | Slower | +| `wait()` | `task.wait()` | Deprecated Roblox API | +| `obj.Method(obj, ...)` | `obj:Method(...)` | Misses fast method call instruction | +| `string:method()` | `string.method(s)` | `string.byte(s)` is faster than `s:byte()` for builtins | + + +**Lint rules to check (via `luau-analyze`):** + +The command should flag these lint warnings as optimization opportunities: + + +| Lint | Name | Optimization impact | +| ---- | ------------------ | ------------------------------------------------------------------------------------------------ | +| 10 | BuiltinGlobalWrite | Overwriting builtins disables fastcall optimization | +| 22 | DeprecatedApi | Deprecated APIs may have performance/correctness issues | +| 23 | TableOperations | `table.insert(t, 0, x)` (wrong index), `table.insert(t, #t+1, x)` (redundant), `#` on non-arrays | +| 3 | GlobalUsedAsLocal | Global used in one function -- should be local for performance | +| 7 | LocalUnused | Dead locals (cleanup candidate) | +| 12 | UnreachableCode | Dead code (removal candidate) | +| 25 | MisleadingAndOr | `a and false or c` bugs -- use `if a then false else c` | + + +### CLI Tool Reference + +All four tools live at `C:\Program Files\Atlas\` and are in PATH. + +#### `luau-compile` -- Static bytecode/codegen analysis (primary tool) + +**Modes** (mutually exclusive, controls output format): + +- `binary` -- Raw bytecode blob (default, not human-readable) +- `text` -- Human-readable bytecode listing with source line annotations. Shows every opcode, register, constant, and jump label per function. +- `remarks` -- Original source with compiler remarks injected as `-- remark:` comments. Shows inlining success/failure (with cost/profit), allocation tracking (`table hash N`, `table array N`). **Single most useful mode for optimization.** +- `codegen` -- Native assembly output. Shows full register allocation, memory ops, branch structure, and outlined helpers. Requires `--target`. + +**Options:** + +- `-O` -- Optimization level 0-2 (default 1). `**-O2` enables automatic inlining.** Always use `-O2` to see production compiler behavior. `-O0` useful as baseline comparison. +- `-g` -- Debug info level 0-2 (default 1). `-g0` strips debug info for smallest bytecode. +- `--target=` -- Architecture for codegen mode: `x64` (Linux/Mac), `x64_ms` (Windows x64 calling convention), `a64` (ARM64), `a64_nf` (ARM64 no-FP). **Use `x64_ms` on Windows.** +- `--record-stats=` -- JSON compilation stats at `total`, `file`, or `function` granularity. At `function` level shows: `bytecodeInstructionCount`, codegen size, parse/compile time, and `lowerStats` (spillsToSlot, spillsToRestore, maxSpillSlotsUsed, skippedFunctions, blocksPreOpt, blocksPostOpt, regAllocErrors, loweringErrors). +- `--bytecode-summary` -- Bytecode operation distribution. Requires `--record-stats=function`. Shows which opcodes dominate. +- `--stats-file=` -- Where to write stats JSON (default `stats.json`). Useful when analyzing multiple files. +- `--timetrace` -- Writes `trace.json` with compiler phase timings (Chrome `chrome://tracing` compatible). +- `--vector-lib=` -- Library name for vector ops (e.g., `"Vector3"`). Tells compiler about vector type for better optimization. +- `--vector-ctor=` -- Constructor function name (e.g., `"new"`). Paired with `--vector-lib` for constant-folding. +- `--vector-type=` -- Vector type name. Completes the vector triplet for full vector optimization. + +**Key invocations for the command (always include Roblox vector flags):** + +``` +luau-compile --remarks -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 +luau-compile --text -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 +luau-compile --codegen --target=x64_ms --record-stats=function -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 # Windows +luau-compile --codegen --target=x64 --record-stats=function -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 # Linux/Mac +luau-compile --bytecode-summary --record-stats=function -O2 --vector-lib=Vector3 --vector-ctor=new --vector-type=Vector3 +``` + +#### `luau-analyze` -- Type checking and linting + +**Modes:** + +- *(omitted)* -- Default: typecheck + lint. Reports type errors and lint warnings to stderr. +- `--annotate` -- Outputs source with all inferred types written inline. Every variable, parameter, and return gets its inferred type. **Critical for finding `any` types that hurt native codegen.** + +**Options:** + +- `--mode=strict` -- Forces `--!strict` type checking even if file lacks the directive. Catches type errors silent in nonstrict mode. +- `--formatter=plain` -- Luacheck-compatible output (machine-parseable, good for counting errors). +- `--formatter=gnu` -- GNU-compatible output (`file:line:col: message`), grep-friendly. +- `--timetrace` -- Writes `trace.json` with analysis phase timings. + +**Key invocations:** + +``` +luau-analyze --mode=strict # Type errors + lint warnings +luau-analyze --annotate # Inferred types (find 'any' types) +``` + +#### `luau` -- Runtime execution, profiling, and coverage + +**Limited utility for Roblox scripts.** Most Roblox scripts depend on engine APIs (`game`, `workspace`, `Players`, etc.) that the standalone `luau` runtime does not provide. This tool is only usable for pure-logic library modules with zero Roblox dependencies. + +**Options:** + +- `-O` -- Optimization level 0-2 (default 1). Same as compile. +- `-g` -- Debug level 0-2 (default 1). Same as compile. +- `--codegen` -- Execute using native code generation. +- `--profile[=N]` -- CPU profiling at N Hz sampling (default 10000). Outputs `profile.out`. +- `--coverage` -- Code coverage collection. Outputs `coverage.out`. Can identify dead code paths. +- `--timetrace` -- Compiler time tracing to `trace.json`. +- `-i, --interactive` -- REPL after script execution. +- `-a, --program-args` -- Pass arguments to the Luau program. + +**Key invocations (only for pure-logic modules without Roblox API dependencies):** + +``` +luau --codegen --profile -O2 # Profile with native codegen +luau --coverage -O2 # Dead code detection +``` + +**For Roblox scripts:** Use Roblox Studio's Script Profiler and `debug.dumpcodesize()` instead. The command should note this when the target file uses Roblox APIs. + +#### `luau-ast` -- AST dump + +**No flags.** Takes a single file, outputs full JSON AST. + +- Node types, source locations, variable scopes, type annotations +- Programmatically parseable for: function nesting depth, function body size (monolith detection), closure captures (upvalue analysis), table construction patterns, loop-invariant expression detection + +**Key invocation:** + +``` +luau-ast # Full JSON AST +``` + +### Reference Documentation + +The command will reference these docs (read during execution if needed for detail): + +**Core optimization:** + +- `.cursor/rules/luau.mdc` -- Luau language rules, optimization tips, performance patterns +- `.cursor/rules/luau/function-inlining.md` -- Inlining RFC (no user `@inline`, automatic at `-O2`, requirements for inlineability) +- `.cursor/rules/luau/syntax-attribute-functions-native.md` -- `@native` per-function attribute RFC +- `.cursor/rules/roblox/en-us/luau/native-code-gen.md` -- Native codegen docs (size limits, type annotation impact, gotchas) + +**Type system (old typechecker only):** + +- `.cursor/rules/roblox/en-us/luau/type-checking.md` -- Type annotation syntax and modes +- `.cursor/rules/luau/generic-functions.md` -- Generic function syntax +- `.cursor/rules/luau/never-and-unknown-types.md` -- `never`/`unknown` for exhaustiveness +- `.cursor/rules/luau/syntax-type-ascription.md` -- `::` cast operator + +**Library optimizations:** + +- `.cursor/rules/luau/function-table-create-find.md` -- `table.create`, `table.find` +- `.cursor/rules/luau/function-table-clear.md` -- `table.clear` for reuse +- `.cursor/rules/luau/function-table-clone.md` -- `table.clone` vs manual copy +- `.cursor/rules/luau/function-table-freeze.md` -- `table.freeze` for readonly +- `.cursor/rules/luau/function-math-lerp.md` -- `math.lerp` properties and correctness +- `.cursor/rules/luau/function-bit32-byteswap.md` -- CPU-level byte swap +- `.cursor/rules/luau/function-bit32-countlz-countrz.md` -- CPU-level bit counting +- `.cursor/rules/luau/function-buffer-bits.md` -- Buffer bit operations +- `.cursor/rules/luau/type-byte-buffer.md` -- `buffer` type for binary data +- `.cursor/rules/luau/vector-library.md` -- Vector library and fastcalls +- `.cursor/rules/luau/function-string-pack-unpack.md` -- Binary string operations +- `.cursor/rules/luau/syntax-floor-division-operator.md` -- `//` dedicated opcode +- `.cursor/rules/luau/syntax-compound-assignment.md` -- `+=` single LHS evaluation + +**Deprecation/avoidance:** + +- `.cursor/rules/luau/deprecate-getfenv-setfenv.md` -- Why fenv disables optimizations +- `.cursor/rules/luau/deprecate-table-getn-foreach.md` -- Deprecated table functions +- `.cursor/rules/luau/generalized-iteration.md` -- Modern iteration patterns + +**Roblox performance (always applicable):** + +- `.cursor/rules/roblox/en-us/performance-optimization/improve.md` -- Script computation, memory, physics, rendering, networking, assets +- `.cursor/rules/roblox/en-us/performance-optimization/design.md` -- Event-driven patterns, frame budgets (~16.67ms at 60 FPS), multithreading +- `.cursor/rules/roblox/en-us/luau/variables.md` -- Local vs global: "global in a time-critical loop can make it perform more than 10% slower" +- `.cursor/rules/roblox/en-us/luau/scope.md` -- Scope performance implications +- `.cursor/rules/roblox/en-us/luau/native-code-gen.md` -- `@native` attribute, Vector3 annotation impact, size limits, `debug.dumpcodesize()` + diff --git a/.cursor/plans/luau-style_ref_paths_845083e1.plan.md b/.cursor/plans/luau-style_ref_paths_845083e1.plan.md new file mode 100644 index 00000000..3fc1a242 --- /dev/null +++ b/.cursor/plans/luau-style_ref_paths_845083e1.plan.md @@ -0,0 +1,1034 @@ +--- +name: Luau-style ref paths +overview: "Migrate Rojo_Ref_* attribute paths from bare absolute paths to Luau require-by-string style: @self/, ./, ../, and @game/. Prefer relative paths for maximum copy/paste portability. Update all syncback, forward sync, and two-way sync codepaths, plus comprehensive tests." +todos: + - id: core-compute + content: Add compute_relative_ref_path() to rojo_ref.rs + unit tests for all prefix selection scenarios + status: completed + - id: resolve-fn + content: Add resolve_ref_path() to tree.rs that handles @game/, @self/, ./, ../ prefixes and .. mid-path + status: completed + - id: forward-sync + content: Update patch_compute.rs and patch_apply.rs to pass source_ref when resolving Rojo_Ref_* paths + status: completed + - id: syncback-placeholders + content: Redesign placeholder system in ref_properties.rs to encode (source, target) pairs, compute relative paths in post-processing + status: completed + - id: syncback-final + content: Update fs_snapshot.rs fix_ref_paths and syncback/mod.rs post-processing to produce relative substitutions + status: completed + - id: two-way-write + content: Update api.rs syncback_updated_properties and added_paths to write relative ref paths + status: completed + - id: change-processor + content: Rewrite update_ref_paths_after_rename in change_processor.rs to resolve-then-recompute instead of string prefix replacement + status: completed + - id: ref-path-index + content: Change RefPathIndex.populate_from_dir to populate_from_tree, store resolved absolute paths, update serve_session.rs call site + status: completed + - id: update-existing-tests + content: Fix all existing test fixtures and assertions to use new prefix format + status: completed + - id: new-test-compute + content: Add comprehensive compute_relative_ref_path unit tests (all scenarios from plan) + status: completed + - id: new-test-resolve + content: Add comprehensive resolve_ref_path unit tests (all prefix types, .., edge cases) + status: completed + - id: new-test-roundtrip + content: Add syncback round-trip integration tests with relative refs (@self, ./, ../, @game) + status: completed + - id: new-test-twoway + content: Add two-way sync integration tests (ref writes, renames, dedup cleanup with relative paths) + status: completed + - id: new-test-edge + content: Add edge case tests (cross-service, ancestor refs, slugified names, dedup suffixes, same-batch add+ref) + status: completed + - id: cleanup + content: Remove dead escape/unescape functions, update documentation comments throughout + status: completed +isProject: false +--- + +# Luau-Style Ref Paths + +## Current State + +`Rojo_Ref_*` attributes store bare absolute paths from DataModel's children: + +```json5 +{ "Rojo_Ref_PrimaryPart": "Workspace/TestModel/Part1.model.json5" } +``` + +## Target State + +Use Luau require-by-string semantics with four prefix types: + +```json5 +// Descendant of source instance +{ "Rojo_Ref_PrimaryPart": "@self/Part1.model.json5" } + +// Sibling (child of parent) +{ "Rojo_Ref_Value": "./Sibling.model.json5" } + +// Navigate up then down (within same service) +{ "Rojo_Ref_Attachment0": "../Attachments/Att1.model.json5" } + +// Cross-service or target is ancestor of source +{ "Rojo_Ref_Value": "@game/ReplicatedStorage/Shared/Module.luau" } +``` + +## Prefix Selection Algorithm + +Given source and target absolute ref paths: + +1. **target == source** --> `@self` (self-reference, rare) +2. **target is descendant of source** --> `@self/` +3. **LCA is DataModel root** (cross-service, `common_prefix_len == 0`) --> `@game/` +4. **target is ancestor of source** (`remaining` is empty after LCA) --> `@game/` +5. **Otherwise** (same service, different branches): + - `ups == 1` (LCA is parent) --> `./` + - `ups >= 2` --> `"../" * (ups - 1)` + `` + +This maximizes portability: subtrees can be copied/moved within a service without breaking refs. + +**Worked examples:** + + +| Source | Target | LCA | Result | +| ---------------------- | ------------------------------- | ------------- | ------------------------------------- | +| `Workspace/A/Script` | `Workspace/A/Script/Child` | (descendant) | `@self/Child` | +| `Workspace/A/Script` | `Workspace/A/Part.model.json5` | `Workspace/A` | `./Part.model.json5` | +| `Workspace/A/Script` | `Workspace/B/Part.model.json5` | `Workspace` | `../B/Part.model.json5` | +| `Workspace/A/B/Script` | `Workspace/C/Part.model.json5` | `Workspace` | `../../C/Part.model.json5` | +| `Workspace/A/Script` | `ReplicatedStorage/Module.luau` | root | `@game/ReplicatedStorage/Module.luau` | +| `Workspace/A/B/Script` | `Workspace/A` | (ancestor) | `@game/Workspace/A` | + + +## Resolution Algorithm + +Per the [Roblox require-by-string docs](https://create.roblox.com/docs/reference/engine/globals/LuaGlobals#require): + +``` +resolve_ref_path(path, source_ref, tree): + @game/rest --> resolve segments from root (game) + @self --> return source_ref (script itself) + @self/rest --> resolve segments from source_ref (script) + ./rest --> resolve segments from parent(source_ref) (script.Parent) + ../rest --> resolve segments from parent(parent(source_ref)) (script.Parent.Parent) + bare_path --> resolve segments from root (backward compat fallback) + + During segment walk: + ".." --> go to parent of previous component + other --> find child by filesystem name (case-insensitive) +``` + +**How `../` chaining works** (from Roblox docs): + +- `../X` = script.Parent.Parent.X (2 ups total) +- `../../X` = script.Parent.Parent.Parent.X (3 ups -- `../` prefix gives 2, then `..` component gives 1 more) +- `../../../X` = 4 ups (prefix gives 2, two `..` components give 2 more) + +**Generation formula** (ups = levels from source to LCA): + +- ups=1: `"./" + remaining` +- ups>=2: `"../" * (ups-1) + remaining` + +Verification: ups=3 --> `"../" * 2` = `"../../"` + remaining. Parse: `../` prefix = 2 ups, one `..` component = 1 more = 3. Correct. + +Bare paths (no prefix) are treated as `@game/` for backward compatibility with old-format files. New writes never produce bare paths. + +--- + +## Phase 1: Core Path Functions + +**File: [src/rojo_ref.rs](src/rojo_ref.rs)** + +- Add `compute_relative_ref_path(source_abs: &str, target_abs: &str) -> String` -- pure string function implementing the prefix selection algorithm above. Both inputs are absolute ref paths (slash-separated filesystem names from root children). Output is the prefixed relative/absolute path. +- Keep `ref_target_path_from_tree()` and `ref_target_path()` for computing absolute paths (they become internal building blocks). +- The escape/unescape functions (`escape_ref_path_segment`, `split_ref_path`, etc.) become dead code since filesystem names can't contain `/`. Remove or deprecate them. + +## Phase 2: Path Resolution (Forward Sync) + +**File: [src/snapshot/tree.rs](src/snapshot/tree.rs)** + +- Add `resolve_ref_path(&self, path: &str, source_ref: Ref) -> Option` -- new public function. Parses prefix, sets starting Ref, then walks segments. Handles `..` as "go to parent" at any position. +- `get_instance_by_path()` remains as internal helper for absolute segment walking (used by `resolve_ref_path` for the `@game/` and bare-path cases). + +**File: [src/snapshot/patch_compute.rs](src/snapshot/patch_compute.rs)** + +- `compute_ref_properties()` currently calls `tree.get_instance_by_path(path)`. Change to call `tree.resolve_ref_path(path, source_ref)` where `source_ref` is the instance that owns the attributes. + +**File: [src/snapshot/patch_apply.rs](src/snapshot/patch_apply.rs)** + +- `finalize_patch_application()` resolves deferred refs in `path_refs_to_rewrite`. Currently uses `tree.get_instance_by_path()`. Change to `tree.resolve_ref_path(path, source_ref)` where source_ref comes from the `MultiMap` key. + +## Phase 3: CLI Syncback (Writing Relative Paths) + +**File: [src/syncback/ref_properties.rs](src/syncback/ref_properties.rs)** + +- **Placeholder redesign:** Placeholders must be unique per (source, target) pair since the same target gets different relative paths from different sources. Change from `__ROJO_REF_`__ to `__ROJO_REF__TO_`__. +- Change `placeholder_to_target: HashMap` to `placeholder_to_source_and_target: HashMap`. +- When `final_paths` IS available, call `compute_relative_ref_path(source_path, target_path)` directly instead of storing the absolute target path. +- `collect_all_paths()` is unchanged (still collects absolute paths for all instances). + +**File: [src/syncback/fs_snapshot.rs](src/syncback/fs_snapshot.rs)** + +- `fix_ref_paths()` substitution map entries are now `(placeholder, relative_path)`. The post-processing step in [src/syncback/mod.rs](src/syncback/mod.rs) must look up BOTH source and target in `ref_path_map`, call `compute_relative_ref_path()`, and produce the substitution. + +**File: [src/syncback/snapshot.rs](src/syncback/snapshot.rs)** + +- `record_ref_path()` is unchanged -- it still records absolute paths in `ref_path_map`. These are used as inputs to `compute_relative_ref_path`. + +## Phase 4: Two-Way Sync (Writing Relative Paths) + +**File: [src/web/api.rs](src/web/api.rs)** + +- `syncback_updated_properties()`: Currently computes target absolute path via `ref_target_path_from_tree()` and stores it directly. Change to also compute source absolute path, then call `compute_relative_ref_path(source_abs, target_abs)` for the attribute value. +- `added_paths` pre-computation: Same change -- compute relative paths from source to the newly-added target. + +**File: [src/change_processor.rs](src/change_processor.rs)** + +- `update_ref_paths_after_rename()`: Currently does string prefix replacement on absolute paths. With relative paths on disk, the approach changes: + 1. Find affected files via RefPathIndex (still indexed by absolute target path) + 2. For each file, parse JSON, resolve each `Rojo_Ref`_* value to absolute (using source instance's absolute path) + 3. If resolved absolute matches old prefix, compute new absolute, then recompute relative path from source + 4. Write updated relative path back +- This requires knowing each file's source instance absolute path. Add a helper that derives this from the file's filesystem path + tree metadata (instigating source lookup). + +## Phase 5: RefPathIndex Updates + +**File: [src/rojo_ref.rs*](src/rojo_ref.rs)* (RefPathIndex) + +The index stores **resolved absolute paths** as keys (not the on-disk relative strings). This preserves the efficient prefix-based lookup. + +- `populate_from_dir()` --> Change to `populate_from_tree(tree: &RojoTree)`. Walk instances in the tree, read `Rojo_Ref`_* attributes, resolve relative paths to absolute using the source instance's position, index the absolute path. Called after initial tree build in [src/serve_session.rs](src/serve_session.rs) line 177-181. +- `add()` / `remove()`: Callers pass resolved absolute paths (same as before). +- `find_by_prefix()` / `update_prefix()`: Unchanged (operate on absolute keys). +- `update_ref_paths_in_file()` in [src/syncback/meta.rs](src/syncback/meta.rs): Rewrite to parse + resolve + recompute instead of string prefix replacement. + +## Phase 6: Tests + +### Update Existing Tests + +All test fixtures and assertions with bare absolute ref paths need updating to use the new prefix format. + +**Key files:** + +- [rojo-test/serve-tests/ref_two_way_sync/](rojo-test/serve-tests/ref_two_way_sync/) -- fixture meta files +- [rojo-test/serve-tests/ref_forward_sync/](rojo-test/serve-tests/ref_forward_sync/) -- fixture meta files +- [tests/tests/two_way_sync.rs](tests/tests/two_way_sync.rs) -- ref assertions +- [tests/tests/serve.rs](tests/tests/serve.rs) -- ref assertions +- [src/rojo_ref.rs](src/rojo_ref.rs) -- unit tests +- [src/syncback/ref_properties.rs](src/syncback/ref_properties.rs) -- unit tests +- [src/syncback/fs_snapshot.rs](src/syncback/fs_snapshot.rs) -- fix_ref_paths tests +- [src/snapshot/tests/](src/snapshot/tests/) -- patch compute/apply tests + +--- + +### Test Specification: `compute_relative_ref_path(source_abs, target_abs)` + +Every test below shows the two absolute path inputs and the exact expected output string. + +#### `@self` -- Target is the source itself or a descendant + + +| # | Source | Target | Expected | Why | +| --- | ----------------- | -------------------------------------------- | ---------------------------------- | ---------------------------------------- | +| 1 | `Workspace/Model` | `Workspace/Model` | `@self` | Self-reference | +| 2 | `Workspace/Model` | `Workspace/Model/Handle.model.json5` | `@self/Handle.model.json5` | Direct child | +| 3 | `Workspace/Model` | `Workspace/Model/SubFolder/Part.model.json5` | `@self/SubFolder/Part.model.json5` | Nested descendant | +| 4 | `Workspace/Model` | `Workspace/Model/A/B/C/D.luau` | `@self/A/B/C/D.luau` | Deeply nested descendant (4 levels) | +| 5 | `Workspace/Tool` | `Workspace/Tool/Handle.model.json5` | `@self/Handle.model.json5` | Tool with Handle (common Roblox pattern) | +| 6 | `Workspace/Model` | `Workspace/Model/Init` | `@self/Init` | Child that is a directory (init-style) | +| 7 | `Workspace/Gun` | `Workspace/Gun/Sounds/Fire.model.json5` | `@self/Sounds/Fire.model.json5` | Tool referencing nested sound | +| 8 | `Workspace/Model` | `Workspace/Model/Hey_Bro.server.luau` | `@self/Hey_Bro.server.luau` | Slugified child name | +| 9 | `Workspace/Model` | `Workspace/Model/Data~2` | `@self/Data~2` | Dedup'd child | + + +#### `./` -- Target is a sibling (shares same parent, 1 level up) + + +| # | Source | Target | Expected | Why | +| --- | ------------------------------------------ | -------------------------------------- | ----------------------- | -------------------------------------- | +| 10 | `Workspace/Folder/ScriptA.server.luau` | `Workspace/Folder/ScriptB.server.luau` | `./ScriptB.server.luau` | Sibling script | +| 11 | `Workspace/Folder/ScriptA.server.luau` | `Workspace/Folder/Config.luau` | `./Config.luau` | Sibling module | +| 12 | `Workspace/Folder/ScriptA.server.luau` | `Workspace/Folder/SubFolder` | `./SubFolder` | Sibling directory | +| 13 | `Workspace/Folder/ScriptA.server.luau` | `Workspace/Folder/SubFolder/Deep.luau` | `./SubFolder/Deep.luau` | Sibling then descend | +| 14 | `Workspace/Folder/ObjectValue.model.json5` | `Workspace/Folder/Target.model.json5` | `./Target.model.json5` | ObjectValue.Value pointing to sibling | +| 15 | `Workspace/Beams/Beam.model.json5` | `Workspace/Beams/Att1.model.json5` | `./Att1.model.json5` | Beam.Attachment0 to sibling attachment | +| 16 | `Workspace/Beams/Beam.model.json5` | `Workspace/Beams/Att2.model.json5` | `./Att2.model.json5` | Beam.Attachment1 to another sibling | +| 17 | `Workspace/Folder/A.luau` | `Workspace/Folder/Hey_Bro.server.luau` | `./Hey_Bro.server.luau` | Sibling with slugified name | +| 18 | `Workspace/Folder/A.luau` | `Workspace/Folder/Data~2` | `./Data~2` | Sibling with dedup suffix | +| 19 | `Workspace/Folder/A.luau` | `Workspace/Folder/Data~3.model.json5` | `./Data~3.model.json5` | Sibling model with dedup suffix | + + +#### `../` -- Navigate up 2+ levels then down (within same service) + + +| # | Source | Target | Expected | Why | +| --- | ----------------------------------------------- | ---------------------------------------------- | -------------------------------- | ------------------------------------------- | +| 20 | `Workspace/A/Script.server.luau` | `Workspace/B/Part.model.json5` | `../B/Part.model.json5` | Parent's sibling's child (2 ups) | +| 21 | `Workspace/A/Script.server.luau` | `Workspace/B` | `../B` | Parent's sibling directory (2 ups) | +| 22 | `Workspace/A/B/Script.server.luau` | `Workspace/C/Part.model.json5` | `../../C/Part.model.json5` | Grandparent's sibling's child (3 ups) | +| 23 | `Workspace/A/B/C/Script.luau` | `Workspace/D/E/Part.model.json5` | `../../../D/E/Part.model.json5` | 4 ups within same service | +| 24 | `Workspace/A/B/C/D/Script.luau` | `Workspace/E/Part.model.json5` | `../../../../E/Part.model.json5` | 5 ups within same service | +| 25 | `Workspace/Models/Car/Body.model.json5` | `Workspace/Models/Truck/Body.model.json5` | `../Truck/Body.model.json5` | Peer model in sibling folder | +| 26 | `Workspace/A/Deep/Script.luau` | `Workspace/A/Other.luau` | `../Other.luau` | Up from deep, target is cousin (2 ups) | +| 27 | `Workspace/A/B/Beam.model.json5` | `Workspace/A/C/Att.model.json5` | `../C/Att.model.json5` | Beam referencing attachment in uncle folder | +| 28 | `Workspace/Systems/Combat/Hitbox.server.luau` | `Workspace/Systems/Audio/HitSound.model.json5` | `../Audio/HitSound.model.json5` | Cross-system ref within same parent | +| 29 | `Workspace/Map/Zone1/Spawns/Spawn1.model.json5` | `Workspace/Map/Zone1/Props/Tree.model.json5` | `../Props/Tree.model.json5` | Same zone, different category | +| 30 | `Workspace/Map/Zone1/Spawns/Spawn1.model.json5` | `Workspace/Map/Zone2/Flag.model.json5` | `../../Zone2/Flag.model.json5` | Cross-zone ref | + + +#### `@game/` -- Cross-service or target is ancestor + + +| # | Source | Target | Expected | Why | +| --- | ----------------------------------------- | -------------------------------------------- | -------------------------------------------------- | ------------------------------------- | +| 31 | `Workspace/Script.server.luau` | `ReplicatedStorage/Module.luau` | `@game/ReplicatedStorage/Module.luau` | Classic cross-service require pattern | +| 32 | `Workspace/Script.server.luau` | `ServerStorage/Data.luau` | `@game/ServerStorage/Data.luau` | Server script to ServerStorage | +| 33 | `ServerScriptService/Main.server.luau` | `ReplicatedStorage/Shared/Utils.luau` | `@game/ReplicatedStorage/Shared/Utils.luau` | SSS to RS (common pattern) | +| 34 | `StarterGui/ScreenGui/Button.client.luau` | `ReplicatedStorage/UI/Theme.luau` | `@game/ReplicatedStorage/UI/Theme.luau` | Client UI to shared module | +| 35 | `Workspace/A/B/C/Script.luau` | `Lighting` | `@game/Lighting` | Ref to a service itself | +| 36 | `Workspace/A/B/C/Script.luau` | `Workspace` | `@game/Workspace` | Ref to own service (ancestor) | +| 37 | `Workspace/A/B/Script.luau` | `Workspace/A` | `@game/Workspace/A` | Ref to own grandparent (ancestor) | +| 38 | `Workspace/A/B/C/Script.luau` | `Workspace/A/B` | `@game/Workspace/A/B` | Ref to own great-grandparent | +| 39 | `Workspace/Model` | `ReplicatedStorage/Assets/Model.model.json5` | `@game/ReplicatedStorage/Assets/Model.model.json5` | Cross-service model ref | +| 40 | `Workspace/Beam.model.json5` | `Lighting/Atmosphere.model.json5` | `@game/Lighting/Atmosphere.model.json5` | Workspace to Lighting | +| 41 | `ReplicatedStorage/A.luau` | `ReplicatedStorage` | `@game/ReplicatedStorage` | Ref to own service (ancestor) | +| 42 | `Workspace/Script.luau` | `SoundService/BGM.model.json5` | `@game/SoundService/BGM.model.json5` | To SoundService | + + +#### Special names: slugified, dedup'd, init-style + + +| # | Source | Target | Expected | Why | +| --- | ------------------------- | -------------------------------------- | ----------------------- | ----------------------------------------------- | +| 43 | `Workspace/Folder/A.luau` | `Workspace/Folder/Hey_Bro.server.luau` | `./Hey_Bro.server.luau` | Sibling with slug (real name "Hey/Bro") | +| 44 | `Workspace/Folder/A.luau` | `Workspace/Other/CON_.luau` | `../Other/CON_.luau` | Windows reserved name (slugified) | +| 45 | `Workspace/A.luau` | `Workspace/Data~2` | `./Data~2` | Dedup'd sibling (Folder) | +| 46 | `Workspace/A.luau` | `Workspace/Data~3.model.json5` | `./Data~3.model.json5` | Dedup'd model sibling | +| 47 | `Workspace/A.luau` | `Workspace/Data~2/Child.luau` | `./Data~2/Child.luau` | Into dedup'd folder | +| 48 | `Workspace/Model` | `Workspace/Model/Scripts` | `@self/Scripts` | Child is init-style directory | +| 49 | `Workspace/Scripts` | `Workspace/Scripts/Foo` | `@self/Foo` | Source is init-style, child is too | +| 50 | `Workspace/MyFolder`_ | `Workspace/MyFolder_/Child.luau` | `@self/Child.luau` | Source name has trailing underscore (from slug) | + + +#### Tricky edge cases + + +| # | Source | Target | Expected | Why | +| --- | ----------------------------------- | ----------------------------------- | ------------------------------------- | ------------------------------------------------------------ | +| 51 | `Workspace` | `Workspace/Model.model.json5` | `@self/Model.model.json5` | Service referencing own child | +| 52 | `Workspace` | `ReplicatedStorage/Module.luau` | `@game/ReplicatedStorage/Module.luau` | Service to service (cross-service) | +| 53 | `Workspace` | `Workspace` | `@self` | Service self-ref | +| 54 | `Workspace` | `Lighting` | `@game/Lighting` | Service to different service | +| 55 | `Workspace/A` | `Workspace/AB` | `./AB` | Prefix ambiguity: "A" is NOT prefix of "AB" in segment terms | +| 56 | `Workspace/Foo.server.luau` | `Workspace/Foo.luau` | `./Foo.luau` | Same stem different extension = different instances | +| 57 | `Workspace/A/B/C/D/E/F/Script.luau` | `Workspace/A/B/C/D/E/G/Target.luau` | `../G/Target.luau` | Deep nesting, close cousins | +| 58 | `Workspace/A/B/C/D/E/F/Script.luau` | `Workspace/X/Target.luau` | `../../../../../X/Target.luau` | Deep nesting, far apart but same service | + + +--- + +### Test Specification: `resolve_ref_path(path, source_ref, tree)` + +Build a RojoTree for each test. The tree structure is shown in comments. Each test verifies that the path resolves to the correct Ref (or None). + +#### Prefix resolution + +Tree for tests 59-78 (instance hierarchy with filesystem names in parens): + +``` +DataModel (root) + Workspace + ModelA (Model, fs: "ModelA") + Script (Script server, fs: "Script.server.luau") + Handle (Part, fs: "Handle.model.json5") + SubFolder (Folder, fs: "SubFolder") + Deep (Part, fs: "Deep.model.json5") + ModelB (Folder, fs: "ModelB") + Part (Part, fs: "Part.model.json5") + ReplicatedStorage + Module (ModuleScript, fs: "Module.luau") + Shared (Folder, fs: "Shared") + Utils (ModuleScript, fs: "Utils.luau") + Lighting +``` + +Key relationships: Script and Handle are **siblings** under ModelA. ModelA **has children** (Script, Handle, SubFolder). ModelA and ModelB are **siblings** under Workspace. + + +| # | Path | Source | Expected | Why | +| --- | ------------------------------------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 59 | `@self/Handle.model.json5` | ModelA | Handle | `@self/` starts at ModelA, child Handle | +| 60 | `@self/SubFolder/Deep.model.json5` | ModelA | Deep | `@self/` nested descent from ModelA | +| 61 | `@self` | Script | Script | Self-reference (no path after prefix) | +| 62 | `./Handle.model.json5` | Script | Handle | `./` starts at Script.Parent = ModelA, child Handle | +| 63 | `./SubFolder/Deep.model.json5` | Script | Deep | `./` starts at ModelA, descend through SubFolder | +| 64 | `./Script.server.luau` | Handle | Script | `./` from Handle.Parent = ModelA, child Script | +| 65 | `../ModelB/Part.model.json5` | Script | Part | `../` starts at Script.Parent.Parent = Workspace, then ModelB/Part | +| 66 | `../../ReplicatedStorage/Module.luau` | Script | Module | `../` starts at Workspace; remaining `../ReplicatedStorage/Module.luau` goes `..` up to DataModel root, then descends. Valid path (just not one we'd generate -- we'd use `@game/` instead) | +| 67 | `@game/ReplicatedStorage/Module.luau` | Script | Module | `@game/` absolute from root | +| 68 | `@game/Workspace/ModelA/Handle.model.json5` | Script | Handle | `@game/` works even when target is nearby | +| 69 | `@game/Lighting` | Script | Lighting | `@game/` to a service | +| 70 | `Workspace/ModelA/Handle.model.json5` | Script | Handle | Bare path (no prefix) = backward compat, resolves like `@game/` | +| 71 | `@game/Workspace/ModelA/NonExistent` | Script | None | Target doesn't exist | +| 72 | `./NonExistent.luau` | Script | None | Sibling doesn't exist | + + +#### `..` mid-path + + +| # | Path | Source | Expected | Why | +| --- | ------------------------------------------- | ------ | -------- | ------------------------------------------------------------------------------------ | +| 73 | `@self/SubFolder/../Handle.model.json5` | ModelA | Handle | Start at ModelA, into SubFolder, `..` back to ModelA, child Handle | +| 74 | `./SubFolder/../Script.server.luau` | Handle | Script | `./` at ModelA, into SubFolder, `..` back to ModelA, child Script | +| 75 | `./SubFolder/../../ModelB/Part.model.json5` | Script | Part | `./` at ModelA, into SubFolder, `..` to ModelA, `..` to Workspace, child ModelB/Part | + + +#### Case-insensitive matching + + +| # | Path | Source | Expected | Why | +| --- | ------------------------------------------- | ------ | -------- | ---------------------------------------- | +| 76 | `@self/handle.model.json5` | ModelA | Handle | Lowercase "handle" matches "Handle" | +| 77 | `./SCRIPT.SERVER.LUAU` | Handle | Script | All-caps matches filesystem name | +| 78 | `@game/workspace/modela/Handle.model.json5` | Script | Handle | Lowercase service and folder names match | + + +#### Filesystem name vs instance name fallback + +Tree with metadata: + +``` +DataModel + Workspace + MyFolder (fs_name: "MyFolder", instance_name: "MyFolder") + Slugged (fs_name: "Hey_Bro.server.luau", instance_name: "Hey/Bro") + Normal.luau (fs_name: "Normal.luau", instance_name: "Normal") +``` + + +| # | Path | Source | Expected | Why | +| --- | ---------------------------------------------- | ------ | -------- | -------------------------- | +| 79 | `./Hey_Bro.server.luau` | Normal | Slugged | Matches by filesystem name | +| 80 | `@game/Workspace/MyFolder/Hey_Bro.server.luau` | Normal | Slugged | Absolute with fs name | + + +--- + +### Test Specification: Syncback Round-Trip + +Each test: build a WeakDom with specific instances and Ref properties, run syncback, verify the on-disk `Rojo_Ref_*` attribute values, then rebuild from disk and verify the Ref properties resolve to the same targets. + +#### RT-1: Model with PrimaryPart (`@self/`) + +``` +Workspace + TestModel (Model) + PrimaryPart → Handle + Handle (Part) + Other (Part) +``` + +**Expected on disk** (`Workspace/TestModel/init.meta.json5`): + +```json5 +{ "className": "Model", "attributes": { "Rojo_Ref_PrimaryPart": "@self/Handle.model.json5" } } +``` + +Rebuild: `TestModel.PrimaryPart` == Handle. Round-trip identical. + +#### RT-2: Cross-service ObjectValue (`@game/`) + +``` +Workspace + OV (ObjectValue) + Value → Target +ReplicatedStorage + Target (ModuleScript, source="return {}") +``` + +**Expected on disk** (`Workspace/OV.model.json5`): + +```json5 +{ "className": "ObjectValue", "attributes": { "Rojo_Ref_Value": "@game/ReplicatedStorage/Target.luau" } } +``` + +#### RT-3: Beam with attachments in same parent (`./`) + +``` +Workspace + Beams (Folder) + Beam (Beam) + Attachment0 → Att1 + Attachment1 → Att2 + Att1 (Attachment) + Att2 (Attachment) +``` + +**Expected on disk** (`Workspace/Beams/Beam.model.json5`): + +```json5 +{ + "className": "Beam", + "attributes": { + "Rojo_Ref_Attachment0": "./Att1.model.json5", + "Rojo_Ref_Attachment1": "./Att2.model.json5" + } +} +``` + +#### RT-4: Beam with attachments in sibling subfolder (`./`) + +``` +Workspace + Beams (Folder) + Beam (Beam) + Attachment0 → Att1 + Attachments (Folder) + Att1 (Attachment) +``` + +Beam and Attachments are siblings under Beams. Source = `Workspace/Beams/Beam.model.json5`. Target = `Workspace/Beams/Attachments/Att1.model.json5`. LCA = `Workspace/Beams`. ups = 1 --> `./`. + +**Expected on disk** (`Workspace/Beams/Beam.model.json5`): + +```json5 +{ + "className": "Beam", + "attributes": { "Rojo_Ref_Attachment0": "./Attachments/Att1.model.json5" } +} +``` + +#### RT-5: Beam with attachments across branches (`../`) + +``` +Workspace + GroupA (Folder) + Beam (Beam) + Attachment0 → Att1 + GroupB (Folder) + Att1 (Attachment) +``` + +**Expected on disk** (`Workspace/GroupA/Beam.model.json5`): + +```json5 +{ "className": "Beam", "attributes": { "Rojo_Ref_Attachment0": "../GroupB/Att1.model.json5" } } +``` + +Source = `Workspace/GroupA/Beam.model.json5`. Target = `Workspace/GroupB/Att1.model.json5`. LCA = `Workspace`. ups = 2. `../GroupB/Att1.model.json5`. + +#### RT-6: Mixed prefix types across multiple meta files + +``` +Workspace + Container (Folder) + Model (Model) + Handle (Part) <-- child of Model + RefToSibling (ObjectValue, Value → Sibling) <-- child of Model + Sibling (Part) <-- child of Container (sibling of Model) +ReplicatedStorage + SharedModule (ModuleScript) +``` + +Two refs produce different prefix types: + +1. **Model.PrimaryPart → Handle**: Source = Model (`Workspace/Container/Model`). Handle is a descendant of Model --> `@self/Handle.model.json5`. Written in `Model/init.meta.json5`. +2. **RefToSibling.Value → Sibling**: Source = RefToSibling (`Workspace/Container/Model/RefToSibling.model.json5`). Target = Sibling (`Workspace/Container/Sibling.model.json5`). LCA = Container. ups = 2 --> `../Sibling.model.json5`. Written in `RefToSibling.model.json5`. + +**Expected** (`Workspace/Container/Model/init.meta.json5`): + +```json5 +{ "className": "Model", "attributes": { "Rojo_Ref_PrimaryPart": "@self/Handle.model.json5" } } +``` + +**Expected** (`Workspace/Container/Model/RefToSibling.model.json5`): + +```json5 +{ "className": "ObjectValue", "attributes": { "Rojo_Ref_Value": "../Sibling.model.json5" } } +``` + +Verifies two different ref prefix types coexist in the same directory tree. + +#### RT-7: Refs through dedup'd siblings + +``` +Workspace + Container (Folder) + Data (Folder, child1) + Data (Folder, child2) --> gets dedup'd to Data~2 + Pointer (ObjectValue) + Value → Data(child2) +``` + +**Expected** (`Workspace/Container/Pointer.model.json5`): + +```json5 +{ "className": "ObjectValue", "attributes": { "Rojo_Ref_Value": "./Data~2" } } +``` + +#### RT-8: Ref to init-style script + +``` +Workspace + Folder + Pointer (ObjectValue, Value → BigScript) + BigScript (ModuleScript with children) + Helper (ModuleScript) +``` + +BigScript has children, so it becomes a directory: `Folder/BigScript/init.luau`. Filesystem name = `BigScript`. + +**Expected** (`Workspace/Folder/Pointer.model.json5`): + +```json5 +{ "className": "ObjectValue", "attributes": { "Rojo_Ref_Value": "./BigScript" } } +``` + +#### RT-9: Ref to instance with slugified name + +``` +Workspace + Folder + Pointer (ObjectValue, Value → Target) + Target (Script, name="Hey:There", server) +``` + +"Hey:There" slugifies to "Hey_There". File = `Hey_There.server.luau`. + +**Expected** (`Workspace/Folder/Pointer.model.json5`): + +```json5 +{ "className": "ObjectValue", "attributes": { "Rojo_Ref_Value": "./Hey_There.server.luau" } } +``` + +#### RT-10: Deep nesting within same service + +``` +Workspace + A (Folder) + B (Folder) + C (Folder) + D (Folder) + Source (ObjectValue, Value → Target) + X (Folder) + Target (Part) +``` + +Source = `Workspace/A/B/C/D/Source.model.json5`. Target = `Workspace/A/X/Target.model.json5`. LCA = `Workspace/A`. ups = 4. --> `../../../X/Target.model.json5`. + +**Expected**: + +```json5 +{ "className": "ObjectValue", "attributes": { "Rojo_Ref_Value": "../../../X/Target.model.json5" } } +``` + +#### RT-11: Multiple instances referencing same target from different depths + +``` +Workspace + Target (Part) + Shallow (ObjectValue, Value → Target) + Deep (Folder) + Mid (Folder) + DeepOV (ObjectValue, Value → Target) +``` + +- Shallow: Source = `Workspace/Shallow.model.json5`, Target = `Workspace/Target.model.json5`. ups = 1. --> `./Target.model.json5` +- DeepOV: Source = `Workspace/Deep/Mid/DeepOV.model.json5`, Target = `Workspace/Target.model.json5`. LCA = `Workspace`. ups = 3. --> `../../Target.model.json5` + +Verifies same target gets different paths from different sources. + +#### RT-12: Duplicate-named attachments (dedup + placeholder system) + +``` +Workspace + Beams (Folder) + Att (Attachment, child1) + Att (Attachment, child2) --> Att~2 + Att (Attachment, child3) --> Att~3 + Beam (Beam) + Attachment0 → Att(child2) + Attachment1 → Att(child3) +``` + +**Expected** (`Workspace/Beams/Beam.model.json5`): + +```json5 +{ + "className": "Beam", + "attributes": { + "Rojo_Ref_Attachment0": "./Att~2.model.json5", + "Rojo_Ref_Attachment1": "./Att~3.model.json5" + } +} +``` + +Verifies source-aware placeholders produce correct per-source relative paths even with dedup. + +--- + +### Test Specification: Two-Way Sync + +#### TW-1: Ref property change writes relative path + +Setup: Serve session with tree: + +``` +Workspace + Model (Model) + Handle (Part) +``` + +Plugin sends: `Model.PrimaryPart = Handle` (Ref property change). + +**Verify** `init.meta.json5` on disk contains: + +```json5 +{ "attributes": { "Rojo_Ref_PrimaryPart": "@self/Handle.model.json5" } } +``` + +#### TW-2: Cross-service ref writes `@game/` + +Setup: + +``` +Workspace + OV (ObjectValue) +ReplicatedStorage + Target (ModuleScript) +``` + +Plugin sends: `OV.Value = Target`. + +**Verify** `OV.model.json5` contains: + +```json5 +{ "attributes": { "Rojo_Ref_Value": "@game/ReplicatedStorage/Target.luau" } } +``` + +#### TW-3: Nil ref removes attribute + +Setup: Model already has `Rojo_Ref_PrimaryPart: "@self/Handle.model.json5"` on disk. + +Plugin sends: `Model.PrimaryPart = nil`. + +**Verify**: `Rojo_Ref_PrimaryPart` is removed from the meta file. + +#### TW-4: Same-batch add + ref + +Plugin sends in one write request: + +- Add: new Part "NewPart" under Model +- Update: `Model.PrimaryPart = NewPart` + +**Verify**: meta file contains `Rojo_Ref_PrimaryPart` with relative path to the newly added instance (should be `@self/NewPart.model.json5`). + +#### TW-5: Rename target updates refs + +Setup: + +``` +Workspace + Folder + Pointer (ObjectValue, Value → OldName) + OldName (Part) +``` + +On disk: `Pointer.model.json5` has `"Rojo_Ref_Value": "./OldName.model.json5"`. + +Plugin renames `OldName` to `NewName`. + +**Verify**: `Pointer.model.json5` updated to `"Rojo_Ref_Value": "./NewName.model.json5"`. + +#### TW-6: Rename source does NOT change its own outgoing refs + +Setup: + +``` +Workspace + Folder + SourceOV (ObjectValue, Value → Target) + Target (Part) +``` + +On disk: `SourceOV.model.json5` has `"Rojo_Ref_Value": "./Target.model.json5"`. + +Plugin renames `SourceOV` to `RenamedOV`. + +**Verify**: `RenamedOV.model.json5` still has `"Rojo_Ref_Value": "./Target.model.json5"` (unchanged -- the relative relationship didn't change). + +#### TW-7: Dedup cleanup rename updates refs + +Setup: + +``` +Workspace + Folder + Data (Folder) + Data~2 (Folder) + Pointer (ObjectValue, Value → Data~2) +``` + +On disk: `Pointer.model.json5` has `"Rojo_Ref_Value": "./Data~2"`. + +Delete `Data` (the base). Dedup cleanup promotes `Data~2` --> `Data`. + +**Verify**: `Pointer.model.json5` updated to `"Rojo_Ref_Value": "./Data"`. + +#### TW-8: Base-name promotion with cross-branch ref + +Setup: + +``` +Workspace + GroupA + Pointer (ObjectValue, Value → Target) + GroupB + Target (Part) + Target~2 (Part) +``` + +On disk: `Pointer.model.json5` has `"Rojo_Ref_Value": "../GroupB/Target~2"`. + +Delete `Target` (base). Cleanup: `Target~2` --> `Target`. + +**Verify**: `Pointer.model.json5` updated to `"Rojo_Ref_Value": "../GroupB/Target"`. + +#### TW-9: Multiple referencing files with different relative paths + +Setup: + +``` +Workspace + Target (Part) + ShallowOV (ObjectValue, Value → Target) + Deep + DeepOV (ObjectValue, Value → Target) +``` + +Rename `Target` to `Renamed`. + +**Verify**: + +- `ShallowOV.model.json5`: `"Rojo_Ref_Value": "./Renamed.model.json5"` (was `./Target.model.json5`) +- `Deep/DeepOV.model.json5`: `"Rojo_Ref_Value":` ../Renamed.model.json5"`(was`../Target.model.json5`) + +Both files updated with correct relative paths from their respective positions. + +#### TW-10: Rename intermediate ancestor + +Setup: + +``` +Workspace + Parent + Child + Target (Part) + Pointer (ObjectValue, Value → Target) +``` + +On disk: `Pointer.model.json5` has `"Rojo_Ref_Value": "./Parent/Child/Target.model.json5"`. + +Rename `Parent` to `RenamedParent`. + +**Verify**: `Pointer.model.json5` updated to `"Rojo_Ref_Value": "./RenamedParent/Child/Target.model.json5"`. + +--- + +### Test Specification: Placeholder System (Syncback) + +#### PH-1: Source-aware placeholders are unique + +Build DOM with two ObjectValues pointing to the same target: + +``` +Workspace + Target (Part) + OV_A (ObjectValue, Value → Target) + OV_B (ObjectValue, Value → Target) +``` + +Call `collect_referents` without `final_paths`. + +**Verify**: + +- OV_A gets placeholder `__ROJO_REF__TO_`__ +- OV_B gets placeholder `__ROJO_REF__TO_`__ +- Placeholders are different strings +- `placeholder_to_source_and_target` maps each to `(source_ref, target_ref)` + +#### PH-2: Post-processing produces different relative paths for same target + +After walk, `ref_path_map` contains: + +- OV_A --> `Workspace/OV_A.model.json5` +- OV_B --> `Workspace/Deep/OV_B.model.json5` +- Target --> `Workspace/Target.model.json5` + +**Verify substitutions**: + +- OV_A's placeholder --> `./Target.model.json5` (sibling) +- OV_B's placeholder --> `../Target.model.json5` (2 ups) + +#### PH-3: Duplicate-named targets with source-aware placeholders + +``` +Workspace + Beams + Att (Attachment, child1) + Att (Attachment, child2) + Beam (Beam, Attachment0 → child1, Attachment1 → child2) +``` + +Both attachments are named "Att" so they get different Refs but tentatively the same path. Placeholders encode both source (Beam) and target (child1, child2) so they're unique. + +After walk with dedup: child1 = `Att.model.json5`, child2 = `Att~2.model.json5`. + +**Verify**: + +- Attachment0: `./Att.model.json5` +- Attachment1: `./Att~2.model.json5` +- No chaining corruption + +#### PH-4: With `final_paths` available, computes relative directly + +Call `collect_referents` WITH `final_paths`: + +``` +OV_A source path: "Workspace/OV_A.model.json5" +Target final path: "Workspace/Target.model.json5" +``` + +**Verify**: No placeholders generated. Path stored directly as `./Target.model.json5`. + +--- + +### Test Specification: RefPathIndex with Relative Paths + +#### RI-1: populate_from_tree indexes resolved absolute paths + +Tree has: + +``` +Workspace/Model/init.meta.json5 with Rojo_Ref_PrimaryPart: "@self/Handle.model.json5" +``` + +Source instance absolute path: `Workspace/Model`. Resolve `@self/Handle.model.json5` from Model = `Workspace/Model/Handle.model.json5`. + +**Verify**: Index contains key `Workspace/Model/Handle.model.json5` mapping to the meta file. + +#### RI-2: find_by_prefix still works with absolute keys + +After RI-1, call `find_by_prefix("Workspace/Model/Handle.model.json5")`. + +**Verify**: Returns the meta file path. + +#### RI-3: Relative refs from different files pointing to same target + +Two files: + +- `Workspace/A/Pointer.model.json5` with `"Rojo_Ref_Value": "./Target.model.json5"` (resolves to `Workspace/A/Target.model.json5`) +- `Workspace/B/Pointer.model.json5` with `"Rojo_Ref_Value": "../A/Target.model.json5"` (resolves to `Workspace/A/Target.model.json5`) + +**Verify**: Both files indexed under key `Workspace/A/Target.model.json5`. + +#### RI-4: Rename updates files with recomputed relative paths + +After RI-3, rename `Workspace/A/Target.model.json5` to `Workspace/A/Renamed.model.json5`. + +**Verify**: + +- `Workspace/A/Pointer.model.json5` updated to `"Rojo_Ref_Value": "./Renamed.model.json5"` +- `Workspace/B/Pointer.model.json5` updated to `"Rojo_Ref_Value": "../A/Renamed.model.json5"` +- Index key updated from `Workspace/A/Target.model.json5` to `Workspace/A/Renamed.model.json5` + +--- + +### Test Specification: `fix_ref_paths` (fs_snapshot.rs) + +#### FP-1: Source-aware substitution with different relative paths + +Snapshot has two meta files: + +``` +/test/ShallowOV.model.json5: + { "attributes": { "Rojo_Ref_Value": "__ROJO_REF_shallow_TO_target__" } } + +/test/Deep/DeepOV.model.json5: + { "attributes": { "Rojo_Ref_Value": "__ROJO_REF_deep_TO_target__" } } +``` + +Substitutions: + +- `__ROJO_REF_shallow_TO_target__` --> `./Target.model.json5` +- `__ROJO_REF_deep_TO_target__` --> `../Target.model.json5` + +**Verify**: Each file gets its own correct relative path. + +#### FP-2: Non-ref lines are never touched + +Meta file has a `properties.Description` containing "Rojo_Ref_" as a string value. + +**Verify**: Only lines with actual `Rojo_Ref`_ attribute keys are substituted. + +#### FP-3: Non-meta files are never touched + +A `.luau` file happens to contain a placeholder string in a comment. + +**Verify**: File is not modified. + +#### FP-4: Multiple refs in same file with mixed prefixes + +```json5 +{ + "attributes": { + "Rojo_Ref_PrimaryPart": "__ROJO_REF_model_TO_handle__", + "Rojo_Ref_Value": "__ROJO_REF_model_TO_target__" + } +} +``` + +Substitutions produce `@self/Handle.model.json5` and `@game/ReplicatedStorage/Target.luau`. + +**Verify**: Both substituted correctly in same file. + +--- + +### Test Specification: Backward Compatibility + +#### BC-1: Old bare paths resolve correctly + +Meta file on disk: `{ "attributes": { "Rojo_Ref_PrimaryPart": "Workspace/Model/Handle.model.json5" } }`. + +**Verify**: Forward sync resolves this to the correct Handle instance (bare path treated as `@game/`). + +#### BC-2: Old bare paths still work during transition + +Build from a project that has old-format `Rojo_Ref_`* values. Refs should resolve correctly. No errors or warnings that break the build. + +#### BC-3: Syncback overwrites old format with new + +Project has old bare paths on disk. Run syncback. **Verify** all `Rojo_Ref_`* values are rewritten with the correct prefix (`@self/`, `./`, `../`, or `@game/`). + +--- + +### Test Specification: Format Transition (syncback_format_transitions) + +#### FT-1: Old absolute --> new relative on re-syncback + +Project has `"Rojo_Ref_PrimaryPart": "Workspace/Model/Handle.model.json5"` (old format). + +Run syncback with same rbxl. + +**Verify**: File updated to `"Rojo_Ref_PrimaryPart": "@self/Handle.model.json5"`. + +#### FT-2: Mixed old and new formats in same project + +Some files have old bare paths, others have already been updated to new format. Run syncback. + +**Verify**: ALL files now use new format. No old bare paths remain. \ No newline at end of file diff --git a/.cursor/plans/matching_algorithm_parity_01524d4c.plan.md b/.cursor/plans/matching_algorithm_parity_01524d4c.plan.md new file mode 100644 index 00000000..e5eec1b7 --- /dev/null +++ b/.cursor/plans/matching_algorithm_parity_01524d4c.plan.md @@ -0,0 +1,275 @@ +--- +name: matching algorithm parity +overview: Fix scoring parity between the 3 matching algorithm implementations (Rust forward sync, Rust syncback, Lua plugin) and add parity/round-trip tests. +todos: + - id: fix-1-ref-scoring + content: "Fix 1: Add Ref-target-aware scoring to Rust countOwnDiffs in both forward sync and syncback matching" + status: completed + - id: fix-2-tags-attrs + content: "Fix 2: Add granular Tags/Attributes scoring to Rust countOwnDiffs in both implementations" + status: completed + - id: fix-3-epsilon + content: "Fix 3: Replace float_cmp::approx_eq! with custom fuzzy_eq matching Lua's 0.0001 epsilon in variant_eq.rs" + status: completed + - id: fix-4-sort + content: "Fix 4: Add matched_indices sort to forward sync build_result; remove TODO from syncback" + status: completed + - id: fix-5-vcache + content: "Fix 5: Cache VCache at session level in Lua matching.lua to avoid redundant decoding" + status: completed + - id: fix-6-tests + content: "Fix 6: Add parity tests (Rust forward vs syncback), round-trip matching test, and Lua parity spec test" + status: completed + - id: run-ci + content: Run /ci after all fixes to verify CI passes + status: completed +isProject: false +--- + +# Matching Algorithm Parity Audit Fixes + +> This plan was generated by `/audit` from a Plan-mode session. +> Refer to `.cursor/rules/atlas.mdc` for full project standards. + +## Standards + +These standards govern every fix in this plan. An implementer MUST read `.cursor/rules/atlas.mdc` before starting. + +### Round-Trip Identity (from `atlas.mdc` Quality Standard) + +Syncback (or two-way sync) writes a directory tree. Building an rbxl from that directory tree and forward-syncing it back must produce a **bit-identical instance tree** -- same names, same classes, same properties, same hierarchy, same ref targets. Any deviation is a bug. + +### CLI Syncback Parity (from `atlas.mdc` Two-Way Sync Strategy) + +Plugin-based sync must produce **exactly the same filesystem output** that `atlas syncback` would give for the same input. Byte-for-byte identical files, identical directory structures, identical naming. Any divergence is a bug until proven otherwise. + +### Code Quality (from `atlas.mdc` Code Quality Standard) + +DRY code that is easy to reason about and hard to get wrong. The same slugify/dedup/meta-update pattern should not be copy-pasted across files. Small refactors (2-3 call sites) are included as fixes. Major rewrites (10+ call sites) are deferred. + +## Context + +The matching algorithm was audited across all 3 parallel implementations: + +- `src/snapshot/matching.rs` (Rust forward sync) +- `src/syncback/matching.rs` (Rust syncback) +- `plugin/src/Reconciler/matching.lua` (Lua plugin) + +The audit found **scoring parity issues** that could cause different implementations to produce different pairings for the same logical input. The most critical is Ref property scoring (UNMATCHED_PENALTY in Lua vs +1 in Rust), followed by Tags/Attributes granularity and float epsilon differences. + +## Fixes + +### Fix 1: Add Ref-Target-Aware Scoring to Rust `countOwnDiffs` + +- **Status:** Approved +- **Finding:** Step 11l: Matching Algorithm Parity -- Ref property scoring +- **Files:** `src/snapshot/matching.rs`, `src/syncback/matching.rs` +- **Problem:** Lua `countOwnDiffs` (matching.lua lines 309-325) scores Ref property mismatches at `UNMATCHED_PENALTY = 10000` by comparing target Name+ClassName. Rust `count_own_diffs` in both files treats Ref properties as regular Variant properties scoring +1 per mismatch. In syncback matching, Ref IDs are always different between WeakDom instances, so ALL Ref properties score +1 regardless of logical equivalence. +- **Solution:** + - In **forward sync** `count_own_diffs` (matching.rs line 452-496): When comparing properties, detect `Variant::Ref` values. For each Ref property: + - If both sides have a Ref value, resolve both Refs to their target instances (via `tree.get_instance(ref)` for tree side, look up snapshot children by snapshot_id for snap side). Compare target Name+ClassName. If different, add `UNMATCHED_PENALTY` instead of 1. + - If one side has Ref and other doesn't (or Ref::none vs valid ref), add `UNMATCHED_PENALTY`. + - Skip `Variant::Ref` from regular property comparison to avoid double-counting. + - In **syncback** `count_own_diffs` (matching.rs line 501-548): Same approach using `new_dom.get_by_ref(ref)` and `old_dom.get_by_ref(ref)` to resolve targets. + - Both functions need access to the DOM/tree for Ref resolution -- add the necessary parameters. +- **Risk:** Adding DOM access to `count_own_diffs` increases its parameter count. The function is `#[inline]`, so ensure the extra parameter doesn't cause performance regression. The Ref resolution is a simple hash lookup (O(1)). +- **Verify round-trip:** Create 3 Models named "Weapon" with different PrimaryPart targets. Syncback, then forward sync. Verify all 3 match to their correct counterparts. +- **Verify syncback parity:** Compare CLI syncback output with two-way sync output for a tree with Ref-differentiated instances. +- **Tests required:** + - Rust unit test: `ref_scoring_forward_sync` -- 2 Models with different PrimaryPart targets, verify correct pairing + - Rust unit test: `ref_scoring_syncback` -- same scenario for syncback matching + - Lua spec test: already has ref scoring test in matching.spec.lua, verify it passes + +### Fix 2: Add Granular Tags/Attributes Scoring to Rust `countOwnDiffs` + +- **Status:** Approved +- **Finding:** Step 11l: Matching Algorithm Parity -- Tags/Attributes granularity +- **Files:** `src/snapshot/matching.rs`, `src/syncback/matching.rs` +- **Problem:** Lua `countOwnDiffs` counts per-tag (+1 each in symmetric difference) and per-attribute (+1 each diff). Rust counts Tags as one Variant property (+1 total) and Attributes as one Variant property (+1 total). An instance with 5 extra tags scores +5 in Lua but +1 in Rust. +- **Solution:** + - In both Rust `count_own_diffs` functions: + - When encountering `Variant::Tags(tags)`, extract and compare granularly: +1 per tag in symmetric difference. Skip from regular property comparison. + - When encountering `Variant::Attributes(attrs)`, extract and compare granularly: +1 per attribute whose value differs (using `variant_eq`), +1 per attribute present only on one side. Skip from regular property comparison. + - The key check is `key == "Tags"` (for `Variant::Tags`) and `key == "Attributes"` (for `Variant::Attributes`). +- **Risk:** Low. Only changes the cost value for Tags/Attributes, making it more discriminating. Existing tests should still pass since they don't depend on exact cost values, only on correct pairings. +- **Verify round-trip:** Instances differing by tags should be paired correctly. +- **Verify syncback parity:** N/A -- Tags/Attributes rarely distinguish instances in typical syncback scenarios. +- **Tests required:** + - Rust unit test: `tags_granular_scoring` -- 2 Parts with different tag sets, verify scoring matches Lua behavior + - Rust unit test: `attributes_granular_scoring` -- 2 Parts with different attribute maps, verify scoring + +### Fix 3: Loosen Rust Float Epsilon to Match Lua + +- **Status:** Approved (Modified: loosen Rust to match Lua) +- **Finding:** Step 11l: Matching Algorithm Parity -- Float epsilon +- **Files:** `src/variant_eq.rs` +- **Problem:** Lua `trueEquals` uses epsilon=0.0001 with absolute+relative check. Rust `variant_eq` uses `float_cmp::approx_eq!` defaults (epsilon~1.19e-7, ulps=4), which is ~1000x tighter. This means Rust considers values different that Lua considers equal, potentially causing different match scores and different pairings. +- **Solution:** + - Replace the `approx_eq!` macro wrapper in `variant_eq.rs` with a custom `fuzzy_eq` function that matches Lua's dual epsilon approach: + +```rust + const EPSILON: f32 = 0.0001; + fn fuzzy_eq_f32(a: f32, b: f32) -> bool { + if a.is_nan() && b.is_nan() { return true; } + let diff = (a - b).abs(); + let max_val = a.abs().max(b.abs()).max(1.0); + diff < EPSILON || diff < max_val * EPSILON + } + + +``` + +- Apply similar `fuzzy_eq_f64` for f64 comparisons. +- Update all float comparison call sites in `variant_eq` (Color3, CFrame, Vector3, UDim, etc.) to use the new function instead of `approx_eq!`. +- Remove the `float-cmp` dependency from `Cargo.toml` if no other code uses it. +- **Risk:** Medium. Loosening epsilon means values that were previously considered different are now equal. This affects: + - `variant_eq` used in matching (intended -- this fix) + - `variant_eq` used in `patch_compute.rs` for property diff detection (side effect -- could suppress legitimate property patches for near-equal values) + - Verify that the 0.0001 epsilon doesn't suppress meaningful Studio property changes. +- **Verify round-trip:** Float properties should round-trip without phantom diffs (the looser epsilon should REDUCE phantom diffs, not create them). +- **Verify syncback parity:** N/A +- **Tests required:** + - Rust unit test: update existing `variant_eq` float tests to verify the new epsilon behavior + - Rust unit test: `fuzzy_eq_matches_lua` -- verify same boundary values as `trueEquals.spec.lua` (0.0001 threshold, relative scaling, NaN handling) + +### Fix 4: Add Sort to Forward Sync `build_result` for Consistency + +- **Status:** Approved +- **Finding:** Code quality -- inconsistent sort between implementations +- **Files:** `src/snapshot/matching.rs`, `src/syncback/matching.rs` +- **Problem:** Syncback `build_result` (line 217) sorts `matched_indices` by new_ref index. Forward sync `build_result` (line 206) does NOT sort. The Lua `matchChildren` returns matched pairs in insertion order (which is group iteration order, then sorted within groups). +- **Solution:** + - Add `matched_indices.sort_by_key(|&(si, _)| si);` to the forward sync `build_result` function (line 203, before the matched Vec construction). + - Remove the TODO comment from the syncback version (line 217): the sort is intentional for consistent downstream iteration order. +- **Risk:** Very low. Only changes the ORDER of matched pairs in the result, not WHICH pairs are matched. Downstream code iterates matched pairs for patch computation, and order should not affect correctness. +- **Verify round-trip:** N/A -- order of matched pairs doesn't affect round-trip identity. +- **Verify syncback parity:** N/A +- **Tests required:** + - No new tests needed. Existing tests verify correct pairings, not ordering. + +### Fix 5: Cache VCache at Session Level in Lua Matching + +- **Status:** Approved +- **Finding:** Performance -- redundant `cacheVirtual` calls +- **Files:** `plugin/src/Reconciler/matching.lua` +- **Problem:** `cacheVirtual` is called during ambiguous group scoring (line 492) AND again inside `computePairCost` (line 362). The same virtual instance's properties are decoded multiple times per session. +- **Solution:** + - Add a `vCacheStore: { [string]: VCache }` field to `MatchingSession`. + - In `cacheVirtual`, check `session.vCacheStore[virtualId]` first. If present, return cached. Otherwise, compute and store. + - Update `computePairCost` to use the session-level cache instead of re-calling `cacheVirtual`. + - The `classKeys` parameter varies by group, but for a given virtualId the classKeys is always the same (determined by ClassName). So caching by virtualId is safe. +- **Risk:** Very low. Pure performance optimization. The VCache content is deterministic for a given virtualId. +- **Verify round-trip:** N/A +- **Verify syncback parity:** N/A +- **Tests required:** + - No new tests needed. Existing matching.spec.lua tests verify correctness. Could add a timing test but not required. + +### Fix 6: Add Parity Tests Between All 3 Implementations + +- **Status:** Approved +- **Finding:** Steps 5-6: Missing parity and round-trip tests +- **Files:** `tests/tests/matching_fixtures.rs` (extend), `plugin/src/Reconciler/matching.spec.lua` (extend) +- **Problem:** No tests verify that the 3 implementations produce identical pairings for the same input. Given Fixes 1-3 close the scoring gaps, parity tests are needed to prevent future divergence. +- **Solution:** + - **Rust integration test** (`tests/tests/matching_fixtures.rs`): Add a test `parity_forward_vs_syncback` that: + 1. Creates a WeakDom with an ambiguous group (e.g., 4 Parts named "Line" with different Transparency values, in reversed order) + 2. Runs syncback matching on `(new_children, old_children)` + 3. Builds a RojoTree from `new_children` snapshots, runs forward sync matching on `(old_children_as_snapshots, tree_children)` + 4. Verifies both produce the same pairings (by comparing matched Name+Property values) + - **Lua spec test** (`matching.spec.lua`): Add a test `parity_with_rust_fixture` that: + 1. Uses the same input data as the Rust test (4 Parts with specific Transparency values) + 2. Runs Lua matching + 3. Verifies the same pairings as the Rust tests (by property comparison) + - **Round-trip matching test** (`tests/tests/matching_fixtures.rs`): Add `round_trip_matching_identity`: + 1. Uses the UFOWave fixture (12 Textures with default-omitted Face) + 2. Runs syncback to produce filesystem + 3. Runs forward sync on the filesystem output + 4. Verifies all 12 Textures are matched to their correct counterparts (by Face value) +- **Risk:** None. Tests only. +- **Verify round-trip:** This IS the round-trip test. +- **Verify syncback parity:** N/A (these are matching-level tests, not filesystem-level) +- **Tests required:** (this fix IS the test) + +## Skipped Fixes + +None. All identified fixes were approved. + +## Accepted Limitations + +### Greedy Matching Not Globally Optimal + +- **Limitation:** The greedy assignment algorithm is not globally optimal. Edge cases exist where a locally-best pick forces a globally-worse assignment. +- **Scenario:** Large ambiguous groups (10+ instances) with specific cost patterns where the Hungarian algorithm would produce different (better) pairings. +- **User decision:** Accepted for all releases (documented in `atlas-dedup.mdc`). +- **Mitigation:** Ambiguous groups are typically small (2-10 instances). The depth limit (MAX_SCORING_DEPTH=3) bounds worst-case performance. Monitoring via test coverage for large groups. + +## Deferred Refactors + +- **Shared matching core:** The 3 implementations share the same algorithm but differ in input types (InstanceSnapshot vs WeakDom vs Studio Instance). A generic/trait-based approach could consolidate the duplicated logic (~2700 total lines), but the type differences are fundamental to each context. Estimated scope: 10+ files, significant API redesign. Deferred. +- **Duplicated grouping pattern:** All 3 implementations repeat the same HashMap-based `(Name, ClassName) -> Vec` grouping. Could be factored into a shared helper, but types differ. Minor DRY violation. Deferred. + +## Test Plan + +### Rust Unit Tests (`#[cfg(test)]` blocks) + +- `ref_scoring_forward_sync` -- 2 Models with different PrimaryPart targets, verify correct pairing (Fix 1) +- `ref_scoring_syncback` -- same scenario for syncback matching (Fix 1) +- `tags_granular_scoring` -- 2 Parts with different tag sets, verify scoring (Fix 2) +- `attributes_granular_scoring` -- 2 Parts with different attribute maps, verify scoring (Fix 2) +- Update existing `variant_eq` float tests for new epsilon behavior (Fix 3) +- `fuzzy_eq_matches_lua` -- verify same boundary values as `trueEquals.spec.lua` (Fix 3) + +### Rust Integration Tests (`tests/tests/`) + +- `parity_forward_vs_syncback` -- same ambiguous group through both Rust matchers, verify identical pairings (Fix 6) +- `round_trip_matching_identity` -- UFOWave fixture through syncback then forward sync, verify correct pairings (Fix 6) + +### Lua Spec Tests (`.spec.lua` files) + +- `parity_with_rust_fixture` -- same input as Rust parity test, verify identical pairings (Fix 6) + +### Snapshot Tests (`insta` crate) + +- None needed for matching fixes. + +### Failing Tests for Known Limitations + +- None. The greedy limitation is accepted and no test is needed to demonstrate it. + +## Test Rules + +The implementer MUST follow these rules when writing tests for this plan: + +### For bugs found and fixed + +Every bug fix MUST have a test that: + +1. Would have **FAILED** before the fix +2. **PASSES** after the fix +3. Prevents the bug from regressing + +### For missing coverage identified + +For each "missing test coverage" gap: + +1. Write the test +2. Verify it passes against the current implementation +3. If it fails, investigate -- it may have found another bug + +### For known limitations (Accepted Limitations section) + +No known-limitation tests needed for this plan. + +### Scope: test every affected layer + +For each finding, determine which test layers are affected and write tests in ALL of them: + +- Does the bug manifest at the plugin level? -> Lua spec test +- Does it manifest at the server level? -> Rust unit test or integration test +- Does it affect the round-trip? -> Integration test with `/api/write` + filesystem verification +- Does it affect forward-sync? -> Serve test with `recv_socket_packet` +- Could it regress from a different code path? -> Test that code path too + +## Final Step: Run CI + +After ALL fixes and tests are implemented, run the `/ci` command (see `.cursor/commands/ci.md`) to execute the full CI pipeline. Every fix must pass CI before the plan is considered complete. Do not skip this step. \ No newline at end of file diff --git a/.cursor/plans/matching_cost_semantics_376e45e2.plan.md b/.cursor/plans/matching_cost_semantics_376e45e2.plan.md new file mode 100644 index 00000000..ce8ad41c --- /dev/null +++ b/.cursor/plans/matching_cost_semantics_376e45e2.plan.md @@ -0,0 +1,324 @@ +--- +name: matching cost semantics +overview: "Split `variant_eq` into two variants: keep fuzzy comparison for Studio-facing operations (patch_compute, api, property_filter), and add a new disk-representation comparison for both Rust matching files where cost should reflect actual file changes." +todos: + - id: expose-fns + content: Make format_f32/format_f64 pub(crate) in json.rs and cleanup_f32 pub(crate) in resolution.rs + status: completed + - id: disk-eq-helpers + content: Add disk_eq_f32, disk_eq_f32_cleaned, disk_eq_f64 zero-allocation helpers to variant_eq.rs + status: completed + - id: variant-eq-disk + content: Implement variant_eq_disk in variant_eq.rs dispatching per Variant type (Float32 standalone vs composite cleanup) + status: completed + - id: update-fwd-matching + content: Replace variant_eq with variant_eq_disk in src/snapshot/matching.rs (count_own_diffs, diff_variant_pair, count_attributes_diff) + status: completed + - id: update-sb-matching + content: Replace variant_eq with variant_eq_disk in src/syncback/matching.rs (count_own_diffs, diff_variant_pair, attribute diff) + status: completed + - id: tests-veq + content: "Add variant_eq_disk unit tests: fuzzy-vs-disk divergence, cleanup normalization, NaN/Inf/-0, all composite types" + status: completed + - id: tests-fwd-large + content: Add 50+ same-named instance matching tests to src/snapshot/matching.rs with 5 variation groups + status: completed + - id: tests-sb-large + content: Add 50+ same-named instance matching tests to src/syncback/matching.rs with 5 variation groups + status: completed + - id: tests-disk-boundary + content: Add tests with values at format_f32 boundaries (close-but-different-on-disk, different-but-same-on-disk) + status: completed + - id: tests-update-existing + content: Update existing matching test assertions to match variant_eq_disk semantics + status: completed +isProject: false +--- + +# Matching Cost Semantics Fix + +## Problem + +`variant_eq` currently uses fuzzy epsilon comparison (absolute + relative, `EPSILON = 0.0001`) in both Rust matching files. This is wrong because: + +- **Rust cost = "will this value result in a file change?"** Two values that format identically on disk should score 0 cost, even if their raw f32 bits differ. Conversely, two values within epsilon but with different disk representations should score 1. +- **Lua cost = "will this change be applied in Studio?"** Fuzzy epsilon is correct here -- it prevents oscillation from Studio float precision rounding. + +The fuzzy epsilon can diverge from disk reality in both directions: + +- `format_f32(1.00005)` = `"1.00005"`, `format_f32(1.00015)` = `"1.00015"` -- different on disk, but `fuzzy_eq_f32` says "equal" (diff = 0.0001) +- `format_f32(0.123456)` could round both `0.1234561` and `0.1234559` to the same string -- same on disk, but `fuzzy_eq_f32` might say "different" + +## Data Flow Analysis + +### Disk serialization pipeline (how floats reach files) + +``` +Variant value + | + v +resolution.rs::from_variant() + |-- Standalone Float32 → AmbiguousValue::Number32(v) [NO cleanup] + |-- Standalone Float64 → AmbiguousValue::Number(v as f64) [NO cleanup] + |-- Composite types (Vector3, Color3, CFrame, UDim, etc.) + | → cleanup_f32(component) then into Array/FullyQualified + v +json.rs::Json5ValueSerializer + |-- serialize_f32 → format_f32(v) [6 significant digits, lexical-write-float] + |-- serialize_f64 → format_f64(v) [full precision] + v +.meta.json5 / .model.json5 on disk +``` + +`**cleanup_f32` matters for composites.** It rounds to 6 decimal places (via `format!("{:.6}")` round-trip), which zeros out very small values. Example: `cleanup_f32(0.0000001)` = `0.0`, so `format_f32(0.0)` = `"0"` vs `format_f32(0.0000001)` = `"1e-7"` -- completely different disk output. + +### Where values come from in each matching file + +`**[src/syncback/matching.rs](src/syncback/matching.rs)`:** + +- `new_dom` = from rbxl/rbxm binary parse. Raw Studio values, NOT through resolution.rs +- `old_dom` = from filesystem build. Values already in disk-normalized form + +`**[src/snapshot/matching.rs](src/snapshot/matching.rs)`:** + +- `snap` = from filesystem re-snapshot. Values read from disk, already in disk form +- `tree` = from RojoTree. Could be from initial forward sync (disk form) or two-way sync push (raw Studio values) + +In both cases, at least one side may not have been through `cleanup_f32 + format_f32`. The comparison must normalize both sides through the full disk pipeline. + +### Other callers of `variant_eq` (KEEP fuzzy -- different semantics) + + +| File | Purpose | Why fuzzy is correct | +| ------------------------------------------------------- | ----------------------------------- | -------------------------------------------------------------- | +| `[patch_compute.rs](src/snapshot/patch_compute.rs)` | Compute patches to send to plugin | Must agree with Lua `diff.lua` (which uses fuzzy `trueEquals`) | +| `[api.rs](src/web/api.rs)` | Filter defaults during two-way sync | "Is this at default?" -- not a disk question | +| `[property_filter.rs](src/syncback/property_filter.rs)` | Filter defaults during syncback | Same rationale | +| `[hash/mod.rs](src/syncback/hash/mod.rs)` | Exclude defaults from content hash | Same rationale | +| `[project.rs](src/snapshot_middleware/project.rs)` | Check project node value drift | Same rationale | + + +## Implementation + +### 1. Expose formatting functions + +- `**[src/json.rs](src/json.rs)`:** Make `format_f32`, `format_f64` and the `Options` constants `pub(crate)` +- `**[src/resolution.rs](src/resolution.rs)`:** Make `cleanup_f32` `pub(crate)` + +### 2. Add disk-representation comparison to `[src/variant_eq.rs](src/variant_eq.rs)` + +Add zero-allocation comparison helpers that write to stack buffers: + +```rust +fn disk_eq_f32(a: f32, b: f32) -> bool { + if a.to_bits() == b.to_bits() { return true; } + // format both into stack buffers, compare byte slices +} + +fn disk_eq_f32_cleaned(a: f32, b: f32) -> bool { + disk_eq_f32(cleanup_f32(a), cleanup_f32(b)) +} + +fn disk_eq_f64(a: f64, b: f64) -> bool { + if a.to_bits() == b.to_bits() { return true; } + // format both into stack buffers, compare byte slices +} +``` + +Then `pub fn variant_eq_disk(a: &Variant, b: &Variant) -> bool` that dispatches: + + +| Variant type | Comparison | +| ----------------------------------------------- | -------------------------------------------------- | +| Float32 | `disk_eq_f32` (no cleanup -- standalone) | +| Float64 | `disk_eq_f64` (no cleanup) | +| Vector3, Color3, CFrame components | `disk_eq_f32_cleaned` (cleanup -- composite) | +| UDim scale / UDim2 scales | `disk_eq_f32_cleaned` (cleanup) | +| NumberRange, Rect min/max | `disk_eq_f32_cleaned` (cleanup) | +| NumberSequence, ColorSequence keypoint floats | `disk_eq_f32_cleaned` (cleanup) | +| PhysicalProperties floats | `disk_eq_f32_cleaned` (cleanup) | +| Ray, Region3 components | `disk_eq_f32_cleaned` (cleanup) | +| Non-float types (Bool, Int, String, Enum, etc.) | Exact equality (no epsilon) | +| Tags | Sorted set comparison (same as now) | +| Attributes | Recursive with `variant_eq_disk` for nested values | + + +### 3. Update both matching files + +`**[src/snapshot/matching.rs](src/snapshot/matching.rs)`:** Replace all `variant_eq` calls in `count_own_diffs`, `diff_variant_pair`, and `count_attributes_diff` with `variant_eq_disk`. + +`**[src/syncback/matching.rs](src/syncback/matching.rs)`:** Same replacements in `count_own_diffs`, `diff_variant_pair`, and attribute comparison. + +### 4. Lua side -- no changes needed (verified) + +The plugin's cost model is correct as-is: + +- `trueEquals` is shared by both `matching.lua` (scoring) and `diff.lua` (patch generation), so they cannot disagree -- if matching says "equal", diff will also say "equal", and the change won't appear in the patch visualizer +- The fuzzy epsilon prevents Studio float-precision oscillation (set 0.5 -> Studio stores 0.49999999 -> reads back as ~0.5 -> trueEquals says "equal" -> no phantom re-diff) +- `matching.lua`'s `countOwnDiffs` fills defaults from `classKeys.defaults` for missing virtual properties, matching what diff/hydrate will do + +The Lua side's cost is correctly "will this change be flagged in the patch visualizer / applied to Studio." + +### 5. Testing Suite + +Existing `variant_eq` tests stay unchanged (they test the fuzzy function which is still used by patch_compute, api, etc.). All new tests below are additive. + +#### 5a. `variant_eq_disk` unit tests (in `src/variant_eq.rs`) + +**Fuzzy-vs-disk divergence tests** -- these are the cases that prove fuzzy is wrong for file-change detection: + +```rust +// Fuzzy says "equal" but disk would differ +assert!(fuzzy_eq_f32(1.00005, 1.00015)); // within epsilon +assert!(!disk_eq_f32(1.00005, 1.00015)); // format_f32 gives different strings + +// Fuzzy says "different" but disk would be identical +assert!(!fuzzy_eq_f32(0.1234561, 0.1234559)); // outside relative epsilon +assert!(disk_eq_f32(0.1234561, 0.1234559)); // format_f32 rounds both the same +``` + +**cleanup_f32 normalization tests** -- composite types get cleaned before formatting: + +```rust +// Very small values zeroed by cleanup +assert!(disk_eq_f32_cleaned(0.0000001, 0.0)); // cleanup makes both 0.0 +assert!(!disk_eq_f32(0.0000001, 0.0)); // without cleanup, different + +// Full Vector3 comparison +assert!(variant_eq_disk( + &Variant::Vector3(Vector3::new(1.0, 2.0, 0.0000001)), + &Variant::Vector3(Vector3::new(1.0, 2.0, 0.0)), +)); // cleanup zeros the tiny z component +``` + +**Edge cases:** + +- NaN == NaN (both format to "NaN") +- Inf == Inf (both format to "Infinity") +- -0.0 == 0.0 (both format to "0") +- All composite types: Vector2, Vector3, Color3, CFrame, UDim, UDim2, Rect, Ray, Region3, NumberRange, NumberSequence, ColorSequence, PhysicalProperties +- Non-float types: exact equality (Bool, Int32, Int64, String, Enum, BrickColor) +- Tags: sorted set comparison +- Attributes: recursive disk comparison of nested Variants + +#### 5b. Large ambiguous group matching -- forward sync (`src/snapshot/matching.rs`) + +**Test: `fifty_same_name_parts_five_variation_groups`** + +50 Parts all named `"Line"` with 5 variation groups of 10 exact duplicates each: + + +| Group | Position | Color | Count | +| ----- | ------------ | ------------------- | ----- | +| A | `[0, 0, 0]` | `[1, 0, 0]` (red) | 10 | +| B | `[0, 5, 0]` | `[1, 0, 0]` (red) | 10 | +| C | `[0, 10, 0]` | `[0, 1, 0]` (green) | 10 | +| D | `[0, 0, 0]` | `[0, 1, 0]` (green) | 10 | +| E | `[0, 5, 0]` | `[0, 0, 1]` (blue) | 10 | + + +- Groups A vs D: same position, different color (color-only diff) +- Groups A vs B: same color, different position (position-only diff) +- Groups B vs E: same position, different color +- Within each group: 10 exact duplicates (any pairing is correct) + +Build snapshots in order A,B,C,D,E. Build tree in **reversed** order E,D,C,B,A. + +**Assertions:** + +1. All 50 matched, 0 unmatched on either side +2. For every matched pair: `Position` values match (within disk equality) +3. For every matched pair: `Color` values match (within disk equality) +4. Completes in < 5 seconds (depth limit prevents exponential blowup) + +**Test: `fifty_parts_position_only_variations`** + +50 Parts named `"Block"` -- same Color on all, 5 distinct Position values (10 duplicates each). Tree reversed. Verify all 50 pair correctly by Position. + +**Test: `fifty_parts_color_only_variations`** + +50 Parts named `"Block"` -- same Position on all, 5 distinct Color values (10 duplicates each). Tree reversed. Verify all 50 pair correctly by Color. + +**Test: `sixty_parts_with_near_float_values`** + +60 Parts named `"Segment"` with Transparency values chosen at `format_f32` boundaries: + + +| Group | Transparency | Count | Notes | +| ----- | ------------ | ----- | ------------------------------------------------ | +| F | `0.100000` | 12 | disk repr: `"0.1"` | +| G | `0.100001` | 12 | disk repr: `"0.100001"` -- different from F | +| H | `0.500000` | 12 | disk repr: `"0.5"` | +| I | `0.500001` | 12 | disk repr differs from H | +| J | `0.999999` | 12 | disk repr rounds to same or different from `1.0` | + + +The exact Transparency values will be chosen by testing `format_f32` output during implementation to find values that: + +- Are within fuzzy epsilon of each other but format differently (must NOT cross-pair) +- Are outside fuzzy epsilon but format the same (must cross-pair) + +This test specifically catches the bug where fuzzy equality would match Group F with Group G instances, causing 12 unnecessary file rewrites. + +#### 5c. Large ambiguous group matching -- syncback (`src/syncback/matching.rs`) + +Same test matrix as 5b but using `WeakDom` + `InstanceBuilder`: + +**Test: `fifty_same_name_parts_five_groups_syncback`** + +Same 5 variation groups (A-E) of 10 Parts each. `new_dom` in order, `old_dom` reversed. + +```rust +let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); +let new_root = new_dom.root_ref(); +for (pos, color) in &groups { + for _ in 0..10 { + new_dom.insert(new_root, InstanceBuilder::new("Part") + .with_name("Line") + .with_property("Position", Variant::Vector3(*pos)) + .with_property("Color", Variant::Color3(*color))); + } +} +``` + +Assertions identical to 5b: all 50 paired, Position+Color match on every pair. + +**Test: `sixty_parts_near_float_syncback`** + +Same boundary test as 5b's `sixty_parts_with_near_float_values`, but in syncback context. + +#### 5d. Disk boundary stress test + +**Test: `disk_representation_boundary_matching`** (in both matching files) + +Explicitly construct instances where fuzzy and disk comparison disagree, and verify matching picks the disk-correct pairing: + +2 Parts named `"Edge"`: + +- Snap/New A: Transparency = `1.00005` (format_f32 = `"1.00005"`) +- Snap/New B: Transparency = `1.00015` (format_f32 = `"1.00015"`) + +Tree/Old (reversed): + +- Tree/Old X: Transparency = `1.00015` +- Tree/Old Y: Transparency = `1.00005` + +With fuzzy equality: both (A,X) and (A,Y) score 0 (within epsilon) -- random pairing, 50% chance of wrong. +With disk equality: (A,Y) scores 0 and (A,X) scores 1 -- deterministically correct. + +Verify A matches Y and B matches X. + +#### 5e. Update existing tests + +Existing matching tests that assert `variant_eq` behavior in matching contexts will continue to pass because: + +- Most use exact integer/enum values (not affected by fuzzy vs disk) +- Float tests use values that are either identical or far apart + +Tests that use float values at fuzzy epsilon boundaries may need value adjustments if the old assertion relied on fuzzy tolerance. Specifically review: + +- `many_same_name_parts_matched_by_properties` (uses `i * 0.1` increments -- safe, these format distinctly) +- `ten_ambiguous_textures_stress` (uses Enum values -- safe, not float) +- `variant_eq_float32_with_new_epsilon` (tests `variant_eq` not `variant_eq_disk` -- unchanged) + +No existing test should break, but verify during implementation. \ No newline at end of file diff --git a/.cursor/plans/matching_session_caching_cbfa7bda.plan.md b/.cursor/plans/matching_session_caching_cbfa7bda.plan.md new file mode 100644 index 00000000..5a3c7eb0 --- /dev/null +++ b/.cursor/plans/matching_session_caching_cbfa7bda.plan.md @@ -0,0 +1,609 @@ +--- +name: Matching session caching +overview: Rewrite the Lua matching module for native codegen performance (--!strict, --!native, --!optimize 2) with session-based caching, clean 2-function mutual recursion, concrete types, and local functions for inlining. Add session caching to both Rust implementations for consistency. +todos: + - id: lua-trueEquals + content: "Rewrite trueEquals.lua: --!strict/--!native/--!optimize 2, single typeof dispatch, dedicated helper functions per type (no temp arrays), adopt DeepEqualsPureUnsafe pattern for table comparison (rawlen/rawget/next, no checkedKeys set), keep fuzzy equality + null-ref + EnumItem cross-type" + status: completed + - id: lua-rewrite + content: "Rewrite matching.lua: --!strict/--!native/--!optimize 2, all functions as local function in dependency order (one forward declare for mutual recursion), 2-function mutual recursion (matchChildren + computePairCost), session caching, totalCost return, O(n) removeMatched, remove pcall in grouping" + status: completed + - id: lua-hydrate-threading + content: Thread session through hydrate.lua, Reconciler/init.lua, and ServeSession.lua (create session before hydrate call at line 628) + status: completed + - id: lua-verify-codegen + content: Run luau-compile --remarks on trueEquals.lua and matching.lua to verify inlining and native codegen quality, iterate until clean + status: completed + - id: lua-test + content: Build plugin, test Lua changes in Studio (hydrate + matching works correctly with session caching) before proceeding to Rust + status: completed + - id: rust-forward-session + content: "AFTER Lua passes: Add MatchingSession to src/snapshot/matching.rs with cost_cache, add total_cost to ForwardMatchResult, thread session through all functions and patch_compute.rs" + status: completed + - id: rust-syncback-session + content: "AFTER Lua passes: Add MatchingSession to src/syncback/matching.rs with cost_cache, add total_cost to MatchResult, thread session through all functions and dir.rs" + status: completed + - id: rust-test-updates + content: Update tests/tests/matching_fixtures.rs, run cargo test to verify Rust changes + status: completed +isProject: false +--- + +# Matching Session Caching + Native Codegen Rewrite + +## Problem + +1. **Redundant computation**: Scoring recurses into children, then hydrate re-calls matchChildren on the same children. The intermediate `_computeChildrenCost` calls matchChildren, iterates matched pairs calling `_computeChangeCount`, which recurses again -- all discarded after scoring. +2. **Poor native codegen structure**: All functions are on the `Matching` table (table lookup per call, uninlineable). No type annotations. No `--!strict`/`--!native`/`--!optimize 2`. A `pcall` with closure allocation in the hot grouping loop. +3. **4-function recursion chain** (`matchChildren -> _computeChildrenCost -> _computeChangeCount -> _computeChildrenCost -> matchChildren`) is complex and hard for both the optimizer and the developer to reason about. + +## Design + +### Conceptual Model + +`matchChildren` is fundamentally `computeSubtreeCost` -- it finds the cost-minimizing pairing of children and returns the total cost. The actual match assignments are a bonus output. The 2-function mutual recursion reflects this: + +- `**matchChildren(session, vChildren, sChildren, virtualInstances, parentVId, parentSInst)`**: Given two child sets, find optimal pairing. Returns `MatchResult` with `totalCost`. +- `**computePairCost(session, virtualId, studioInstance, virtualInstances, bestSoFar)`**: Given one virtual and one studio instance, compute total cost including subtree. Calls `matchChildren` for children cost. + +`_computeChildrenCost` and `_computeChangeCount` are eliminated. Their logic is absorbed into these two functions. + +### MatchingSession + +```lua +type MatchingSession = { + matchCache: { [string]: { [Instance]: MatchResult } }, + costCache: { [string]: { [Instance]: number } }, +} +``` + +- **matchCache**: keyed by `(parentVirtualId, parentStudioInstance)`. Parent uniquely determines children set. +- **costCache**: keyed by `(virtualId, studioInstance)`. Only stored when `cost < bestSoFar` (exact, not early-exited). + +Caching replaces the need for a depth limit: each unique pair/children-set is computed at most once. + +### bestSoFar Interaction + +- **costCache**: only store when computation was complete (`cost < bestSoFar`). If cached, return immediately regardless of caller's `bestSoFar` (the cached value IS the exact cost). +- **matchCache**: `matchChildren` always completes (no `bestSoFar` on it), so results are always cacheable. + +--- + +## Lua Plugin Rewrite: [matching.lua](plugin/src/Reconciler/matching.lua) + +### Module Header + +```lua +--!strict +--!native +--!optimize 2 +``` + +### Type Definitions (all concrete, minimal `any`) + +```lua +type MatchPair = { + virtualId: string, + studioInstance: Instance, +} + +type MatchResult = { + matched: { MatchPair }, + unmatchedVirtual: { string }, + unmatchedStudio: { Instance }, + totalCost: number, +} + +type MatchingSession = { + matchCache: { [string]: { [Instance]: MatchResult } }, + costCache: { [string]: { [Instance]: number } }, +} + +type VCache = { + props: { [string]: any }, + extraProps: { string }?, + tags: { [string]: boolean }?, + attrs: { [string]: any }?, + childCount: number, +} + +type SCache = { + instance: Instance, + props: { [string]: any }, + tags: { [string]: boolean }?, + attrs: { [string]: any }?, + children: { Instance }, + childCount: number, +} +``` + +### All Functions as `local function` in Dependency Order + +Every function is a `local function` defined in dependency order. No `Matching.foo()` table lookups. One forward declaration for the mutual recursion pair. + +```lua +local UNMATCHED_PENALTY: number = 10000 + +local function newSession(): MatchingSession + return { matchCache = {}, costCache = {} } +end + +local function cacheVirtual(vInst: any, classKeys: any): VCache + -- same logic as current _cacheVirtual, with return type annotation +end + +local function cacheStudio(studioInstance: Instance, classKeys: any, extraProps: { string }?): SCache + -- same logic as current _cacheStudio + -- ADDITION: store sCache.instance = studioInstance +end + +local function countOwnDiffs(vCache: VCache, sCache: SCache, classKeys: any): number + -- same logic as current _countOwnDiffsCached +end + +local function removeMatched(arr: { any }, matchedIndices: { [number]: boolean }): () + -- O(n) in-place compaction instead of O(n^2) table.remove loop + local write = 1 + for read = 1, #arr do + if not matchedIndices[read] then + arr[write] = arr[read] + write += 1 + end + end + for i = write, #arr do + arr[i] = nil + end +end +``` + +### Single Forward Declaration for Mutual Recursion + +`computePairCost` calls `matchChildren` and vice versa. Define `matchChildren` first (forward-declared), then `computePairCost` as a `local function`, then assign `matchChildren`: + +```lua +local matchChildren -- forward declare (matchChildren is defined after computePairCost) + +local function computePairCost(...) + -- can reference matchChildren (captured as upvalue, assigned before first call) +end + +matchChildren = function(...) + -- can reference computePairCost (already defined above) +end +``` + +### computePairCost (replaces `_computeChangeCount` + `_computeChildrenCost`) + +```lua +computePairCost = function( + session: MatchingSession, + virtualId: string, + studioInstance: Instance, + virtualInstances: { [string]: any }, + bestSoFar: number +): number + -- 1. Cache lookup + local vc = session.costCache[virtualId] + if vc then + local cached = vc[studioInstance] + if cached ~= nil then return cached end + end + + local vInst = virtualInstances[virtualId] + if not vInst then return UNMATCHED_PENALTY end + + -- 2. Own diffs + local classKeys = RbxDom.getClassComparisonKeys(vInst.ClassName) + local vCache = cacheVirtual(vInst, classKeys) + local sCache = cacheStudio(studioInstance, classKeys, vCache.extraProps) + local cost: number = countOwnDiffs(vCache, sCache, classKeys) + if cost >= bestSoFar then return cost end + + -- 3. Children cost (direct matchChildren call, no intermediate function) + local vChildren = vInst.Children + local studioKids = sCache.children + + if (not vChildren or #vChildren == 0) and #studioKids == 0 then + -- leaf: no children on either side + elseif not vChildren or #vChildren == 0 then + cost += #studioKids * UNMATCHED_PENALTY + elseif #studioKids == 0 then + for _, childId in vChildren do + if virtualInstances[childId] then cost += UNMATCHED_PENALTY end + end + else + local validVChildren: { string } = {} + for _, childId in vChildren do + if virtualInstances[childId] then + table.insert(validVChildren, childId) + end + end + local childResult = matchChildren( + session, validVChildren, studioKids, virtualInstances, + virtualId, studioInstance + ) + cost += childResult.totalCost + end + + -- 4. Cache if exact (no early exit) + if cost < bestSoFar then + if not session.costCache[virtualId] then + session.costCache[virtualId] = {} + end + session.costCache[virtualId][studioInstance] = cost + end + + return cost +end +``` + +### matchChildren (restructured, returns totalCost) + +```lua +matchChildren = function( + session: MatchingSession, + virtualChildren: { string }, + studioChildren: { Instance }, + virtualInstances: { [string]: any }, + parentVirtualId: string?, + parentStudioInstance: Instance? +): MatchResult + -- 1. Cache lookup + if parentVirtualId and parentStudioInstance then + local pc = session.matchCache[parentVirtualId] + if pc then + local cached = pc[parentStudioInstance] + if cached then return cached end + end + end + + local matched: { MatchPair } = {} + local remainingVirtual: { string } = table.clone(virtualChildren) + local remainingStudio: { Instance } = table.clone(studioChildren) + + -- 2. Group by (Name, ClassName) -- NO pcall, direct property access + local vByKey: { [string]: { number } } = {} + for i, id in remainingVirtual do + local vInst = virtualInstances[id] + if vInst then + local key = vInst.Name .. "\0" .. vInst.ClassName + local group = vByKey[key] + if not group then + group = {} + vByKey[key] = group + end + table.insert(group, i) + end + end + + local sByKey: { [string]: { number } } = {} + for i, inst in remainingStudio do + local key = inst.Name .. "\0" .. inst.ClassName + local group = sByKey[key] + if not group then + group = {} + sByKey[key] = group + end + table.insert(group, i) + end + + -- 3. 1:1 instant match + local matchedV: { [number]: boolean } = {} + local matchedS: { [number]: boolean } = {} + + for key, vIndices in vByKey do + local sIndices = sByKey[key] + if sIndices and #vIndices == 1 and #sIndices == 1 then + local vi, si = vIndices[1], sIndices[1] + if not matchedV[vi] and not matchedS[si] then + table.insert(matched, { + virtualId = remainingVirtual[vi], + studioInstance = remainingStudio[si], + }) + matchedV[vi] = true + matchedS[si] = true + end + end + end + + removeMatched(remainingVirtual, matchedV) + removeMatched(remainingStudio, matchedS) + + -- 4. Ambiguous groups (same structure as before but using computePairCost) + if #remainingVirtual > 0 and #remainingStudio > 0 then + -- Rebuild groups, pre-compute caches, N*M scoring... + -- Score via: ownDiffs (from pre-computed caches) + childrenCost + -- For childrenCost: call computePairCost (which calls matchChildren recursively) + -- BUT: the ambiguous loop uses pre-computed caches for ownDiffs directly + -- (cacheVirtual/cacheStudio/countOwnDiffs), then adds children cost via + -- a direct matchChildren call (same as computePairCost but with caches already built) + -- + -- The inner scoring for an ambiguous (vi, si) pair: + -- cost = countOwnDiffs(vCaches[vi], sCaches[si], classKeys) + -- if cost < bestSoFar then + -- local vInst = virtualInstances[remainingVirtual[vi]] + -- local studioKids = sCaches[si].children + -- -- filter valid virtual children, call matchChildren, add totalCost + -- end + -- + -- Greedy assign sorted by cost ascending (tie-break by insertion index) + end + + -- 5. Compute totalCost for ALL matched pairs (uses session cache) + local totalCost: number = 0 + for _, pair in matched do + totalCost += computePairCost( + session, pair.virtualId, pair.studioInstance, virtualInstances, math.huge + ) + end + totalCost += (#remainingVirtual + #remainingStudio) * UNMATCHED_PENALTY + + local result: MatchResult = { + matched = matched, + unmatchedVirtual = remainingVirtual, + unmatchedStudio = remainingStudio, + totalCost = totalCost, + } + + -- 6. Cache result + if parentVirtualId and parentStudioInstance then + if not session.matchCache[parentVirtualId] then + session.matchCache[parentVirtualId] = {} + end + session.matchCache[parentVirtualId][parentStudioInstance] = result + end + + return result +end +``` + +### pcall Removal in Studio Grouping + +The current code wraps `inst.Name, inst.ClassName` in a `pcall(function() ... end)`. This allocates a closure per studio child and prevents inlining. During matching, studio children come from `:GetChildren()` on a live instance within a single-threaded sync frame -- they cannot be destroyed mid-operation. Replace with direct property access. + +### O(n) removeMatched + +Current `_removeMatched` uses `table.remove` in a reverse loop -- O(n^2) due to element shifting. Replace with single-pass in-place compaction (shown above). + +### Module Export + +```lua +return { + newSession = newSession, + matchChildren = matchChildren, +} +``` + +Only two symbols exported. All helpers are local (invisible outside module, inlineable by compiler). + +--- + +## Lua Callers + +### [hydrate.lua](plugin/src/Reconciler/hydrate.lua) + +```lua +local Matching = require(script.Parent.matching) + +local function hydrate(instanceMap, virtualInstances, rootId, rootInstance, session) + -- ... + local result = Matching.matchChildren( + session, validVirtualIds, existingChildren, virtualInstances, + rootId, rootInstance + ) + for _, pair in result.matched do + hydrate(instanceMap, virtualInstances, pair.virtualId, pair.studioInstance, session) + end + -- ... +end +``` + +### [Reconciler/init.lua](plugin/src/Reconciler/init.lua) + +```lua +function Reconciler:hydrate(virtualInstances, rootId, rootInstance, session) + Timer.start("Reconciler:hydrate") + local result = hydrate(self.__instanceMap, virtualInstances, rootId, rootInstance, session) + Timer.stop() + return result +end +``` + +### [ServeSession.lua](plugin/src/ServeSession.lua) + +Create session before hydrate at line 628: + +```lua +local Matching = require(script.Parent.Reconciler.matching) +-- ... +local session = Matching.newSession() +self.__reconciler:hydrate(readResponseBody.instances, serverInfo.rootInstanceId, game, session) +``` + +--- + +## trueEquals Rewrite: [trueEquals.lua](plugin/src/Reconciler/trueEquals.lua) + +### Module Header + +```lua +--!strict +--!native +--!optimize 2 +``` + +### Problems in Current Implementation + +1. **Redundant type dispatch**: Every branch checks both `typeA` and `typeB` (`typeA == "CFrame" and typeB == "CFrame"`). Check type once, early-out if they differ. +2. **Heap allocations in hot path**: `{ a:GetComponents() }`, `{ a.X, a.Y, a.Z }`, `{ a.X, a.Y }` build temporary arrays for every CFrame/Vector3/Vector2 comparison. +3. `**checkedKeys` set**: Allocates a table for every table-vs-table comparison. Unnecessary -- second loop only needs to check for extra keys in `b`. +4. **No type annotations**, no `--!strict`, no `--!native`, no `--!optimize 2`. + +### Structure: Single typeof Dispatch + Dedicated Helpers + +Adopt the [DeepEqualsPureUnsafe](escape-tsunami reference) pattern: `rawequal` identity check, `rawlen` for quick array length, `rawget`/`next` for table traversal, no temporary allocations. Bake in fuzzy equality for the matching use case. + +```lua +local EPSILON: number = 0.0001 + +local function fuzzyEq(a: number, b: number): boolean + local diff = math.abs(a - b) + local maxVal = math.max(math.abs(a), math.abs(b), 1) + return diff < EPSILON or diff < maxVal * EPSILON +end + +local function color3Eq(a: Color3, b: Color3): boolean + return math.floor(a.R * 255) == math.floor(b.R * 255) + and math.floor(a.G * 255) == math.floor(b.G * 255) + and math.floor(a.B * 255) == math.floor(b.B * 255) +end + +local function vector2Eq(a: Vector2, b: Vector2): boolean + return fuzzyEq(a.X, b.X) and fuzzyEq(a.Y, b.Y) +end + +local function vector3Eq(a: Vector3, b: Vector3): boolean + return fuzzyEq(a.X, b.X) and fuzzyEq(a.Y, b.Y) and fuzzyEq(a.Z, b.Z) +end + +local function cframeEq(a: CFrame, b: CFrame): boolean + local ax, ay, az, aR00, aR01, aR02, aR10, aR11, aR12, aR20, aR21, aR22 = a:GetComponents() + local bx, by, bz, bR00, bR01, bR02, bR10, bR11, bR12, bR20, bR21, bR22 = b:GetComponents() + return fuzzyEq(ax, bx) and fuzzyEq(ay, by) and fuzzyEq(az, bz) + and fuzzyEq(aR00, bR00) and fuzzyEq(aR01, bR01) and fuzzyEq(aR02, bR02) + and fuzzyEq(aR10, bR10) and fuzzyEq(aR11, bR11) and fuzzyEq(aR12, bR12) + and fuzzyEq(aR20, bR20) and fuzzyEq(aR21, bR21) and fuzzyEq(aR22, bR22) +end +``` + +All helpers are small `local function`s -- the compiler can inline them into `trueEquals`. + +### Main Function: Single typeof Dispatch + +```lua +local function trueEquals(a: any, b: any): boolean + if rawequal(a, b) then return true end + + -- Null-ref equivalence: nil == {Ref = "000...0"} + if a == nil then + return type(b) == "table" and rawget(b, "Ref") == "00000000000000000000000000000000" + end + if b == nil then + return type(a) == "table" and rawget(a, "Ref") == "00000000000000000000000000000000" + end + + local t = typeof(a) + if t ~= typeof(b) then + -- EnumItem/number cross-type + if t == "number" and typeof(b) == "EnumItem" then return a == (b :: EnumItem).Value end + if t == "EnumItem" and typeof(b) == "number" then return (a :: EnumItem).Value == b end + return false + end + + -- Same-type dispatch (no redundant second type check) + if t == "table" then + -- DeepEqualsPureUnsafe pattern with fuzzy recursive equality + if rawlen(a) ~= rawlen(b) then return false end + for k, v in next, a do + local ov = rawget(b, k) + if ov == nil or not trueEquals(v, ov) then return false end + end + for k in next, b do + if rawget(a, k) == nil then return false end + end + return true + end + + if t == "number" then return fuzzyEq(a, b) end + if t == "Color3" then return color3Eq(a, b) end + if t == "Vector3" then return vector3Eq(a, b) end + if t == "Vector2" then return vector2Eq(a, b) end + if t == "CFrame" then return cframeEq(a, b) end + + -- NaN check (number case already handled above, this catches any remaining NaN-like values) + if a ~= a and b ~= b then return true end + + return false +end +``` + +**Key improvements:** + +- **Zero heap allocations**: No temporary arrays for CFrame/Vector3/Vector2. CFrame uses 12 direct `fuzzyEq` calls. Vector3/Vector2 use direct component access. +- **No `checkedKeys` set**: Second table loop only checks existence via `rawget`, doesn't compare values (already done in first loop). +- **Single `typeof` call path**: Check type of `a` once, verify `b` matches, then dispatch. No redundant `typeA == X and typeB == X` at every branch. +- `**rawequal` identity check first**: Catches same-reference case before any type checking. +- `**rawlen` fast-path for tables**: Different array lengths = not equal, no iteration needed. + +--- + +## Codegen Verification + +After each rewrite, run: + +```bash +luau-compile --remarks plugin/src/Reconciler/trueEquals.lua +luau-compile --remarks plugin/src/Reconciler/matching.lua +``` + +**trueEquals.lua** -- check that: + +- `fuzzyEq`, `color3Eq`, `vector2Eq`, `vector3Eq`, `cframeEq` are inlined into `trueEquals` +- No "can't inline" remarks on any helper + +**matching.lua** -- check that: + +- `countOwnDiffs`, `cacheVirtual`, `cacheStudio`, `removeMatched` are inlined at call sites +- `computePairCost` inlines or at minimum shows no optimization barriers +- No generic dispatch warnings for typed parameters + +Iterate on any functions that fail to inline (split if too large, simplify control flow). + +--- + +## Rust Forward Sync: [src/snapshot/matching.rs](src/snapshot/matching.rs) + +Add session struct and thread through. Rust already has the clean 2-function structure (`match_forward` + `compute_change_count` / `match_children_for_scoring`). + +```rust +pub struct MatchingSession { + cost_cache: RefCell>, +} +``` + +- Add `total_cost: u32` to `ForwardMatchResult` +- Add `session: &MatchingSession` param to `match_forward`, `compute_change_count`, `match_children_for_scoring` +- Cache `compute_change_count` results by `(snap.snapshot_id, tree_ref)`, only when `cost < best_so_far` +- Create session in [patch_compute.rs](src/snapshot/patch_compute.rs) `compute_patch_set`, thread through `compute_patch_set_internal` and `compute_children_patches` + +## Rust Syncback: [src/syncback/matching.rs](src/syncback/matching.rs) + +Same pattern. Session struct, `total_cost` on `MatchResult`, session param on all functions. + +- Create session in [dir.rs](src/snapshot_middleware/dir.rs) before `match_children` call + +## Test Updates: [matching_fixtures.rs](tests/tests/matching_fixtures.rs) + +Pass `&MatchingSession::new()` to `match_children` and `match_forward` calls. + +--- + +## Summary of Optimizations + + +| Change | Impact | +| ------------------------------------------------------- | ----------------------------------------------------------------- | +| **trueEquals**: single typeof dispatch | Eliminates redundant type checks at every branch | +| **trueEquals**: dedicated type helpers | Zero heap allocations (no temp arrays for CFrame/Vector3/Vector2) | +| **trueEquals**: DeepEqualsPureUnsafe table pattern | No checkedKeys set; rawlen/rawget/next | +| **trueEquals**: `--!strict`/`--!native`/`--!optimize 2` | Native codegen for the most-called function in matching | +| **matching**: local functions (not table methods) | Enables compiler inlining | +| **matching**: `--!strict`/`--!native`/`--!optimize 2` | Native codegen, type-guided optimization | +| **matching**: concrete types on all params/returns | Specialized native code paths | +| **matching**: remove pcall in grouping | Eliminates closure + pcall overhead per child | +| **matching**: 2-function recursion (from 4) | Simpler for optimizer, easier to reason about | +| **matching**: session caching (matchCache + costCache) | Eliminates redundant subtree scoring across recursion levels | +| **matching**: totalCost on matchChildren | Eliminates _computeChildrenCost re-iteration of matched pairs | +| **matching**: O(n) removeMatched | Linear vs quadratic element removal | + + diff --git a/.cursor/plans/merge_plugin_upload_workflow_0943cf6c.plan.md b/.cursor/plans/merge_plugin_upload_workflow_0943cf6c.plan.md new file mode 100644 index 00000000..33a1b785 --- /dev/null +++ b/.cursor/plans/merge_plugin_upload_workflow_0943cf6c.plan.md @@ -0,0 +1,161 @@ +--- +name: Merge plugin upload workflow +overview: Manually apply the upstream commit "Upload plugin as part of release workflow" to the Atlas fork, adapting all file references and metadata to match the fork's diverged codebase. +todos: + - id: new-files + content: Create .lune/.config.luau, .lune/scripts/plugin-upload.luau, .lune/upload-plugin.luau (adapted for atlas), plugin/UploadDetails.json (with Atlas values) + status: completed + - id: submodule + content: Add opencloud-execute submodule at .lune/opencloud-execute + status: completed + - id: build-rs + content: Add UploadDetails.json to plugin snapshot in build.rs + status: completed + - id: project-json + content: Add UploadDetails node to plugin.project.json + status: completed + - id: rokit + content: Add lune tool to rokit.toml + status: completed + - id: release-yml + content: Add Roblox upload step to release.yml build-plugin job + status: completed + - id: plugin-rs + content: Apply trivial test change in src/cli/plugin.rs + status: completed + - id: local-scripts + content: Replace scripts/publish-plugin.ps1 and add scripts/publish-plugin.sh to use lune-based upload + status: completed +isProject: false +--- + +# Merge Plugin Upload Release Workflow + +Cherry-picking will produce conflicts in 6+ files due to heavy divergence. Manual application is the cleanest path. + +## Upstream bugs to fix + +**Swapped env vars in `release.yml`:** The upstream commit has `RBX_UNIVERSE_ID` mapped to `vars.PLUGIN_CI_PLACE_ID` and `RBX_PLACE_ID` mapped to `vars.PLUGIN_CI_UNIVERSE_ID` -- clearly crossed. We will use the correct mapping. + +## Local scripts replacement + +`scripts/publish-plugin.ps1` currently uses `cargo run -- upload`. Replace it (and add missing `.sh`) to use the same Lune-based upload pipeline locally. The `.lune/upload-plugin.luau` script already handles the local case (builds the plugin via `atlas build` when `CI != "true"`), so the wrapper scripts just load `.env` and run `lune run upload-plugin Atlas.rbxm`. + +`.env` uses the same names as GitHub Actions secrets/vars: + +``` +PLUGIN_UPLOAD_TOKEN=your-opencloud-api-key +PLUGIN_CI_UNIVERSE_ID=your-universe-id +PLUGIN_CI_PLACE_ID=your-place-id +``` + +The local scripts map these to `RBX_API_KEY`/`RBX_UNIVERSE_ID`/`RBX_PLACE_ID` before calling lune, matching what the CI workflow `env:` block does. + +## What the commit adds + +Automated plugin upload to Roblox as part of the GitHub release workflow, using [Lune](https://lune-org.github.io/docs) + OpenCloud Luau Execute to publish the built `.rbxm` to Roblox's plugin marketplace. + +## New files (create as-is, with Atlas adaptations) + +### `.lune/.config.luau` + +Lune config with strict mode and typedefs alias. Copy verbatim from upstream. + +### `.lune/scripts/plugin-upload.luau` + +Roblox engine script that deserializes the plugin `.rbxm`, reads `UploadDetails`, and calls `AssetService:CreateAssetVersionAsync`. Copy verbatim -- it's generic (reads values from UploadDetails at runtime). + +### `.lune/upload-plugin.luau` + +Lune entry point that reads the `.rbxm`, sends it to OpenCloud for execution. One adaptation needed: change `process.exec("rojo", ...)` to `process.exec("atlas", ...)` on the local-testing path (line 25 in upstream). + +### `plugin/UploadDetails.json` + +Atlas-specific values per user input: + +```json +{ + "assetId": 111151139098227, + "name": "Atlas", + "description": "The plugin portion of Atlas, a tool to enable professional tooling for Roblox developers.", + "creatorId": 422368266, + "creatorType": "Group" +} +``` + +## Submodule addition + +### `.lune/opencloud-execute` + +Add git submodule: `https://github.com/Dekkonot/opencloud-luau-execute-lune.git` at `.lune/opencloud-execute`. This also adds an entry to `[.gitmodules](.gitmodules)`. + +## Existing file modifications + +### `[.github/workflows/release.yml](.github/workflows/release.yml)` + +Two additions to the `build-plugin` job: + +1. **Add Rokit install step** (after "Install Rust", before "Build Atlas") so `lune` is on PATH: + +```yaml +- name: Install Rokit + uses: CompeyDev/setup-rokit@v0.1.2 +``` + +1. **Add upload step** (after "Upload Plugin to Artifacts" at line 47) with **corrected** env var mapping (upstream had these swapped): + +```yaml +- name: Upload Plugin to Roblox + env: + RBX_API_KEY: ${{ secrets.PLUGIN_UPLOAD_TOKEN }} + RBX_UNIVERSE_ID: ${{ vars.PLUGIN_CI_UNIVERSE_ID }} + RBX_PLACE_ID: ${{ vars.PLUGIN_CI_PLACE_ID }} + run: lune run upload-plugin Atlas.rbxm +``` + +### `[build.rs](build.rs)` + +Add `"UploadDetails.json"` entry to the plugin snapshot hashmap, after `"Version.txt"` on line 98: + +```rust +"UploadDetails.json" => snapshot_from_fs_path(&plugin_dir.join("UploadDetails.json"))?, +``` + +### `[plugin.project.json](plugin.project.json)` + +Add `UploadDetails` child node after `Version` (line 25): + +```json +"UploadDetails": { + "$path": "plugin/UploadDetails.json" +} +``` + +### `[rokit.toml](rokit.toml)` + +Add lune tool (latest version) after the existing entries: + +```toml +lune = "lune-org/lune@0.10.4" +``` + +### `[src/cli/plugin.rs](src/cli/plugin.rs)` + +Trivial test change on line 101: `assert!(initialize_plugin().is_ok())` becomes `let _ = initialize_plugin().unwrap();` + +### `[scripts/publish-plugin.ps1](scripts/publish-plugin.ps1)` + +Replace entirely. New version: load `.env`, export `RBX_API_KEY`/`RBX_UNIVERSE_ID`/`RBX_PLACE_ID`, run `lune run upload-plugin Atlas.rbxm`. + +### `scripts/publish-plugin.sh` (new) + +Bash equivalent: source `.env`, export vars, run `lune run upload-plugin Atlas.rbxm`. + +## Required GitHub secrets/vars + +After merging, the repo needs these configured in GitHub settings: + +- **Secret:** `PLUGIN_UPLOAD_TOKEN` -- Roblox OpenCloud API key +- **Variable:** `PLUGIN_CI_UNIVERSE_ID` -- Universe ID for the place used by OpenCloud Execute +- **Variable:** `PLUGIN_CI_PLACE_ID` -- Place ID within that universe + diff --git a/.cursor/plans/parallel_filesystem_io_39399341.plan.md b/.cursor/plans/parallel_filesystem_io_39399341.plan.md new file mode 100644 index 00000000..d7eae94d --- /dev/null +++ b/.cursor/plans/parallel_filesystem_io_39399341.plan.md @@ -0,0 +1,343 @@ +--- +name: Parallel filesystem IO +overview: Add parallel file reading via VFS prefetch cache, parallel snapshot building via rayon at directory level, and parallel RefPathIndex population to speed up all project loading paths. +todos: + - id: vfs-prefetch-cache + content: Add PrefetchCache struct (files + canonical maps), prefetch_cache field on VfsInner, read_raw() helper, wire into read()/read_to_string()/canonicalize() + status: completed + - id: vfs-prefetch-api + content: Add Vfs::set_prefetch_cache() and Vfs::clear_prefetch_cache() public methods + status: completed + - id: vfs-prefetch-tests + content: Add unit tests for prefetch cache (file hit/miss, read_to_string, canonicalize hit/miss, watch registration, clear) + status: completed + - id: prefetch-builder + content: Write prefetch_project_files() using walkdir + rayon to parallel-read all files under root + status: completed + - id: parallel-snapshot-build + content: Add rayon par_iter to snapshot_dir_no_meta in dir.rs for parallel child processing + status: completed + - id: env-var-fallback + content: Add ATLAS_SEQUENTIAL env var to disable all parallelization for debugging + status: completed + - id: integrate-serve-session + content: Call prefetch in ServeSession::init_tree() before snapshot_from_vfs() + status: completed + - id: parallel-refpathindex + content: Parallelize RefPathIndex::populate_from_dir() using walkdir + rayon for file reads/parsing + status: completed + - id: test-determinism + content: Add parallel snapshot determinism test - run snapshot_from_vfs N times, verify identical output + status: completed + - id: test-and-verify + content: Run full cargo test, verify no regressions, add timing logs to confirm speedup + status: completed +isProject: false +--- + +# Parallel Filesystem I/O for Project Loading + +## Problem + +Every file read goes through `Vfs` -> `Mutex` -> `StdBackend::read()` -> `fs_err::read()`. This serializes all I/O. For a 3500-file project, startup is bottlenecked by ~3500 sequential syscalls, plus sequential CPU-bound parsing of each file. + +**Affected commands:** `serve`, `build`, `upload`, `syncback`, `sourcemap` -- all call `snapshot_from_vfs()` which recursively reads every file through the VFS one at a time. + +## Current Sequential Hotspots + +1. `**snapshot_from_vfs()` initial tree build** ([src/serve_session.rs](src/serve_session.rs) line 141) -- The biggest bottleneck. Recursive directory traversal in [src/snapshot_middleware/dir.rs](src/snapshot_middleware/dir.rs) lines 54-63, each child read goes through VFS mutex. +2. `**RefPathIndex::populate_from_dir()`** ([src/rojo_ref.rs](src/rojo_ref.rs) lines 372-431) -- Sequential walk + `std::fs::read()` for every `.meta.json5`/`.model.json5` file. Runs after tree build on `serve`. +3. **Two-way sync writes in `api.rs`** -- Sequential `fs::write()` calls (low priority, small batches). + +## Strategy: Full Pipeline + +Three layers, all using `rayon` (already a dependency): + +1. **VFS Prefetch Cache** -- parallel I/O: read all files upfront +2. **Parallel Snapshot Build** -- parallel CPU: parse files concurrently at directory level +3. **Parallel RefPathIndex** -- parallel I/O+CPU: read and parse meta/model files concurrently + +Combined timeline (vs current sequential): + +``` +Current: [read→parse→read→parse→read→parse→read→parse→...] (serial everything) + +Pipelined: [=====parallel read all files=====] (Phase 1: I/O) + ......[==parallel parse+build snapshots==] (Phase 2: CPU) +``` + +### Part 1: VFS Prefetch Cache + +**Concept:** Before `snapshot_from_vfs()` runs, walk the directory tree and read all file contents in parallel using `std::fs` + rayon (bypassing the VFS lock). Store results in a HashMap on VfsInner. The subsequent parallel snapshot build hits the cache (HashMap lookup, no I/O) instead of disk. + +**Changes to [crates/memofs/src/lib.rs](crates/memofs/src/lib.rs):** + +- Add `PrefetchCache` struct with `files: HashMap>` and `canonical: HashMap` +- Add `prefetch_cache: Option` field to `VfsInner` +- Extract a shared `read_raw(&mut self, path) -> io::Result>` helper that checks `prefetch_cache.files` first (using `HashMap::remove` to free memory on hit), falls back to backend +- Wire BOTH `VfsInner::read()` AND `VfsInner::read_to_string()` through `read_raw()` +- Add cache check to `VfsInner::canonicalize()`: check `prefetch_cache.canonical` first (using `HashMap::remove`), fall back to backend +- Add `Vfs::set_prefetch_cache(cache)` and `Vfs::clear_prefetch_cache()` + +```rust +pub struct PrefetchCache { + pub files: HashMap>, + pub canonical: HashMap, +} +``` + +**Critical: `read_to_string_lf_normalized` coverage.** The `Vfs::read_to_string_lf_normalized()` method (used by ALL `.luau` files via `lua.rs:55`) calls `VfsInner::read_to_string()` internally. Since we wire `VfsInner::read_to_string()` through `read_raw()`, LF-normalized reads hit the cache automatically. The flow: `Vfs::read_to_string_lf_normalized` -> lock -> `VfsInner::read_to_string` -> `read_raw` (cache hit) -> UTF-8 convert -> unlock -> CRLF->LF normalize. + +**Critical: `canonicalize` coverage.** Every middleware calls `vfs.canonicalize(path)` once, storing the result in `InstanceMetadata::relevant_paths` for file-change-to-instance mapping. During init, these are parallelized in the prefetch phase. After init, the cache is cleared and `change_processor.rs` calls `canonicalize()` through the backend for fresh results. + +**What is NOT cached** (and why that's acceptable): + +- `read_dir()`: directory listing syscall, ~10us each, ~500 dirs = ~5ms total +- `metadata()`: stat syscall, ~~1-10us each. `get_dir_middleware` probes up to 13 init files per directory (~~6500 calls for 500 dirs = ~6.5ms) +- `exists()`: only used by `Project::load_initial_project` (1-2 calls before prefetch starts) + +**No stale canonicalize entries:** The cache exists only during `init_tree()`. During this window, the filesystem is static (no watcher running, no mutations). After `clear_prefetch_cache()`, all canonicalize calls go through the backend -- the `ChangeProcessor` always gets fresh results for rename/move detection. **No memory leak:** `clear_prefetch_cache()` drops the entire `PrefetchCache` struct. + +**New helper (in `src/` or inline in `serve_session.rs`):** + +```rust +fn prefetch_project_files(root: &Path) -> io::Result { + use memofs::PrefetchCache; + use rayon::prelude::*; + use walkdir::WalkDir; + + let entries: Vec = WalkDir::new(root) + .follow_links(true) // Match VFS behavior: StdBackend::read() follows symlinks + .into_iter() + .filter_map(|e| e.ok()) + .collect(); + + // Parallel file reads (files only) + let file_data: Vec<_> = entries + .par_iter() + .filter(|e| e.file_type().is_file()) + .filter_map(|e| { + let path = e.path().to_path_buf(); + std::fs::read(&path).ok().map(|c| (path, c)) + }) + .collect(); + + // Parallel canonicalize (files AND directories) + let canonical_data: Vec<_> = entries + .par_iter() + .filter_map(|e| { + let path = e.path().to_path_buf(); + std::fs::canonicalize(&path).ok().map(|c| (path, c)) + }) + .collect(); + + Ok(PrefetchCache { + files: file_data.into_iter().collect(), + canonical: canonical_data.into_iter().collect(), + }) +} +``` + +`walkdir` (already a dependency) handles the directory walk. `.follow_links(true)` ensures symlinked files are prefetched (walkdir handles symlink loop detection). `rayon::par_iter` parallelizes both file reads AND canonicalize syscalls. The canonicalize cache covers both files and directories since `dir.rs:66` canonicalizes directory paths. + +**Integration in [src/serve_session.rs](src/serve_session.rs) `init_tree()`:** + +```rust +fn init_tree(vfs: &Vfs, start_path: &Path) -> Result<...> { + let root_project = Project::load_initial_project(vfs, start_path)?; + + if std::env::var("ATLAS_SEQUENTIAL").is_err() { + let cache = prefetch_project_files(root_project.folder_location())?; + vfs.set_prefetch_cache(cache); + } + + let snapshot = snapshot_from_vfs(&instance_context, vfs, start_path)?; + + vfs.clear_prefetch_cache(); + // ... rest unchanged +} +``` + +**Note:** The serve command calls `init_tree()` twice -- once via `new_oneshot()` for config (port/address), once via `new()` for the real session. Both get prefetch. The oneshot is immediately dropped so its prefetch is "wasted", but the overhead is small (~50ms) and the code stays simple. + +**Why this is safe:** The prefetch reads files with `std::fs` (no VFS lock). The cache is loaded into VfsInner behind the mutex. The snapshot build then acquires the mutex per-read as before, but each read is a HashMap remove (nanoseconds) instead of a disk syscall. Existing behavior is preserved for cache misses (falls through to backend). Watch registration still happens normally (gated by `watch_enabled`). + +`**StdBackend::read_dir()` is eager:** It collects and sorts all directory entries before returning the `ReadDir` iterator (`std_backend.rs:283-296`). The iterator does not hold any reference into the backend. This means `read_dir` results are safe to use from rayon threads after the VFS lock is released. + +### Part 2: Parallel Snapshot Build at Directory Level + +**Concept:** In `snapshot_dir_no_meta`, process each directory's children in parallel using rayon. With the prefetch cache warm, each rayon thread acquires the VFS mutex briefly for a cache hit (~1us), then parses the file content lock-free (the expensive part). Mutex contention is negligible since critical sections are just HashMap removes. + +**Changes to [src/snapshot_middleware/dir.rs](src/snapshot_middleware/dir.rs) `snapshot_dir_no_meta()`:** + +```rust +// Current (sequential): +for entry in vfs.read_dir(path)? { + let entry = entry?; + if !passes_filter_rules(&entry) { continue; } + if let Some(child) = snapshot_from_vfs(context, vfs, entry.path())? { + snapshot_children.push(child); + } +} + +// Parallel (with order preservation): +use rayon::prelude::*; + +let entries: Vec<_> = vfs.read_dir(path)? + .filter_map(|e| e.ok()) + .collect(); + +let results: Vec>> = entries + .par_iter() + .map(|entry| { + if !passes_filter_rules(entry) { + return Ok(None); + } + snapshot_from_vfs(context, vfs, entry.path()) + }) + .collect(); + +let mut snapshot_children = Vec::new(); +for result in results { + if let Some(snapshot) = result? { + snapshot_children.push(snapshot); + } +} +``` + +**Why `map` instead of `filter_map`:** rayon's `par_iter().map().collect()` preserves original order (map is an indexed operation). `filter_map` converts to an unindexed iterator where order is implementation-dependent. Using `map` + sequential flatten ensures children appear in the same order as `read_dir` returns them. + +**Why this is safe for concurrent VFS access:** Each VFS call (metadata, read, read_dir) independently acquires/releases the Mutex -- no nested locking, so no deadlock risk. `&InstanceContext` is `Sync` (contains `Arc>` + `Vec`). `&Vfs` is `Sync` (wraps `Mutex`). `anyhow::Error` is `Send` (required for rayon's `collect::>>()`). + +**Multiple VFS calls per middleware:** Each middleware makes several VFS calls during snapshot building. For example, `lua.rs` calls `read_to_string_lf_normalized` (cache hit), `canonicalize` (backend), and reads companion `.meta.json5` (cache hit). With parallel builds, the cached reads are fast (~1us lock + HashMap remove), while `metadata` and `canonicalize` still serialize on the mutex with real syscalls. This limits maximum parallelism but the file-read savings dominate. + +**Recursion:** Since `snapshot_from_vfs` is recursive, a directory with 10 subdirectories kicks off 10 parallel subtrees, each of which parallelizes their own children. rayon's work-stealing keeps all cores busy across the entire tree. + +**Rayon thread pool:** The `sourcemap` command configures a global rayon pool with `max(num_cpus, 6)` threads at `src/cli/sourcemap.rs:93`, but this happens AFTER `init_tree()` completes. Our parallel snapshot build runs before that configuration, using rayon's default pool (CPU count). No conflict. + +**Environment variable gate:** + +```rust +if std::env::var("ATLAS_SEQUENTIAL").is_ok() { + // Sequential fallback (existing code) +} else { + // Parallel path +} +``` + +### Part 3: Parallel RefPathIndex Population + +**Changes to [src/rojo_ref.rs](src/rojo_ref.rs) `populate_from_dir()`:** + +Current: sequential stack-based walk + `std::fs::read()` per file. + +New approach: + +1. Walk tree with `walkdir`, collect all `.meta.json5`/`.model.json5` paths (sequential, fast -- just filename checks) +2. Read + parse + extract `Rojo_Ref_*` entries in parallel with rayon. `&RojoTree` is read-only and safe to share across threads (`get_ids_at_path()` and `ref_target_path_from_tree()` are in-memory lookups). +3. Collect `Vec<(String, PathBuf)>` results, then insert into index (sequential) + +## Race Condition Analysis + +### No risk (by design) + +- **VFS data races:** Impossible. `Mutex` serializes all access. Multiple rayon threads contend briefly on cache lookups (~1us critical sections), which is safe. +- **Deadlock from re-entrant locking:** Impossible. Each VFS call (metadata, read, read_dir) acquires and releases the lock independently. No VFS call internally calls another VFS method. `ReadDir` iterator is fully materialized before lock release. +- **InMemoryFs thread safety:** `InMemoryFs` wraps `Arc>`, additionally protected by VFS's outer Mutex. Unit tests using InMemoryFs are safe with parallel snapshot building. + +### Mitigated risks + +- **Child ordering non-determinism:** `par_iter().filter_map().collect()` does NOT guarantee order (filter_map makes the iterator unindexed). We use `par_iter().map().collect()` instead, which preserves input order because `map` on an indexed iterator stays indexed. The sequential flatten step then filters Nones. A determinism test (run N times, compare output) validates this. +- **Prefetch cache stale reads:** If a file is modified between prefetch and snapshot_from_vfs, the cached content is stale. In practice this cannot happen during the few milliseconds of startup. Same time-of-check-time-of-use window exists with sequential reads. No additional risk. +- **Error propagation in parallel:** `par_iter().map().collect::>>()` collects all results. Errors are propagated sequentially in the flatten step via `?`. First error wins, consistent with sequential behavior. + +### Covered by environment variable fallback + +`ATLAS_SEQUENTIAL=1` disables both prefetch cache and parallel snapshot building, reverting to the exact current sequential code path. Useful for: + +- Debugging if parallelization causes platform-specific issues +- A/B performance comparison +- CI environments requiring deterministic sequential execution + +## Testing Strategy + +### Existing tests that automatically exercise the new system + +All integration tests spawn the CLI binary, which goes through `init_tree()` -> prefetch + parallel snapshot: + +- **Build tests** ([tests/tests/build.rs](tests/tests/build.rs)) -- Run `rojo build` on real filesystem fixtures. Exercise full prefetch + parallel snapshot pipeline. +- **Serve tests** ([tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs) `TestServeSession`) -- Spawn `atlas serve` on temp dirs. Exercise prefetch + parallel snapshot + RefPathIndex. +- **Syncback/roundtrip tests** ([tests/tests/syncback.rs](tests/tests/syncback.rs), [tests/tests/syncback_roundtrip.rs](tests/tests/syncback_roundtrip.rs)) -- Full CLI syncback pipeline. +- **Two-way sync tests** ([tests/tests/two_way_sync.rs](tests/tests/two_way_sync.rs)) -- Serve session + write API. +- **Matching fixture tests** ([tests/tests/matching_fixtures.rs](tests/tests/matching_fixtures.rs)) -- Call `snapshot_from_vfs()` directly with `Vfs::new_default()`. Exercise parallel snapshot building in dir.rs but NOT prefetch (no init_tree call). +- **Unit tests in dir.rs** -- Use `InMemoryFs`. Exercise parallel snapshot building but not prefetch. This is correct -- InMemoryFs files don't exist on the real filesystem. + +### New tests to add + +1. **VFS prefetch cache unit tests** (in [crates/memofs/src/lib.rs](crates/memofs/src/lib.rs) test module): + - Cache hit: `set_prefetch_cache` with known path -> `read()` returns cached data + - Cache depletion: second `read()` of same path falls through to backend (entry was removed on first read) + - Cache miss: `read()` of path not in cache falls through to backend + - `read_to_string` uses cache: verify UTF-8 conversion from cached bytes + - Watch registration: if `watch_enabled`, watches are registered even on cache hits + - `clear_prefetch_cache`: after clear, all reads go through backend + - Empty/None cache behaves identically to no cache +2. **Parallel snapshot determinism test** (in [tests/tests/build.rs](tests/tests/build.rs) or new file): + - Run `snapshot_from_vfs` on a non-trivial fixture 5 times + - Assert all 5 snapshots are identical (catches ordering bugs from rayon) + - Use a fixture with multiple children per directory to stress the parallel path +3. **Prefetch + parallel integration sanity** (implicit): + - All existing insta snapshot tests serve as regression tests. If prefetch returns wrong data or parallel build reorders children, snapshot comparisons fail immediately. + +### Tests we do NOT need + +- **VFS mutex stress test:** The Mutex is a well-tested primitive. Contention with ~1us critical sections and rayon's thread pool is a solved problem. +- **Prefetch TOCTOU test:** The time-of-check-time-of-use window between prefetch and read is identical to the existing sequential window. No additional test. +- **InMemoryFs concurrency test:** InMemoryFs is protected by both its own Mutex and the VFS's outer Mutex. Double-locked by design. + +## What About the File Watcher? + +The ChangeProcessor handles individual file events one at a time. Single-file reads don't benefit from parallelization. Two scenarios where deeper parallelism could help: + +- **Batch events after `git checkout`**: Many files change at once, processed sequentially. Could batch and parallelize, but adds complexity. Deferred. +- **Rescan after watcher critical error**: Re-snapshots the full tree. Would benefit from prefetch, but rescan is rare. Can be added later. + +## Other Sequential I/O (Lower Priority, Not in This Plan) + +- **Two-way sync writes** (`api.rs`): Small batches (1-10 files). Not worth the complexity. +- **Build output encoding**: Single file write. Cannot parallelize. +- **Upload encoding**: Single buffer + HTTP upload. Network-bound. +- `**validate_tree()` / `check_tree_freshness()`**: Re-snapshots full tree. Could use prefetch. Secondary benefit, can add later. + +## Files Changed + + +| File | Change | +| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------- | +| [crates/memofs/src/lib.rs](crates/memofs/src/lib.rs) | Add prefetch cache field, `read_raw()` helper, `set/clear_prefetch_cache()` methods | +| [src/snapshot_middleware/dir.rs](src/snapshot_middleware/dir.rs) | Add rayon `par_iter` for parallel child snapshot building | +| [src/serve_session.rs](src/serve_session.rs) | Call prefetch before snapshot build in `init_tree()` | +| [src/rojo_ref.rs](src/rojo_ref.rs) | Parallelize `populate_from_dir()` with rayon | + + +No changes to VFS backend trait. No new dependencies (rayon, walkdir already present). + +## Stack Compatibility Checklist + +Verified across the full stack: + +- `Vfs` is `Sync` (wraps `Mutex`, `VfsBackend: Send`) +- `InstanceContext` is `Sync` (contains `Arc>` + `Vec`) +- `anyhow::Error` is `Send` (required for rayon error collection) +- `StdBackend::read_dir()` eagerly collects entries before returning (`std_backend.rs:283-296`) -- no lazy iteration holding the lock +- No `vfs.lock()` calls in any snapshot middleware -- only syncback writes use `VfsLock` +- `InMemoryFs` is thread-safe (`Arc>`) -- unit tests with parallel dir.rs are safe +- `read_to_string_lf_normalized` routes through `VfsInner::read_to_string` -> `read_raw` -- cache hit for all `.luau` files +- `VfsInner::canonicalize` checks `prefetch_cache.canonical` -- eliminates ~400ms Windows serialization point +- Canonicalize cache safe: only alive during `init_tree()` (no mutations), `ChangeProcessor` always gets fresh backend results after cache clear +- `walkdir` with `.follow_links(true)` matches `StdBackend::read` symlink behavior; handles loop detection +- `Project::load_initial_project` reads via VFS BEFORE prefetch starts -- correct, only reads 1 project file +- Rayon global pool not configured until after `init_tree` completes (sourcemap configures at `sourcemap.rs:93`) +- `ReadDir` iterator fully materialized -- safe to use from rayon threads after lock release +- No stateful side-effects in any snapshot middleware -- all are pure functions of (context, vfs, path) + diff --git a/.cursor/plans/parallel_io_audit_fixes_35f49f83.plan.md b/.cursor/plans/parallel_io_audit_fixes_35f49f83.plan.md new file mode 100644 index 00000000..88f4c885 --- /dev/null +++ b/.cursor/plans/parallel_io_audit_fixes_35f49f83.plan.md @@ -0,0 +1,267 @@ +--- +name: Parallel IO audit fixes +overview: "Fix three issues found during audit of the parallelization/optimization commits: a data loss bug in the orphan scanner, a duplicate-children bug in the prefetch cache, and a nested-detection correctness issue, plus a small DRY refactor." +todos: + - id: fix1-hidden-dirs + content: Fix orphan scanner walkdir to prune hidden directories using filter_entry() + status: completed + - id: fix2-dedup-roots + content: Deduplicate overlapping $path roots in prefetch_project_files before walking + status: completed + - id: fix3-nested-detection + content: Replace broken binary search nested detection with O(n log n) ancestor-tracking approach + status: completed + - id: fix4-unc-refactor + content: Extract shared strip_unc_prefix helper from 3 inline copies + status: completed + - id: test-hidden-dir + content: Add integration test for hidden directory survival in clean syncback + status: completed + - id: test-overlap + content: Add test for overlapping $path roots with prefetch cache + status: completed + - id: test-nested + content: Add unit test for nested path detection correctness + status: completed + - id: run-ci + content: Run full CI pipeline (cargo test, clippy, fmt) + status: completed +isProject: false +--- + +# Parallel I/O Audit Fixes + +> This plan was generated by `/audit` from a Plan-mode session. +> Refer to `.cursor/rules/atlas.mdc` for full project standards. + +## Standards + +These standards govern every fix in this plan. An implementer MUST read `.cursor/rules/atlas.mdc` before starting. + +### Round-Trip Identity (from `atlas.mdc` - Quality Standard) + +Syncback (or two-way sync) writes a directory tree. Building an rbxl from that directory tree and forward-syncing it back must produce a **bit-identical instance tree** -- same names, same classes, same properties, same hierarchy, same ref targets. Any deviation is a bug. + +### CLI Syncback Parity (from `atlas.mdc` - Two-Way Sync Strategy) + +Plugin-based sync must produce **exactly the same filesystem output** that `atlas syncback` would give for the same input. Byte-for-byte identical files, identical directory structures, identical naming. Any divergence is a bug until proven otherwise. + +### Code Quality (from `atlas.mdc` - Code Quality Standard) + +DRY code that is easy to reason about and hard to get wrong. The same slugify/dedup/meta-update pattern should not be copy-pasted across files. Small refactors (2-3 call sites) are included as fixes. Major rewrites (10+ call sites) are deferred. + +## Context + +Audited the last 7 commits on the current branch (from `1ffed121` through `56e1ab4a`), which add: + +- VFS prefetch cache for parallel file reading at startup +- Parallel snapshot building via rayon in `dir.rs` +- Parallel RefPathIndex population via rayon + walkdir +- Parallel tree hashing in syncback +- Orphan scanner rewrite using walkdir +- Placeholder-based ref path substitution optimization +- ServeCommand refactor and dead code cleanup + +## Fixes + +### Fix 1: Prune hidden directories from walkdir orphan scanner + +- **Status:** Approved +- **Finding:** Orphan scanner walkdir replacement doesn't prune hidden directories +- **Files:** `[src/syncback/mod.rs](src/syncback/mod.rs)` +- **Problem:** Lines ~406-424. The `walkdir::WalkDir` iterator yields ALL entries including children of hidden directories. The hidden-name filter only checks `file_name().starts_with('.')`, so `settings.json` inside `.vscode/` passes the filter and gets added to `existing_paths`. In clean mode, these files would be deleted. +The old recursive `scan_directory` function skipped hidden directories AND never recursed into them. The walkdir replacement must replicate this behavior. +- **Solution:** Replace the flat `.flatten()` + post-hoc hidden check with `filter_entry()` to prune the walk at hidden directories: + +```rust + for entry in walkdir::WalkDir::new(&canonical_dir) + .follow_links(true) + .into_iter() + .filter_entry(|e| { + if e.depth() == 0 { + return true; + } + e.file_name() + .to_str() + .map_or(true, |n| !n.starts_with('.') || n == ".gitkeep") + }) + .flatten() + { + if entry.depth() == 0 { + continue; + } + let path = entry.path().to_path_buf(); + if !is_valid_path(&ignore_patterns, project_path, &path) { + continue; + } + // Hidden check no longer needed here -- filter_entry handles it + paths.insert(path); + } + + +``` + + `filter_entry` controls which directories walkdir descends into. When it returns `false` for a directory, walkdir skips the directory AND all its descendants. This exactly matches the old recursive behavior. + +- **Risk:** Low -- only touches the orphan scanner loop. `filter_entry` is walkdir's standard pruning mechanism. +- **Verify round-trip:** Clean mode syncback with a `.vscode/` directory inside a `$path` source dir should NOT delete `.vscode/` contents. +- **Verify syncback parity:** N/A -- this only affects clean mode orphan removal, not the sync pipeline. +- **Tests required:** + - Integration test: create a temp project with `src/.hidden/keep.txt`, run clean-mode syncback, verify `.hidden/keep.txt` survives. + +### Fix 2: Deduplicate overlapping roots in prefetch cache + +- **Status:** Approved +- **Finding:** Overlapping `$path` roots cause duplicate children in prefetch cache +- **Files:** `[src/serve_session.rs](src/serve_session.rs)` +- **Problem:** Lines ~136-156 in `prefetch_project_files`. `collect_path_roots` collects ALL `$path` entries including nested ones. For a project with `$path: "src"` and `$path: "src/shared"`, walkdir walks `src/shared` twice -- once as part of `src` and once as its own root. The `children_map` accumulates duplicate entries for `src/shared`'s children. When consumed by VFS `read_dir`, these duplicates cause duplicate instance snapshots. +- **Solution:** After collecting roots in `prefetch_project_files`, deduplicate by removing any root that is a descendant of another root: + +```rust + let mut roots: Vec = Vec::new(); + collect_path_roots(&project.tree, folder, &mut roots); + + // Remove roots that are descendants of other roots to avoid + // walking the same directory twice and producing duplicate children. + roots.sort(); + let mut deduped: Vec = Vec::new(); + for root in roots { + if !deduped.iter().any(|existing| root.starts_with(existing)) { + deduped.push(root); + } + } + let roots = deduped; + + +``` + + After sorting, a parent path always sorts before its descendants. By iterating in order and only keeping roots that are NOT descendants of already-kept roots, we eliminate all overlaps in O(n^2) worst case but typically O(n) since most projects have few `$path` entries. + +- **Risk:** Low -- only modifies which directories are walked during prefetch. The dedup logic is simple and well-understood. +- **Verify round-trip:** Build a project with overlapping `$path` entries. Verify snapshot tree has no duplicate instances. Run build 5 times, verify deterministic output. +- **Verify syncback parity:** N/A -- prefetch cache only affects forward sync initial load. +- **Tests required:** + - Unit test in `tests/tests/build.rs`: create a project with `$path: "src"` and a child with `$path: "src/shared"`. Run `snapshot_from_vfs` with prefetch cache. Verify no duplicate children under the `shared` directory instance. + +### Fix 3: Fix nested path detection in orphan removal + +- **Status:** Approved (O(n log n) approach) +- **Finding:** Binary search nested detection misses sibling paths +- **Files:** `[src/syncback/mod.rs](src/syncback/mod.rs)` +- **Problem:** Lines ~869-888. The optimization uses `binary_search` to find the current path's position, then checks only `sorted_removals[idx - 1]`. This misses cases where a sibling path sits between the parent and the current path in sorted order. +Example: `sorted = ["a/b", "a/b/c", "a/b/d"]`. For `a/b/d` at idx 2, the code checks `sorted[1] = "a/b/c"` (a sibling, not an ancestor). The actual ancestor `"a/b"` at idx 0 is not checked. +- **Solution:** Replace binary search with a sorted single-pass using an `current_ancestor` tracker: + +```rust + let mut sorted_removals: Vec<_> = paths_to_remove.iter().cloned().collect(); + sorted_removals.sort(); + + let mut current_ancestor: Option<&PathBuf> = None; + for old_path in &sorted_removals { + if let Some(ancestor) = current_ancestor { + if old_path.starts_with(ancestor) && old_path != ancestor { + continue; // nested under current ancestor, skip + } + } + current_ancestor = Some(old_path); + + let relative_path = old_path + .strip_prefix(&canonical_project_path) + .unwrap_or(old_path); + + log::debug!("Removing orphaned path: {}", relative_path.display()); + if old_path.is_dir() { + fs_snapshot.remove_dir(relative_path); + } else { + fs_snapshot.remove_file(relative_path); + } + } + + +``` + + In sorted path order, all descendants of a path appear consecutively after it (because `Path` comparison is component-based). So a single variable tracking the current top-level ancestor is sufficient. When we encounter a non-descendant, it becomes the new ancestor. This is O(n log n) for the sort + O(n) for the walk. + + This also removes the intermediate `paths_vec` and the `binary_search` code, simplifying the function. + +- **Risk:** Low -- the logic is simpler than what it replaces. The correctness argument is straightforward: sorted paths guarantee all descendants appear consecutively. +- **Verify round-trip:** Clean mode syncback with nested directories. Verify parent-only removal (no redundant child removals). +- **Verify syncback parity:** N/A -- orphan removal is internal to clean mode. +- **Tests required:** + - Unit test: create paths `["a/b", "a/b/c", "a/b/d", "x/y", "x/y/z"]`, verify only `"a/b"` and `"x/y"` are top-level removals. + +### Fix 4: Extract shared `strip_unc_prefix` helper + +- **Status:** Approved +- **Finding:** Duplicated UNC prefix stripping logic in 3 locations +- **Files:** `[src/syncback/mod.rs](src/syncback/mod.rs)` +- **Problem:** The same UNC prefix stripping pattern appears at: + 1. Lines ~396-404 (orphan scanner: canonicalize `dirs_to_scan` entries) + 2. Lines ~649-656 (`strip_unc_prefix` helper definition inside orphan removal block) + 3. Lines ~703-711 (inline canonicalize in `collect_all_path_refs_and_mappings`) + Location 2 defines a proper `fn strip_unc_prefix`, but locations 1 and 3 inline the same logic. +- **Solution:** Move `strip_unc_prefix` to module level (or just above the clean-mode block), and replace all 3 inline instances with calls to it. The function is already defined at location 2: + +```rust + fn strip_unc_prefix(p: PathBuf) -> PathBuf { + let path_str = p.to_string_lossy(); + if path_str.starts_with(r"\\?\") || path_str.starts_with("//?/") { + PathBuf::from(&path_str[4..]) + } else { + p + } + } + + +``` + + Move this definition before the `existing_paths` block (around line ~258), then replace the inline code at lines ~396-404 with `strip_unc_prefix(std::fs::canonicalize(dir).unwrap_or_else(|_| dir.clone()))` and at lines ~703-711 with `strip_unc_prefix(p)`. + +- **Risk:** Zero -- pure refactor, no behavior change. +- **Verify round-trip:** N/A -- refactor only. +- **Verify syncback parity:** N/A -- refactor only. +- **Tests required:** None -- existing tests cover this code path. + +## Skipped Fixes + +### Skipped: Serve command loads project twice + +- **Finding:** `src/cli/serve.rs` loads the project file via `Project::load_initial_project` for port/address, then `ServeSession::new` loads it again in `init_tree`. +- **Problem:** Minor performance -- project file read twice from disk at startup. +- **Reason skipped:** Acceptable overhead. Project files are small and this only happens once. + +### Skipped: Commented-out git config in init.rs + +- **Finding:** Four commented-out git config lines in `src/cli/init.rs` lines 134-137. +- **Reason skipped:** User indicated these are planned for future enablement. + +## Accepted Limitations + +- **Limitation:** Symlinks within project `$path` directories may cause path mismatches +- **Scenario:** If a symlink inside a `$path` source directory points elsewhere, walkdir-collected paths won't match `added_paths` during clean-mode orphan removal. Files behind the symlink could be incorrectly removed. +- **User decision:** Accepted for this release +- **Mitigation:** Symlinks within source directories are not a supported configuration in Roblox development. Root-level symlinks (project folder itself) still work because `canonicalize` is called on the root. + +## Deferred Refactors + +None identified beyond the small `strip_unc_prefix` extraction (included as Fix 4). + +## Test Plan + +### Rust Integration Tests (`tests/tests/`) + +- **Hidden directory survival in clean syncback** (Fix 1): Create temp project with `src/.hidden/keep.txt`, run clean-mode syncback, assert `.hidden/keep.txt` still exists after syncback. +- **Overlapping `$path` roots don't create duplicates** (Fix 2): Create project with overlapping `$path` entries (`"src"` and `"src/shared"`), run `snapshot_from_vfs` with prefetch cache, verify no duplicate children. + +### Rust Unit Tests + +- **Nested path detection correctness** (Fix 3): Test the ancestor-tracking algorithm with paths `["a/b", "a/b/c", "a/b/d", "x/y", "x/y/z"]`, verify only `"a/b"` and `"x/y"` are emitted as top-level. + +## Test Rules + +- Every bug fix MUST have a test that would have FAILED before the fix and PASSES after. +- Fix 4 (refactor) does not need new tests -- existing test coverage is sufficient. + +## Final Step: Run CI + +After ALL fixes and tests are implemented, run the full CI pipeline (`cargo test --locked`, `cargo clippy`, `cargo fmt --check`). Every fix must pass CI before the plan is considered complete. \ No newline at end of file diff --git a/.cursor/plans/reconciler_robustness_overhaul_2f440e06.plan.md b/.cursor/plans/reconciler_robustness_overhaul_2f440e06.plan.md new file mode 100644 index 00000000..789fa159 --- /dev/null +++ b/.cursor/plans/reconciler_robustness_overhaul_2f440e06.plan.md @@ -0,0 +1,286 @@ +--- +name: Reconciler robustness overhaul +overview: Fix the Atlas plugin Reconciler to handle MeshPart creation via AssetService, MeshId swapping, and CSG reconstruction -- modeled after the proven rbxsync Sync.luau / CSGHandler.luau implementation. +todos: + - id: tags-attrs-verify + content: Verify Tags/Attributes work through existing RbxDom customProperties.lua path (read-only audit, no code changes) + status: pending + - id: meshpart-create + content: Add MeshPart creation via AssetService:CreateMeshPartAsync in reify.lua, skip MeshId in property loop + status: pending + - id: meshid-swap + content: "Add MeshId change detection in applyPatch.lua: recreate MeshPart, copy children/properties, swap in instanceMap" + status: pending + - id: csg-handler + content: Port CSGHandler from rbxsync to Atlas plugin (Plugin:Separate for decomposition, UnionAsync/SubtractAsync for reconstruction) + status: pending + - id: csg-twoway-decompose + content: "Add CSG decomposition to ChangeBatcher/encodeInstance: when encoding a CSG part, call Plugin:Separate and send component parts to server" + status: pending + - id: csg-server-store + content: "Server-side: store decomposed CSG component parts on disk and send them back to plugin during forward sync" + status: pending + - id: csg-reify-reconstruct + content: "Plugin-side: detect CSG component data in patches, reconstruct via CSGHandler in reify.lua" + status: pending + - id: csg-warning + content: Add warning for .rbxm-sourced CSG parts that have never been decomposed (appear as empty shells until first two-way sync) + status: pending +isProject: false +--- + +# Reconciler Robustness Overhaul + +## Tags/Attributes: Already Working (Verification Only) + +Tags and Attributes ARE correctly handled through the RbxDom `customProperties.lua` system ([plugin/rbx_dom_lua/customProperties.lua](plugin/rbx_dom_lua/customProperties.lua) lines 51-120): + +- **Tags** (lines 97-120): Uses `CollectionService:GetTags()` for read, `CollectionService:AddTag()/RemoveTag()` for write. Properly diffs existing tags against new tags. +- **Attributes** (lines 51-96): Uses `instance:GetAttributes()` for read, `instance:SetAttribute()` for write. Clears removed attributes. Skips reserved `RBX` prefixed names. + +They arrive in the `Properties` map from the server (e.g. `Properties.Tags`, `Properties.Attributes`) and flow through `setProperty` -> RbxDom `PropertyDescriptor:write` -> custom handler. The property application order within a single instance doesn't matter per user confirmation. + +No code changes needed -- just verify during implementation that this path is exercised. + +--- + +## Problem 1: MeshPart Creation Fails Silently + +**File:** [plugin/src/Reconciler/reify.lua](plugin/src/Reconciler/reify.lua) line 63 + +```lua +local createSuccess, instance = pcall(Instance.new, virtualInstance.ClassName) +``` + +`Instance.new("MeshPart")` creates an instance with empty MeshId. **MeshId is read-only** after creation in plugins. The server sends MeshId in Properties, but `setProperty` fails silently (property added to `unappliedProperties`). Result: every MeshPart synced from server has no mesh geometry. + +**rbxsync reference:** [Sync.luau](d:\UserGenerated\rbxsync\plugin\src\Sync.luau) lines 624-681 use `AssetService:CreateMeshPartAsync(meshId)`, then apply remaining properties with MeshId excluded. + +**Fix in reify.lua:** Before `Instance.new`, check for MeshPart and use `AssetService:CreateMeshPartAsync`: + +```lua +local instance +local meshIdAlreadySet = false + +if virtualInstance.ClassName == "MeshPart" then + local meshIdValue = virtualInstance.Properties.MeshId + if meshIdValue then + local decodeSuccess, meshId = decodeValue(meshIdValue, instanceMap) + if decodeSuccess and type(meshId) == "string" and meshId ~= "" then + local AssetService = game:GetService("AssetService") + local ok, meshPart = pcall(function() + return AssetService:CreateMeshPartAsync(meshId) + end) + if ok and meshPart then + instance = meshPart + meshIdAlreadySet = true + end + end + end +end + +if instance == nil then + local createSuccess + createSuccess, instance = pcall(Instance.new, virtualInstance.ClassName) + if not createSuccess then + addAllToPatch(unappliedPatch, virtualInstances, id) + return + end +end +``` + +Then in the property loop, skip MeshId if already set: + +```lua +if meshIdAlreadySet and propertyName == "MeshId" then + continue +end +``` + +--- + +## Problem 2: MeshId Swap on Existing MeshPart + +**File:** [plugin/src/Reconciler/applyPatch.lua](plugin/src/Reconciler/applyPatch.lua) lines 250-277 + +When an update changes MeshId on an existing MeshPart, `setProperty` fails (read-only). The property ends up in `unappliedUpdate` -- the MeshPart keeps the old mesh. + +**rbxsync reference:** [Sync.luau](d:\UserGenerated\rbxsync\plugin\src\Sync.luau) lines 852-941 detect MeshId change, recreate via `CreateMeshPartAsync`, copy properties/children, destroy old. + +**Fix in applyPatch.lua:** Add a MeshPart recreation block before the normal update logic: + +```lua +-- Before the normal property update loop: +if instance:IsA("MeshPart") and update.changedProperties + and update.changedProperties.MeshId then + + local decodeSuccess, newMeshId = decodeValue(update.changedProperties.MeshId, instanceMap) + + if decodeSuccess and type(newMeshId) == "string" and newMeshId ~= "" then + local AssetService = game:GetService("AssetService") + local ok, newMeshPart = pcall(function() + return AssetService:CreateMeshPartAsync(newMeshId) + end) + + if ok and newMeshPart then + -- Apply name + newMeshPart.Name = update.changedName or instance.Name + + -- Apply changed properties (excluding MeshId) + for propName, propValue in pairs(update.changedProperties) do + if propName ~= "MeshId" then + local ds, dv = decodeValue(propValue, instanceMap) + if ds then setProperty(newMeshPart, propName, dv) end + end + end + + -- Move children + for _, child in ipairs(instance:GetChildren()) do + child.Parent = newMeshPart + end + + -- Swap in instanceMap + local parent = instance.Parent + newMeshPart.Parent = parent + instanceMap:insert(update.id, newMeshPart) + instance.Parent = nil -- Not Destroy(), allows undo + + continue -- Skip normal update path + end + end +end +``` + +--- + +## Problem 3: CSG Parts Created as Empty Shells + +### Root Cause + +The server filters out CSG geometry data before sending to the plugin: + +- `**MeshData**` (BinaryString) -- can't be serialized to JSON/MessagePack ([src/web/api.rs](src/web/api.rs) line 3606-3608) +- `**PhysicalConfigData**` (SharedString) -- explicitly filtered ([src/web/interface.rs](src/web/interface.rs) lines 147-153) + +Even if these bytes reached the plugin, they're **not scriptable** -- no Roblox API exists to set `MeshData` or `PhysicalConfigData` on instances from Lua. No plugin API exists to load raw .rbxm bytes into instances either. + +**The only way to create CSG geometry in Studio is `BasePart:UnionAsync()` / `BasePart:SubtractAsync()` from component parts.** This is exactly what rbxsync does. + +### CSG Data Flow + +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ TWO-WAY SYNC (Studio → Server → Studio) │ +│ │ +│ 1. User edits CSG in Studio │ +│ 2. Plugin detects change via ChangeBatcher │ +│ 3. Plugin calls Plugin:Separate() to decompose → component parts │ +│ 4. Plugin encodes component parts + CSG metadata │ +│ 5. Server stores component data alongside instance │ +│ 6. On next forward sync: server sends component data │ +│ 7. Plugin reconstructs via UnionAsync/SubtractAsync │ +│ │ +│ LIMITATION: .rbxm-sourced CSG that was NEVER synced from Studio │ +│ will appear as empty shells until first two-way sync. │ +│ A warning is logged for these cases. │ +└─────────────────────────────────────────────────────────────────────┘ +``` + +### Implementation: 4 parts + +#### Part A: Port CSGHandler module + +Create [plugin/src/CSGHandler.lua](plugin/src/CSGHandler.lua), ported from [rbxsync CSGHandler.luau](d:\UserGenerated\rbxsync\plugin\src\CSGHandler.luau). + +Key functions: + +- `CSGHandler.isCSGOperation(instance)` -- check if UnionOperation/IntersectOperation/NegateOperation +- `CSGHandler.separate(pluginInstance, union)` -- decompose via `Plugin:Separate()`, returns `{addParts, subtractParts}` +- `CSGHandler.reconstruct(basePart, additionalParts, subtractParts)` -- rebuild via `UnionAsync/SubtractAsync` + +#### Part B: CSG decomposition in two-way sync (Plugin → Server) + +**File:** [plugin/src/ChangeBatcher/encodeInstance.lua](plugin/src/ChangeBatcher/encodeInstance.lua) + +When encoding a CSG instance for syncback, decompose it: + +1. Call `CSGHandler.separate(plugin, instance)` to get component parts +2. Encode each component part as a child instance under a special `_csg` key +3. Include CSG metadata (operation types: add vs subtract) +4. Send to server as part of the normal write patch + +The server write handler ([src/web/api.rs](src/web/api.rs)) needs to recognize the `_csg` data and store it. + +#### Part C: Server-side CSG storage and retrieval + +**File:** [src/web/api.rs](src/web/api.rs) and [src/change_processor.rs](src/change_processor.rs) + +When the server receives CSG component data via `/api/write`: + +1. Store component part snapshots alongside the CSG instance (e.g., as metadata or in a `_csg/` subdirectory in the snapshot tree) +2. During forward sync, when sending a CSG instance to the plugin, include the component data in the instance's properties/metadata + +This needs a new mechanism -- CSG component data is not a normal property. Options: + +- Add a `csgComponents` field to the `Instance` struct in [src/web/interface.rs](src/web/interface.rs) +- Or encode as a special metadata entry in `InstanceMetadata` + +#### Part D: CSG reconstruction in reify.lua + +**File:** [plugin/src/Reconciler/reify.lua](plugin/src/Reconciler/reify.lua) and [plugin/src/Reconciler/applyPatch.lua](plugin/src/Reconciler/applyPatch.lua) + +When reifying a CSG instance that has component data: + +1. Instead of `Instance.new("UnionOperation")`, queue it for deferred CSG reconstruction +2. Create the component parts first (as normal instances, unparented) +3. After all instances are created, reconstruct CSG via `CSGHandler.reconstruct()` +4. Apply properties (skip CFrame -- geometry defines position, per rbxsync line 1499-1509) +5. Parent the reconstructed union + +For CSG instances WITHOUT component data (from .rbxm, never two-way synced): + +- Create the empty shell via `Instance.new()` as today +- Log a warning: "CSG instance 'X' has no component data -- appears as empty shell" + +--- + +## Implementation Order + +The changes are independent enough to implement incrementally: + +1. **MeshPart creation** (reify.lua) -- immediate high-value fix, no server changes +2. **MeshId swap** (applyPatch.lua) -- immediate high-value fix, no server changes +3. **CSGHandler module** (new file) -- pure plugin code, self-contained +4. **CSG warning** (reify.lua) -- trivial addition +5. **CSG decomposition** (encodeInstance.lua + server) -- requires server changes +6. **CSG server storage** (api.rs + interface.rs) -- requires protocol thinking +7. **CSG reconstruction** (reify.lua) -- requires steps 5+6 + +Steps 1-4 can ship as one PR. Steps 5-7 require more design and could be a separate PR. + +--- + +## Files Modified + + +| File | Changes | +| ------------------------------------------------------------------------------------------ | ----------------------------------------------------------- | +| [plugin/src/Reconciler/reify.lua](plugin/src/Reconciler/reify.lua) | MeshPart creation, CSG warning, CSG deferred reconstruction | +| [plugin/src/Reconciler/applyPatch.lua](plugin/src/Reconciler/applyPatch.lua) | MeshId swap detection + recreation | +| [plugin/src/CSGHandler.lua](plugin/src/CSGHandler.lua) | **NEW** -- ported from rbxsync | +| [plugin/src/ChangeBatcher/encodeInstance.lua](plugin/src/ChangeBatcher/encodeInstance.lua) | CSG decomposition during two-way sync | +| [src/web/interface.rs](src/web/interface.rs) | CSG component data in Instance struct | +| [src/web/api.rs](src/web/api.rs) | CSG storage on write, CSG data on read | + + +## Files Verified (no changes) + + +| File | Status | +| ---------------------------------------------------------------------------------- | ------------------------------------------------------------------ | +| [plugin/rbx_dom_lua/customProperties.lua](plugin/rbx_dom_lua/customProperties.lua) | Tags (lines 97-120) and Attributes (lines 51-96) correctly handled | +| [plugin/src/Reconciler/setProperty.lua](plugin/src/Reconciler/setProperty.lua) | Routes to RbxDom descriptors, including custom handlers | +| [plugin/src/Reconciler/decodeValue.lua](plugin/src/Reconciler/decodeValue.lua) | Decodes all property types via RbxDom | +| [plugin/src/Reconciler/diff.lua](plugin/src/Reconciler/diff.lua) | Compares Tags/Attributes via Properties map | + + diff --git a/.cursor/plans/ref_path_and_dedup_integration_81a47dfe.plan.md b/.cursor/plans/ref_path_and_dedup_integration_81a47dfe.plan.md new file mode 100644 index 00000000..ace63727 --- /dev/null +++ b/.cursor/plans/ref_path_and_dedup_integration_81a47dfe.plan.md @@ -0,0 +1,88 @@ +--- +name: Ref path and dedup integration +overview: "Two deferred items from the ambiguous path handling feature: (1) migrate collect_referents() to build filesystem-name-compatible paths so ref round-trips work for slugified/dedup'd instances, and (2) integrate dedup_suffix.rs cleanup rules into the change_processor deletion handler.\"" +todos: + - id: ref-path-tentative-fs + content: Create tentative_fs_path() function and replace inst_path() calls in collect_referents/collect_all_paths + status: completed + - id: dedup-cleanup-change-processor + content: Integrate dedup_suffix cleanup rules into change_processor handle_tree_event removal handling + status: completed +isProject: false +--- + +# Syncback Ref Path Migration and Dedup Cleanup Integration + +## Problem 1: Ref paths break for slugified/dedup'd instance names + +### Root cause + +`collect_referents()` in [src/syncback/ref_properties.rs](src/syncback/ref_properties.rs) calls `inst_path()` which delegates to the legacy `ref_target_path()`. This produces escaped instance-name paths like `"Workspace/Hey\/Bro"`. But `get_instance_by_path()` in [src/snapshot/tree.rs](src/snapshot/tree.rs) now uses simple `path.split('/')` without escape handling. An instance named `"Hey/Bro"` gets path `"Workspace/Hey\/Bro"`, which splits into `["Workspace", "Hey\\/Bro"]` -- the escaped slash is NOT unescaped, so resolution fails. + +The fallback in `get_instance_by_path()` tries case-insensitive instance name matching, but the segment `"Hey\\/Bro"` does not match instance name `"Hey/Bro"`. + +### Fix: Build filesystem-compatible paths in collect_referents + +Create a `tentative_fs_path()` function that builds paths compatible with the new `get_instance_by_path()`: + +- Walk ancestors from target to root (same as `ref_target_path`) +- For each instance, compute a **tentative filesystem name**: `slugify(name)` + extension based on class/children +- Join with `/` (no escaping needed since slugified names can't contain `/`) + +The extension mapping uses the same class-to-middleware logic as `get_best_middleware()`: + +- `Script` -> check RunContext: `.server.luau` / `.client.luau` / `.plugin.luau` / `.legacy.luau` / `.local.luau`; if has children, directory (no ext) +- `LocalScript` -> `.local.luau` or directory +- `ModuleScript` -> `.luau` or directory +- `Folder`, `Configuration`, `Tool`, GUI classes -> directory (no ext) +- `StringValue` -> `.txt` +- `LocalizationTable` -> `.csv` +- Everything else -> `.model.json5` or directory if has children + +**Files to change:** + +- [src/syncback/ref_properties.rs](src/syncback/ref_properties.rs): Replace `inst_path()` calls with new `tentative_fs_path()`. Update `collect_all_paths()` similarly. +- [src/syncback/snapshot.rs](src/syncback/snapshot.rs): Update `inst_path()` to use the new path builder (or deprecate it for ref contexts). + +### How resolution works after this fix + +A ref path like `"Workspace/Hey_Bro.server.luau"`: + +1. Primary lookup: `filesystem_name_for()` returns `"Hey_Bro.server.luau"` (from instigating_source) -- **match** +2. If no instigating_source: fallback matches instance name `"Hey/Bro"` against segment `"Hey_Bro.server.luau"` -- no match, but this is expected for file-backed instances + +For directory instances like services: `"Workspace"` matches both filesystem name and instance name -- works as before. + +--- + +## Problem 2: Dedup cleanup not triggered on deletion in change_processor + +### Current state + +[src/syncback/dedup_suffix.rs](src/syncback/dedup_suffix.rs) provides `compute_cleanup_action()` with gap-tolerant, base-name-promotion, and group-to-1 rules. But `handle_tree_event()` in [src/change_processor.rs](src/change_processor.rs) (line 1043) only logs removals -- it doesn't check if a deletion leaves a dedup group needing suffix cleanup. + +### Fix: Add dedup cleanup after removals + +In `handle_tree_event()`, after the removal loop (line ~1070), for each removed instance: + +1. Find the parent instance (before the removal is applied to the tree) +2. Enumerate the parent's remaining children (excluding the removed one) +3. Group remaining children by dedup base name (parse `~N` suffixes from filesystem names via `parse_dedup_suffix()`) +4. For each dedup group affected by the removal, call `compute_cleanup_action()` +5. If the action is `RemoveSuffix` or `PromoteLowest`: + a. Call `suppress_path()` for both old and new paths + b. Rename the file on disk + c. Update meta file `name` field if needed + d. Update `RefPathIndex` via `update_ref_paths_after_rename()` + +**Files to change:** + +- [src/change_processor.rs](src/change_processor.rs): Add dedup cleanup logic after the removal loop in `handle_tree_event()`. Import `dedup_suffix::{parse_dedup_suffix, compute_cleanup_action}`. + +### Edge cases + +- The removed instance's parent must still exist in the tree when we enumerate siblings +- ProjectNode instances are excluded (already guarded) +- The cleanup rename must happen BEFORE `apply_patch_set()` removes the instance from the tree (since we need the parent relationship) +- VFS suppression must cover both the old path (being renamed from) and the new path (being renamed to) + diff --git a/.cursor/plans/remove_plugin_animations_f4a43647.plan.md b/.cursor/plans/remove_plugin_animations_f4a43647.plan.md new file mode 100644 index 00000000..f42eb850 --- /dev/null +++ b/.cursor/plans/remove_plugin_animations_f4a43647.plan.md @@ -0,0 +1,101 @@ +--- +name: Remove Plugin Animations +overview: Replace every `Flipper.Spring.new(...)` with `Flipper.Instant.new(...)` across all plugin components to make all UI transitions instant, and remove the 0.3s double-click delay in DomLabel expand/collapse. +todos: + - id: domlabel + content: "DomLabel.lua: Replace 2 Spring goals with Instant, remove task.delay(0.3) wrapper from expand logic" + status: completed + - id: page + content: "Page.lua: Replace Spring goal with Instant" + status: completed + - id: touchripple + content: "TouchRipple.lua: Replace 3 Spring goals with Instant, remove EXPAND_SPRING constant" + status: completed + - id: textbutton + content: "TextButton.lua: Replace 3 Spring goals with Instant, remove SPRING_PROPS constant" + status: completed + - id: textinput + content: "TextInput.lua: Replace 3 Spring goals with Instant, remove SPRING_PROPS constant" + status: completed + - id: notification + content: "Notification.lua: Replace 2 Spring goals with Instant" + status: completed + - id: dropdown + content: "Dropdown.lua: Replace Spring goal with Instant" + status: completed + - id: iconbutton + content: "IconButton.lua: Replace 2 Spring goals with Instant, remove HOVER_SPRING_PROPS constant" + status: completed + - id: checkbox + content: "Checkbox.lua: Replace Spring goal with Instant" + status: completed +isProject: false +--- + +# Remove All Animations From Plugin + +## Strategy + +Every animation in the plugin uses Flipper motors with `Flipper.Spring.new(value, { frequency, dampingRatio })` goals. The Flipper library already has `Flipper.Instant.new(value)` which sets the motor value immediately with no interpolation. The fix is a mechanical replacement of every `Flipper.Spring.new(...)` with `Flipper.Instant.new(...)`. + +This preserves the binding/motor infrastructure (no structural refactoring needed) while making every transition instant. + +Additionally, the DomLabel expand/collapse has a `task.delay(0.3)` that waits to distinguish single-click from double-click before expanding. This delay should be removed so expansion is immediate on click. + +## Files to Change (10 files, ~20 replacements) + +### 1. [DomLabel.lua](plugin/src/App/Components/PatchVisualizer/DomLabel.lua) -- Expand/collapse + +- Line 436: `Flipper.Spring.new(24, {...})` -> `Flipper.Instant.new(24)` +- Line 602: `Flipper.Spring.new(goalHeight, {...})` -> `Flipper.Instant.new(goalHeight)` +- Lines 545-608: Remove the `task.delay(0.3)` wrapper so expansion logic runs immediately on single click (keep double-click detection, just don't delay the expand) + +### 2. [Page.lua](plugin/src/App/Page.lua) -- Page transitions + +- Line 65: `Flipper.Spring.new(...)` -> `Flipper.Instant.new(...)` + +### 3. [TouchRipple.lua](plugin/src/App/Components/TouchRipple.lua) -- Click ripple effect + +- Line 84: `Flipper.Spring.new(1, EXPAND_SPRING)` -> `Flipper.Instant.new(1)` +- Line 85: `Flipper.Spring.new(1, EXPAND_SPRING)` -> `Flipper.Instant.new(1)` +- Line 93: `Flipper.Spring.new(0, {...})` -> `Flipper.Instant.new(0)` +- Remove unused `EXPAND_SPRING` constant + +### 4. [TextButton.lua](plugin/src/App/Components/TextButton.lua) -- Hover/enabled + +- Line 36: `Flipper.Spring.new(...)` -> `Flipper.Instant.new(...)` +- Line 64: `Flipper.Spring.new(1, SPRING_PROPS)` -> `Flipper.Instant.new(1)` +- Line 70: `Flipper.Spring.new(0, SPRING_PROPS)` -> `Flipper.Instant.new(0)` +- Remove unused `SPRING_PROPS` constant + +### 5. [TextInput.lua](plugin/src/App/Components/TextInput.lua) -- Hover/enabled + +- Line 34: `Flipper.Spring.new(...)` -> `Flipper.Instant.new(...)` +- Line 101: `Flipper.Spring.new(1, SPRING_PROPS)` -> `Flipper.Instant.new(1)` +- Line 107: `Flipper.Spring.new(0, SPRING_PROPS)` -> `Flipper.Instant.new(0)` +- Remove unused `SPRING_PROPS` constant + +### 6. [Notification.lua](plugin/src/App/Components/Notifications/Notification.lua) -- Appear/dismiss + +- Line 43: `Flipper.Spring.new(0, {...})` -> `Flipper.Instant.new(0)` +- Line 50: `Flipper.Spring.new(1, {...})` -> `Flipper.Instant.new(1)` + +### 7. [Dropdown.lua](plugin/src/App/Components/Dropdown.lua) -- Open/close + +- Line 39: `Flipper.Spring.new(...)` -> `Flipper.Instant.new(...)` + +### 8. [IconButton.lua](plugin/src/App/Components/IconButton.lua) -- Hover circle + +- Line 41: `Flipper.Spring.new(1, HOVER_SPRING_PROPS)` -> `Flipper.Instant.new(1)` +- Line 45: `Flipper.Spring.new(0, HOVER_SPRING_PROPS)` -> `Flipper.Instant.new(0)` +- Remove unused `HOVER_SPRING_PROPS` constant + +### 9. [Checkbox.lua](plugin/src/App/Components/Checkbox.lua) -- Active state + +- Line 26: `Flipper.Spring.new(...)` -> `Flipper.Instant.new(...)` + +## Not Touched + +- **Spinner.lua** -- Uses `RenderStepped` rotation, not Flipper springs. This is a loading indicator, not a decorative animation. Keeping it. +- **bindingUtil.lua** -- `mapLerp`, `blendAlpha`, `fromMotor` are utility functions still needed by the instant-value bindings. No changes needed. +- **Flipper package** -- Vendored, not modified. diff --git a/.cursor/plans/rewrite_matching_algorithm_df45bbe1.plan.md b/.cursor/plans/rewrite_matching_algorithm_df45bbe1.plan.md new file mode 100644 index 00000000..403f9dd3 --- /dev/null +++ b/.cursor/plans/rewrite_matching_algorithm_df45bbe1.plan.md @@ -0,0 +1,156 @@ +--- +name: Rewrite matching algorithm +overview: "Replace the multi-pass matching algorithm in all 3 implementations with a recursive change-count scoring function and greedy assignment. Sort key is a tuple: (change_count, child_index_A, child_index_B). Fast-paths for Ref pins and (Name, ClassName) grouping." +todos: + - id: rewrite-lua-matching + content: "Rewrite Lua matching.lua: recursive computeChangeCount + greedy assign with Ref pin and (Name, ClassName) fast-paths" + status: completed + - id: rewrite-rust-forward + content: "Rewrite Rust forward sync matching.rs: recursive count_changes + greedy assign using variant_eq" + status: completed + - id: rewrite-rust-syncback + content: "Rewrite Rust syncback matching.rs: recursive count_changes + greedy assign with hash fast-path" + status: completed + - id: verify-tests + content: Run cargo test to verify all 714 tests still pass + status: completed +isProject: false +--- + +# Rewrite Matching Algorithm + +Replace all 3 matching implementations with one recursive scoring function and greedy assignment. + +## matchChildren (entry point, called per parent) + +``` +matchChildren(sideA, sideB, parentA, parentB) -> (matched, unmatchedA, unmatchedB): + + // Fast-path 1: Ref pin (confirmed identity, highest priority) + // Scan parentA and parentB for Ref properties (PrimaryPart, Value, etc.) + // that point to children. If parentA.PrimaryPart = childX on sideA and + // parentB.PrimaryPart = childY on sideB, pin (X, Y) as matched. + // Remove pinned children from sideA/sideB. + for each Ref property name shared by parentA and parentB: + targetA = parentA[refProp] + targetB = parentB[refProp] + if targetA in sideA and targetB in sideB: + match (targetA, targetB) + remove both from sideA/sideB + + // Fast-path 2: Group remaining by (Name, ClassName) + // 1:1 groups get instant-matched with zero scoring. + groups = group sideA and sideB by (Name, ClassName) + for each group: + if exactly 1 on each side: instant match, remove both + + // Ambiguous groups: score + greedy assign + for each group with multiple on at least one side: + pairs = [] + bestSoFar = infinity + for each (A, B) in cross-product of group: // iteration order = child order + cost = computeChangeCount(A, B, bestSoFar) // RECURSIVE, early-exits + pairs.append((cost, A, B)) + if cost < bestSoFar: bestSoFar = cost + + stable sort pairs by cost ascending + // Stable sort preserves insertion order on ties = original child order + greedy assign: pick first unmatched, remove both, repeat + + // Leftovers = new/deleted +``` + +The two fast-paths resolve the vast majority of children with zero scoring. Only truly ambiguous groups (same Name, same ClassName, multiple candidates) ever call `computeChangeCount`. + +## computeChangeCount (recursive scoring function) + +One function. Returns the total number of changes the reconciler would need to make to turn A into B, **including the entire subtree**. + +``` +computeChangeCount(A, B, bestSoFar) -> number: + // Cheap: all own diffs in one shot (flat loops, no branching) + cost = countOwnDiffs(A, B) // properties + tags + attributes + if cost >= bestSoFar: return cost + + // Expensive: recursive children matching + childResult = matchChildren(A.children, B.children, A, B) // RECURSE + for each (childA, childB) in childResult.matched: + cost += computeChangeCount(childA, childB, bestSoFar - cost) + if cost >= bestSoFar: return cost + cost += count(childResult.unmatched) * UNMATCHED_PENALTY + + return cost + +countOwnDiffs(A, B) -> number: + cost = 0 + for each property key in union(A.properties, B.properties): + if values differ or key missing on one side: cost += 1 + for each tag in symmetricDifference(A.Tags, B.Tags): cost += 1 + for each attr key in union(A.Attributes, B.Attributes): + if missing or value differs: cost += 1 + return cost +``` + +`computeChangeCount` and `matchChildren` are **mutually recursive**. Base case: leaf nodes (no children). + +Name and ClassName are NOT in the change count -- the (Name, ClassName) grouping in `matchChildren` already guarantees they match for any pair that reaches `computeChangeCount`. + +## Unmatched Child Penalty + +Each unmatched child (create or delete an entire instance) adds a **large constant** (`UNMATCHED_PENALTY`) to the parent pair's cost. No need to recursively calculate the subtree -- creating/deleting an instance is categorically more expensive than tweaking properties, so a flat constant correctly prioritizes pairings that avoid creates/deletes. + +`UNMATCHED_PENALTY` should be large enough that any number of property diffs on matched children is still cheaper than one unmatched child. A value like `10,000` works (no real instance has 10K properties). + +## Sort Key and Stability + +Sort key is `change_count`, ascending. On ties, original child order is preserved. + +- **Rust:** `slice::sort_by` is stable natively. No extra work needed. +- **Lua:** `table.sort` is NOT stable (Luau uses non-stable quicksort). Use a simple ad-hoc stable sort: pairs are built with their insertion index, and the Lua sort comparator breaks ties by index. Something like `if a.cost == b.cost then return a.idx < b.idx end; return a.cost < b.cost`. + +## Known Limitation: Greedy vs Optimal + +The greedy algorithm (pick lowest-cost pair first) is NOT globally optimal. Edge cases exist where a locally-best pick forces a globally-worse assignment. The Hungarian algorithm (O(N^3)) would give optimal results, but ambiguous groups are typically small (2-10 instances) and the greedy works correctly for the common case. Flagged for revisiting if real-world games hit this edge case. + +## What Counts as a Change (+1 each) + +- **Each differing property:** iterate union of property keys on both sides. Each key where values differ or one side missing = +1. Includes CFrame, Size, Color3, Source, Material, Ref properties (PrimaryPart, Value), everything. Tags and Attributes are excluded from this loop (counted separately below). +- **Tags:** count individual tag operations. Each tag that needs to be added or removed = +1. If A has `{"Foo", "Bar"}` and B has `{"Bar", "Baz"}`, cost = 2 (remove "Foo", add "Baz"). +- **Attributes:** count individual attribute operations. Each attribute that needs to be added, removed, or changed = +1. If A has `{X=1, Y=2}` and B has `{Y=3, Z=4}`, cost = 3 (remove X, change Y, add Z). +- **Recursive children cost:** total from `matchChildren` (matched pair costs + unmatched subtree costs). + +Things NOT in the change count: + +- Name -- grouping guarantees match. +- ClassName -- grouping guarantees match. +- Parent -- structural. + +## Files to Rewrite + +### 1. Lua -- `[plugin/src/Reconciler/matching.lua](plugin/src/Reconciler/matching.lua)` + +Complete rewrite. Three functions: `matchChildren`, `computeChangeCount`, `costOfEntireInstance`. Property comparison via pcall reads from Studio instances vs virtual `.Properties`. Ref pin scans parent's Ref properties. (Name, ClassName) grouping as fast-path. + +### 2. Rust forward sync -- `[src/snapshot/matching.rs](src/snapshot/matching.rs)` + +Complete rewrite. Same three functions. Property comparison using `variant_eq()` from `[src/variant_eq.rs](src/variant_eq.rs)`. Operates on `InstanceSnapshot` vs `RojoTree` instances. + +### 3. Rust syncback -- `[src/syncback/matching.rs](src/syncback/matching.rs)` + +Complete rewrite. Same three functions. Hash-match fast-path: if `new_hash == old_hash`, return 0 without checking properties. Otherwise `variant_eq()` per property. Operates on `WeakDom` instances. + +## What Gets Deleted + +- All multi-pass infrastructure (Pass 1/2/3, `_pass1NameAndClass`, `_pass3Similarity`, etc.) +- `compute_similarity` / `compute_forward_similarity` / `_computeSimilarity` +- `collect_ref_property_targets`, `pass2_ref_discriminators` +- Replaced by: `matchChildren`, `computeChangeCount`, `costOfEntireInstance` + +## Correctness + +- Ref-pinned pairs are pulled out first -- confirmed identity, never wrong +- (Name, ClassName) grouping prevents cross-class or cross-name pairing +- Within ambiguous groups, recursive change counting picks the pair requiring fewest total reconciler operations across the entire subtree +- Child order tiebreaker ensures determinism on equal scores +- Unmatched subtree cost correctly penalizes pairings that cause cascading creates/deletes + diff --git a/.cursor/plans/rokit_install_audit_fixes_e465275a.plan.md b/.cursor/plans/rokit_install_audit_fixes_e465275a.plan.md new file mode 100644 index 00000000..e02ce9cf --- /dev/null +++ b/.cursor/plans/rokit_install_audit_fixes_e465275a.plan.md @@ -0,0 +1,390 @@ +--- +name: Rokit Install Audit Fixes +overview: Fix the EPERM crash when deleting a globally installed atlas.exe, and harden the entire rokit installation pipeline in the VS Code extension for reliability across Windows, macOS, and Linux. +todos: + - id: bug1-elevated-delete + content: Add elevatedDelete() helper and fix the 'Switch to Rokit' flow in openMenu.ts to handle EPERM/EACCES/EBUSY with platform-specific elevation + status: completed + - id: bug2-progress + content: Wrap installRojo() in vscode.window.withProgress() with stage reporting + status: completed + - id: bug3-github-api + content: Replace eryn.io proxy URL with official GitHub API URL in installRojo.ts + status: completed + - id: bug4-temp-cleanup + content: Delete temp rokit binary after successful self-install + status: completed + - id: bug5-path-dedup + content: Deduplicate PATH entry and verify rokit accessibility before rokit add + status: completed + - id: bug6-stream + content: Replace manual stream piping with stream.promises.pipeline() + status: completed + - id: bug7-rokit-add-existing + content: Use 'rokit add --global --force' + 'rokit trust' + verify atlas --version + non-fatal plugin install + status: completed + - id: feat-check-updates + content: Add 'Check for Updates' menu item in openMenu.ts for rokit-managed installs, handle project-level + global rokit update, reinstall Studio plugin after update + status: completed + - id: feat-auto-update-check + content: Add background update check on activation using rokit update --check (once per day, project + global), prompt user with Update button that triggers rokit update + plugin reinstall + status: completed +isProject: false +--- + +# Rokit Installation Audit and Fixes + +## Audit Findings + +### How Rokit Installation Works (No Elevation Needed) + +Rokit installs entirely to `~/.rokit/` (user-level directory). `rokit self-install` modifies user-level PATH environment variables (HKCU on Windows, shell rc files on Unix). **Neither rokit installation nor `rokit add` requires elevation.** + +The only scenario requiring elevation is the "Switch to Rokit" migration flow, which tries to delete an existing globally installed `atlas.exe` from a protected directory like `C:\Program Files\Atlas` (written by [install.ps1](install.ps1)) or `/usr/local/bin/atlas` (written by [install.sh](install.sh)). + +### Issues Found (by severity) + +--- + +**BUG 1 -- CRITICAL: `fs.unlink` fails without elevation, crashes the entire flow** + +In [vscode-rojo/src/commands/openMenu.ts](vscode-rojo/src/commands/openMenu.ts) line 133: + +```typescript +return fs.unlink(install.resolvedPath) +``` + +When atlas.exe is in `C:\Program Files\Atlas` or `/usr/local/bin/`, this throws EPERM/EACCES. The `.then()` error handler on line 139 catches it but just shows a generic message and the flow dies. Additionally, on Windows the exe may be **locked by a running process** (atlas serve), causing EBUSY even with elevation. + +**Fix:** Add an `elevatedDelete()` helper that: + +1. Tries `fs.unlink()` first +2. On EPERM/EACCES, uses platform-specific elevated delete: + - **Windows:** `powershell -Command "Start-Process powershell -ArgumentList '-Command Remove-Item \"path\"' -Verb RunAs -Wait"` (triggers UAC prompt) + - **macOS:** `osascript -e 'do shell script "rm path" with administrator privileges'` (triggers password dialog) + - **Linux:** `pkexec rm path` (triggers PolicyKit dialog) +3. On EBUSY (Windows exe locked), shows a specific error message telling the user to stop atlas serve first + +--- + +**BUG 2 -- HIGH: No progress indicator during installation** + +In [vscode-rojo/src/installRojo.ts](vscode-rojo/src/installRojo.ts) `installRojo()`: + +The entire flow (download rokit zip ~3.5MB, extract, self-install, `rokit add` atlas) runs silently. Users see zero feedback and may think the extension froze. + +**Fix:** Wrap the body of `installRojo()` in `vscode.window.withProgress()` with `ProgressLocation.Notification` and `cancellable: false`. Report progress at each stage: + +- "Downloading Rokit..." +- "Installing Rokit..." +- "Installing Atlas via Rokit..." + +--- + +**BUG 3 -- HIGH: Third-party release URL is a single point of failure** + +In [vscode-rojo/src/installRojo.ts](vscode-rojo/src/installRojo.ts) line 113: + +```typescript +const latestReleaseResponse = await fetch( + "https://latest-github-release.eryn.io/rojo-rbx/rokit", +) +``` + +This third-party proxy (`eryn.io`) is not under our control. If it goes down, installation breaks. The official rokit install scripts use the GitHub API directly. + +**Fix:** Replace with the official GitHub API URL: `https://api.github.com/repos/rojo-rbx/rokit/releases/latest`. Verified: response schema is identical, both return the same `GitHubRelease` shape. Add an `Accept: application/vnd.github+json` header per GitHub API best practice. + +--- + +**BUG 4 -- MEDIUM: Temp files never cleaned up** + +In [vscode-rojo/src/installRojo.ts](vscode-rojo/src/installRojo.ts) line 143-170: + +The rokit binary is downloaded to `os.tmpdir()` but never deleted after `rokit self-install` succeeds. + +**Fix:** Add `fs.promises.unlink(tempPath).catch(() => {})` after the `exec(self-install)` call succeeds. + +--- + +**BUG 5 -- MEDIUM: PATH update is fragile** + +In [vscode-rojo/src/installRojo.ts](vscode-rojo/src/installRojo.ts) lines 177-181: + +```typescript +if ("PATH" in process.env) { + const envPath = process.env.PATH!.split(path.delimiter) + envPath.push(path.join(os.homedir(), ".rokit", "bin")) + process.env.PATH = envPath.join(path.delimiter) +} +``` + +This only updates the extension's own process environment. But the subsequent `rokit add --global` on line 184 runs via `exec()` which inherits this env, so it should work. The concern is: + +- It doesn't check if the path is already present (could add duplicates on repeated attempts) +- The rokit binary path might differ on some systems + +**Fix:** Check for duplicates before appending. Also verify the rokit binary is actually accessible after the PATH update by doing a quick `which("rokit")` check before proceeding to `rokit add`. + +--- + +**BUG 6 -- LOW: Stream error handling is fragile** + +In [vscode-rojo/src/installRojo.ts](vscode-rojo/src/installRojo.ts) lines 51-60, 149-163: + +`promisifyStream` resolves on whichever of "close"/"finish"/"end" fires first. The pipeline/pipe chain has multiple potential failure modes that aren't properly propagated. + +**Fix:** Use `stream.promises.pipeline()` (available in Node 15+, which VS Code ships with) instead of manually piping and tracking events. This gives proper error propagation and backpressure handling. Alternatively, simplify to `await promisify(stream.pipeline)(download.body!, unzipper.ParseOne(), fs.createWriteStream(tempPath))`. + +--- + +**BUG 7 -- HIGH: `rokit add --global` crashes if atlas is already configured** + +In [vscode-rojo/src/installRojo.ts](vscode-rojo/src/installRojo.ts) line 184: + +```typescript +await exec("rokit add --global UserGeneratedLLC/rojo atlas") +``` + +This is the **only** action after rokit is confirmed installed. It fails if atlas is already in the project's `rokit.toml` or already added globally. The scenario: + +1. Project has `rokit.toml` with atlas listed but `rokit install` hasn't been run yet +2. `which("atlas")` finds the rokit shim at `~/.rokit/bin/atlas` +3. `atlas --version` fails (rokit says tool not installed), stderr contains "rokit" -> `getRojoInstall` returns null +4. User sees "Install Atlas now", clicks it +5. `isRokitInstalled()` returns true (skips rokit download) +6. `rokit add --global` fails because atlas is already configured -> **onboarding dies** + +**Fix:** Use `--force` to handle "already exists", trust the tool, then verify and install the plugin: + +```typescript +await exec("rokit add --global --force UserGeneratedLLC/rojo atlas") +await exec("rokit trust UserGeneratedLLC/rojo") +await exec("atlas --version") + +// Install the Studio plugin (non-fatal -- don't crash onboarding if Studio isn't installed) +try { + await exec("atlas plugin install") +} catch (e: any) { + vscode.window.showWarningMessage( + `Atlas installed successfully, but the Studio plugin could not be installed: ${e.stderr || e}. ` + + `You can install it later from the Atlas menu or by running "atlas plugin install".` + ) +} +``` + +- `--force` on `rokit add` handles the "already exists" case cleanly -- no try/catch needed, it re-adds/updates regardless +- `rokit trust` marks `UserGeneratedLLC/rojo` as trusted so rokit won't prompt the user for confirmation when running atlas (without trust, `rokit install` and first-run shims can block waiting for user input) +- `atlas --version` verifies the tool is actually usable after add +- `atlas plugin install` writes the embedded Studio plugin to disk (non-fatal if Studio isn't installed yet) + +--- + +--- + +**FEATURE: "Check for Updates" menu item for Rokit-managed installs** + +In [vscode-rojo/src/commands/openMenu.ts](vscode-rojo/src/commands/openMenu.ts), the `generateProjectMenu()` return array (line 315) has action items like Studio, Syncback, Sourcemap, Install Plugin. There is no way to update Atlas from the extension. + +`generateProjectMenu()` already tracks `installType` (line 168) and whether it's `InstallType.rokit`. We use this to conditionally show the item. + +**Implementation:** + +1. Add a menu item in `generateProjectMenu()` return array, conditionally included when `installType === InstallType.rokit`: + +```typescript +...(installType === InstallType.rokit + ? [ + { + label: "$(sync) Check for Updates", + description: `Atlas v${allRojoVersions[0]}`, + action: "checkUpdates", + projectFile: projectFiles[0], + }, + ] + : []), +``` + +1. Add `"checkUpdates"` case in the `onDidAccept` handler (after the `"install"` case). The handler needs to support both project-level `rokit.toml` and global installations: + +```typescript +case "checkUpdates": { + if (!selectedItem.projectFile) return + const updateFolder = path.dirname(selectedItem.projectFile.path.fsPath) + + input.hide() + + await vscode.window.withProgress( + { location: vscode.ProgressLocation.Notification, title: "Checking for Atlas updates..." }, + async (progress) => { + // Try project-level update first, fall back to global + let updateOutput: string + try { + const result = await exec("rokit update atlas", { cwd: updateFolder }) + updateOutput = result.stdout || result.stderr + } catch { + try { + const result = await exec("rokit update --global atlas") + updateOutput = result.stdout || result.stderr + } catch (e: any) { + vscode.window.showErrorMessage(`Could not update Atlas: ${e.stderr || e}`) + return + } + } + + // Re-install the Studio plugin so it matches the new binary. + // The plugin is embedded in the atlas binary at compile time + // (include_bytes! in src/cli/plugin.rs), so updating the binary + // without re-installing leaves a stale AtlasManagedPlugin.rbxm + // in Studio's plugins folder. Mismatched versions break live sync + // due to protocol version enforcement. + progress.report({ message: "Reinstalling Studio plugin..." }) + try { + await exec("atlas plugin install") + } catch (e: any) { + vscode.window.showWarningMessage( + `Atlas updated, but plugin reinstall failed: ${e.stderr || e}. ` + + `Run "atlas plugin install" manually to avoid version mismatch.` + ) + } + + vscode.window.showInformationMessage(updateOutput.trim() || "Atlas is up to date.") + }, + ) + break +} +``` + +The strategy: try `rokit update atlas` in the project directory first. If atlas isn't in the project's `rokit.toml`, this fails and we fall back to `rokit update --global atlas`. Rokit's output tells the user whether an update was applied or they're already current. + +After a successful update, immediately runs `atlas plugin install` to overwrite the stale `AtlasManagedPlugin.rbxm` in Studio's plugins folder. The plugin is baked into the atlas binary via `include_bytes!` at compile time (`src/cli/plugin.rs:12`), so the `.rbxm` in Studio never auto-updates -- it must be explicitly reinstalled. If the plugin reinstall fails (e.g. Studio not installed), a warning is shown with manual instructions rather than failing the whole update. + +--- + +**FEATURE: Background update check on extension activation** + +In [vscode-rojo/src/extension.ts](vscode-rojo/src/extension.ts) `activate()`: + +Rokit has `rokit update atlas --check` which checks for updates without installing. Supports both project-level (`rokit update atlas --check` in the project dir, reads `rokit.toml`) and global (`rokit update --global atlas --check`). + +**Implementation:** Add a new file `vscode-rojo/src/checkForUpdates.ts` with a `checkForAtlasUpdates(context)` function, called fire-and-forget from `activate()`: + +```typescript +export async function checkForAtlasUpdates(context: vscode.ExtensionContext) { + // Throttle: once per day + const lastCheck = context.globalState.get("atlas::lastUpdateCheck") + const ONE_DAY = 24 * 60 * 60 * 1000 + if (lastCheck && Date.now() - lastCheck < ONE_DAY) return + + // Check if rokit is available + const rokitPath = await which("rokit").catch(() => null) + if (!rokitPath) return + + const workspaceFolder = vscode.workspace.workspaceFolders?.[0]?.uri.fsPath + + // rokit update --check prints update info if available, or "up to date" if current. + // Check both project-level and global. + let checkOutput: string | null = null + let updateScope: "project" | "global" = "global" + + // Try project-level first (if workspace is open) + if (workspaceFolder) { + try { + const result = await exec("rokit update atlas --check", { cwd: workspaceFolder }) + const output = (result.stdout + result.stderr).trim() + if (output && !output.toLowerCase().includes("up to date")) { + checkOutput = output + updateScope = "project" + } + } catch { + // atlas not in project rokit.toml -- fall through + } + } + + // Try global if no project-level update found + if (!checkOutput) { + try { + const result = await exec("rokit update --global atlas --check") + const output = (result.stdout + result.stderr).trim() + if (output && !output.toLowerCase().includes("up to date")) { + checkOutput = output + updateScope = "global" + } + } catch { + // atlas not installed globally via rokit + } + } + + context.globalState.update("atlas::lastUpdateCheck", Date.now()) + + if (!checkOutput) return + + const choice = await vscode.window.showInformationMessage( + `An Atlas update is available. ${checkOutput}`, + "Update", + "Dismiss", + ) + + if (choice !== "Update") return + + await vscode.window.withProgress( + { location: vscode.ProgressLocation.Notification, title: "Updating Atlas..." }, + async (progress) => { + try { + if (updateScope === "project" && workspaceFolder) { + await exec("rokit update atlas", { cwd: workspaceFolder }) + } else { + await exec("rokit update --global atlas") + } + } catch (e: any) { + vscode.window.showErrorMessage(`Could not update Atlas: ${e.stderr || e}`) + return + } + + progress.report({ message: "Reinstalling Studio plugin..." }) + try { + await exec("atlas plugin install") + } catch { + vscode.window.showWarningMessage( + "Atlas updated, but Studio plugin reinstall failed. Run 'atlas plugin install' manually.", + ) + } + + vscode.window.showInformationMessage("Atlas updated successfully.") + }, + ) +} +``` + +Called from `activate()` as fire-and-forget (no `await`) so it never blocks extension startup: + +```typescript +// In activate(), after all synchronous setup is done: +checkForAtlasUpdates(context).catch(() => {}) +``` + +Key design decisions: + +- **Uses `rokit update --check`**: Lets rokit handle version resolution natively -- no GitHub API calls, no manual semver parsing +- **Project-level first, then global**: Checks the project's `rokit.toml` first (if workspace is open), falls back to global. When updating, uses the same scope that found the update +- **Throttle**: Once per day via `globalState`, so frequent restarts don't spam the user +- **Silent failures**: Every step is wrapped in try/catch that returns early. Missing rokit, no atlas configured, network errors -- all silently ignored +- **Non-blocking**: Called without `await` so extension activation is never delayed + +--- + +### Summary of What's Working Correctly + +- Asset name regex (`-(?\w+)-(?\w+)\.zip$`) correctly matches all 6 rokit 1.2.0 release assets (linux/macos/windows x x86_64/aarch64, all `.zip`) +- Platform/arch mapping covers all common Node.js platforms (win32, darwin, linux) and architectures (x64, arm64) +- `rokit self-install` correctly sets up `~/.rokit/bin` in user PATH on all platforms +- `chmod 0o755` is correctly applied on Unix after extraction +- `getRojoInstall` correctly detects rokit-managed vs globally installed atlas + +## Files to Change + +- [vscode-rojo/src/installRojo.ts](vscode-rojo/src/installRojo.ts) -- Bugs 2, 3, 4, 5, 6, 7 +- [vscode-rojo/src/commands/openMenu.ts](vscode-rojo/src/commands/openMenu.ts) -- Bug 1, Check for Updates menu item +- [vscode-rojo/src/checkForUpdates.ts](vscode-rojo/src/checkForUpdates.ts) -- NEW: background update check logic +- [vscode-rojo/src/extension.ts](vscode-rojo/src/extension.ts) -- Call checkForAtlasUpdates() on activation + diff --git a/.cursor/plans/strip_dedup_suffix_automatically_3d35c847.plan.md b/.cursor/plans/strip_dedup_suffix_automatically_3d35c847.plan.md new file mode 100644 index 00000000..2deafdfa --- /dev/null +++ b/.cursor/plans/strip_dedup_suffix_automatically_3d35c847.plan.md @@ -0,0 +1,117 @@ +--- +name: Strip dedup suffix automatically +overview: Eliminate unnecessary `.meta.json5` files for dedup'd instances by having forward sync automatically strip `~N` dedup suffixes from filenames when deriving instance names, and having syncback skip writing meta `name` fields for dedup-only cases. +todos: + - id: strip-fn + content: Add `strip_dedup_suffix()` to `dedup_suffix.rs` and ensure it is importable from snapshot code + status: completed + - id: forward-files + content: Modify `file_name_for_path()` in `metadata.rs` to strip `~N` dedup suffixes + status: completed + - id: forward-dirs + content: Modify `get_dir_middleware()` in `snapshot_middleware/mod.rs` to strip `~N` from directory names + status: completed + - id: syncback-meta + content: Change `name_for_inst()` needs_meta condition to `needs_slugify` only (remove dedup check) + status: completed + - id: change-processor + content: Update file rename handler in `change_processor.rs` to use `slugified_new_name` for meta name conditions + status: completed + - id: update-tests + content: Update existing tests and fixtures for new dedup-only behavior (no meta name) + status: completed + - id: new-tests + content: "Add new tests: forward sync stripping, edge cases, syncback dedup-only, roundtrip without meta" + status: completed +isProject: false +--- + +# Auto-Strip Dedup Suffixes to Eliminate Redundant Meta Files + +## Problem + +When syncback encounters duplicate instance names (e.g., two instances named "LocalScript"), it creates: + +- `LocalScript.local.luau` (first) +- `LocalScript~2.local.luau` (second, with dedup suffix) +- `LocalScript~2.meta.json5` with `name: "LocalScript"` (meta file to recover real name) + +The meta file is redundant because `~N` is a well-defined system convention -- the `~` character is forbidden in instance names (always slugified to `_`), so `~N` can only come from the dedup system. Forward sync can strip it automatically. + +## Safety Argument + +- The `~` character is in `SLUGIFY_CHARS` (`[file_names.rs](src/syncback/file_names.rs)`), so any instance literally named `Foo~2` would be slugified to `Foo_2` during syncback. Natural tilde names cannot produce the `~N` pattern. +- Existing projects with meta files still work: meta `name` field takes priority over filename-derived name (`[meta_file.rs](src/snapshot_middleware/meta_file.rs)` `apply_name`). +- Non-dedup patterns like `~0`, `~abc`, `~` are NOT stripped (only `~N` where N > 0). + +## Changes + +### 1. Add `strip_dedup_suffix()` utility + +Add to `[src/syncback/dedup_suffix.rs](src/syncback/dedup_suffix.rs)` alongside `parse_dedup_suffix`: + +```rust +pub fn strip_dedup_suffix(name: &str) -> &str { + parse_dedup_suffix(name).map_or(name, |(base, _)| base) +} +``` + +### 2. Forward sync: strip dedup from file names + +In `[src/snapshot/metadata.rs](src/snapshot/metadata.rs)`, `SyncRule::file_name_for_path()` (line 330) -- after stripping the middleware extension, also strip `~N`: + +```rust +pub fn file_name_for_path<'a>(&self, path: &'a Path) -> anyhow::Result<&'a str> { + let name = /* existing logic to strip middleware extension */; + Ok(strip_dedup_suffix(name)) +} +``` + +The return type stays `&'a str` because stripping `~N` returns a sub-slice of the same string. + +### 3. Forward sync: strip dedup from directory names + +In `[src/snapshot_middleware/mod.rs](src/snapshot_middleware/mod.rs)`, `get_dir_middleware()` (line 116) -- strip `~N` from `dir_name` before returning: + +```rust +let dir_name = strip_dedup_suffix(dir_name); +``` + +Return type stays `&'path str` (sub-slice of the original path). + +### 4. Syncback: don't require meta name for dedup-only + +In `[src/syncback/file_names.rs](src/syncback/file_names.rs)`, `name_for_inst()` line 79 -- change: + +```rust +// Before: +let needs_meta = needs_slugify || deduped_slug != base; +// After: +let needs_meta = needs_slugify; +``` + +When only dedup happened (no slugification), forward sync will recover the correct name by stripping `~N`. + +### 5. Two-way sync: update meta name conditions in change_processor + +In `[src/change_processor.rs](src/change_processor.rs)`, the file rename handler (lines 1569, 1581) currently uses `deduped_new_name != *new_name` to decide whether meta name is needed. Change both to `slugified_new_name != *new_name`, matching the directory handler (line 1401) which already does this correctly. + +### 6. Module visibility + +Ensure `strip_dedup_suffix` is importable from snapshot code. May need to adjust `pub` visibility on the `syncback::dedup_suffix` module, or alternatively place the function in a shared location if cross-module import is problematic. + +### 7. Update existing tests and fixtures + +- `**src/syncback/file_names.rs` tests:** Update tests like `name_for_inst_dedup_collision` (line 826) and `nfi_clean_name_dedup_still_needs_meta_dir` (line 1175) -- these should now assert `needs_meta = false` for dedup-only cases. +- `**rojo-test/build-tests/dedup_suffix_with_meta/`:** Update fixture to remove now-unnecessary meta files for dedup-only cases. +- `**tests/tests/syncback_roundtrip.rs`:** The `dedup_suffix_with_meta` roundtrip test (line 299) and idempotency test (line 362) may need updated expected output. + +### 8. Add new tests + +- **Forward sync unit test:** File `Foo~2.luau` without meta -> instance name is `"Foo"` +- **Forward sync unit test:** Directory `Foo~2/` without meta -> instance name is `"Foo"` +- **Forward sync edge cases:** `Foo~0.luau` -> `"Foo~0"`, `Foo~abc.luau` -> `"Foo~abc"` (not valid dedup, kept as-is) +- **Syncback test:** Dedup-only case produces no meta `name` field +- **Syncback test:** Slugify + dedup still produces meta `name` field (e.g., `"Hey/Bro"` -> `Hey_Bro~2.luau` with meta name `"Hey/Bro"`) +- **Roundtrip test:** Dedup'd names without meta files survive build -> syncback -> rebuild + diff --git a/.cursor/plans/syncback_via_rbxm_ad0231a7.plan.md b/.cursor/plans/syncback_via_rbxm_ad0231a7.plan.md new file mode 100644 index 00000000..8db42fca --- /dev/null +++ b/.cursor/plans/syncback_via_rbxm_ad0231a7.plan.md @@ -0,0 +1,394 @@ +--- +name: Syncback via RBXM +overview: Add a "Syncback" button to the plugin that serializes the DataModel per-service via SerializationService, sends the rbxm chunks to a totally segregated server endpoint, which tears down the entire serve system, runs CLI-equivalent clean-mode syncback, and restarts serve -- all within the same process. +todos: + - id: server-exit-reason + content: Modify LiveServer::start() to return ServerExitReason via tokio select + shutdown channel (src/web/mod.rs) + status: completed + - id: api-types + content: Add SyncbackPayload/ServiceChunk types to src/web/interface.rs + status: completed + - id: api-endpoint + content: Add POST /api/syncback endpoint handler in src/web/api.rs (fully segregated, no session validation) + status: completed + - id: serve-loop + content: Wrap ServeCommand::run() in a loop that handles SyncbackRequested (src/cli/serve.rs) + status: completed + - id: reconstruct-dom + content: "Implement run_live_syncback: reconstruct WeakDom from service chunks, run syncback_loop (clean mode, all CLI defaults), write files" + status: completed + - id: plugin-button + content: Add Syncback button (Danger style, layoutOrder 2) to NotConnected.lua + status: completed + - id: plugin-confirm + content: Add confirmation dialog (StudioPluginGui, danger-themed) in App/init.lua + status: completed + - id: plugin-serialize + content: Implement DataModel serialization per SYNCBACK_SERVICES list using SerializationService + status: completed + - id: plugin-api + content: Add standalone syncback HTTP post function (no ApiContext/session needed) + status: completed + - id: plugin-flow + content: "Wire up full flow in App/init.lua: confirm -> serialize -> POST /api/syncback -> done (fire and forget)" + status: completed + - id: test-helpers + content: Add post_api_syncback, wait_to_come_back_online, make_service_chunk, make_rbxl_from_chunks, run_cli_syncback_on_chunks to serve_util.rs + status: completed + - id: test-fixtures + content: Create live_syncback, live_syncback_place_ids, and live_syncback_complex serve-test fixtures + status: completed + - id: test-parity + content: "CLI parity tests (byte-for-byte): scripts, models+properties, refs, special names, duplicates, deep hierarchy, mixed types, comprehensive properties, multi-service" + status: completed + - id: test-validation + content: "Rejection tests: bad protocol, bad version, blocked place, unlisted place, whitelisted place, unrestricted" + status: completed + - id: test-lifecycle + content: "Lifecycle tests: server comes back functional (WebSocket + file watch), syncback twice with different data, replaces all old files" + status: completed + - id: test-roundtrip + content: "Round-trip identity test: build -> decompose -> live syncback -> rebuild -> compare instance trees" + status: completed +isProject: false +--- + +# Syncback via RBXM + +## Architecture + +```mermaid +sequenceDiagram + participant User + participant Plugin + participant Server + participant Filesystem + + User->>Plugin: Click "Syncback" button + Plugin->>Plugin: Show danger confirmation dialog + User->>Plugin: Confirm + Plugin->>Plugin: SerializeInstancesAsync per service + Plugin->>Server: POST /api/syncback (msgpack, segregated endpoint) + Server->>Plugin: 200 OK + Note over Server: Store payload, signal shutdown + Note over Server: Full teardown: drop ServeSession, VFS watcher, ChangeProcessor, WebSocket connections + Note over Server: Reconstruct WeakDom from service chunks + Note over Server: Run syncback_loop (clean mode, all CLI defaults) + Server->>Filesystem: Write files + Note over Server: Create fresh ServeSession + LiveServer + Note over Server: Bind to same address:port + Plugin->>Plugin: Poll GET /api/rojo until server is back + Plugin->>Server: Normal connect flow +``` + + + +## Server Changes + +### 1. Make `LiveServer::start()` return instead of blocking forever + +`[src/web/mod.rs](src/web/mod.rs)`: Change `start()` to return a `ServerExitReason` enum. Add a shutdown mechanism via `Arc` (wrapping a `Mutex>` + `tokio::sync::Notify`). The accept loop uses `tokio::select!` to watch for the notify signal alongside `listener.accept()`. When signaled, it breaks and returns. + +```rust +pub enum ServerExitReason { + SyncbackRequested(SyncbackPayload), +} + +pub struct SyncbackSignal { + payload: Mutex>, + notify: tokio::sync::Notify, +} +``` + +The `SyncbackSignal` is shared between `LiveServer` (which watches it) and the API handler (which deposits data into it). When the accept loop exits, the Tokio runtime drops, all spawned tasks (including active WebSocket connections) are cancelled, and the `Arc` drops -- tearing down the ChangeProcessor thread, VFS watcher, and all in-memory state. + +### 2. New segregated API endpoint: `POST /api/syncback` + +`[src/web/api.rs](src/web/api.rs)`: Totally separate from the connect/sync flow. No session ID needed -- the plugin doesn't need to be "connected" to use this. But it does validate version and place ID. + +Request body is msgpack-encoded: + +```rust +pub struct SyncbackRequest { + pub protocol_version: u64, + pub server_version: String, + pub place_id: Option, + pub services: Vec, +} +pub struct ServiceChunk { + pub class_name: String, + pub data: Vec, // raw rbxm bytes +} +// Payload passed to the serve loop (version info stripped after validation) +pub struct SyncbackPayload { + pub services: Vec, +} +``` + +Types live in `[src/web/interface.rs](src/web/interface.rs)`. + +Handler flow: + +1. Deserialize msgpack body into `SyncbackRequest` +2. **Validate protocol version** -- reject with 400 if `request.protocol_version != PROTOCOL_VERSION` +3. **Validate server version** -- reject with 400 if major.minor doesn't match `SERVER_VERSION` +4. **Validate place ID** against the project's `servePlaceIds` (whitelist) and `blockedPlaceIds` (blocklist). Reject with 403 if not allowed. +5. Extract `SyncbackPayload` (just the services), store in `SyncbackSignal` -> notify -> return 200 `{"status": "syncback_initiated"}` + +The response is sent before any teardown begins (the API handler is a separate tokio task from the accept loop). + +### 3. Wrap `ServeCommand::run()` in a loop + +`[src/cli/serve.rs](src/cli/serve.rs)`: The `run()` method becomes a loop. Each iteration creates a fresh VFS, ServeSession, and LiveServer. When `start()` returns `SyncbackRequested`, the old session is fully dropped (VFS watcher stops, ChangeProcessor thread joins, tree is freed), syncback runs, and the loop continues to create everything fresh: + +```rust +loop { + let (vfs, errors) = Vfs::new_default_with_errors(); + let session = Arc::new(ServeSession::new(vfs, project_path.clone(), Some(errors))?); + let server = LiveServer::new(session); + + match server.start(addr) { + ServerExitReason::SyncbackRequested(payload) => { + log::info!("Live syncback requested, running..."); + run_live_syncback(&project_path, payload)?; + log::info!("Syncback complete, restarting serve..."); + continue; + } + } +} +``` + +IP and port are resolved once before the loop (they don't change). + +### 4. Reconstruct WeakDom + run syncback (CLI-equivalent) + +New function `run_live_syncback(project_path: &Path, payload: SyncbackPayload) -> anyhow::Result<()>` in `src/cli/serve.rs`: + +1. **Build WeakDom** -- Create `DataModel` root, for each `ServiceChunk` parse rbxm via `rbx_binary::from_reader`, create a service instance of the correct class, deep-clone all children from the parsed dom into the target dom. Maintain a `HashMap` ref-map to fix up Ref properties across chunks. +2. **Run syncback** -- Identical to what CLI `atlas syncback` does: + - Create oneshot VFS: `Vfs::new_oneshot()` + - Load old tree: `ServeSession::new_oneshot(vfs, project_path)` + - Call `syncback_loop(vfs, &mut old_tree, new_dom, project, incremental: false)` -- clean mode + - Write: `fs_snapshot.write_to_vfs_parallel(base_path, vfs)` + - `refresh_git_index(base_path)` + +Same code path as CLI syncback, same defaults, same clean mode. The only difference is the WeakDom source. + +## Plugin Changes + +### 5. Syncback button on NotConnected page + +`[plugin/src/App/StatusPages/NotConnected.lua](plugin/src/App/StatusPages/NotConnected.lua)`: Add "Syncback" button with `style = "Danger"` and `layoutOrder = 2`, shift "Sync" to `layoutOrder = 3`: + +``` +[Settings] [Syncback] [Sync] +``` + +Fires `self.props.onSyncback`. + +### 6. Confirmation dialog + +`[plugin/src/App/init.lua](plugin/src/App/init.lua)`: Floating `StudioPluginGui` dialog (same pattern as `AcceptConfirm` in Confirming.lua): + +- Title: "Full Syncback" +- Message: "This will overwrite your project files with the current Studio state. This cannot be undone." +- Buttons: "Cancel" (Bordered) | "Syncback" (Danger) +- State: `self.state.showingSyncbackConfirm` + +### 7. DataModel serialization + +After confirmation, serialize per-service. Hardcode the same `VISIBLE_SERVICES` list from the Rust side: + +```lua +local SYNCBACK_SERVICES = { + "Lighting", "MaterialService", "ReplicatedFirst", "ReplicatedStorage", + "ServerScriptService", "ServerStorage", "SoundService", "StarterGui", + "StarterPack", "StarterPlayer", "Teams", "TextChatService", + "VoiceChatService", "Workspace", +} + +local services = {} +for _, className in SYNCBACK_SERVICES do + local service = game:FindService(className) + if service and #service:GetChildren() > 0 then + local buf = SerializationService:SerializeInstancesAsync(service:GetChildren()) + table.insert(services, { + className = className, + data = buf, -- msgpack-luau natively encodes buffer as bin format (zero-copy) + }) + end +end +``` + +The msgpack-luau library handles `buffer` types natively -- when `type(data) == "buffer"`, it encodes as msgpack bin 8/16/32 using `buffer.copy` directly. No string conversion needed. On the Rust side, `rmp_serde` deserializes msgpack bin into `Vec` automatically. + +### 8. Direct HTTP post (no ApiContext needed) + +Since this endpoint is fully segregated and doesn't need a session, the plugin just needs host:port from the address entry fields. A standalone function posts directly: + +```lua +local function postSyncback(host, port, services) + local url = ("http://%s:%s/api/syncback"):format(host, port) + local body = Http.msgpackEncode({ + protocolVersion = Config.protocolVersion, + serverVersion = Config.expectedServerVersionString, + placeId = game.PlaceId, + services = services, + }) + return Http.post(url, body):andThen(rejectFailedRequests) +end +``` + +No `ApiContext` instance needed. This keeps it completely independent from the connect flow. + +### 9. Full flow in App + +`[plugin/src/App/init.lua](plugin/src/App/init.lua)`: The `onSyncback` handler: + +1. Show confirmation dialog +2. On confirm: serialize DataModel per-service +3. POST to `/api/syncback` with the chunks +4. On 200: show a notification ("Syncback data sent. Server is processing.") +5. Done. Stay on NotConnected page. User clicks "Sync" manually whenever they're ready. + +Fire and forget -- the plugin's job ends after the POST succeeds. No polling, no auto-reconnect, no special state. + +## Key Design Decisions + +- **Fully segregated endpoint**: `/api/syncback` has zero coupling to the session/connect system. No session ID, no auth. Plugin fires and forgets. +- **Complete teardown**: When the signal fires, the entire serve system is destroyed -- Tokio runtime drops, ServeSession drops (joining the ChangeProcessor thread, stopping VFS watcher), all WebSocket connections are cancelled. Nothing survives from the previous serve. +- **CLI-equivalent syncback**: Uses `syncback_loop` with `incremental: false` (clean mode), non-interactive. Identical behavior to `atlas syncback --input file.rbxl` with all defaults. +- **Same process, same port**: The `atlas serve` process never exits. After syncback completes, a fresh serve starts on the same address:port. The user sees log output in their terminal throughout. +- **Danger styling**: Uses existing `Danger` theme (`#E74C3C` red) defined in `[plugin/src/App/Theme.lua](plugin/src/App/Theme.lua)`. +- **Per-service serialization**: 14 visible services from `VISIBLE_SERVICES` constant, matching both Rust and Lua sides. Cross-service Ref properties are lost in transit but syncback's `Rojo_Ref_`* system resolves refs from filesystem structure. + +## Tests + +The core invariant: **live syncback must produce byte-identical filesystem output to CLI syncback given the same input data.** If it doesn't, something is broken in the serialize -> transmit -> reconstruct -> syncback pipeline. Every test that verifies data accuracy runs BOTH live syncback and CLI syncback on the same input, then compares output with `assert_dirs_equal` (byte-for-byte file comparison from `[tests/rojo_test/roundtrip_util.rs](tests/rojo_test/roundtrip_util.rs)`). + +Tests go in a dedicated `tests/tests/live_syncback.rs`. They use `run_serve_test` from `[tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs)` plus the round-trip utilities. + +### Test helpers (add to `serve_util.rs`) + +`**post_api_syncback_raw`** -- Sends a raw msgpack body to `POST /api/syncback`. Returns the full `reqwest::blocking::Response` so tests can assert on status codes and error bodies. + +`**post_api_syncback`** -- Convenience wrapper that fills in correct protocol version + server version. Takes `place_id: Option` and `services: Vec`. Panics if response is not 200. + +`**wait_to_come_back_online`** -- Polls `GET /api/rojo` with exponential backoff (longer timeout than initial startup since syncback runs between teardown and restart). Returns the new `ServerInfoResponse`. Panics if server doesn't come back. + +`**make_service_chunk`** -- Builds a `ServiceChunk` from `InstanceBuilder` children: + +```rust +fn make_service_chunk(class_name: &str, children: Vec) -> ServiceChunk { + let mut dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let root = dom.root_ref(); + for child in children { + dom.insert(root, child); + } + let refs: Vec = dom.root().children().to_vec(); + let mut buf = Vec::new(); + rbx_binary::to_writer(&mut buf, &dom, &refs).unwrap(); + ServiceChunk { class_name: class_name.to_string(), data: buf } +} +``` + +`**make_rbxl_from_chunks**` -- Builds a complete rbxl file from the same service chunks, for CLI parity testing. Creates a WeakDom with DataModel root + properly-classed service instances + children, serializes to rbxl. This gives CLI syncback the exact same input data as live syncback received. + +`**run_cli_syncback_on_chunks**` -- End-to-end helper: copies a fixture to a fresh temp dir, calls `make_rbxl_from_chunks`, runs `run_rojo_syncback_clean`, returns the temp dir path. Used by every parity test as the CLI reference. + +### Fixtures + +`**rojo-test/serve-tests/live_syncback/**` -- Multi-service project: + +``` +default.project.json5 # RS -> src/shared, SSS -> src/server, Workspace -> src/workspace +src/shared/OldModule.luau +src/server/OldScript.server.luau +src/workspace/OldPart.model.json5 +``` + +`**rojo-test/serve-tests/live_syncback_place_ids/**` -- Same layout + `servePlaceIds: [123]`, `blockedPlaceIds: [999]`. + +`**rojo-test/serve-tests/live_syncback_complex/**` -- All 14 visible services mapped to separate `$path` dirs. Used for comprehensive parity tests. + +### CLI parity tests (data accuracy) + +Every parity test follows this pattern: + +```rust +fn assert_live_matches_cli( + fixture_name: &str, + chunks: Vec, + place_id: Option, +) { + run_serve_test(fixture_name, |session, _| { + // 1. Live syncback + session.post_api_syncback(place_id, chunks.clone()); + session.wait_to_come_back_online(); + + // 2. CLI syncback on fresh copy of same fixture with same data + let cli_dir = run_cli_syncback_on_chunks(fixture_name, &chunks); + + // 3. Byte-for-byte comparison of every file + assert_dirs_equal(session.path(), &cli_dir); + }); +} +``` + +`**parity_scripts**` -- ModuleScript (`return {}`), Script/Server (`print("server")`), Script/Client (`print("client")`), LocalScript. Verifies Source content preserved exactly, correct file extensions (`.luau`, `.server.luau`, `.client.luau`, `.local.luau`), RunContext encoded in extension. + +`**parity_models_with_properties**` -- Model containing Parts with: Anchored (Bool), Size (Vector3), Position (Vector3), Color (Color3), BrickColor, Shape (Enum), Transparency (Float64). Verifies every property value round-trips through rbxm -> reconstruct -> syncback -> meta/model files identically. + +`**parity_ref_properties**` -- Model with PrimaryPart -> child Part. ObjectValue with Value -> sibling Part. Verifies `Rojo_Ref_*` attribute paths in meta/model files match CLI output exactly. + +`**parity_special_names**` -- Instances named `"What?"`, `"Key:Value"`, `"A/B"`, `""`, `'He said "hi"'`. Verifies slugified filenames (`What_`, `Key_Value`, `A_B`, etc.) and meta `name` fields match CLI. + +`**parity_duplicate_names**` -- 3x Folder named `"Data"`, 2x Script named `"Handler"`. Verifies dedup suffixes (`Data/`, `Data~2/`, `Data~3/`, `Handler.server.luau`, `Handler~2.server.luau`) and meta files match CLI. + +`**parity_deep_hierarchy**` -- 6 levels: Folder > Folder > Folder > Folder > Model > Part. Verifies nested directory structure is identical. + +`**parity_mixed_file_types**` -- StringValue (-> `.txt`), LocalizationTable (-> `.csv`), JSON data module (-> `.json5`), binary model (-> `.rbxm`), Folder (-> directory). All under one service. Byte-for-byte match with CLI. + +`**parity_comprehensive_properties**` -- Single Part with every testable property type set to non-default values: Bool, Int32, Float32, Float64, String, Vector2, Vector3, Vector2int16, Vector3int16, UDim, UDim2, Color3, Color3uint8, BrickColor, NumberRange, Rect, Enum, Font, Tags, Attributes (with nested types). Verifies meta.json5 property serialization matches CLI exactly. + +`**parity_multi_service**` -- Data under all 14 visible services simultaneously. Each service gets a unique set of instances (scripts, models, folders). Verifies every service's `$path` directory matches CLI output. + +`**parity_empty_services_mixed**` -- Some services have children, some don't, some are absent from the payload entirely. Verifies populated services are correct and empty/absent services result in clean directories. + +### Validation / rejection tests + +`**rejects_bad_protocol**` -- Send `protocol_version: 999`. Expect non-200. GET /api/rojo still returns same session ID (server unaffected). + +`**rejects_bad_version**` -- Send `server_version: "0.0.0"`. Expect non-200. Server unaffected. + +`**rejects_blocked_place**` -- Fixture has `blockedPlaceIds: [999]`. Send `placeId: 999`. Expect 403. Server unaffected. + +`**rejects_unlisted_place**` -- Fixture has `servePlaceIds: [123]`. Send `placeId: 456`. Expect 403. Server unaffected. + +`**allows_whitelisted_place**` -- `servePlaceIds: [123]`, send `placeId: 123`. Expect 200. Syncback runs, server restarts. + +`**allows_any_place_when_unrestricted**` -- No servePlaceIds/blockedPlaceIds. Send `placeId: 99999`. Expect 200. + +### Lifecycle / edge case tests + +`**server_comes_back_functional**` -- After live syncback, verify the restarted server is fully operational: GET /api/rojo works, can read instances via /api/read, WebSocket connects and receives patches when a file is modified on disk. + +`**syncback_twice_different_data**` -- Run live syncback with data A. Wait for restart. Run live syncback AGAIN with data B. Wait for restart. Verify filesystem matches data B (not A). Verify server is functional. Tests the serve loop handles multiple iterations. + +`**syncback_replaces_all_old_files**` -- Pre-populate the fixture with files that should NOT survive clean mode (orphan scripts, old models). After live syncback, verify none of the old files remain. Only files from the new data exist. + +### Round-trip identity test + +The ultimate accuracy test: + +`**roundtrip_build_syncback_rebuild**` -- Using the `live_syncback_complex` fixture: + +1. `atlas build` -> rbxl_A +2. Decompose rbxl_A into per-service chunks (parse with `rbx_binary::from_reader`, extract children of each service, re-serialize as rbxm chunks) +3. POST chunks to live syncback, wait for restart +4. `atlas build` -> rbxl_B +5. Parse both rbxl_A and rbxl_B into WeakDoms +6. Walk both trees recursively: compare instance names, classes, property values (using the same fuzzy equality from `variant_eq` for floats/CFrames), and child counts +7. Any divergence = test failure with detailed diff + +This tests the full pipeline: build -> decompose into service chunks -> transmit -> reconstruct -> syncback -> build again. If the output matches, the live syncback system preserves 100% data fidelity. \ No newline at end of file diff --git a/.cursor/plans/update_audit_and_rules_0625423a.plan.md b/.cursor/plans/update_audit_and_rules_0625423a.plan.md new file mode 100644 index 00000000..7549ab2f --- /dev/null +++ b/.cursor/plans/update_audit_and_rules_0625423a.plan.md @@ -0,0 +1,110 @@ +--- +name: Update audit and rules +overview: Update `.cursor/commands/audit.md` to cover the live syncback pipeline, new files (encodeHelpers, encodeService, new test suites), and refactored encoding architecture. Fix discrepancies in rule MDC files where they don't reflect recent code changes. +todos: + - id: audit-live-syncback + content: Add Live Syncback Pipeline section to audit.md with full file tree covering plugin encoding, transport, server handling, and serve loop restart + status: completed + - id: audit-encode-helpers + content: Add encodeHelpers.lua and encodeService.lua entries to the Plugin->Server ChangeBatcher section in audit.md + status: completed + - id: audit-new-tests + content: Add live_syncback.rs, git_sync_defaults.rs, matching_fixtures.rs to the Shared Infrastructure test section in audit.md + status: completed + - id: audit-11n-live-syncback + content: Add section 11n (Live Syncback Parity) audit checklist to audit.md + status: completed + - id: fix-atlas-api-mdc + content: Add encodeHelpers.lua to atlas-api.mdc ChangeBatcher table and update encodeInstance.lua description + status: completed + - id: fix-atlas-plugin-mdc + content: Add encodeHelpers.lua to atlas-plugin.mdc ChangeBatcher listing and fix SyncbackConfirm placement + status: completed + - id: fix-atlas-dedup-mdc + content: Add encodeHelpers.lua to atlas-dedup.mdc Key Files Map table + status: completed +isProject: false +--- + +# Update Audit Command and Rule Files + +## Summary of Discrepancies Found + +After comparing all 8 rule MDC files and the audit command against the actual source code and recent git history (40+ commits including live syncback, matching session caching, encodeHelpers refactor, float precision, git sync defaults), here are the issues: + +### audit.md - Major Gaps + +1. **No live syncback pipeline** -- The audit command describes only two pipelines (CLI syncback, two-way sync). The live syncback pipeline (`POST /api/syncback`) is a third distinct flow involving `encodeService.lua`, `SyncbackConfirm` UI, `handle_api_syncback`, `build_dom_from_chunks`, and `run_live_syncback`. Not covered at all. +2. **Missing `encodeHelpers.lua`** -- New shared helper module (`plugin/src/ChangeBatcher/encodeHelpers.lua`) that consolidates `encodeAttributes`, `encodeTags`, and `forEachEncodableProperty`. Both `encodeInstance.lua` and `encodeService.lua` import from it. Not mentioned anywhere in audit.md. +3. **Missing `encodeService.lua`** -- New file (`plugin/src/ChangeBatcher/encodeService.lua`) for service-level property encoding during live syncback. Not mentioned. +4. **Missing 3 new test suites** -- `tests/tests/live_syncback.rs`, `tests/tests/git_sync_defaults.rs`, `tests/tests/matching_fixtures.rs` are not referenced in the audit's test infrastructure section. +5. **Missing `SyncbackConfirm` UI** -- The floating confirmation dialog for live syncback (rendered in `App/init.lua`) is not mentioned in the plugin UI or data flow sections. +6. **Missing live syncback audit checklist** -- No equivalent of sections 11a-11m for live syncback-specific concerns (service property encoding, ObjectValue carrier lifecycle, rbxm blob reconstruction, clean-mode-only semantics, server restart behavior). + +### atlas-api.mdc - Issues + +1. **Missing `encodeHelpers.lua`** in the ChangeBatcher file table (section "ChangeBatcher"). The table lists `init.lua`, `createPatchSet.lua`, `encodePatchUpdate.lua`, `encodeInstance.lua`, `encodeService.lua`, `encodeProperty.lua`, `propertyFilter.lua` -- but `encodeHelpers.lua` is missing. +2. `**encodeInstance.lua` description** says "includes duplicate detection helpers" but doesn't note that property iteration was refactored to use `encodeHelpers.forEachEncodableProperty()`. + +### atlas-plugin.mdc - Issues + +1. **Missing `encodeHelpers.lua`** from the file listing under "4. ChangeBatcher" section. The `encodeHelpers.lua` file exports `encodeAttributes`, `encodeTags`, `forEachEncodableProperty`. +2. `**SyncbackConfirm**` is listed under StatusPages (`SyncbackConfirm` - "Floating dialog for live syncback confirmation (rendered by `App/init.lua`)") which is slightly misleading. It's NOT a StatusPage file -- it's rendered inline in `App/init.lua` as a `StudioPluginGui` with `isEphemeral = true`. The parenthetical note is correct but the placement under StatusPages implies a separate file. + +### atlas-dedup.mdc - Issues + +1. **Missing `encodeHelpers.lua`** from the Key Files Map table. This file is relevant because it contains the shared property filtering logic (`forEachEncodableProperty`) used during instance encoding for both two-way sync additions and live syncback. + +## Changes + +### Changes to `audit.md` + +**Section "Sync System Layout" -- add Live Syncback Pipeline** after the Two-Way Sync section: + +Add a third pipeline section covering: + +- Plugin-side: `SYNCBACK_SERVICES` in `App/init.lua`, `encodeService.lua`, `performSyncback()`, `SyncbackConfirm` dialog +- Transport: `POST /api/syncback` (MessagePack) +- Server-side: `handle_api_syncback` in `api.rs`, `SyncbackSignal`/`LiveServer` in `mod.rs`, `build_dom_from_chunks`/`run_live_syncback` in `serve.rs` +- Wire types: `SyncbackRequest`, `ServiceChunk`, `SyncbackPayload` in `interface.rs` + +**Section "Plugin --> Server" -- add `encodeHelpers.lua`** to the Change Batching subsection: + +- Add entry for `plugin/src/ChangeBatcher/encodeHelpers.lua` describing `encodeAttributes`, `encodeTags`, `forEachEncodableProperty` +- Add entry for `plugin/src/ChangeBatcher/encodeService.lua` describing service-level encoding for live syncback + +**Section "Shared Infrastructure" -- add new test files:** + +- `tests/tests/live_syncback.rs` -- Live syncback parity, validation, lifecycle, edge cases +- `tests/tests/git_sync_defaults.rs` -- Git-based sync default selection tests +- `tests/tests/matching_fixtures.rs` -- Matching algorithm fixture tests + +**Add new audit section 11n: Live Syncback Parity** covering: + +- Service property encoding completeness (`encodeService.lua` vs `filter_properties`) +- ObjectValue carrier lifecycle (created, serialized, destroyed) +- rbxm blob reconstruction (`build_dom_from_chunks` child distribution) +- Clean mode semantics (always `incremental=false`) +- Server restart behavior (session invalidation, reconnection) +- `SYNCBACK_SERVICES` list consistency with `VISIBLE_SERVICES` in Rust +- Protocol/version/placeId validation +- Round-trip: live syncback output must match CLI syncback output + +**Update Audit Reading Order** (step 9 area) to include live syncback flow. + +### Changes to Rule MDC Files + +`**atlas-api.mdc`:** + +- Add `encodeHelpers.lua` row to ChangeBatcher file table with description: "Shared encoding helpers: `encodeAttributes`, `encodeTags`, `forEachEncodableProperty`. Used by `encodeInstance.lua` and `encodeService.lua`." +- Update `encodeInstance.lua` row to note it uses `encodeHelpers.forEachEncodableProperty()` for property iteration + +`**atlas-plugin.mdc`:** + +- Add `encodeHelpers.lua` to the "4. ChangeBatcher" file listing with description: "Shared property encoding helpers (`encodeAttributes`, `encodeTags`, `forEachEncodableProperty`)" +- Move `SyncbackConfirm` from the StatusPages list to its own note under the App section, clarifying it's rendered inline in `App/init.lua` as a floating `StudioPluginGui`, not a separate StatusPage file + +`**atlas-dedup.mdc`:** + +- Add `plugin/src/ChangeBatcher/encodeHelpers.lua` to the Key Files Map table with subsystem "Encoding" and purpose "Shared property filtering/encoding helpers used by `encodeInstance.lua` and `encodeService.lua`" + diff --git a/.cursor/prompts/Ambiguous Path Handling.txt b/.cursor/prompts/Ambiguous Path Handling.txt new file mode 100644 index 00000000..5b43c6bc --- /dev/null +++ b/.cursor/prompts/Ambiguous Path Handling.txt @@ -0,0 +1,350 @@ +Ambiguous Path Handling + +We're looking to finally add support for Ambiguous Paths. + +Naming +* Duplicate file names are saved as ~1, ~2 etc - an expansion of the slugify system. + - ~1, ~2 suffixes: the existing deduplicate_name() in file_names.rs already does this. But currently it's only used for slug collisions (e.g., What?Script and What_Script both slugify to What_Script). I'm proposing to extend it to true duplicate names. The meta name field then stores the real name. + - Note: `~` is in SLUGIFY_CHARS, so instance names naturally containing `~` get it replaced with `_` BEFORE dedup runs. "Foo~1" (natural name) slugifies to "Foo_1", which is always distinct from the dedup suffix "Foo~1". This invariant is already tested in file_names.rs. +* Name field will be populated in the meta/model definitions +* IMPORTANT: Matching groups and dedup groups are DIFFERENT concepts with different grouping criteria: + - Matching groups (for pairing instances during sync): built by true Instance.Name. Two siblings with the same Name are in the same matching group regardless of ClassName. Used by the matching algorithm in all three locations. + - Dedup groups (for filesystem ~N suffixes): built by the FILESYSTEM SLUG that the middleware would produce. Only apply ~N suffixes when the filesystem representation would actually collide. A Folder "Foo" (becomes Foo/) and a Script "Foo" (becomes Foo.server.luau) do NOT collide on disk and do NOT need dedup suffixes. Two Folders both named "Foo" DO collide (both become Foo/) and need dedup. The dedup key is the lowercased final filesystem path component (slug + extension or directory name) that the middleware assigns. + - Init-style collisions: when a directory-style instance (Folder) and an init-style instance (Script with children) both want the same directory name, they DO collide -- apply dedup suffixes to the directories (Foo/ and Foo~1/). The middleware must compute whether each instance would be a directory or file BEFORE dedup grouping runs. + - In short: compute what each instance's filesystem name WOULD be (via middleware rules), group by lowercased collision on that name, then apply ~N suffixes within each collision group. +* Suffix cleanup is GAP-TOLERANT: when a dedup'd sibling is deleted, do NOT renumber remaining suffixes to fill gaps. If you have Attachment, Attachment~1, Attachment~2 and delete ~1, keep Attachment and Attachment~2 as-is. Gaps are harmless and avoiding renumbering prevents cascading file renames + mass ref path updates. + - The ONE exception: when the dedup group shrinks to exactly 1 remaining instance, remove its suffix entirely (rename to clean name). This keeps the filesystem clean for the common case. The ref path update system must handle this single rename. + - If the base-name holder (no suffix) is deleted and multiple remain, the lowest-numbered suffix (~1, or ~2 if ~1 is already gone) is promoted to base name. This is the only rename that occurs on deletion besides the group→1 cleanup. +* Base-name assignment: the matching algorithm should preserve existing filesystem assignments. Whichever instance currently holds the base name (no suffix) on disk keeps it when updating. New instances get the next available ~N suffix. In clean syncback (no existing files), Roblox child order determines base-name assignment (first child in the DOM children list gets the base name). +* Tiebreaker ordering: when the matching algorithm has tied candidates (identical similarity scores), Roblox child order (Instance:GetChildren() index on plugin side, DOM child order on Rust side) is the canonical tiebreaker for deterministic results. +* VFS event handling: when change_processor renames files for dedup suffix cleanup (e.g., Attachment~1 → Attachment after base-name holder deleted), tag these as self-initiated renames. compute_patch_set should recognize and ignore self-initiated VFS events to prevent feedback loops. + +Refs +* PREREQUISITE: Ref resolution must happen BEFORE the matching algorithm runs. Ref resolution is cheap on the Rust side (pointer lookups). On the Lua plugin side, reading Ref properties from Studio instances requires pcall (some instances have restricted access) -- still fast but add error handling. Missing or invalid Refs should log a warning and default to nil -- don't error, don't block matching. +* FILE FORMAT: Keep the Rojo_Ref_* attribute prefix with slug paths. Ref paths are stored as attributes: `Rojo_Ref_PrimaryPart: "Workspace/Model.model.json5/Attachment~1"`. Each path segment is the FULL FILESYSTEM NAME of the instance (slug + dedup suffix + extension for files, directory name + dedup suffix for directories). Examples: + - Folder "Foo" (directory) → segment `"Foo"` + - ModuleScript "Foo" → segment `"Foo.luau"` + - Script "Foo" (server) → segment `"Foo.server.luau"` + - Script "Foo" with children (init-style directory) → segment `"Foo"` + - Dedup'd ModuleScript "Foo" → segment `"Foo~1.luau"` + This guarantees ref path segments are always unique among siblings (because filesystem names are unique). The tradeoff: ref paths are coupled to middleware/extension choices, so script type transitions (ModuleScript→Script changes .luau→.server.luau) and file-to-folder transitions ALSO require ref path updates for all refs passing through that instance. This is additional maintenance but eliminates the ambiguity that extension-free paths would create for same-name siblings with different classes. + The path starts from the DataModel's children (DataModel excluded, so `game.Workspace.Model.Part` becomes `"Workspace/Model.model.json5/Part"`). This keeps the existing prefix-based detection (scan for keys starting with `Rojo_Ref_`) which is simple and doesn't require reflection data. +* NOTE: The Rojo_Ref_* system did NOT exist on origin/master -- it was added on the rbxm-ambiguous branch. So this is effectively a new addition, not a modification. The original Rojo $id / Rojo_Target_* / Rojo_Id system stays for backward compatibility with existing projects. When the slug-path system is available, prefer it. Fall back to the $id system for instances without filesystem backing (see below). +* The ref path is always unambiguous because each segment is the full filesystem name, which is unique among siblings by definition (filesystem can't have two files/dirs with the same name). Two Folders named "Attachment" become "Attachment" and "Attachment~1" on disk, producing distinct segments. A Folder "Foo" and Script "Foo" produce segments "Foo" and "Foo.server.luau" -- distinct without dedup. +* Ref path resolution: + - Syncback (ref_properties.rs): When writing Rojo_Ref_* attributes, build the ref path from the full filesystem names assigned during syncback (the same names used for files/directories on disk, INCLUDING extensions). + - Forward sync (patch_apply.rs): When resolving Rojo_Ref_* paths, walk the tree matching each segment against the full filesystem name of each child (derived from instigating_source filename/dirname). get_instance_by_path() in tree.rs currently resolves using instance names -- it needs a filesystem-name resolver instead. For instances without instigating_source (e.g., inside user .rbxm files, newly added during session), fall back to computing what the filesystem name WOULD be (slugify name + middleware extension). + - Forward sync Ref pre-resolution for matching (patch_compute.rs): When pre-resolving Refs before the matching algorithm, try the old RojoTree FIRST (most refs target existing instances). If the path can't be resolved in the old tree (target is a new instance), fall back to walking the snapshot tree. The snapshot is only a subtree (the re-snapshotted directory), so this fallback only works for refs within the same subtree. Refs to new instances outside the current snapshot remain unresolved and are skipped during matching (treated as nil). + - Two-way sync (change_processor.rs, api.rs): When the plugin sends a Ref property change, the server receives a UID. The server must compute the slug path for the target to write it to disk. For file-backed instances, derive the slug from instigating_source. For instances without filesystem backing (inside .rbxm files, newly added instances before VFS catches up), fall back to the $id / Rojo_Target_* system instead of slug paths. +* Cleanup: Remove the escape_ref_path_segment() / split_ref_path() infrastructure in rojo_ref.rs. With filesystem-name-based paths, `/` in instance names is already slugified to `_` by slugify_name(). The `\/` escaping system was needed because the old system used raw instance names -- filesystem-name paths don't need it. +* IMPORTANT: Any time an instance is moved, renamed, or has its dedup suffix reassigned, ALL Rojo_Ref_* paths across ALL meta/model files that reference that instance (or any of its descendants) must be updated. This includes: + - Renames: instance name changes -> slug changes -> all refs to it and its children update + - Dedup reassignment: matching algorithm reassigns ~1 to ~2, or suffix removed entirely -> all refs update + - Reparenting: instance moves to a different parent -> entire ancestor path changes -> all refs update + - Suffix cleanup: dedup suffix removed because sibling was deleted -> all refs update + - Script type transitions: ClassName change (e.g., ModuleScript→Script) changes extension (.luau→.server.luau), which changes the ref segment -> all refs through this instance update + - File-to-folder transitions: instance gains/loses children, flipping between file (Foo.server.luau) and directory (Foo/) representation -> ref segment changes -> all refs update + - The existing RefPathIndex infrastructure tracks which files contain refs to which targets. Extend it to handle all cases above. + - Batch ref path writes: when dedup reassignment causes mass ref path updates (many siblings renamed, each referenced by multiple files), collect ALL affected ref updates first, then write all affected files in one batch. This avoids redundant I/O when multiple refs in the same file need updating. + +Matching System +* We need to create a Rust and Lua matching algorithm that produces the same pairing given equivalent instance data. The data shapes differ (Rust: UstrMap, Lua: Studio property access via pcall) but the algorithm logic and output must be equivalent. +* The matching algorithm attempts to assign a list of child instances to the current corresponding state, whether that is the actual Instances in the tree in Roblox Studio or the files on the file system (the dom). +* It tries it's BEST to find what will effectively requires the fewest number of changes of each child, when duplicates are encountered. Things which are 1:1 can be considered an instant match. +* You need to consider literally everything about the instance. Name, ClassName, Properties, Attributes, Tags, ITS children, etc. To avoid combinatorial explosion, use a bottom-up subtree hash computed once and cached. The hash infrastructure already exists in syncback/hash/. + - Rust side: blake3 (existing infrastructure). Full subtree hash of name+class+sorted property hashes+children hashes. + - Lua plugin side: use XXH32 (impl attached at bottom of this file) for the STRUCTURAL subtree fingerprint only (child count + sorted child name+class pairs). Do NOT hash full property values (especially Source) -- the serialize-to-buffer cost dominates. For actual property comparison in Pass 3 similarity scoring, compare properties directly with early-exit ordering: ClassName → Tags → Attributes → scalar properties (numbers, bools, enums) → Source last (most expensive, least likely needed after cheaper signals differentiate). + - Hashes are a fast-path signal, NOT a definitive equality proof. If hashes differ → definitely different. If hashes match → very likely identical, score as max similarity. The matching algorithm must not rely solely on hash equality for correctness -- other discriminators (Refs, ClassName, Tags) always participate. This makes 32-bit collision risk irrelevant in practice (ambiguous groups are small, typically 2-10 instances). + - Lua-side caching: compute structural fingerprints LAZILY -- only for instances that actually enter Pass 3 (ambiguous groups not resolved by Refs). Cache fingerprints for the duration of the matching call (not across calls). On the Rust side, hash_tree() precomputes all hashes bottom-up in one pass before matching begins -- this is cheap with blake3. +* Ambiguous groups are built by the TRUE Instance.Name (as finally resolved for each instance). Two siblings with the same true Name are in the same ambiguous group regardless of ClassName. On the plugin/Rust-tree side, Instance.Name / instance.name() is readily available. On the filesystem side, resolving the true Name is non-trivial: it may come from the `name` field in .meta.json5 (if present, overrides filename), the `name` field in .model.json5, or the filename stem after stripping extensions and script suffixes (if no meta override). The matching algorithm must resolve all true Names before it can build groups. +* The matching algorithm should work in passes (Refs must be pre-resolved before any pass runs -- see Refs section above): + - Pass 1: Match all instances that have a unique true Name among their siblings (no ambiguity, O(n)). Within a same-Name group, ClassName is a fast first discriminator for narrowing candidates. + - Pass 2: For remaining unmatched instances (same-Name groups), use the pre-resolved Refs as discriminators. Refs are the strongest signal because they carry structural identity -- if instance A has a Ref property pointing to UID-7, and UID-7 was already matched in Pass 1, then we know instance A's Ref "means" something specific. If only one candidate on the other side also has a Ref resolving to that same matched target, that's a near-certain match. On the Rust side, Refs are already resolved to specific Refs in the tree. On the Lua side, Refs sent from the server are literal UIDs that map to specific Studio instances via the instanceMap. Also consider incoming Refs TO these instances (other instances pointing at them) -- if instance X is the only candidate that other already-matched instances reference, that's equally strong signal. + - Unmatched Ref targets: if a Ref target is itself in an unmatched ambiguous group (not resolved in Pass 1), treat it as a weaker signal -- compare target names/classes without requiring a confirmed match. Do NOT recursively match the target's group (adds complexity for an extremely rare edge case). If the weaker signal is insufficient, the instance falls through to Pass 3 naturally. + - Pass 3: For still-unmatched same-Name groups, use ClassName as a fast discriminator to narrow candidates, then compute pairwise similarity scores (using precomputed subtree hashes on Rust side, direct comparison on Lua side) and greedily assign best-match-first (pick highest score, remove both from pool, repeat). +* Instances which were extra from the incoming changes are considered NEW and then extra from the existing are considered DELETED +* The server assigns every instance a unique Ref (UID) in the RojoTree. When the plugin connects, the matching algorithm pairs server UIDs to Studio instances and caches this mapping in the instanceMap. Once matched, that pairing is stable for the duration of the sync session -- the matching algorithm should not reshuffle existing matched pairs unnecessarily. This cache allows subsequent operations (property changes, renames) to be fast lookups rather than re-running the full matching algorithm. +* We don't need a "UUID" system saved as attributes on instances because of this matching system, even though that would allow this algorithm to work as-is but faster, it's not desired as it adds a lot of bloat. +* Performance of this is really important. Precompute hashes and resolve Refs once before entering the matching loop. Avoid recomputing anything per-pair in Pass 3 -- all signals should be cached before pairwise comparison begins. +* FAST PATH: if Pass 1 matches ALL children (no ambiguous groups exist among siblings), skip Passes 2 and 3 entirely. This is the common case -- most parents have uniquely-named children. The fast path makes forward sync (called on every VFS event) effectively free for non-ambiguous subtrees, degrading to the same cost as the current name+class matching. +* Matching priority within an ambiguous group (same true Name): Refs (pre-resolved -- strongest signal), ClassName, Tags, Attributes, Properties (non-Ref), Children hash. +* The matching needs to handle partial matches -- instance exists in both but has property changes. In this case, just pick the best of each. If candidates are completely equal (identical similarity scores), use Roblox child order as the deterministic tiebreaker (first in DOM children list / Instance:GetChildren() index wins). +* Matching should be run in 3 locations. These are 3 SEPARATE IMPLEMENTATIONS of the same algorithm pattern (2 Rust, 1 Lua), NOT shared code. The data shapes are too different to share implementation. Equivalence is guaranteed by shared test vectors (see Testing section), not by code reuse. + - hydrate.lua -- plugin connects, matches server virtual instances (encoded properties, UIDs for Refs) to Studio instances (read via pcall, direct Instance Ref pointers). Lua implementation. + - compute_patch_set in patch_compute.rs -- VFS event triggers re-snapshot, matches snapshot children (InstanceSnapshot with Rojo_Ref_* as string paths) to tree children (RojoTree instances with Refs as UIDs). Rust implementation #1. + - syncback_dir_no_meta in dir.rs -- syncback matches new instance children (WeakDom from Roblox file) to existing filesystem children (WeakDom from old tree). Rust implementation #2. +* IDEA: We can build a cache progressively by assigning children to specific UIDs. This can help on running time of the algorithm. We need to ensure things don't become stale in this LUT cache. The cache can be built up during actual application. The cache invalidates per-parent when a child is added, removed, or renamed under that parent. During two-way sync, each /api/write mutation should invalidate the affected parent's cache entry. The cache maps parent_ref -> (children_list, match_assignments) and is cheap to rebuild for a single parent. + +Syncback: +* ARCHITECTURAL CHANGE: the current syncback flow processes children one-at-a-time via with_joined_path(). Collision-based dedup requires knowing ALL children's filesystem names before assigning suffixes. This requires restructuring to TWO PASSES: + 1. First pass: iterate all children, select middleware for each (get_best_middleware), compute tentative filesystem name (slug + extension or directory name). + 2. Second pass: group by lowercased collision on tentative name, assign ~N suffixes within each collision group, then actually process each child with its final dedup'd name. + The current with_joined_path() does middleware selection + filename generation + dedup in one call per child. This must be split so dedup grouping has the full picture before any suffix is assigned. +* Needs to be able to output the duplicate files with ~N dedup suffixes +* The matching algorithm runs FIRST to pair incoming instances to existing filesystem children. Then files are written/updated based on the pairing: + - Matched pairs: update the existing file/directory in place (overwrite content, update meta). The matching algorithm should preserve existing base-name assignments -- whoever held the base name on disk keeps it. + - Unmatched incoming instances: create new files (with dedup suffixes if Name collides) + - Unmatched existing files: delete them (instance was removed) +* When a dedup suffix changes (base-name holder deleted → lowest suffix promoted, or group shrinks to 1 → suffix removed), rename the file on disk and update all ref paths that referenced the old slug. Gaps in suffix numbers are left as-is (no renumbering). +* Clean syncback rebuilds everything from scratch -- matching is only needed in incremental mode to preserve existing file paths where possible. In clean mode, base-name assignment for duplicate names uses Roblox child order (first child in DOM children list gets the base name, second gets ~1, etc.). +* It should not just keep "adding" more files each time a syncback happens, it needs to CORRECT/UPDATE them, utilizing the matching algorithm +* When a third duplicate is added (e.g., existing "Attachment" and "Attachment~1", now a third "Attachment" appears), the matching algorithm pairs the two existing ones, and the new one gets "Attachment~2". Existing suffixes are never renumbered. + +Forward Sync: +* When a user edits Attachment~1/init.meta.json5 on disk, the server re-snapshots. The snapshot produces two children both named "Attachment" (from their meta name fields). compute_patch_set must use the matching algorithm to pair them correctly against the tree's existing Attachment instances, rather than greedy first-match which may cross-pair them. + +API +* Leverage the UID we are sending it from the rust side, and vice-versa for newly created instances - make sure those are session persistant and utilized. We need to utilize this to keep things straight when we send data back to the rust side so it knows where to place things. +* The UID I'm speaking of is the Ref that the server assigns to each instance in the RojoTree. These are sent to the plugin via the API when connecting, and the plugin sends them back for newly created instances. This is the existing system -- no new UID mechanism is needed. +* Processing order in /api/write: use a two-pass approach. First pass: apply ALL mutations (renames, property changes, adds, removes) to the tree. Second pass: recompute all affected ref slug paths based on the final tree state. This ensures refs always use the newest slugs, even when a rename and a ref update targeting the renamed instance arrive in the same request. + +Notes +* Case-insensitive instances need to be differentiated on the filesystem to properly support all operating systems. On Linux (case-sensitive FS), "Script" and "script" can coexist as separate files, but for cross-platform compatibility they must still be deduped with ~N suffixes. The matching algorithm handles these trivially since the true Names are different. +* This system needs to work for all sync directions/systems - syncback, two-way sync, forward sync, one-shot, rbxl construction from project, etc. +* Project file instances: project keys are unique by JSON definition (duplicate keys in JSON/JSON5 are not valid), so project-sourced instances do NOT participate in the dedup system. They always have unique names within their sibling group. +* Performance target: system must handle large games (10K+ instances) with <100ms latency per operation. Optimization can be deferred but the architecture must not have O(n^2) bottlenecks on the full tree. The matching algorithm should only run on the affected subtree when possible. +* We need to literally remove the ENTIRE system responsible for blacklisting duplicately named paths on the plugin and in the rust code, it is no longer relevant. + +rbxm-ambiguous +* Additionally, you should review the changes done in the branch rbxm-ambiguous. I'm abandoning that branch, but these two problems are closely related. I don't really ever want projects to have rbxm files being outputted for syncback or two-way sync anymore, but I still want to fully support loading them, but they should also get removed/replaced properly when you do a syncback on clean mode. +* Best to do a full diff from origin/master +* We specifically mention / probably want to take these features which were mentioned specifically in this plan: + - The plugin relaxations (diff.lua, encodeInstance.lua duplicate-skip removal) -- these are needed regardless of approach + - The has_duplicate_children helper and case-insensitive detection + - The project.rs case-insensitive duplicate detection. Note: with the dedup-suffix approach, ProjectNode children with duplicate names CAN now be written as individual files with ~N suffixes -- the warn-and-skip guard from the rbxm branch may no longer be needed. The implementer should evaluate whether to remove it or keep it as a safety net. + - The roundtrip test improvements (compare_children Vec-based, Tags check) + - The macOS ref_properties test ignore + - The DuplicateChildrenError typed error pattern (or equivalent) +* Don't take: ambiguous_container metadata, find_rbxm_container, reserialize_rbxm_container, rbxm fallback block, all ambiguous test fixtures. We are specifically trying to avoid rbxm-ing things. +* No special transition logic needed for old .rbxm files. Syncback rewrites the filesystem from scratch (clean) or matches instances to existing files (incremental) -- either way, the rbxm just doesn't get recreated. User-intentional .rbxm files that users placed manually in their project still load fine via snapshot_rbxm during forward sync -- that's unchanged. + +Sync System Invariants (from audit.md -- these apply to EVERY change in this feature) + +* ROUND-TRIP IDENTITY: Syncback (or two-way sync) writes a directory tree. Building an rbxl from that directory tree and forward-syncing it back must produce a bit-identical instance tree -- same names, same classes, same properties, same hierarchy, same ref targets. The dedup system, matching algorithm, and ref path system must ALL preserve this invariant. Any deviation is a bug. + +* CLI SYNCBACK PARITY: Plugin-based two-way sync must produce EXACTLY the same filesystem output that `atlas syncback` would give for the same input. Byte-for-byte identical files, identical directory structures, identical naming, identical dedup suffixes. This means the dedup logic, matching algorithm, and ref path computation must be shared or equivalent between the two paths. Any divergence is a bug. + +* 4 FILE FORMAT BRANCHES: syncback_updated_properties() in api.rs has 4 branches for writing property changes. The dedup/naming system must work correctly in ALL 4: + 1. Directory format: init.meta.json5 inside the instance's directory + 2. Standalone script: adjacent ScriptName.meta.json5 (strips .server/.client/.plugin/.local/.legacy suffix from stem) + 3. Model file: inline in .model.json5 (writes directly into the file, NOT adjacent meta) + 4. Other file types (.txt, .csv, .toml): adjacent FileName.meta.json5 + Each branch must: compute the correct dedup'd filename, write the correct meta name field, and produce ref paths consistent with the dedup assignment. + +* INSTIGATING SOURCE CONSISTENCY: every instance in the RojoTree tracks where it came from via InstigatingSource. When creating files with dedup suffixes, the InstigatingSource path must reflect the dedup'd path (including ~N suffix). Pre-seed taken_names from existing tree children via their instigating_source paths (using strip_middleware_extension), NOT from instance names. This prevents slug collisions with existing files on disk. + +* FILE-TO-FOLDER TRANSITIONS: instances can flip between file representation (Foo.server.luau) and directory representation (Foo/init.server.luau + children). This transition changes the dedup group key (file stem vs directory name) and must update: + - The dedup suffix (may change or be added/removed) + - The meta file location (adjacent → init.meta.json5 inside directory, or vice versa) + - All Rojo_Ref_* paths that pass through this instance (slug changes) + - The InstigatingSource path + Round-trip: adding a child to a file instance then removing it must produce identical filesystem state to the original. Treat any code touching these transitions as high-risk. + +* SCRIPT TYPE TRANSITIONS: ClassName changes (ModuleScript → Script) cause file extension changes (.luau → .server.luau). This changes the dedup group key since the filesystem representation changes. A ModuleScript "Foo" (.luau) and Script "Foo" (.server.luau) don't collide -- but if the ModuleScript becomes a Script, it now collides with the existing Script. The dedup system must handle this mid-session during two-way sync. + +* DUAL WRITE PATH CONSISTENCY: handle_api_write in api.rs does TWO things: (1) writes files to disk, and (2) sends a PatchSet to the ChangeProcessor to update the in-memory tree. These MUST stay consistent. If the file write uses dedup suffix ~1 but the PatchSet doesn't reflect that, the tree and filesystem diverge silently. + +* MULTIPLE PLUGIN ENTRY POINTS: property changes flow through the plugin via TWO separate paths: + 1. ChangeBatcher flow (live two-way sync): Instance.Changed → InstanceMap → ChangeBatcher → encodePatchUpdate → ApiContext:write() + 2. Confirmation dialog flow (initial sync pull): ServeSession.__confirmAndApplyInitialPatch → encodePatchUpdate → ApiContext:write() + Both paths must handle duplicate-named instances correctly. A function modified in path 1 may not have been updated in path 2. + +* REVIEW DIALOGUE INVARIANT: No two-way sync change may bypass the review dialogue in Always and Initial sync modes. Dedup suffix renames, ref path updates, and meta file changes triggered by the matching algorithm must all be cataloged in the confirmation UI. No silent application. + +* VFS ECHO PREVENTION: the existing suppress_path() system uses per-path counters. For dedup operations: suppress_path() MUST be called before any file write/rename during dedup suffix changes. New instance additions must NOT suppress (VFS watcher needs to pick up new files). The VFS event tagging mentioned in the Naming section should integrate with this existing mechanism. + +* ATOMICITY: dedup suffix cleanup involves multiple filesystem operations (rename file, update meta, update ref paths). There is a window where the VFS watcher could fire between operations and snapshot an inconsistent intermediate state. Minimize race windows by ordering operations carefully and using suppress_path() to cover all intermediate states. + +* DETERMINISM: given the same instance tree, the dedup system must ALWAYS produce the same filesystem output. Child ordering must be stable (sorted/deterministic), dedup suffix assignment must be deterministic, and generated identifiers must be reproducible. Non-determinism = different builds from same source = git churn = invariant violation. + +* COMPOUND FILE EXTENSIONS: .server.luau, .client.luau, .model.json5, .meta.json5, .project.json5 are multi-part extensions. Dedup suffixes are applied to the STEM before the compound extension: Foo~1.server.luau (correct), NOT Foo.server~1.luau. Meta pairing: Foo~1.server.luau pairs with Foo~1.meta.json5 (strip script suffix from stem, then add .meta.json5). The suffix list in change_processor.rs must include all compound parts for rename operations. + +Testing +* This system is extremely critical -- subtle bugs here silently corrupt user projects. Test the hell out of it at every layer: e2e, plugin, api, syncback, forward sync, two-way sync, build, roundtrip. +* Ensure we try all combinations, folder/meta swaps, etc. Try to make the system conflict with the existing slugify system. +* Define shared test vectors as structured data (e.g., two lists of instance descriptions -> expected match pairing) that both the Rust and Lua implementations must produce identical results for. This is the strongest guarantee that the two implementations don't diverge. +* The scenarios below are MINIMUM REQUIRED tests, not exhaustive. The implementer should add additional edge cases they discover during implementation. Think adversarially -- what combinations of operations could break invariants? +* Specific scenarios to test: + - Two identical Folders with same Name (simplest dedup case) + - Two Folders same Name, different children (matching must pair correctly by content) + - Two Folders same Name, identical content (tiebreaker: Roblox child order wins -- first in DOM children list gets base name) + - Folder "Foo" next to Script "Foo" (same Name, different ClassName -- NO dedup suffix because Foo/ and Foo.server.luau don't collide on disk. Both keep clean names.) + - Folder "Foo" next to Script "Foo" where Script has children (init-style: Foo/ vs Foo/init.server.luau -- these DO collide, dedup suffix applied) + - Instance naturally named "Foo~1" next to two "Foo" instances -- "Foo~1" slugifies to "Foo_1" (tilde replaced with underscore), which is distinct from the dedup suffix "Foo~1". Verify filesystem has Foo/, Foo~1/, Foo_1/ (or Foo_1.server.luau etc.) + - Third duplicate added to existing dedup group (no renumbering of existing suffixes) + - Dedup suffix cleanup (group→1): only two instances "Foo" and "Foo~1", base deleted, ~1 renamed to clean "Foo" (group shrank to 1) + - Gap tolerance: three instances Foo, Foo~1, Foo~2 -- delete ~1 -- verify Foo and Foo~2 remain unchanged (no renumbering) + - Base-name promotion: three instances Foo, Foo~1, Foo~2 -- delete Foo (base) -- lowest suffix ~1 promoted to base name, ~2 stays as ~2 + - Gap tolerance with refs: Foo~2 has refs pointing to it, ~1 deleted, verify Foo~2's refs are untouched (no path changes since no rename occurred) + - Idempotency: syncback twice produces identical filesystem output + - Roundtrip: syncback -> build -> compare trees +* Ref path integrity tests (CRITICAL -- refs becoming stale/broken is the highest-risk failure mode): + - Basic ref survives syncback: Model.PrimaryPart points to a Part, syncback writes the ref path, build resolves it back to the same Part. Roundtrip identity preserved. + - Ref to a dedup'd instance: PrimaryPart points to "Attachment~1". Syncback writes the slug path. Build resolves it correctly (not to the other Attachment). + - Ref survives gap-tolerant deletion: Three "Attachment" instances (Attachment, Attachment~1, Attachment~2). Delete the middle one (~1). Attachment and Attachment~2 remain unchanged -- NO ref path updates needed. Verify all Rojo_Ref_* paths still resolve correctly. + - Ref survives base-name promotion: Three "Attachment" instances. Delete the base (Attachment). Attachment~1 is promoted to base name "Attachment". ALL Rojo_Ref_* paths that pointed to the old "Attachment~1" must now say "Attachment". Attachment~2 is unchanged. Verify by building. + - Ref survives group→1 cleanup: Two "Attachment" instances (Attachment, Attachment~1). Delete Attachment. Attachment~1 renamed to Attachment (group shrank to 1). ALL Rojo_Ref_* paths must update. Verify. + - Ref survives rename: Instance "Foo" is renamed to "Bar". All ref paths containing "Foo" segment must update to "Bar". Verify across meta files, model files, and project files. + - Ref survives reparent: Instance moves from Parent_A to Parent_B. The entire ancestor path changes. All ref paths must update. Verify. + - Ref to deeply nested instance through multiple dedup layers: A/B~1/C/D~2 where both B and D have dedup suffixes. Ref resolves correctly through the full chain. + - Cross-file ref update: File_A.meta.json5 has a ref to an instance whose slug changes. File_B.model.json5 also has a ref to the same instance. BOTH files must be updated. Verify by reading both files after the operation. + - Ref to instance that gets deleted: Ref path points to an instance that no longer exists after syncback. Should resolve to nil, not crash, not point to wrong instance. + - Ref path with instance whose natural name contains tilde: Instance named "Foo~1" (real name). Slug is "Foo_1" (tilde slugified). If there are also two "Foo" instances, filesystem has Foo/, Foo~1/ (dedup), Foo_1/ (slugified). Ref to the "Foo~1" instance uses slug path "Parent/Foo_1". Verify the resolver finds the correct instance. + - Two-way sync ref update: Plugin sends a property change that triggers a dedup suffix reassignment on the server. Server must update all Rojo_Ref_* paths in affected files AND the patch sent back to the plugin must reflect the correct ref targets. + - Concurrent ref + rename in same batch: In a single /api/write request, one instance is renamed (changing its slug) and another instance has a Ref property update pointing to the renamed instance. The ref path must use the NEW slug, not the old one. + - Forward sync ref resolution after manual edit: User manually edits a ref path in a meta file on disk (changes "Part/Attachment~1" to "Part/Attachment~2"). Server re-snapshots and resolves the ref. Verify it points to the correct instance or logs a warning if invalid. + - Fallback to $id system: Instance inside a user .rbxm file has no instigating_source. Ref to this instance should use the $id/Rojo_Target_* system, not a slug path. Verify the ref resolves correctly after build. + - Mass ref update stress test: 50+ instances all referencing the same target. Target gets renamed. All 50 Rojo_Ref_* paths must update. Verify none are missed. +* Matching algorithm tests (CRITICAL -- wrong pairing silently corrupts projects): + - Matching with ClassName discriminator: two "Part" instances, one is Part and one is MeshPart. Match by ClassName in Pass 1 (unique within same-Name group per class). + - Matching with Ref discriminator: two identical Folders "Data", one has a Ref from an ObjectValue pointing to it. Match via Ref in Pass 2. + - Matching with property similarity: two Scripts "Handler" with different Source. Match the one with closer Source in Pass 3. + - Matching stability: run matching on same input twice, verify identical pairing both times. + - Matching preserves existing pairings: during two-way sync, matched pairs should not reshuffle when a new sibling is added. + - Fast path verification: parent with all uniquely-named children skips Passes 2-3 entirely (verify via timing or instrumentation). +* Dedup group collision tests: + - Folder "Foo" + ModuleScript "Foo" → no collision (Foo/ vs Foo.luau), no dedup suffix + - Folder "Foo" + Folder "Foo" → collision (both Foo/), dedup applied + - Script "Foo" (server) + Script "Foo" (client) → no collision (Foo.server.luau vs Foo.client.luau), no dedup suffix + - Script "Foo" (server) + Script "Foo" (server) → collision (both Foo.server.luau), dedup applied + - ModuleScript "Foo" + ModuleScript "Foo" → collision (both Foo.luau), dedup applied + - Instance "Foo?" + Instance "Foo*" → both slugify to "Foo_", filesystem collision, dedup applied. Different matching groups (different true names) but same dedup group (same slug). + - Instance "Foo" + Instance "foo" → case-insensitive collision on Windows/macOS, dedup applied for cross-platform safety. +* Cross-system integration tests: + - Full roundtrip with duplicates: create place with duplicate-named instances → syncback → build → compare trees bit-for-bit + - Incremental syncback after rename: syncback, rename one duplicate in Studio, syncback again -- verify correct file is updated (not the wrong sibling) + - Two-way sync add duplicate: during live sync, add a third instance with same name -- verify correct ~N suffix assigned and refs intact + - Two-way sync delete from middle: during live sync, delete an instance from a dedup group -- verify gap tolerance (no renumbering) + - Forward sync edit dedup'd file: edit Attachment~1/init.meta.json5 on disk -- verify compute_patch_set pairs it to the correct tree instance + - Slugify + dedup interaction: instance named "What?Script" alongside "What_Script" alongside another "What?Script" -- first two slugify to same thing, third is a true duplicate of the first. Verify all three get correct unique filesystem names and refs resolve. + - Project file + filesystem duplicates: project defines child "Foo" via $path, filesystem sibling also named "Foo" -- project key wins (unique by definition), filesystem child coexists without dedup. +* Sync invariant tests (from audit.md concepts): + - CLI syncback parity: run CLI syncback on a place with duplicates, then run two-way sync on the same place -- verify byte-identical filesystem output. Any divergence is a bug. + - 4 file format branches: verify dedup works for (1) directory with init.meta.json5, (2) standalone script with adjacent .meta.json5, (3) model file with inline name, (4) .txt/.csv with adjacent .meta.json5. Each must produce correct dedup'd names and meta name fields. + - Compound extension dedup: Script~1.server.luau pairs with Script~1.meta.json5 (not Script.server~1.luau or Script~1.server.meta.json5). Verify meta pairing is correct. + - InstigatingSource after dedup: verify that instances written with ~N suffixes have InstigatingSource::Path pointing to the dedup'd path (e.g., src/Foo~1.server.luau, not src/Foo.server.luau). + - File-to-folder transition with dedup: Script "Foo" (flat, Foo.server.luau) gets a child added → becomes Foo/init.server.luau. If another "Foo" folder already exists, dedup suffix must be applied to the new directory. All ref paths must update. + - Script type transition in dedup group: ModuleScript "Foo" and Script "Foo" don't collide (Foo.luau vs Foo.server.luau). Change ModuleScript to Script → now both are .server.luau → dedup collision created mid-session. Verify dedup suffix added and refs updated. + - Dual write path consistency: during /api/write, verify the file written to disk uses the same dedup suffix as the PatchSet sent to ChangeProcessor. Kill the server mid-sync and verify build from filesystem matches the in-memory tree state. + - Review dialogue with dedup: in Always sync mode, adding a duplicate-named instance shows the dedup assignment in the confirmation UI before applying. + - Atomicity: rename Attachment~1 to Attachment (suffix cleanup) while VFS watcher is active. Verify no intermediate state is snapshotted where both Attachment and Attachment~1 exist or neither exists. + - Determinism: build the same project 3 times from scratch -- verify all 3 outputs are bit-identical. No non-deterministic suffix assignment or child ordering. + - Idempotency: syncback twice on same input with no changes in between -- verify zero filesystem modifications on second run. + - Pre-seeded dedup: add a new instance "Foo" to a parent that already has Foo.server.luau on disk. Verify taken_names is seeded from instigating_source paths and the new instance gets ~1 (not overwriting the existing file). + +This is a large refactor, and I may be missing some critical details in this - ask many questions to make sure there is no ambiguity. + + + +XXH32.luau: + +--!strict +--!optimize 2 +--!native + +local function XXH32(Message: buffer, Seed: number?): number + local PRIME_1, PRIME_1_HIGH, PRIME_1_LOW = 0x9e3779B1, 40503, 31153 + local PRIME_2, PRIME_2_HIGH, PRIME_2_LOW = 0x85ebca77, 34283, 51831 + local PRIME_3_HIGH, PRIME_3_LOW = 49842, 44605 + local PRIME_4_HIGH, PRIME_4_LOW = 10196, 60207 + local PRIME_5, PRIME_5_HIGH, PRIME_5_LOW = 0x165667b1, 5718, 26545 + + local UsedSeed = Seed or 0 + local MessageLength = buffer.len(Message) + local Digest: number + local CurrentOffset = 0 + + if MessageLength >= 16 then + local Accumulator1 = UsedSeed + PRIME_1 + PRIME_2 + local Accumulator2 = UsedSeed + PRIME_2 + local Accumulator3 = UsedSeed + local Accumulator4 = UsedSeed - PRIME_1 + + while CurrentOffset <= MessageLength - 16 do + local Word1 = buffer.readu32(Message, CurrentOffset) + local Word2 = buffer.readu32(Message, CurrentOffset + 4) + local Word3 = buffer.readu32(Message, CurrentOffset + 8) + local Word4 = buffer.readu32(Message, CurrentOffset + 12) + + local AHigh1, ALow1 = bit32.rshift(Word1, 16), bit32.band(Word1, 65535) + local Mult1 = bit32.lshift((AHigh1 * PRIME_2_LOW) + (ALow1 * PRIME_2_HIGH), 16) + (ALow1 * PRIME_2_LOW) + + local Temp1 = bit32.lrotate(Accumulator1 + Mult1, 13) + local AHigh1_2, ALow1_2 = bit32.rshift(Temp1, 16), bit32.band(Temp1, 65535) + Accumulator1 = bit32.lshift((AHigh1_2 * PRIME_1_LOW) + (ALow1_2 * PRIME_1_HIGH), 16) + (ALow1_2 * PRIME_1_LOW) + + local AHigh2, ALow2 = bit32.rshift(Word2, 16), bit32.band(Word2, 65535) + local Mult2 = bit32.lshift((AHigh2 * PRIME_2_LOW) + (ALow2 * PRIME_2_HIGH), 16) + (ALow2 * PRIME_2_LOW) + + local Temp2 = bit32.lrotate(Accumulator2 + Mult2, 13) + local AHigh2_2, ALow2_2 = bit32.rshift(Temp2, 16), bit32.band(Temp2, 65535) + Accumulator2 = bit32.lshift((AHigh2_2 * PRIME_1_LOW) + (ALow2_2 * PRIME_1_HIGH), 16) + (ALow2_2 * PRIME_1_LOW) + + local AHigh3, ALow3 = bit32.rshift(Word3, 16), bit32.band(Word3, 65535) + local Mult3 = bit32.lshift((AHigh3 * PRIME_2_LOW) + (ALow3 * PRIME_2_HIGH), 16) + (ALow3 * PRIME_2_LOW) + + local Temp3 = bit32.lrotate(Accumulator3 + Mult3, 13) + local AHigh3_2, ALow3_2 = bit32.rshift(Temp3, 16), bit32.band(Temp3, 65535) + Accumulator3 = bit32.lshift((AHigh3_2 * PRIME_1_LOW) + (ALow3_2 * PRIME_1_HIGH), 16) + (ALow3_2 * PRIME_1_LOW) + + local AHigh4, ALow4 = bit32.rshift(Word4, 16), bit32.band(Word4, 65535) + local Mult4 = bit32.lshift((AHigh4 * PRIME_2_LOW) + (ALow4 * PRIME_2_HIGH), 16) + (ALow4 * PRIME_2_LOW) + + local Temp4 = bit32.lrotate(Accumulator4 + Mult4, 13) + local AHigh4_2, ALow4_2 = bit32.rshift(Temp4, 16), bit32.band(Temp4, 65535) + Accumulator4 = bit32.lshift((AHigh4_2 * PRIME_1_LOW) + (ALow4_2 * PRIME_1_HIGH), 16) + (ALow4_2 * PRIME_1_LOW) + + CurrentOffset += 16 + end + + Digest = bit32.lrotate(Accumulator1, 1) + bit32.lrotate(Accumulator2, 7) + bit32.lrotate(Accumulator3, 12) + bit32.lrotate(Accumulator4, 18) + else + Digest = UsedSeed + PRIME_5 + end + + Digest += MessageLength + + while CurrentOffset <= MessageLength - 4 do + if CurrentOffset + 4 <= buffer.len(Message) then + local Word = buffer.readu32(Message, CurrentOffset) + + local AHigh_w, ALow_w = bit32.rshift(Word, 16), bit32.band(Word, 65535) + local Mult_w = bit32.lshift((AHigh_w * PRIME_3_LOW) + (ALow_w * PRIME_3_HIGH), 16) + (ALow_w * PRIME_3_LOW) + + Digest += Mult_w + + local Rotated = bit32.lrotate(Digest, 17) + local AHigh_r, ALow_r = bit32.rshift(Rotated, 16), bit32.band(Rotated, 65535) + Digest = bit32.lshift((AHigh_r * PRIME_4_LOW) + (ALow_r * PRIME_4_HIGH), 16) + (ALow_r * PRIME_4_LOW) + end + CurrentOffset += 4 + end + + while CurrentOffset < MessageLength do + if CurrentOffset < buffer.len(Message) then + local ByteValue = buffer.readu8(Message, CurrentOffset) + + local AHigh_b, ALow_b = bit32.rshift(ByteValue, 16), bit32.band(ByteValue, 65535) + local Mult_b = bit32.lshift((AHigh_b * PRIME_5_LOW) + (ALow_b * PRIME_5_HIGH), 16) + (ALow_b * PRIME_5_LOW) + + Digest += Mult_b + + local Rotated_b = bit32.lrotate(Digest, 11) + local AHigh_rb, ALow_rb = bit32.rshift(Rotated_b, 16), bit32.band(Rotated_b, 65535) + Digest = bit32.lshift((AHigh_rb * PRIME_1_LOW) + (ALow_rb * PRIME_1_HIGH), 16) + (ALow_rb * PRIME_1_LOW) + end + CurrentOffset += 1 + end + + local XorResult1 = bit32.bxor(Digest, bit32.rshift(Digest, 15)) + local AHigh_f1, ALow_f1 = bit32.rshift(XorResult1, 16), bit32.band(XorResult1, 65535) + Digest = bit32.lshift((AHigh_f1 * PRIME_2_LOW) + (ALow_f1 * PRIME_2_HIGH), 16) + (ALow_f1 * PRIME_2_LOW) + + local XorResult2 = bit32.bxor(Digest, bit32.rshift(Digest, 13)) + local AHigh_f2, ALow_f2 = bit32.rshift(XorResult2, 16), bit32.band(XorResult2, 65535) + Digest = bit32.lshift((AHigh_f2 * PRIME_3_LOW) + (ALow_f2 * PRIME_3_HIGH), 16) + (ALow_f2 * PRIME_3_LOW) + + return bit32.bxor(Digest, bit32.rshift(Digest, 16)) +end + +return XXH32 \ No newline at end of file diff --git a/.cursor/rules/atlas-api.mdc b/.cursor/rules/atlas-api.mdc index d704f798..ab1484f2 100644 --- a/.cursor/rules/atlas-api.mdc +++ b/.cursor/rules/atlas-api.mdc @@ -1,11 +1,6 @@ --- -description: "Web API and plugin communication: WebSocket protocol, live sync architecture, two-way sync, patch format, ServeSession state management, and ChangeBatcher." -globs: - - "src/web/**/*.rs" - - "src/serve_session.rs" - - "plugin/src/ServeSession.lua" - - "plugin/src/ChangeBatcher/**/*.lua" - - "plugin/src/ApiContext.lua" +globs: src/web/**,src/serve_session.rs,src/git.rs,,plugin/src/** +alwaysApply: false --- # Atlas API & Communication @@ -50,6 +45,7 @@ Studio → ChangeBatcher → Plugin → WebSocket → Server → Filesystem | `/api/write` | POST | Two-way sync: plugin sends changes | | `/api/open/:id` | POST | Open script in editor (`{"cursor": [line, col]}`) | | `/api/ref-patch/:id` | GET | Fix Ref properties after syncback | +| `/api/syncback` | POST | Live syncback: full Roblox → filesystem sync | | `/api/socket/:cursor` | GET | WebSocket upgrade for live patch streaming | ### Server Info Response (`GET /api/rojo`) @@ -68,10 +64,22 @@ Studio → ChangeBatcher → Plugin → WebSocket → Server → Filesystem "visibleServices": ["Workspace", "ServerScriptService"], "placeId": 123456, "gameId": 789012, - "ignoreHiddenServices": true + "ignoreHiddenServices": true, + "gitMetadata": { + "changedIds": ["ref-1", "ref-2"], + "scriptCommittedHashes": { + "ref-1": ["a1b2c3...40-char-sha1-hex", "d4e5f6...staged-sha1"] + } + } } ``` +**`gitMetadata`** is `null` when the project is not in a git repo. Contains: +- `changedIds`: Instance Refs whose backing files have changes relative to HEAD (staged + unstaged + untracked) +- `scriptCommittedHashes`: For changed script instances, 1-2 SHA1 hashes (HEAD version and staged version if different) computed using the git blob format (`"blob \0"`). Plugin checks if Studio `script.Source` matches ANY hash to default "Atlas". + +Recomputed fresh on each `GET /api/rojo` connect. Uses a two-phase tree lock: snapshot path mappings briefly, then release before running git subprocesses. + ### Write Request (`POST /api/write`) ```json @@ -79,12 +87,108 @@ Studio → ChangeBatcher → Plugin → WebSocket → Server → Filesystem "sessionId": "uuid", "removed": ["id-1"], "added": { "id-2": { "parent": "parent-id", "className": "Folder", "name": "New", "properties": {}, "children": [] } }, - "updated": [{ "id": "id-3", "changedName": "Renamed", "changedClassName": "Script", "changedProperties": { "Source": "print('hi')" }, "changedMetadata": {} }] + "updated": [{ "id": "id-3", "changedName": "Renamed", "changedClassName": "Script", "changedProperties": { "Source": "print('hi')" }, "changedMetadata": {} }], + "stageIds": ["ref-1", "ref-2"] } ``` +**`stageIds`**: Instance IDs whose backing files should be staged via `git add` after writing. Includes all push-accepted IDs + auto-selected pull-accepted IDs. The server resolves these to filesystem paths and runs `git add`. Staging is split: `api.rs` stages additions/removals/push files directly; `change_processor` stages Source writes after they complete (via `stage_ids` on PatchSet). + **Format detection:** `Name/init.luau` → directory | `Name.luau` → file | neither → use `has_children` (`.lua` also supported) +### Live Syncback Endpoint (`POST /api/syncback`) + +One-click full sync from Studio to filesystem. Does **not** require an active WebSocket connection -- the plugin POSTs directly from the NotConnected page using the host/port fields. + +**Request (`SyncbackRequest`):** + +```rust +pub struct SyncbackRequest { + pub protocol_version: f64, // f64 because Luau has no integer type + pub server_version: String, + pub place_id: Option, + pub data: Vec, // Single rbxm blob containing ALL children from ALL services + pub services: Vec, +} +``` + +**`ServiceChunk`:** + +```rust +pub struct ServiceChunk { + pub class_name: String, + pub child_count: u32, + pub ref_target_count: u32, + pub properties: HashMap, // Service-level properties (omitted if empty) + pub refs: HashMap, // Ref property name → 1-based carrier index (omitted if empty) +} +``` + +**Data format:** The `data` field is a single rbxm binary blob containing all children from all services concatenated. Each service's range in the blob is `child_count + ref_target_count` consecutive entries. First `child_count` entries are real children (parented to service). Next `ref_target_count` entries are temporary ObjectValue carriers encoding Ref properties (destroyed after resolving). The `refs` map connects property names to carrier indices. + +**Validation:** +- Protocol version must match `PROTOCOL_VERSION` (6) → 400 Bad Request +- Server version MAJOR.MINOR must match → 400 Bad Request +- Place ID must pass `servePlaceIds`/`blockedPlaceIds` checks if configured → 403 Forbidden + +**Response:** `{"status": "syncback_initiated"}` + +**Server-side types:** + +```rust +pub struct SyncbackPayload { + pub data: Vec, + pub services: Vec, +} + +pub enum ServerExitReason { + SyncbackRequested(SyncbackPayload), +} +``` + +**`SyncbackSignal` (`src/web/mod.rs`):** Thread-safe signal for API handler → serve loop communication. Uses `Mutex>` + `Notify`. `fire()` deposits payload and notifies; `take_payload()` retrieves it. `LiveServer::start()` uses `tokio::select!` to wait for either new connections OR the syncback signal, exiting the accept loop when fired. + +### Live Syncback Data Flow + +``` +Plugin (NotConnected page) + │ User clicks "Syncback" button → SyncbackConfirm dialog + │ User confirms → performSyncback() + │ Encodes services via encodeService() + serializes children via SerializationService + ▼ +HTTP POST /api/syncback (MessagePack body) + │ + ▼ +handle_api_syncback() [src/web/api.rs] + │ Validates protocol, version, place ID + │ Fires SyncbackSignal + ▼ +LiveServer::start() receives notification [src/web/mod.rs] + │ Exits accept loop + │ Returns ServerExitReason::SyncbackRequested(payload) + ▼ +run_live_syncback() [src/cli/serve.rs] + │ build_dom_from_chunks(payload) → reconstructs DOM from rbxm + service chunks + │ syncback_loop(incremental = false) → clean mode, full filesystem rewrite + │ write_to_vfs_parallel() → writes files + │ refresh_git_index() → updates git index + ▼ +Serve loop restarts (new session, ready for connections) +``` + +### Live Syncback vs Two-Way Sync + +| Aspect | Live Syncback (`/api/syncback`) | Two-Way Sync (`/api/write`) | +|--------|--------------------------------|---------------------------| +| **Purpose** | Full Roblox → filesystem sync | Incremental property/instance changes | +| **Scope** | Entire instance tree (all services) | Individual instance changes | +| **Mode** | Clean mode (replaces everything) | Incremental (add/update/remove) | +| **Server restart** | Yes (restarts serve loop) | No (continues serving) | +| **Data format** | Single rbxm blob + service chunks | Patch-based (added/updated/removed) | +| **Connection** | No WebSocket needed (HTTP POST) | Requires active WebSocket session | +| **When used** | User-initiated one-click syncback | Real-time bidirectional sync | +| **Session** | Creates new session after restart | Uses existing live session | + ## WebSocket Protocol (v6) **Connection:** `ws://host:port/api/socket/:cursor` @@ -177,7 +281,7 @@ for id, data in patch.added do instanceMap:insert(id, Reconciler.reify(data)) en for _, update in patch.updated do Reconciler.hydrate(instanceMap:fromId(update.id), update) end ``` -## Two-Way Sync (Experimental) +## Two-Way Sync **Setting:** `twoWaySync` (default: true) | Syncs Studio changes → filesystem in real-time @@ -229,13 +333,21 @@ for _, update in patch.updated do Reconciler.hydrate(instanceMap:fromId(update.i │ │ Looks up InstigatingSource::Path → fs::remove_file() │ │ ▼ │ │ For each added: syncback_added_instance() │ +│ │ Groups by parent, pre-seeds sibling_slugs, dedup on collision │ │ │ Determines file format, writes to filesystem │ │ ▼ │ +│ Build added_paths map (for same-batch Ref resolution) │ +│ │ Maps Ref → filesystem path via added_instance_fs_segment() │ +│ ▼ │ +│ For each updated: syncback_updated_properties() │ +│ │ Extracts Ref props → Rojo_Ref_* attrs (path from tree/added_paths) │ +│ │ merge_or_build_meta() writes props/attrs (supports attr removal) │ +│ ▼ │ │ Sends PatchSet to tree_mutation_sender channel │ │ ▼ │ │ ChangeProcessor::handle_tree_event() [src/change_processor.rs] │ -│ │ For removed: fs::remove_file(path) + remove meta │ -│ │ For updated: Source writes, renames, ClassName changes, properties │ +│ │ For removed: dedup cleanup (gap-tolerant, base-name promotion) │ +│ │ For updated: Source writes, renames, ClassName changes, ref paths │ │ ▼ │ │ apply_patch_set() → Updates RojoTree │ │ ▼ │ @@ -293,17 +405,21 @@ end | File | Purpose | |------|---------| -| `init.lua` | Main batcher: accumulator, RenderStepped loop, pause/resume | -| `createPatchSet.lua` | Converts pending changes → PatchSet | -| `encodePatchUpdate.lua` | Encodes property updates for existing instances | -| `encodeInstance.lua` | Full instance encoding (for additions/syncback) | +| `init.lua` | Main batcher: accumulator, RenderStepped loop, pause/resume, deferred refs | +| `createPatchSet.lua` | Converts pending changes → PatchSet (passes instanceMap for Ref resolution) | +| `encodePatchUpdate.lua` | Encodes property updates including Ref properties via InstanceMap | +| `encodeInstance.lua` | Full instance encoding (for additions/syncback); uses `encodeHelpers` for property iteration; includes duplicate detection helpers (`findDuplicateNames`, `hasDuplicateSiblings`, `isPathUnique`) | +| `encodeHelpers.lua` | Shared encoding helpers: `encodeAttributes`, `encodeTags`, `forEachEncodableProperty`. Used by `encodeInstance.lua` and `encodeService.lua`. Defines `SKIP_PROPERTIES` (Parent, Name, Archivable). | +| `encodeService.lua` | Encodes service-level properties/attributes/refs for live syncback; uses ObjectValue carriers for Ref properties | | `encodeProperty.lua` | Single property value encoding via RbxDom | +| `propertyFilter.lua` | Property type filtering (only `UniqueId` blocked; `Ref` is allowed) | **Core State:** ```lua ChangeBatcher = { __accumulator = 0, -- Time since last flush __pendingPropertyChanges = {}, -- Instance → { PropertyName → true } + __pendingUnresolvedRefs = {}, -- targetInstance → [{sourceInstance, propertyName}] __syncSourceOnly = false, -- Only sync Source property __paused = false, -- Pause during confirmation __renderSteppedConnection = conn, -- Frame loop connection @@ -338,10 +454,20 @@ end - Reduces server load and WebSocket traffic - Still feels responsive to users +**Deferred Ref Mechanism:** + +When a Ref property targets an instance not yet in the InstanceMap (e.g., newly pulled instance whose server ID hasn't arrived), the property is deferred: + +```lua +-- In encodePatchUpdate: onUnresolvedRef callback defers the property +-- In ChangeBatcher: __pendingUnresolvedRefs[targetInstance] = {sourceInstance, propertyName} +-- When target appears (onInstanceInserted), re-queues the property change for next batch +``` + ### createPatchSet.lua: Patch Generation ```lua -function createPatchSet(instanceMap, propertyChanges, syncSourceOnly) +function createPatchSet(instanceMap, propertyChanges, syncSourceOnly, onUnresolvedRef) local patch = PatchSet.newEmpty() for instance, properties in propertyChanges do @@ -353,20 +479,18 @@ function createPatchSet(instanceMap, propertyChanges, syncSourceOnly) if properties.Parent then if instance.Parent == nil then - -- Instance was deleted in Studio table.insert(patch.removed, instanceId) else - Log.warn("Cannot sync non-nil Parent changes yet") -- Reparenting not supported + Log.warn("Cannot sync non-nil Parent changes yet") end else if syncSourceOnly then - -- Server in source-only mode: only sync Source property if properties.Source then - local update = encodePatchUpdate(instance, instanceId, { Source = true }) + local update = encodePatchUpdate(instance, instanceId, { Source = true }, instanceMap, onUnresolvedRef) if update then table.insert(patch.updated, update) end end else - local update = encodePatchUpdate(instance, instanceId, properties) + local update = encodePatchUpdate(instance, instanceId, properties, instanceMap, onUnresolvedRef) if update then table.insert(patch.updated, update) end end end @@ -378,8 +502,10 @@ end ### encodePatchUpdate.lua: Update Encoding +Encodes property updates for existing instances. Now handles Ref properties via InstanceMap lookup. + ```lua -function encodePatchUpdate(instance, instanceId, properties) +function encodePatchUpdate(instance, instanceId, properties, instanceMap, onUnresolvedRef) local update = { id = instanceId, changedProperties = {}, @@ -392,15 +518,30 @@ function encodePatchUpdate(instance, instanceId, properties) local descriptor = RbxDom.findCanonicalPropertyDescriptor(instance.ClassName, propertyName) if not descriptor then continue end - local success, encoded = encodeProperty(instance, propertyName, descriptor) - if success then - update.changedProperties[propertyName] = encoded + if descriptor.dataType == "Ref" then + -- Ref property: resolve target via InstanceMap + local target = instance[propertyName] + if target == nil then + update.changedProperties[propertyName] = { Ref = NULL_REF } + else + local targetId = instanceMap.fromInstances[target] + if targetId then + update.changedProperties[propertyName] = { Ref = targetId } + elseif onUnresolvedRef then + onUnresolvedRef(target, instance, propertyName) + end + end + else + local success, encoded = encodeProperty(instance, propertyName, descriptor) + if success then + update.changedProperties[propertyName] = encoded + end end end end if next(update.changedProperties) == nil and update.changedName == nil then - return nil -- No actual changes + return nil end return update @@ -409,65 +550,31 @@ end ### encodeInstance.lua: Full Instance Encoding (Syncback/Additions) -Used when pulling new instances from Studio to filesystem. **Critical: handles duplicate detection.** +Used when pulling new instances from Studio to filesystem. Uses `encodeHelpers.forEachEncodableProperty()` for property iteration (shared with `encodeService.lua`). Duplicate-named instances are encoded and sent to the server; the server handles filesystem collisions via the dedup suffix system (`~N`). Includes duplicate detection helpers (`findDuplicateNames`, `hasDuplicateSiblings`, `isPathUnique`). ```lua --- Duplicate handling (matches server's ref_properties.rs::is_path_unique) -local function isPathUnique(instance) - local current = instance - while current do - if hasDuplicateSiblings(current) then - return false -- Any duplicate in ancestor chain = ambiguous path - end - current = current.Parent - if current == game then break end - end - return true -end - function encodeInstance(instance, parentId, _skipPathCheck) - -- Skip instances with ambiguous paths (duplicate-named siblings) - if not _skipPathCheck and not isPathUnique(instance) then - Log.warn("Skipped '{}' - path contains duplicate-named siblings", instance:GetFullName()) - return nil - end - local properties = {} - -- Encode Attributes and Tags (work on all instance types) - encodeAttributes(instance, properties) - encodeTags(instance, properties) + Helpers.encodeAttributes(instance, properties) + Helpers.encodeTags(instance, properties) - -- For scripts, Source is required if SCRIPT_CLASS_NAMES[instance.ClassName] then local success, encoded = encodeProperty(instance, "Source", descriptor) if not success then return nil end properties.Source = encoded end - -- Encode all serializable properties - local classDescriptor = RbxDom.findClassDescriptor(instance.ClassName) - for propertyName, meta in classDescriptor.properties do - if SKIP_PROPERTIES[propertyName] then continue end -- Parent, Name, Archivable - if SKIP_PROPERTIES_BY_CLASS[instance.ClassName]?[propertyName] then continue end -- Source, ScriptGuid, etc. - - local isReadable = meta.scriptability == "ReadWrite" or meta.scriptability == "Read" - local doesSerialize = meta.serialization ~= "DoesNotSerialize" - - if isReadable and doesSerialize then - local success, encoded = encodeProperty(instance, propertyName, descriptor) - if success and encoded ~= nil then - properties[propertyName] = encoded - end + Helpers.forEachEncodableProperty(instance.ClassName, SKIP_PROPERTIES_BY_CLASS, function(propertyName, descriptor) + local success, encoded = encodeProperty(instance, propertyName, descriptor) + if success and encoded ~= nil then + properties[propertyName] = encoded end - end + end) - -- Recursively encode children (skipping duplicates at each level) local children = {} - local childDuplicates = findDuplicateNames(instance:GetChildren()) for _, child in instance:GetChildren() do - if childDuplicates[child.Name] then continue end - local encoded = encodeInstance(child, nil, true) -- Skip path check, already validated + local encoded = encodeInstance(child, nil, true) if encoded then table.insert(children, encoded) end end @@ -521,19 +628,31 @@ true, false, 42, "string" ### Server-Side: ChangeProcessor (`src/change_processor.rs`) -Receives patches from plugin via `tree_mutation_receiver` channel. Handles removals, Source writes, renames (with slugify/dedup/meta lifecycle), ClassName changes (script type transitions via file renames), and property writes to meta/model files. +Receives patches from plugin via `tree_mutation_receiver` channel. Handles removals (with dedup cleanup), Source writes, renames (with slugify/dedup/meta lifecycle), ClassName changes (script type transitions via file renames), property writes to meta/model files, and ref path updates. **Capabilities:** - **Removals:** Deletes file + associated meta file via `InstigatingSource::Path` +- **Dedup cleanup after removals:** When a deletion leaves a dedup group needing suffix cleanup, performs renames (gap-tolerant, base-name promotion, group-to-1). Uses `removed_set` (HashSet) to exclude co-removed siblings from sibling enumeration. Calls `compute_cleanup_action()` from `dedup_suffix.rs`. - **Source writes:** Updates script file contents - **Renames:** Slugifies new name, deduplicates against siblings, renames file on disk, creates/updates/removes meta `name` field as needed. Handles both standalone files and init-file directories. - **ClassName changes:** Script-to-script transitions (e.g. ModuleScript → Script) via file extension renames - **Property writes:** Attributes and other properties written to meta/model files via `syncback_updated_properties` in `api.rs` +- **Ref path updates:** `RefPathIndex` tracks which meta/model files contain `Rojo_Ref_*` paths. `update_ref_paths_after_rename()` efficiently updates all affected files when an instance is renamed or dedup-cleaned. +- **VFS suppression for dedup renames:** `suppress_path_any(&from)` + `suppress_path(&to)` prevents the VFS watcher from re-triggering on files renamed during dedup cleanup. +- **InstigatingSource updates:** After dedup cleanup renames, the renamed sibling's `InstigatingSource::Path` and metadata path indexes are updated in-memory. **InstigatingSource:** Every instance in RojoTree tracks where it came from: - `InstigatingSource::Path(PathBuf)` - Created from a file, can be written back - `InstigatingSource::ProjectNode { path, name, node, parent_class }` - Defined in project file, **cannot be modified via two-way sync** +**`handle_tree_event()` flow:** +1. Process removals (log but don't delete -- API already deleted files) +2. Dedup cleanup (runs before `apply_patch_set` to use parent/sibling relationships) +3. Process updates: renames, ClassName changes, Source writes, ref path updates +4. `apply_patch_set()` to update RojoTree +5. Apply metadata updates (InstigatingSource paths for both regular and dedup renames) +6. Send applied patch to message queue for plugin broadcast + ### Pause Mechanism: Critical for Consistency **Why pause is essential:** @@ -584,6 +703,25 @@ end - Safe for production environments - Team viewing same project simultaneously +### Git-Based Sync Direction Defaults + +When the plugin shows the sync confirmation dialog, items get smart default selections based on git status: + +- **File has no git changes**: default to "Studio" (pull) -- any diff is because Studio was changed +- **File has git changes AND is a script AND Studio Source matches committed/staged hash**: default to "Atlas" (push) -- developer changed the file, Studio hasn't diverged +- **Otherwise**: `nil` -- user must decide (both sides changed, or non-script with changes) +- **No git repo**: all defaults remain `nil` (feature is no-op) + +Only **Updated** items get defaults. Added/Removed items remain `nil`. + +**Hash computation**: Both sides use `SHA1("blob \0")` (git blob format). Server sends up to 2 hashes per changed script (HEAD + staged if different). Plugin computes hash of `script.Source` (non-draft version) and checks if ANY hash matches. + +**Auto-staging**: After confirmation, accepted files are staged via `git add`: +- Push-accepted items: always staged (all types, including non-scripts) +- Pull-accepted items: only staged if auto-selected (`defaultSelection ~= nil`); manually-chosen pulls are left unstaged for manual diff + +**Implementation files**: `src/git.rs` (server git helpers), `plugin/src/PatchTree.lua` (default selection logic), `plugin/src/SHA1.luau` (hash computation), `src/web/interface.rs` (`GitMetadata` struct, `stage_ids` on WriteRequest/PatchSet) + ### Known Limitations (Detailed) | Issue | Cause | Impact | @@ -592,8 +730,23 @@ end | **Rapid changes interleave** | 200ms batch window may split logically related changes | Partial updates can leave inconsistent state | | **Reparenting not supported** | `Parent` property changes to non-nil values are ignored | Moving instances in Studio doesn't sync | | **Project node instances immutable** | InstigatingSource::ProjectNode cannot be modified | Instances from project file can't be changed | -| **Duplicate-named siblings skipped** | Path ambiguity makes reliable sync impossible | Instances with same-name siblings ignored | -| **Ref properties complex** | Instance references may become invalid when instances move | ObjectValue.Value may not sync correctly | +| **Greedy matching not globally optimal** | Ambiguous groups (same Name+ClassName) use greedy assignment | Rare edge cases may produce suboptimal pairing and rename churn | +| **Ref paths with ancestor-level ambiguity** | `Rojo_Ref_*` path-based refs written to disk may resolve to wrong sibling if duplicates exist at any ancestor level | Ref may point to wrong instance after rebuild; in-memory tree is correct during session | + +### Reconciler Matching Algorithm (`plugin/src/Reconciler/`) + +The plugin uses a recursive change-count scoring algorithm (implemented in `matching.lua`) to pair incoming virtual instances with existing Studio instances during hydration. + +**Entry point:** `Matching.matchChildren(virtualChildren, studioChildren, virtualInstances)` + +**Algorithm:** +1. **Fast path:** Group by (Name, ClassName). 1:1 groups get instant-matched with zero scoring. +2. **Ambiguous groups:** For groups with multiple candidates, compute pairwise `computePairCost()` (recursive, scores entire subtree). Greedy assignment sorted by cost ascending, child-order tiebreaker. +3. **Leftovers:** Unmatched virtual = new instances, unmatched Studio = deleted. + +**Performance:** File-level `--!native` and `--!optimize 2` directives for native codegen. `MatchingSession` caches scoring results across recursive calls (`matchCache` for match results, `costCache` for pair costs), avoiding redundant computation when hydrate re-calls `matchChildren` on the same children. `cacheVirtual()` filters children to only those existing in `virtualInstances`, preventing crashes on incomplete virtual instance trees. + +**Value equality:** Shared `trueEquals.lua` module used by both `matching.lua` and `diff.lua` (fuzzy equality for floats with epsilon 0.0001, Color3, CFrame, Vector3; NaN handling; nil/null-ref equivalence). ## Protocol Enforcement @@ -635,6 +788,10 @@ Browser: `http://localhost:34872` → Network tab → WebSocket messages | Server | Plugin | |--------|--------| | `src/web/api.rs` - Endpoints | `plugin/src/ServeSession.lua` - Connection | -| `src/web/interface.rs` - Packets | `plugin/src/ApiContext.lua` - HTTP/WS client | -| `src/serve_session.rs` - Session | `plugin/src/ChangeBatcher/` - Two-way sync | -| | `plugin/src/Reconciler/` - Patch application | +| `src/web/interface.rs` - Packets, syncback types | `plugin/src/ApiContext.lua` - HTTP/WS client | +| `src/web/mod.rs` - SyncbackSignal, LiveServer | `plugin/src/ChangeBatcher/` - Two-way sync | +| `src/serve_session.rs` - Session | `plugin/src/ChangeBatcher/encodeService.lua` - Live syncback encoding | +| `src/cli/serve.rs` - Live syncback handler | `plugin/src/Reconciler/` - Patch application | +| `src/change_processor.rs` - FS writer | `plugin/src/Reconciler/matching.lua` - Matching | +| `src/rojo_ref.rs` - Ref paths | `plugin/src/PatchTree.lua` - Default selections | +| `src/git.rs` - Git integration | `plugin/src/App/init.lua` - Live syncback UI | diff --git a/.cursor/rules/atlas-dedup.mdc b/.cursor/rules/atlas-dedup.mdc new file mode 100644 index 00000000..fc6cb7ef --- /dev/null +++ b/.cursor/rules/atlas-dedup.mdc @@ -0,0 +1,579 @@ +--- +globs: src/syncback/file_names.rs,src/syncback/dedup_suffix.rs,src/syncback/matching.rs,src/syncback/ref_properties.rs,src/snapshot/matching.rs,src/snapshot/tree.rs,src/snapshot/patch_compute.rs,src/rojo_ref.rs,src/variant_eq.rs,src/change_processor.rs,src/web/api.rs,plugin/src/Reconciler/matching.lua,plugin/src/Reconciler/trueEquals.lua,plugin/src/Reconciler/hydrate.lua,plugin/src/Reconciler/diff.lua,plugin/src/XXH32.luau,tests/tests/two_way_sync.rs,tests/tests/syncback_roundtrip.rs +alwaysApply: false +--- + +# Atlas Dedup System + +> Atlas is a fork of Rojo. Many internal identifiers, API endpoints, and struct names still reference the original "Rojo" naming. + +## Quality Standard + +This is a production-grade synchronization system. The filesystem must represent a 1:1 copy of the Studio place. The core invariant: + +**Round-trip identity:** Syncback (or two-way sync) writes a directory tree. Building an rbxl from that directory tree and forward-syncing it back must produce a bit-identical instance tree -- same names, same classes, same properties, same hierarchy, same ref targets. Any deviation is a bug. + +**CLI syncback parity:** Plugin-based two-way sync must produce exactly the same filesystem output as `atlas syncback`. Byte-for-byte identical files, identical directory structures, identical naming. Any divergence is a bug until proven otherwise. + +## Architecture Overview + +The dedup system replaced the old "skip duplicates" blacklist with three interlocking subsystems: + +``` +Matching Algorithm → Dedup System → Ref Path System +(pairs instances) (assigns FS names) (paths use final FS names) +``` + +**Execution order per sync operation:** Matching runs first (pairs instances), dedup runs second (assigns filesystem names), ref paths are computed last (using final filesystem names). + +### Three Implementations of Matching + +The matching algorithm exists in 3 parallel implementations that must produce identical pairings: + +1. **Rust syncback** -- `src/syncback/matching.rs` (operates on `WeakDom` instances) +2. **Rust forward sync** -- `src/snapshot/matching.rs` (operates on `InstanceSnapshot` vs `RojoTree`) +3. **Lua plugin** -- `plugin/src/Reconciler/matching.lua` (operates on Studio instances vs virtual instances) + +All three use the same recursive change-count scoring + greedy assignment algorithm. Changes to the algorithm must be applied to all three. + +All three now have **`MatchingSession` caching** to avoid redundant recursive scoring: +- **Rust:** `cost_cache: RefCell>` -- caches `compute_change_count` results +- **Lua:** `matchCache` (parent virtual ID → Studio instance → MatchResult), `costCache` (virtual ID → Studio instance → cost) + +### Key Files Map + +| File | Subsystem | Purpose | +|------|-----------|---------| +| `src/syncback/file_names.rs` | Dedup | `slugify_name`, `deduplicate_name_with_ext`, `name_for_inst`, `name_needs_slugify` | +| `src/syncback/dedup_suffix.rs` | Dedup | `compute_cleanup_action`, `DedupCleanupAction`, `parse_dedup_suffix`, `build_dedup_name` | +| `src/syncback/matching.rs` | Matching | Syncback matching: `match_children` (WeakDom) | +| `src/snapshot/matching.rs` | Matching | Forward sync matching: `match_forward` (Snapshot vs Tree) | +| `plugin/src/Reconciler/matching.lua` | Matching | Lua matching: `Matching.matchChildren` (Virtual vs Studio) | +| `src/rojo_ref.rs` | Ref Paths | `ref_target_path_from_tree`, `compute_relative_ref_path`, `resolve_ref_path_to_absolute`, `RefPathIndex`, `Rojo_Ref_*` constants | +| `src/snapshot/tree.rs` | Ref Paths | `get_instance_by_path`, `filesystem_name_for` | +| `src/syncback/ref_properties.rs` | Ref Paths | `collect_referents`, `tentative_fs_path`, `RefLinks` | +| `src/variant_eq.rs` | Matching | `variant_eq` property comparison (Rust) | +| `plugin/src/Reconciler/trueEquals.lua` | Matching | `trueEquals` property comparison (Lua) | +| `src/change_processor.rs` | Integration | Dedup cleanup on deletion, VFS suppression, ref path updates | +| `src/web/api.rs` | Integration | `added_paths`, `syncback_updated_properties`, ref-to-attribute conversion | +| `plugin/src/ChangeBatcher/encodeHelpers.lua` | Encoding | Shared property filtering/encoding helpers (`encodeAttributes`, `encodeTags`, `forEachEncodableProperty`). Used by `encodeInstance.lua` and `encodeService.lua`. | + +--- + +## Subsystem 1: Slugification and Dedup (`file_names.rs`) + +### slugify_name + +Pure, stateless. Replaces forbidden filesystem characters with `_`. + +**Constants:** +```rust +const FORBIDDEN_CHARS: [char; 9] = ['<', '>', ':', '"', '/', '|', '?', '*', '\\']; +const SLUGIFY_CHARS: [char; 10] = ['<', '>', ':', '"', '/', '|', '?', '*', '\\', '~']; +const DANGEROUS_SUFFIXES: [&str; 8] = [".server", ".client", ".plugin", ".local", ".legacy", ".meta", ".model", ".project"]; +const INVALID_WINDOWS_NAMES: [&str; 22] = ["CON", "PRN", "AUX", "NUL", "COM1".."COM9", "LPT1".."LPT9"]; +``` + +**Behavior:** +- Replaces each `SLUGIFY_CHARS` char with `_` (including `~` to prevent natural names from looking like dedup suffixes) +- Strips leading/trailing spaces and dots +- Neutralizes Windows reserved names (`CON` → `CON_`) +- Neutralizes dangerous suffixes (`.server` etc.) that would be misinterpreted as middleware extensions + +**Key invariant:** `slugify_name` is idempotent. Applying it twice produces the same result. + +### name_needs_slugify + +Returns `true` if the name requires slugification. Checks for: `SLUGIFY_CHARS`, control characters, leading/trailing spaces, trailing dots, dangerous suffixes (`.server` etc.), Windows reserved names (`CON` etc.), and empty string. Used to decide whether metadata `name` field is needed. + +### deduplicate_name_with_ext + +Stateful collision resolver. Takes a base slug and extension, checks against `taken_names` set (case-insensitive). + +```rust +pub fn deduplicate_name_with_ext( + base: &str, + extension: Option<&str>, + taken_names: &HashSet, +) -> (String, String) +// Returns: (deduped_slug, full_fs_name) +// "Foo" + ".server.luau" with collision → ("Foo~2", "Foo~2.server.luau") +``` + +**Key invariant:** Suffix goes on the stem BEFORE the compound extension: `Foo~2.server.luau`, NOT `Foo.server~2.luau`. + +### Dedup Key Semantics + +The dedup key is the **full filesystem name component** (slug + extension), lowercased. This means: + +- `Folder "Foo"` (key: `"foo"`) and `Script "Foo"` (key: `"foo.server.luau"`) do NOT collide +- `Script "Foo" (server)` and `Script "Foo" (server)` both key `"foo.server.luau"` → collision, dedup applied +- `Folder "Foo"` and init-style `Script "Foo"` both key `"foo"` (both directories) → collision, dedup applied + +### name_for_inst + +The primary entry point for generating filesystem names during syncback: + +```rust +pub fn name_for_inst<'a>( + middleware: Middleware, + new_inst: &'a Instance, + old_inst: Option>, + taken_names: &HashSet, +) -> anyhow::Result<(Cow<'a, str>, bool, String)> +// Returns: (filename, needs_meta_name, dedup_key) +``` + +- In incremental mode (`old_inst` is `Some`): preserves existing filesystem path +- In clean mode: slugifies name, adds middleware extension, deduplicates against `taken_names` +- Returns `needs_meta_name = true` when slug differs from real name (including dedup suffix) + +### parse_dedup_suffix (in `dedup_suffix.rs`) + +Parses `~N` suffix from a stem: `"Foo~3"` → `Some(("Foo", 3))`, `"Foo"` → `None`. Returns `None` for `~0` (invalid) and `~abc` (non-numeric). + +### Metadata `name` Field + +When the filesystem name differs from the instance name (due to slugification or dedup), the real name is stored in `.meta.json5` or `.model.json5` `name` field. This is the authoritative instance name during forward sync. + +``` +Instance "Hey/Bro" → Hey_Bro.luau + meta: {"name": "Hey/Bro"} +Instance "Folder" → Folder/ (no meta needed, name == slug) +Instance "Folder" → Folder~2/ + meta: {"name": "Folder"} +``` + +--- + +## Subsystem 2: Dedup Suffix Cleanup (`dedup_suffix.rs`) + +When instances are deleted, the remaining dedup group may need suffix adjustments. + +### DedupCleanupAction + +```rust +pub enum DedupCleanupAction { + None, + RemoveSuffix { from: PathBuf, to: PathBuf }, + PromoteLowest { from: PathBuf, to: PathBuf }, +} +``` + +### Three Cleanup Rules + +1. **Gap-tolerant deletion:** Deleting `~2` from `{Foo, Foo~2, Foo~3}` leaves `{Foo, Foo~3}`. No renumbering. Gaps are tolerated. + +2. **Base-name promotion:** Deleting `Foo` (base) from `{Foo, Foo~2, Foo~3}` promotes `Foo~2` to `Foo`. `Foo~3` stays. The lowest-numbered suffix becomes the new base. + +3. **Group-to-1 cleanup:** Reducing the group to 1 member removes the suffix entirely. Deleting from `{Foo, Foo~2}` leaves one instance → renamed to clean `Foo`. + +### compute_cleanup_action + +```rust +pub fn compute_cleanup_action( + base_stem: &str, + extension: Option<&str>, + remaining_stems: &[String], + deleted_was_base: bool, + parent_dir: &Path, +) -> DedupCleanupAction +``` + +**Critical integration detail:** The `remaining_stems` input must exclude co-removed siblings (instances being deleted in the same batch). The `change_processor.rs` builds a `removed_set: HashSet` and skips those refs during sibling enumeration. + +--- + +## Subsystem 3: Matching Algorithm + +### Entry Points + +**Rust syncback:** +```rust +pub fn match_children( + new_children: &[Ref], old_children: &[Ref], + new_dom: &WeakDom, old_dom: &WeakDom, + new_hashes: Option<&HashMap>, + old_hashes: Option<&HashMap>, +) -> MatchResult +``` + +**Rust forward sync:** +```rust +pub fn match_forward( + snapshot_children: Vec, + tree_children: &[Ref], + tree: &RojoTree, +) -> ForwardMatchResult +``` + +**Lua plugin:** +```lua +function Matching.matchChildren( + virtualChildren: { string }, -- array of virtual instance IDs + studioChildren: { Instance }, -- array of Studio Instance objects + virtualInstances: { [string]: any } -- map of virtual ID → virtual instance data +) -> { matched, unmatchedVirtual, unmatchedStudio } +``` + +### Algorithm (All Three Implementations) + +**Fast-path: (Name, ClassName) Grouping.** Group children by `(Name, ClassName)`. 1:1 groups get instant-matched with zero scoring. + +**Ambiguous Groups: Recursive Scoring + Greedy Assignment.** For groups with multiple candidates on at least one side: + +``` +for each (A, B) in cross-product: + cost = computeChangeCount(A, B, bestSoFar) + if cost < bestSoFar: bestSoFar = cost + +stable sort by cost ascending (ties broken by child order) +greedy assign: pick lowest-cost pair, remove both, repeat +``` + +### computeChangeCount (Recursive) + +`computeChangeCount` and `matchChildren` are **mutually recursive**: + +``` +computeChangeCount(A, B, bestSoFar): + cost = countOwnDiffs(A, B) // properties + tags + attributes + if cost >= bestSoFar: return cost // early exit + + childResult = matchChildren(A.children, B.children) // RECURSE + for matched pair: cost += computeChangeCount(childA, childB, bestSoFar - cost) + cost += count(unmatched) * UNMATCHED_PENALTY + return cost +``` + +### Constants + +```rust +const UNMATCHED_PENALTY: u32 = 10_000; // Unmatched child = categorically expensive +const MAX_SCORING_DEPTH: u32 = 3; // Depth limit prevents O(n^k) explosion +``` + +### countOwnDiffs + +Counts property-level changes needed to turn A into B: +- Each differing property = +1 (using `variant_eq` / `trueEquals`) +- Each tag in symmetric difference = +1 +- Each differing attribute = +1 +- Name and ClassName NOT counted (grouping guarantees match) + +### Hash Fast-Path (Syncback Only) + +Syncback matching has precomputed blake3 subtree hashes. If `new_hash == old_hash`, return cost 0 immediately (skips property comparison and recursion). + +### Sort Stability + +- **Rust:** `slice::sort_by` is natively stable. Ties preserve child order. +- **Lua:** `table.sort` is NOT stable. Comparator breaks ties by insertion index: `if a.cost == b.cost then return a.idx < b.idx end`. + +### Property Comparison + +**Rust (`variant_eq`):** Type check first; float comparison uses `approx_eq!` with epsilon. Special handling for CFrame, Color3, Vector3, ColorSequence, NumberSequence, Attributes (sorted), Tags (sorted). + +**Lua (`trueEquals`):** Recursive deep equality. Treats `nil` and null ref (`{Ref = "000...0"}`) as equal. Floats use epsilon 0.0001 (absolute + relative). CFrame/Vector3 component-wise. Color3 via RGB ints. EnumItem via `.Value`. + +### Session Caching + +All three implementations wrap scoring in a `MatchingSession` that caches results: +- **Rust:** `compute_change_count` checks `cost_cache` before scoring. If `(ref_a, ref_b)` was scored before, returns cached value. +- **Lua:** `computePairCost` checks `costCache`; `matchChildren` checks `matchCache`. Avoids redundant work when hydrate re-calls matching on the same children. + +The Lua matching file uses `--!native` and `--!optimize 2` file-level directives for native codegen performance. `cacheVirtual()` filters children to only those existing in `virtualInstances`, preventing crashes on incomplete virtual instance trees. + +### Known Limitation: Greedy vs Optimal + +The greedy algorithm is NOT globally optimal. Edge cases exist where a locally-best pick forces a globally-worse assignment. The Hungarian algorithm (O(N^3)) would give optimal results, but ambiguous groups are typically small (2-10 instances). Accepted for current release. + +--- + +## Subsystem 4: Ref Path System + +### Path Format + +Ref paths use **full filesystem names** including extensions and dedup suffixes: + +``` +Folder "Foo" → segment "Foo" +ModuleScript "Foo" → segment "Foo.luau" +Script "Foo" (server) → segment "Foo.server.luau" +Dedup'd ModuleScript → segment "Foo~2.luau" +Init-style Script → segment "Foo" (directory) +``` + +Absolute paths are slash-separated from DataModel's children downward: `"Workspace/TestModel/Part1.model.json5"`. + +### Luau-Style Relative Ref Paths + +Syncback now writes **relative** ref paths using Luau require-by-string semantics via `compute_relative_ref_path()`: + +- `@self` -- target is the source instance itself +- `@self/Child.luau` -- target is a descendant of source +- `./Sibling.luau` -- target is a same-parent sibling +- `../Uncle.luau` -- target is a parent's sibling +- `@game/Service/Path` -- cross-service reference + +Forward sync resolves both absolute and relative formats via `resolve_ref_path_to_absolute()` + `get_instance_by_path()`. + +**Placeholder system:** During syncback, before dedup suffixes are finalized, ref paths are written as `__ROJO_REF__TO___` placeholders. After the syncback walk assigns final filesystem names, `fix_ref_paths()` substitutes placeholders with actual relative paths. + +### Key Functions + +**Building paths:** +```rust +// From RojoTree (two-way sync, change_processor): +pub fn ref_target_path_from_tree(tree: &RojoTree, target_ref: Ref) -> String +// Uses filesystem_name_for() for each ancestor + +// From WeakDom (CLI syncback, legacy instance-name paths): +pub fn ref_target_path(dom: &WeakDom, target_ref: Ref) -> String +// Uses dom.ancestors_of() with escape_ref_path_segment (instance names, NOT filesystem names) + +// Tentative path (before dedup suffixes assigned): +fn tentative_fs_path(dom: &WeakDom, target_ref: Ref) -> String +// Uses slugified name + extension based on class +``` + +**Resolving paths:** +```rust +// In tree.rs: +pub fn get_instance_by_path(&self, path: &str) -> Option +// Splits by "/", matches segments case-insensitive against filesystem_name_for() +// Fallback: matches against instance name (case-insensitive) + +pub fn filesystem_name_for(&self, id: Ref) -> String +// Returns filename from instigating_source, falls back to instance name +``` + +**Computing filesystem names for refs:** +```rust +// In ref_properties.rs: +fn tentative_fs_name(inst: &Instance) -> String +// Slugified name + extension based on class and children count +// Used before dedup suffixes are assigned + +pub fn collect_referents( + dom: &WeakDom, + pre_prune_paths: &HashMap, + final_paths: Option<&HashMap>, +) -> RefLinks +// Uses final_paths (dedup-aware) when available, otherwise tentative_fs_path +``` + +### Attribute Format + +Ref properties are stored as `Rojo_Ref_*` attributes in meta/model files: + +```json5 +{ + "attributes": { + "Rojo_Ref_PrimaryPart": "Workspace/TestModel/Part1.model.json5" + } +} +``` + +Constants in `rojo_ref.rs`: +```rust +pub const REF_PATH_ATTRIBUTE_PREFIX: &str = "Rojo_Ref_"; +pub const REF_POINTER_ATTRIBUTE_PREFIX: &str = "Rojo_Target_"; +pub const REF_ID_ATTRIBUTE_NAME: &str = "Rojo_Id"; +``` + +### RefPathIndex + +Tracks which filesystem files contain which ref paths, enabling efficient updates on rename: + +```rust +pub struct RefPathIndex { + paths_to_files: HashMap>, +} +``` + +Used by `update_ref_paths_after_rename` in `change_processor.rs` to find and update all files containing a given ref path prefix when a dedup rename occurs. + +--- + +## Integration Point 1: change_processor.rs (Dedup Cleanup) + +### Deletion Flow + +When instances are removed via two-way sync, `handle_tree_event` runs dedup cleanup **before** `apply_patch_set` removes instances from the tree: + +1. Build `removed_set: HashSet` from ALL removals in the batch +2. For each removed instance: + a. Get parent ref and filesystem name + b. Parse dedup suffix from stem + c. Enumerate remaining siblings (skipping co-removed via `removed_set`) + d. Group by extension (prevents cross-middleware false matches) + e. Call `compute_cleanup_action` +3. Execute cleanup action (rename file, suppress VFS, update ref paths) + +### VFS Suppression + +Dedup renames must suppress VFS events to prevent feedback loops: +- `suppress_path_any(&old_path)` -- old path may trigger REMOVE (Linux/Windows) or stale CREATE (macOS) +- `suppress_path(&new_path)` -- new path triggers Create/Write +- `unsuppress_path(&new_path)` -- rollback if rename fails + +**Critical:** New instance additions must NOT be suppressed. The VFS watcher needs to pick up new files for forward sync. + +### Ref Path Updates After Dedup Rename + +After successful dedup rename (e.g., `Foo~2` → `Foo`): +1. Compute old and new ref path segments +2. Call `update_ref_paths_after_rename(&old_prefix, &new_prefix)` +3. This function uses `RefPathIndex` to find affected files +4. Updates `Rojo_Ref_*` attribute values in each affected file +5. Updates the index: both path keys and filesystem paths +6. Suppresses VFS events for each file write + +### Ref Path Updates After User-Initiated Rename + +User-initiated renames (via the plugin's two-way sync) also update `Rojo_Ref_*` paths. The flow: +1. `old_ref_path` is captured BEFORE the rename (line ~1268, `ref_target_path_from_tree`) +2. Rename handler sets `new_ref_segment` to the new filesystem name (directory rename line ~1350, file rename line ~1512) +3. After rename handling, `update_ref_paths_after_rename(old_ref_path, new_ref_path)` is called (line ~1812) to update all meta/model files containing refs to the renamed instance + +This ensures both dedup-triggered and user-initiated renames maintain ref path integrity. + +### Metadata Consistency + +After dedup rename, the survivor's metadata must be updated: +- `InstigatingSource::Path` updated to new filesystem path +- Path indexes updated in-memory +- Suppression counters kept consistent with updated path + +--- + +## Integration Point 2: api.rs (Two-Way Sync Write) + +### added_paths (Same-Batch Ref Targets) + +When `/api/write` adds instances and sets Ref properties targeting them in the same batch, `added_paths` pre-computes filesystem paths: + +```rust +let added_paths: HashMap = request.added.iter() + .filter_map(|(guid, added)| { + let parent_path = ref_target_path_from_tree(&tree, added.parent?); + let fs_name = added_instance_fs_segment(added); // slug + extension + Some((*guid, format!("{}/{}", parent_path, fs_name))) + }) + .collect(); +``` + +`added_instance_fs_segment` mirrors `tentative_fs_name` logic but operates on `AddedInstance` fields (class_name, children, properties). + +### Ref-to-Attribute Conversion + +`syncback_updated_properties` converts `Variant::Ref` properties to `Rojo_Ref_*` attributes: + +1. Extract Ref properties from `changed_properties` BEFORE `filter_properties_for_meta` +2. For each Ref property: + - **Nil ref:** Mark `Rojo_Ref_PropertyName` for removal + - **Valid target in tree:** Compute path via `ref_target_path_from_tree`, add to `ref_attributes` + - **Target in `added_paths`:** Use pre-computed path + - **Target not found:** Log warning and skip +3. Merge `ref_attributes` into attributes map +4. Pass `remove_attributes` to `merge_or_build_meta` + +### merge_or_build_meta Attribute Lifecycle + +``` +1. Remove stale attributes (remove_attributes list) +2. Merge new attributes (including Rojo_Ref_* from ref properties) +3. Remove wins over add if same key in both lists? No -- adds happen AFTER removes, so add wins +``` + +### syncback_added_instance + +When adding instances, dedup is handled via `sibling_slugs: &mut HashSet`: +- Tracks filesystem names claimed in the current batch +- Prevents duplicate slug assignment when multiple instances added simultaneously +- Passed to `syncback_instance_to_path_with_stats` for dedup suffix assignment + +--- + +## Key Invariants + +1. **Round-trip identity:** syncback → build → compare = bit-identical trees +2. **CLI syncback parity:** two-way sync produces byte-identical filesystem output to CLI syncback +3. **Determinism:** same input always produces same filesystem output (sorted child ordering, deterministic suffix assignment) +4. **InstigatingSource consistency:** dedup'd paths reflected in `InstigatingSource::Path` +5. **Compound extensions:** suffix before compound extension (`Foo~2.server.luau`, NOT `Foo.server~2.luau`) +6. **Meta pairing:** `Foo~2.server.luau` pairs with `Foo~2.meta.json5` (strip script suffix, keep dedup suffix) +7. **Dedup key = full FS name:** collisions checked using complete filesystem name (slug + extension), lowercased +8. **Co-removed exclusion:** batch removal skips co-removed siblings when computing cleanup actions + +--- + +## Known Limitations (Accepted) + +### ProjectNode Duplicate-Named Children + +Duplicate-named children under ProjectNode are currently skipped during syncback with warnings (`src/snapshot_middleware/project.rs`). Project keys are unique by JSON definition, but filesystem children under project nodes could collide. Accepted for current release. + +### Greedy Matching Not Globally Optimal + +The greedy assignment may produce suboptimal pairing in rare ambiguous groups. A globally optimal algorithm (Hungarian, O(N^3)) would be better but is deferred since ambiguous groups are typically small. + +### ~~Matching Cache Not Implemented~~ (Resolved) + +`MatchingSession` caching is now implemented in all three matching implementations. See "Session Caching" in Subsystem 3. The per-parent invalidation cache (mapping `parent_ref → (children_list_hash, match_assignments)`) remains deferred. + +### ~~Stale Ref Paths After Rename~~ (Resolved) + +Both dedup-triggered renames and user-initiated renames now update `Rojo_Ref_*` paths automatically. See "Ref Path Updates" in the Integration Point 1 section for details. + +--- + +## Deferred Refactors + +- **Shared filesystem-segment derivation:** Consolidate tentative filesystem name computation into one shared helper used by `rojo_ref.rs`, `tree.rs`, `api.rs`, and `ref_properties.rs`. Currently duplicated across 4+ call sites. +- **Optimal assignment algorithm:** Replace greedy with Hungarian for large ambiguous groups. +- **Per-parent matching invalidation cache:** `MatchingSession` cost/match caching is implemented, but the per-parent cache with `children_list_hash` invalidation (avoids re-running matching when children haven't changed) remains deferred. +- **Duplicated name→index grouping:** All three matching implementations repeat the same HashMap pattern. Could be factored but types differ. + +--- + +## Testing Strategy + +### Test Locations + +- Rust unit tests: `#[cfg(test)]` blocks in source files +- Rust integration tests: `tests/tests/two_way_sync.rs`, `tests/tests/syncback_roundtrip.rs` +- Test fixtures: `rojo-test/syncback-tests/`, `rojo-test/build-tests/` +- Lua spec tests: `.spec.lua` files +- Snapshot tests: `cargo insta review` + +### Critical Test Scenarios + +**Dedup:** Same-named Folders, Folders with different children, Folder + Script same name (no dedup), Folder + init-style Script (dedup), natural tilde name, third duplicate added, suffix cleanup (group-to-1), gap tolerance, base-name promotion, idempotency, roundtrip. + +**Ref paths:** Basic ref survives syncback, ref to dedup'd instance, ref survives cleanup rename, ref survives base-name promotion, ref survives group-to-1, deeply nested through dedup layers, cross-file ref update, tilde-named instance ref, two-way sync ref update, forward sync manual edit. + +**Matching:** ClassName discriminator, property similarity scoring, matching stability across runs, preserves existing pairings, fast path verification, recursive depth limit, hash fast-path short circuit. + +**Dedup collision:** Folder + ModuleScript (no collision, different keys), Server + Client script (no collision), ModuleScript + ModuleScript (collision), slug collision ("Foo?" + "Foo*"), case-insensitive collision ("Foo" + "foo"). + +**Cross-system:** Full roundtrip with duplicates, two-way sync add/delete, forward sync edit dedup'd file, project file + filesystem duplicates, batch-remove 2 of 3 same-named siblings, same-batch add + ref. + +--- + +## Historical Context + +The dedup system was built across these plans (chronological): + +1. **nuke_path_encoding** -- Replaced `%NAME%` encoding with slugify + dedup + metadata `name` field foundation +2. **two-way_sync_refs** -- Added Ref property support to two-way sync with `Rojo_Ref_*` path-based attributes +3. **atlas_fork_isolation** -- Renamed binary/plugin for coexistence (preserved `Rojo_Ref_*` attribute names for backward compatibility) +4. **ambiguous_path_handling** -- Core feature: 3-pass matching algorithm, filesystem-collision dedup, filesystem-name ref paths, dedup suffix cleanup +5. **ref_path_and_dedup_integration** -- Migrated `collect_referents` to filesystem-name paths, integrated cleanup rules into change_processor +6. **rewrite_matching_algorithm** -- Replaced multi-pass with recursive change-count scoring + greedy assignment +7. **audit_ambiguous_path_handling** -- First audit: fixed forward sync re-matching, case sensitivity, ProjectNode paths, Lua require crash, dead code, ref format mismatch +8. **ambiguous_paths_audit_fixes** -- Second audit: recursive matching, `added_paths` format, dedup-aware ref collection, full-FS-name cleanup grouping, shared `trueEquals` +9. **audit_ambiguous_paths_followup** -- Third audit: metadata drift after cleanup rename, co-removed sibling exclusion, `added_paths` extension/suffix, attribute-scoped final paths +10. **audit_ref_two-way_sync** -- Ref audit: round-trip fidelity, path computation correctness, meta lifecycle, VFS suppression, wire format +11. **matching_session_caching** -- Session-scoped caching for all three matching implementations, `--!native`/`--!optimize 2` for Lua, valid children tracking +12. **luau-style_ref_paths** -- Relative ref paths (`@self`, `./`, `../`, `@game/`), source-aware placeholder system +13. **live_syncback** -- One-click full sync from Studio via `POST /api/syncback`, service property encoding, server restart diff --git a/.cursor/rules/atlas-plugin.mdc b/.cursor/rules/atlas-plugin.mdc new file mode 100644 index 00000000..ca131f9d --- /dev/null +++ b/.cursor/rules/atlas-plugin.mdc @@ -0,0 +1,373 @@ +--- +globs: plugin/** +alwaysApply: false +--- + +# Rojo Studio Plugin + +Built with Roact (React-like), handles WebSocket sync with server, patch application to Studio, and UI. + +**Entry:** `plugin/src/init.server.lua` | **Protocol:** WebSocket v6 | **Must match server MAJOR.MINOR** + +## Architecture + +### Key Components + +**1. UI Layer** (`App/`) +- **init.lua** - Root component, manages connection state, auto-reconnect, sync reminders, playtest server auto-connect. Also renders `SyncbackConfirm` floating dialog (StudioPluginGui with `isEphemeral=true`, not a StatusPage file). Contains `performSyncback()` and `SYNCBACK_SERVICES` constant. +- **Theme.lua** - Studio-matched color palette (auto light/dark) +- **Page.lua** - Page wrapper component +- **Components/** - Reusable UI components + - `PatchVisualizer/` - Tree view of changes with property diffs (`init.lua`, `DomLabel.lua`, `DisplayValue.lua`, `ChangeList.lua`) + - `StringDiffVisualizer/` - Side-by-side script diffs (`init.lua`, `StringDiff.lua`) + - `TableDiffVisualizer/` - Attributes and Tags diffs (`init.lua`, `Dictionary.lua`, `Array.lua`) + - `Notifications/` - Toast notifications (`init.lua`, `Notification.lua`, `FullscreenNotification.lua`) + - `Studio/` - Studio API wrappers (`StudioPluginAction`, `StudioPluginContext`, `StudioPluginGui`, `StudioToggleButton`, `StudioToolbarContext`, `StudioToolbar`) + - General: `Header`, `TextInput`, `TextButton`, `Checkbox`, `Dropdown`, `Tag`, `IconButton`, `Tooltip`, `TouchRipple`, `VirtualScroller`, `ScrollingFrame`, `Spinner`, `BorderedContainer`, `EditableImage`, `SlicedImage`, `ClassIcon` +- **StatusPages/** - Main UI pages + - `NotConnected.lua` - Connection form (host/port input), "Syncback" button (Danger style) + - `Connecting.lua` - Loading screen + - `Confirming.lua` - Patch confirmation dialog with push/pull/ignore per-node selections + - `Connected.lua` - Status display with patch history + - `Error.lua` - Error display + - `Settings/` - Settings UI (`init.lua`, `Setting.lua`) +- Uses Roact for declarative UI + +**2. Communication** (`ServeSession.lua`, `ApiContext.lua`) +- WebSocket connection to server via `/api/socket/:cursor` +- Handles patches, errors, reconnection +- Protocol version checking +- Confirmation flow with push/pull/ignore +- One-shot mode support + +**3. Reconciler** (`Reconciler/`) +- **applyPatch.lua** - Applies server patches to Studio tree +- **reify.lua** - Creates instances from snapshots (parent last) +- **hydrate.lua** - Updates InstanceMap with existing Studio instances +- **diff.lua** - Computes patch between virtual and Studio state (for two-way sync) +- **matching.lua** - Recursive change-count scoring algorithm for pairing virtual instances with Studio instances +- **trueEquals.lua** - Shared value equality (fuzzy floats, Color3, CFrame, Vector3; NaN handling; nil/null-ref equivalence) +- **decodeValue.lua** - Converts encoded values to Roblox types +- **setProperty.lua** - Sets properties safely (handles unscriptable) +- **getProperty.lua** - Gets properties with fallbacks +- **Error.lua** - Reconciler error handling + +**4. ChangeBatcher** (`ChangeBatcher/`) +- **init.lua** - Main batcher: accumulates changes every 200ms, pause/resume, deferred Ref resolution +- **createPatchSet.lua** - Converts pending changes to PatchSet +- **encodePatchUpdate.lua** - Encodes property updates including Ref properties via InstanceMap +- **encodeInstance.lua** - Full instance encoding (for additions/syncback); uses `encodeHelpers` for property iteration; includes duplicate detection helpers (`findDuplicateNames`, `hasDuplicateSiblings`, `isPathUnique`) +- **encodeHelpers.lua** - Shared property encoding helpers (`encodeAttributes`, `encodeTags`, `forEachEncodableProperty`). Used by `encodeInstance.lua` and `encodeService.lua`. Defines `SKIP_PROPERTIES` (Parent, Name, Archivable). +- **encodeService.lua** - Encodes service-level properties/attributes/refs for live syncback; uses ObjectValue carriers for Ref properties +- **encodeProperty.lua** - Single property value encoding via RbxDom +- **propertyFilter.lua** - Property type filtering (only `UniqueId` blocked; `Ref` is allowed) + +**5. State Management** +- **Settings.lua** - Persistent plugin settings (prefixed `Atlas_` in plugin store) +- **InstanceMap.lua** - Bidirectional Server ID <-> Studio Instance mapping with change detection +- **PatchSet.lua** / **PatchTree.lua** - Patch representation and visualization tree (PatchTree.build accepts optional `gitMetadata` for smart default selections) +- **Types.lua** - Type definitions and runtime validation +- **ChangeMetadata.lua** - Change metadata tracking + +**6. Utilities** +- **Timer.lua** - Performance timing +- **DiffUtil.lua** - Diff utilities +- **Version.lua** - Version parsing and comparison +- **Dictionary.lua** - Dictionary utilities +- **Assets.lua** - Asset references +- **SHA1.luau** - SHA1 hashing (used for git blob format hash comparison in PatchTree default selections) +- **XXH32.luau** - XXH32 hash function +- **soundPlayer.lua** - Sound effects +- **preloadAssets.lua** - Asset preloading +- **ignorePlaceIds.lua** - Place ID ignore list +- **timeUtil.lua** - Time formatting +- **strict.lua** - Strict module wrapper +- **invariant.lua** - Assertion utilities + +**7. Testing** +- Unit tests: `.spec.lua` files (TestEZ) throughout all modules +- Integration tests: `integration/` directory (`confirmBehavior`, `oneShotSync`, `syncflow`, `twoWaySync`) +- Stress tests: `.stress.spec.lua` files (applyPatch, diff, reify, encodeInstance, PatchSet) +- Chaos tests: `chaos/` directory (`stressPatterns.spec.lua`) +- Test utilities: `testUtils/` (`MockApiContext`, `PatchGenerator`, `LargeTreeGenerator`) +- Test runner: `runTests.lua` + +## Build Structure (`plugin.project.json`) + +The plugin is built from `plugin.project.json` at the repo root: + +```json +{ + "name": "Rojo", + "tree": { + "$className": "Folder", + "Plugin": { "$path": "plugin/src" }, + "Packages": { + "$path": "plugin/Packages", + "Log": { "$path": "plugin/log" }, + "Http": { "$path": "plugin/http" }, + "Fmt": { "$path": "plugin/fmt" }, + "RbxDom": { "$path": "plugin/rbx_dom_lua" } + }, + "Version": { "$path": "plugin/Version.txt" } + } +} +``` + +Vendored packages live under `plugin/Packages/`. Custom modules (`Log`, `Http`, `Fmt`, `RbxDom`) are injected into the Packages tree from their own directories at build time. + +## Key Configuration (`Config.lua`) + +```lua +{ + isDevBuild = false, -- Detected from ROJO_DEV_BUILD instance + codename = "Epiphany", + version = { major, minor, patch, metadata }, -- Parsed from Version.txt + expectedServerVersionString = "7.7 or newer", + protocolVersion = 6, -- MUST match server + defaultHost = "localhost", + defaultPort = "34873", + ignoredClassNames = { -- Auto-created instances to skip + TouchTransmitter = true, + Camera = true, + }, +} +``` + +**When changing protocol:** Bump `protocolVersion` in both `plugin/src/Config.lua` AND `src/web/interface.rs`. + +## Settings + +Persistent settings stored with `Atlas_` prefix in Studio plugin store: + +```lua +{ + openScriptsExternally = false, + twoWaySync = true, + oneShotSync = true, + autoReconnect = false, + showNotifications = false, + enableSyncFallback = false, + syncReminderMode = "Notify", -- "None" | "Notify" | "Fullscreen" + syncReminderPolling = true, + checkForUpdates = false, + checkForPrereleases = false, + autoConnectPlaytestServer = false, + confirmationBehavior = "Always", -- "Never" | "Initial" | "Always" | "Large Changes" | "Unlisted PlaceId" + largeChangesConfirmationThreshold = 5, + playSounds = true, + typecheckingEnabled = false, + logLevel = "Info", + timingLogsEnabled = false, + priorEndpoints = {}, +} +``` + +## WebSocket Protocol (v6) + +1. Connect to `ws://host:port/api/socket/:cursor` +2. Receive `ServerInfo` handshake (protocol version, project name, session ID) +3. Plugin sends: `{type = "Subscribe"}` +4. Server sends patches on filesystem changes (MessagePack encoded) + +**Patch structure:** +```lua +{ + type = "ApplyPatch", + sessionId = "uuid", + patch = { + removed = {"id-1", "id-2"}, + added = { + ["id-3"] = { + id = "id-3", className = "Folder", name = "New", + properties = {}, metadata = { ignoreUnknownInstances = false }, + children = {"child-1"}, + }, + }, + updated = { + { id = "id-4", changedName = "Renamed", changedProperties = { Source = "code" } }, + }, + } +} +``` + +## ServeSession (`ServeSession.lua`) + +Manages the lifecycle of a connection to the server. + +**Status flow:** `NotStarted` -> `Connecting` -> `Connected` -> `Disconnected` + +**Key methods:** +- `new(options)` - Creates session with `ApiContext` and `twoWaySync` flag +- `start()` - Connects, computes initial patch, shows confirmation +- `stop()` - Disconnects and cleans up +- `__onWebSocketMessage()` - Handles incoming patches +- `__applyPatch()` - Applies patches via Reconciler +- `__confirmAndApplyInitialPatch()` - Handles user confirmation with push/pull/ignore +- `__computeInitialPatch()` - Computes catch-up patch +- `hookPrecommit()` / `hookPostcommit()` - Lifecycle hooks + +**Confirmation flow:** + +1. Receive patch (initial or streaming) +2. Check `confirmationBehavior` setting +3. If confirmation needed: pause ChangeBatcher, build PatchTree with git-based default selections (from `serverInfo.gitMetadata`), show Confirming page with PatchVisualizer +4. User reviews defaults and selects push/pull/ignore per node +5. Build `stageIds` list (all push-accepted + auto-selected pull-accepted), send with write request +6. Apply via Reconciler, resume ChangeBatcher + +**One-shot mode (`oneShotSync: true`):** +- ChangeBatcher still collects changes (for diff visualization) +- Automatic outgoing writes are blocked +- Only explicit "pull" selections in confirmation dialog can write +- Disconnects after initial sync completes + +## Reconciler (`Reconciler/`) + +**applyPatch.lua** - Main patch application: +1. Remove instances in `removed` +2. Create instances in `added` via `reify` +3. Update instances in `updated` via `hydrate` +4. Handle deferred Ref properties +5. Collect errors, return unapplied patch + +**reify.lua** - Create instances (parent last - Roblox best practice) +**hydrate.lua** - Update InstanceMap with existing instances (uses matching algorithm) +**diff.lua** - Compute patch between virtual instances and Studio state +**setProperty.lua** / **getProperty.lua** - Handle unscriptable properties + +### Matching Algorithm (`matching.lua`) + +Pairs incoming virtual instances with existing Studio instances during hydration. + +**Entry point:** `Matching.matchChildren(virtualChildren, studioChildren, virtualInstances)` + +**Algorithm:** +1. **Fast path:** Group by (Name, ClassName). 1:1 groups get instant-matched with zero scoring. +2. **Ambiguous groups:** For groups with multiple candidates, compute pairwise `computePairCost()` (recursive, scores entire subtree). Greedy assignment sorted by cost ascending, child-order tiebreaker for stability. +3. **Leftovers:** Unmatched virtual = new instances, unmatched studio = deleted. + +**Performance:** File-level `--!native` and `--!optimize 2` directives for native codegen. `MatchingSession` caches results across recursive calls: +```lua +type MatchingSession = { + matchCache: { [string]: { [Instance]: MatchResult } }, + costCache: { [string]: { [Instance]: number } }, +} +``` + +**Valid children tracking:** `cacheVirtual()` filters children to only those existing in `virtualInstances`, preventing crashes on incomplete virtual instance trees. + +**Sort stability note:** `table.sort` is NOT stable in Luau. Comparator breaks ties by insertion index to ensure deterministic ordering. + +**Value equality:** Shared `trueEquals.lua` module used by both `matching.lua` and `diff.lua` -- fuzzy equality for floats (epsilon 0.0001, absolute + relative), Color3 via RGB ints, CFrame/Vector3 component-wise, NaN handling, nil/null-ref equivalence, EnumItem/number interop. + +## InstanceMap (`InstanceMap.lua`) + +Bidirectional Server ID <-> Studio Instance mapping with integrated change detection: + +```lua +InstanceMap = { + fromIds = {}, -- Server ID -> Studio Instance + fromInstances = {}, -- Studio Instance -> Server ID + pausedUpdateInstances = {}, -- Set of instances to ignore changes from + instancesToSignal = {}, -- Instance -> RBXScriptConnection(s) + onInstanceChanged = fn, -- Callback: (instance, propertyName) -> void +} +``` + +**Key methods:** +- `insert(id, instance)` - Add mapping and connect change signals +- `removeId(id)` / `removeInstance(instance)` - Remove mapping and disconnect signals +- `destroyInstance(instance)` - Parent to nil (preserves undo history) +- `fromId(id)` / `fromInstance(instance)` - Look up mapping +- `pauseInstance(instance)` / `unpauseAllInstances()` - Pause/resume change detection + +**Signal connection (`__connectSignals`):** +- Regular instances: `instance.Changed` event (fires for any property) +- ValueBase instances: Explicit `GetPropertyChangedSignal` for Name, Value, Parent (ValueBase.Changed only fires for Value) + +**Change guards (`__maybeFireInstanceChanged`):** +- Skipped if instance is in `pausedUpdateInstances` +- Skipped if `onInstanceChanged` callback is nil +- Skipped if `RunService:IsRunning()` (Play mode) + +**Pause mechanism:** During incoming patch application, instances are paused to prevent their changes from being sent back to the server (feedback loop prevention). `unpauseAllInstances()` is called each frame in `ChangeBatcher:__cycle()`. + +## Two-Way Sync + +Watches Studio changes and sends them to the server in real-time. Enabled by default (`twoWaySync: true`). + +**Data flow:** Studio property change -> InstanceMap signal -> ServeSession callback -> ChangeBatcher -> 200ms batch flush -> `POST /api/write` (MessagePack) -> Server writes to filesystem + +**ChangeBatcher state:** +```lua +{ + __accumulator = 0, -- Time since last flush + __pendingPropertyChanges = {}, -- Instance -> { PropertyName -> true } + __pendingUnresolvedRefs = {}, -- targetInstance -> [{sourceInstance, propertyName}] + __syncSourceOnly = false, -- Only sync Source property + __paused = false, -- Pause during confirmation + __renderSteppedConnection = conn, -- Frame loop connection + __instanceMap = instanceMap, + __onChangesFlushed = callback, +} +``` + +**Deferred Ref resolution:** When a Ref property targets an instance not yet in the InstanceMap, the property is deferred in `__pendingUnresolvedRefs`. When the target appears via `onInstanceInserted`, the property change is re-queued for the next batch. + +**Guards:** +- `oneShotSync` setting blocks automatic outgoing writes +- `__paused` flag blocks flushes during confirmation dialog +- `RunService:IsRunning()` blocks during Play mode + +## Live Syncback + +One-click full sync from Studio to filesystem. Available from the **NotConnected page** -- does not require an active WebSocket session. Sends directly to the server via HTTP POST using the host/port fields from the connection form. + +**`SYNCBACK_SERVICES` constant (14 services):** Lighting, MaterialService, ReplicatedFirst, ReplicatedStorage, ServerScriptService, ServerStorage, SoundService, StarterGui, StarterPack, StarterPlayer, Teams, TextChatService, VoiceChatService, Workspace. + +**Flow:** +1. User clicks "Syncback" button on NotConnected page → `SyncbackConfirm` floating dialog appears +2. Dialog warns: "This will overwrite your project files with the current Studio state. This cannot be undone." +3. User confirms → `performSyncback()`: + a. Iterates `SYNCBACK_SERVICES`, calls `encodeService(service)` for each + b. Collects all children instances and ObjectValue Ref carriers + c. Serializes all children via `SerializationService:SerializeInstancesAsync()` into a single buffer + d. Destroys temporary ObjectValue carriers + e. POSTs to `http://host:port/api/syncback` (MessagePack): `{protocolVersion, serverVersion, placeId, data, services}` +4. Shows success/error notification + +**`encodeService(service)`** (`ChangeBatcher/encodeService.lua`): +- Encodes service-level properties via `RbxDom.findClassDescriptor()` +- Skips: `Parent`, `Name`, `Archivable`, `Attributes`, `Tags` (handled separately), class-specific excludes (`Lighting.ClockTime`) +- Ref properties: creates temporary `ObjectValue` carriers (Name = property name, Value = target instance), stores 1-based index in `refs` map +- Returns `(chunk, refTargets)` where `chunk = {className, childCount, refTargetCount, properties?, refs?}` + +## Dependencies + +### Vendored Packages (`plugin/Packages/`) + +Roact, Flipper, Promise, TestEZ, t, Highlighter, msgpack-luau, json5. **Don't edit vendored packages.** + +### Custom Modules (injected via `plugin.project.json`) + +| Module | Path | Purpose | +|--------|------|---------| +| Log | `plugin/log` | Structured logging with configurable levels | +| Http | `plugin/http` | HTTP client utilities | +| Fmt | `plugin/fmt` | String formatting with debug output | +| RbxDom | `plugin/rbx_dom_lua` | Reflection database, property descriptors, encoded value encode/decode | + +## Common Issues + +**Plugin not reloading:** Enable Studio setting "Reload plugins on file change" +**Connection fails:** Check server running, host/port match, protocol versions match +**Properties not syncing:** Verify scriptable in rbx_reflection_database +**Duplicates:** InstanceMap corruption - disconnect, restart Studio +**Two-way sync feedback loops:** Check pause mechanism is working; instances must be paused during reconciliation + +## Debugging + +Settings -> Log Level -> "Trace" for verbose output. `ROJO_DEV_BUILD` instance enables auto-tests via TestEZ. + +**Timing:** Enable `timingLogsEnabled` setting to see `Timer.start`/`Timer.stop` measurements for Reconciler and other hot paths. diff --git a/.cursor/rules/atlas-project.mdc b/.cursor/rules/atlas-project.mdc index 1b1c20cf..6cf66c8c 100644 --- a/.cursor/rules/atlas-project.mdc +++ b/.cursor/rules/atlas-project.mdc @@ -1,12 +1,5 @@ --- -description: "Project file format specification: .project.json5 schema, file extensions, middleware types, init files, sync rules, and property reference." -globs: - - "**/*.project.json5" - - "**/*.project.json" - - "**/*.meta.json5" - - "**/*.meta.json" - - "**/*.model.json5" - - "**/*.model.json" +alwaysApply: true --- # Atlas Project File Format @@ -198,7 +191,7 @@ Configure `syncbackRules` object for Roblox → filesystem sync: | `ignoreProperties` | object | `{}` | Properties to skip per class: `{"ClassName": ["Prop1", "Prop2"]}` | | `ignoreClasses` | string[] | `[]` | Instance classes to skip entirely | | `syncCurrentCamera` | bool | `false` | Whether to sync `Workspace.CurrentCamera` | -| `syncUnscriptable` | bool | `true` | Whether to sync unscriptable properties | +| `syncUnscriptable` | bool | `false` | Whether to sync unscriptable properties | | `ignoreReferents` | bool | `false` | Whether to skip Ref properties (e.g., `Model.PrimaryPart`) | | `createIgnoreDirPaths` | bool | `false` | Whether `/**` in ignore paths also matches the directory itself | | `ignoreHiddenServices` | bool | — | Override root-level `ignoreHiddenServices` for syncback | @@ -215,7 +208,7 @@ Configure `syncbackRules` object for Roblox → filesystem sync: }, "ignoreClasses": ["Terrain", "Camera"], "syncCurrentCamera": false, - "syncUnscriptable": true, + "syncUnscriptable": false, "ignoreReferents": false, "createIgnoreDirPaths": false, "warnDuplicateNames": false @@ -254,7 +247,9 @@ Add metadata to file-based instances. Meta files use **camelCase keys** (no `$` **Key differences from ProjectNode:** Meta uses `className`, `properties`, `attributes` (camelCase, no `$` prefix). Adjacent meta does NOT support `className`. -**`name` field (slugify support):** When an instance name contains forbidden filesystem characters (`: / ? * < > | " \`), syncback slugifies the filename (forbidden chars → `_`) and stores the real name in the `name` field. On forward sync, `name` overrides the filename-derived instance name. On collisions (e.g. `A/B` and `A_B` both slugify to `A_B`), `~1`, `~2` suffixes are appended and meta `name` fields distinguish them. +**`name` field (slugify support):** When an instance name contains forbidden filesystem characters (`: / ? * < > | " \`), syncback slugifies the filename (forbidden chars → `_`) and stores the real name in the `name` field. On forward sync, `name` overrides the filename-derived instance name. On collisions (e.g. `A/B` and `A_B` both slugify to `A_B`), `~2`, `~3` suffixes are appended and meta `name` fields distinguish them. + +**`Rojo_Ref_*` attributes:** Meta files may also contain `Rojo_Ref_PropertyName` attributes written by syncback/two-way sync for Ref property linking. These are managed automatically and should not be hand-edited. **Meta files CAN'T:** Set `$path`, have children. @@ -331,6 +326,33 @@ Same format as properties. Limitations: no Ref, no NumberSequence/ColorSequence, ## Ref Property Linking +Two systems for linking instance references, both supported: + +### Path-Based: `Rojo_Ref_*` (preferred, written by syncback/two-way sync) + +Uses filesystem-name paths (slugified, dedup'd) to reference target instances: + +```json5 +// In meta file for an instance with PrimaryPart pointing to Handle +{ + "attributes": { + "Rojo_Ref_PrimaryPart": "Workspace/MyModel/Handle.model.json5" + } +} +``` + +**Attribute format:** `Rojo_Ref_ = ""` + +Syncback writes **Luau-style relative paths**: +- `@self` / `@self/Child.luau` -- target is same instance or a descendant +- `./Sibling.luau` -- target is a same-parent sibling +- `../Uncle.luau` -- target is a parent's sibling +- `@game/Service/Path` -- cross-service reference + +Forward sync resolves both relative and absolute formats via `resolve_ref_path_to_absolute()` + `get_instance_by_path()`. Path segments use full filesystem names (slug + extension + dedup suffix), matched case-insensitive with instance name fallback. + +### ID-Based: `Rojo_Target_*` + `$id` (legacy, still read by forward sync) + Link instances via `$id` and special attributes: ```json5 @@ -347,6 +369,8 @@ Link instances via `$id` and special attributes: **Attribute format:** `Rojo_Target_` +**Priority:** When both `Rojo_Ref_*` and `Rojo_Target_*` exist for the same property, `Rojo_Ref_*` takes precedence. + ## Property Types `String`, `Bool`, `Int32`, `Int64`, `Float32`, `Float64`, `Vector2`, `Vector3`, `Vector2int16`, `Vector3int16`, `UDim`, `UDim2`, `CFrame`, `Color3`, `Color3uint8`, `BrickColor`, `NumberRange`, `Rect`, `NumberSequence`, `ColorSequence`, `Enum`, `Ref`, `Content`, `Font`, `PhysicalProperties`, `Axes`, `Faces`, `Tags`, `Attributes` diff --git a/.cursor/rules/rbx-dom.mdc b/.cursor/rules/atlas-rbx-dom.mdc similarity index 96% rename from .cursor/rules/rbx-dom.mdc rename to .cursor/rules/atlas-rbx-dom.mdc index a807d4b8..5ec16ca5 100644 --- a/.cursor/rules/rbx-dom.mdc +++ b/.cursor/rules/atlas-rbx-dom.mdc @@ -1,9 +1,6 @@ --- -description: "rbx-dom submodule: separate Git repo for Roblox format libraries. Has own CI/CD and release cycle." -globs: - - "rbx-dom/**/*.rs" - - "rbx-dom/**/*.lua" - - "rbx-dom/**/*.toml" +globs: rbx-dom/** +alwaysApply: false --- # rbx-dom Submodule diff --git a/.cursor/rules/src.mdc b/.cursor/rules/atlas-src.mdc similarity index 75% rename from .cursor/rules/src.mdc rename to .cursor/rules/atlas-src.mdc index 074e98a0..a914ecf5 100644 --- a/.cursor/rules/src.mdc +++ b/.cursor/rules/atlas-src.mdc @@ -1,8 +1,6 @@ --- -description: "Rust server/CLI implementation: snapshot system, middleware, commands, syncback, and web API architecture." -globs: - - "src/**/*.rs" - - "crates/**/*.rs" +globs: src/**,crates/** +alwaysApply: false --- # Rojo Server/CLI (Rust) @@ -19,8 +17,9 @@ Pure functions: filesystem → instance tree → minimal patches → apply chang **Key files:** - `mod.rs` - System overview with detailed explanation -- `patch_compute.rs` - Generate diffs -- `patch_apply.rs` - Apply changes +- `patch_compute.rs` - Generate diffs (uses matching algorithm) +- `patch_apply.rs` - Apply changes (resolves `Rojo_Ref_*` and `Rojo_Target_*` refs) +- `matching.rs` - Forward sync matching algorithm (recursive change-count scoring, `MatchingSession` caching) - `instance_snapshot.rs` - Core snapshot type ```rust @@ -73,14 +72,14 @@ pub struct InstanceContext { **Filename handling (syncback/two-way sync):** -Instance names with forbidden filesystem chars (`<>:"/\|?*`) are handled by: +Instance names with forbidden filesystem chars (`<>:"/\|?*~`) are handled by: 1. `slugify_name()` -- replaces forbidden chars with `_` (pure, stateless) -2. `deduplicate_name()` -- appends `~1`, `~2`, etc. on collision (stateful) +2. `deduplicate_name_with_ext()` -- dedup key = full filesystem name (slug + extension), case-insensitive; appends `~2`, `~3`, etc. on collision (stateful) 3. Metadata `name` field -- stores the real instance name in `.meta.json5`/`.model.json5` when the slug differs ### 3. Commands (`src/cli/`) -- `serve.rs` - Live sync with WebSocket +- `serve.rs` - Live sync with WebSocket; live syncback via `run_live_syncback()` + `build_dom_from_chunks()` (server restarts after syncback) - `build.rs` - Generate rbxl/rbxm/rbxmx/rbxlx - `upload.rs` - Upload to Roblox.com - `sourcemap.rs` - Generate sourcemaps @@ -93,13 +92,19 @@ Reverse: Roblox file → filesystem - `snapshot.rs` - Convert instances to snapshots - `fs_snapshot.rs` - Filesystem representation +- `matching.rs` - Syncback matching algorithm (recursive change-count scoring, `MatchingSession` caching) +- `dedup_suffix.rs` - Dedup cleanup rules (gap-tolerant, base-name promotion, group-to-1) +- `meta.rs` - Meta file read/write helpers (upsert/remove name, update ref paths) +- `ref_properties.rs` - Ref property path linking (`Rojo_Ref_*`, tentative_fs_path, placeholder system) - `hash/` - Content hashing for change detection -- `property_filter.rs` - Apply syncback rules +- `fs_snapshot.rs` - Filesystem representation, parallel writes (`write_to_vfs_parallel`), ref path post-processing (`fix_ref_paths`) +- `property_filter.rs` - Apply syncback rules (default: `sync_unscriptable = false`) ### 5. Web Server (`src/web/`) -- `api.rs` - HTTP/WebSocket endpoints -- `interface.rs` - Browser UI +- `api.rs` - HTTP/WebSocket endpoints, including `POST /api/syncback` handler (`handle_api_syncback`) +- `interface.rs` - Wire types: patches, `SyncbackRequest`, `ServiceChunk`, `SyncbackPayload`, `ServerExitReason` +- `mod.rs` - `SyncbackSignal` (Mutex + Notify pattern), `LiveServer` (uses `tokio::select!` to wait for connections or syncback signal) - Protocol version enforcement ### 6. Project System (`src/project.rs`) @@ -120,6 +125,8 @@ Parse/validate `.project.json5` (or legacy `.project.json`), handle nested proje **Key:** `src/serve_session.rs` manages active sessions +**Live syncback integration:** The serve loop checks for `ServerExitReason::SyncbackRequested(payload)`. When received, calls `run_live_syncback()` which reconstructs the DOM from the plugin's rbxm blob + service chunks via `build_dom_from_chunks()`, runs `syncback_loop(incremental=false)` (clean mode), writes files via `write_to_vfs_parallel()`, refreshes git index, then restarts the serve loop. + ### `rojo build` 1. Load project @@ -146,7 +153,7 @@ pub struct SyncbackRules { ignore_properties: IndexMap>, // Class → properties ignore_classes: Vec, // Classes to skip entirely sync_current_camera: Option, // Default: false - sync_unscriptable: Option, // Default: true + sync_unscriptable: Option, // Default: false ignore_referents: Option, // Skip Ref properties, default: false } ``` @@ -217,13 +224,14 @@ Review: `cargo insta review` **HTTP:** - `GET /` - Browser UI - `GET /api/rojo` - Server info (version, project name, protocol) +- `POST /api/syncback` - Live syncback: full Roblox → filesystem sync (validates protocol, version, place ID) **WebSocket:** `ws://host:port/api/rojo` -**Protocol v5:** +**Protocol v6:** ```json // Handshake -{"type": "ServerInfo", "protocolVersion": 5, "serverVersion": "7.7.0"} +{"type": "ServerInfo", "protocolVersion": 6, "serverVersion": "7.7.0"} // Patch { @@ -309,6 +317,7 @@ to_writer(&mut buffer, &dom, dom.root())?; **Core:** rbx_binary, rbx_xml, rbx_dom_weak, memofs (local crate) **Async:** tokio, hyper, hyper-tungstenite **Serialization:** serde, json5, toml, yaml-rust2, csv +**Float precision:** lexical-write-float (for roundtrip-safe f32/f64 formatting in JSON) **Utilities:** clap, anyhow, thiserror, crossbeam-channel, rayon, globset ## Module Organization @@ -367,13 +376,22 @@ RUST_LOG=rojo=debug rojo build ``` src/ - cli/ - Commands - snapshot/ - Core system - snapshot_middleware/ - File handlers - syncback/ - Reverse sync - web/ - HTTP/WebSocket - lib.rs - Public API - main.rs - CLI entry + cli/ - Commands + snapshot/ - Core system + matching.rs - Forward sync matching algorithm + snapshot_middleware/ - File handlers + syncback/ - Reverse sync + matching.rs - Syncback matching algorithm + dedup_suffix.rs - Dedup cleanup rules + meta.rs - Meta file helpers + web/ - HTTP/WebSocket (incl. SyncbackSignal, LiveServer) + interface.rs - Wire types (patches, syncback requests) + change_processor.rs - Two-way sync filesystem writer + git.rs - Git integration (changed files, SHA1 hashing, auto-staging) + rojo_ref.rs - Ref path system (Rojo_Ref_*, RefPathIndex, Luau-style relative paths) + variant_eq.rs - Property value comparison + lib.rs - Public API + main.rs - CLI entry crates/ memofs/ - Virtual filesystem diff --git a/.cursor/rules/atlas-syncback.mdc b/.cursor/rules/atlas-syncback.mdc index d0da6c07..43607d93 100644 --- a/.cursor/rules/atlas-syncback.mdc +++ b/.cursor/rules/atlas-syncback.mdc @@ -1,8 +1,6 @@ --- -description: "Syncback architecture: Roblox→filesystem synchronization, incremental vs clean modes, hash-based change detection, middleware selection, and property filtering." -globs: - - "src/syncback/**/*.rs" - - "src/cli/syncback.rs" +globs: src/syncback/**,src/cli/syncback.rs +alwaysApply: false --- # Atlas Syncback Architecture @@ -48,15 +46,17 @@ Filesystem (Lua files, models, directories) ``` syncback/ - mod.rs - Main loop, orchestration - snapshot.rs - Convert instances → snapshots - fs_snapshot.rs - Filesystem representation - file_names.rs - Generate valid filenames - meta.rs - Meta file read/write helpers + mod.rs - Main loop, orchestration, ref path post-processing + snapshot.rs - Convert instances → snapshots, record_ref_path() + fs_snapshot.rs - Filesystem representation, fix_ref_paths() + file_names.rs - slugify_name(), deduplicate_name_with_ext(), name_for_inst() + matching.rs - Recursive change-count matching algorithm + dedup_suffix.rs - Dedup cleanup rules (gap-tolerant, base-name promotion, group-to-1) + meta.rs - Meta file read/write helpers (upsert/remove name, update ref paths) property_filter.rs - Filter properties per syncback rules - ref_properties.rs - Handle Ref properties (cross-references) + ref_properties.rs - Ref property path linking (Rojo_Ref_*, tentative_fs_path) hash/ - Content hashing for change detection - mod.rs - Hash variants + mod.rs - Hash tree computation variant.rs - Variant-specific hashing stats.rs - Statistics tracking ``` @@ -66,13 +66,16 @@ syncback/ **Flow:** 1. **Parse input file** - Read rbxl/rbxm with `rbx_binary` -2. **Prune tree** - Remove instances not in original project -3. **Filter hidden services** - Remove internal Roblox services (AdService, Analytics, etc.) -4. **Collect pre-prune paths** - Track all instances before pruning (for external refs) -5. **Hash comparison** - Compare hashes of old tree vs new tree -6. **Recursive syncback** - Walk trees, matching instances to files -7. **Write filesystem** - Create/update files based on changes -8. **Link Ref properties** - Resolve cross-references between instances +2. **Collect pre-prune paths** - `collect_all_paths()` maps all Refs to filesystem-name paths before pruning (for external refs) +3. **Prune tree** - Remove instances not in original project +4. **Filter hidden services** - Remove internal Roblox services (AdService, Analytics, etc.) +5. **Collect referents** - `collect_referents()` finds Ref properties and writes `Rojo_Ref_*` attributes using tentative paths +6. **Filter properties** - Apply syncback rules (ignore classes, properties, etc.) +7. **Hash comparison** - Compare hashes of old tree vs new tree +8. **Recursive syncback walk** - Match instances to files via matching algorithm, assign filenames with dedup +9. **Record ref path map** - During the walk, `record_ref_path()` maps each Ref to its final dedup'd filesystem path +10. **Post-process ref paths** - `fix_ref_paths()` compares tentative paths vs final paths, substitutes in written meta/model files +11. **Write filesystem** - Apply FsSnapshot to disk (sequential or parallel) **Signature:** ```rust @@ -111,6 +114,67 @@ pub fn syncback_loop( **Implementation detail:** `old_ref` is ignored in `SyncbackSnapshot::with_joined_path` when `incremental=false`. +### Live Syncback Mode (via `atlas serve`) + +Triggered via `POST /api/syncback` during an active `atlas serve` session. The plugin encodes the Studio state (service properties + children as rbxm blob) and POSTs directly -- no WebSocket connection required. + +**Server-side flow:** +1. `handle_api_syncback()` validates protocol, version, place ID +2. Fires `SyncbackSignal` → `LiveServer` exits accept loop +3. `run_live_syncback()` in `src/cli/serve.rs`: + a. `build_dom_from_chunks()` reconstructs DOM from rbxm blob + `ServiceChunk` metadata + b. Runs `syncback_loop(incremental=false)` -- always clean mode + c. `write_to_vfs_parallel()` writes files with Windows retry logic + d. `refresh_git_index()` updates git +4. Serve loop restarts (new session) + +**Key differences from CLI syncback:** +- Runs during an active serve session (not standalone) +- Receives data via HTTP (not file-based) +- Always clean mode (replaces everything) +- Server restarts after completion + +**Plugin-side encoding:** `encodeService()` encodes service properties/attributes/refs (ObjectValue carriers for Ref), `SerializationService:SerializeInstancesAsync()` serializes all children into a single rbxm buffer. + +See `atlas-api.mdc` for wire format details (`SyncbackRequest`, `ServiceChunk`). + +## Matching Algorithm (`matching.rs`) + +Pairs new instances (from Roblox file) with old instances (from filesystem) to determine which files to update vs create. + +**Entry point:** `match_children(new_children, old_children, new_dom, old_dom, hashes)` + +**Algorithm:** +1. **Fast path:** Group by (Name, ClassName). 1:1 groups get instant-matched with zero scoring. +2. **Ambiguous groups:** For groups with multiple candidates on either side, compute pairwise `compute_change_count()` -- a recursive scoring function that counts property diffs + recursive children cost. Greedy assignment sorted by cost ascending, child-order tiebreaker for stability. +3. **Hash fast-path:** If `new_hash == old_hash`, return 0 cost immediately (skips property comparison and recursion). + +**Constants:** +- `UNMATCHED_PENALTY = 10,000` - Cost per unmatched child (create/delete is categorically more expensive than property tweaks) +- `MAX_SCORING_DEPTH = 3` - Depth limit for recursive scoring to prevent O(n^k) explosion + +**`compute_change_count()` and `match_children()` are mutually recursive.** Base case: leaf nodes (no children). + +**Output:** `MatchResult { matched: Vec<(new_ref, old_ref)>, unmatched_new: Vec, unmatched_old: Vec }` + +**Session caching:** `MatchingSession` wraps a `cost_cache: RefCell>` to cache `compute_change_count` results, avoiding redundant recursive scoring when the same pairs are encountered at different tree levels. + +**Known limitation:** Greedy assignment is not globally optimal. The Hungarian algorithm (O(N^3)) would give optimal results, but ambiguous groups are typically small (2-10 instances). + +## Dedup Suffix Management (`dedup_suffix.rs`) + +Manages `~N` suffix cleanup after instance deletions during two-way sync. + +**Key functions:** +- `parse_dedup_suffix(stem) -> Option<(base, N)>` - Parse `~N` from filename stem +- `build_dedup_name(base, suffix, ext)` - Build filename with optional `~N` suffix +- `compute_cleanup_action(removed_stem, remaining_stems) -> DedupCleanupAction` - Determine what to do after a deletion + +**Cleanup rules:** +- **Gap-tolerant:** Deleting `~2` from `{Foo, Foo~2, Foo~3}` leaves `{Foo, Foo~3}`. No renumbering. +- **Base-name promotion:** Deleting `Foo` (base) from `{Foo, Foo~2, Foo~3}` promotes `Foo~2` to `Foo`. `Foo~3` stays. +- **Group-to-1 cleanup:** If group shrinks to 1 member, remove its suffix entirely. + ## Syncback Rules Configured in `syncbackRules` section of project file: @@ -133,7 +197,7 @@ pub struct SyncbackRules { /// Whether to sync Workspace.CurrentCamera (default: false) pub sync_current_camera: Option, - /// Whether to sync unscriptable properties (default: true) + /// Whether to sync unscriptable properties (default: false) pub sync_unscriptable: Option, /// Whether to skip Ref properties (default: false) @@ -246,12 +310,13 @@ snapshot.with_new_path(path, new_ref, old_ref) ``` **Filename generation** (`file_names.rs`): -- Slugifies instance name (`slugify_name()`: forbidden chars → `_`) -- Deduplicates against taken names (`deduplicate_name()`: appends `~1`, `~2`, etc.) -- Adds appropriate extension based on middleware -- Stores real name in `.meta.json5` `name` field when slug differs +- `slugify_name()`: forbidden chars (`<>:"/\|?*~`) → `_`. Also strips leading/trailing spaces/dots, neutralizes dangerous suffixes (`.server`, `.meta`, etc.), handles Windows reserved names (`CON`, `PRN`, etc.) +- `deduplicate_name_with_ext()`: Dedup key = full filesystem name (slug + extension), case-insensitive. `Folder "Foo"` (key `"foo"`) and `Script "Foo"` (key `"foo.server.luau"`) do NOT collide. Appends `~2`, `~3`, etc. on collision, with suffix inserted before compound extension: `Foo~2.server.luau` +- `name_for_inst()` returns `(filename, needs_meta_name, dedup_key)`. `needs_meta_name` is true when the filesystem name differs from the real instance name. +- `~` is in `SLUGIFY_CHARS` so natural tilde names (e.g., `"Foo~1"`) get slugified to `"Foo_1"`, avoiding confusion with dedup suffixes. - Example: `Script` named "My Script" → `My Script.server.luau` - Example: `Script` named "Hey/Bro" → `Hey_Bro.server.luau` + meta with `name: "Hey/Bro"` +- Example: Two `Folder "Data"` → `Data/` + `Data~2/` (meta: `name: "Data"`) ### Property Filtering @@ -301,6 +366,10 @@ fs_snapshot.merge(other) // Combine snapshots // Apply to filesystem fs_snapshot.write_to_vfs(base_path, vfs)?; +fs_snapshot.write_to_vfs_parallel(base_path, vfs)?; // Parallel writes (see below) + +// Post-process ref paths +fs_snapshot.fix_ref_paths(&substitutions); // Replace placeholder paths with final relative paths // Query fs_snapshot.is_empty() @@ -310,26 +379,57 @@ fs_snapshot.removed_paths() ## Ref Properties (`ref_properties.rs`) -**Challenge:** Ref properties (like `ObjectValue.Value`) point to other instances by Ref, but Refs are different between old and new trees. +**Challenge:** Ref properties (like `ObjectValue.Value`, `Model.PrimaryPart`) point to other instances by Ref, but Refs are different between old and new trees. -**Solution:** +**Solution: Path-based `Rojo_Ref_*` attributes (preferred)** -1. **Collect paths** - Map every Ref → full instance path (e.g., `Workspace.Part1`) -2. **Collect referents** - Find all properties that are Refs, store as (source_path, target_path) -3. **Link referents** - After syncback, use Atlas's `$id` / `Rojo_Target_PropertyName` system to link instances +Syncback writes **Luau-style relative paths** as `Rojo_Ref_PropertyName` attributes via `compute_relative_ref_path()`: -**Atlas linking system:** -```json +```json5 { - "id": "some-unique-id", "properties": {}, "attributes": { - "Rojo_Target_Value": "some-unique-id" + "Rojo_Ref_PrimaryPart": "./Handle.model.json5" } } ``` -Plugin reads `Rojo_Target_PropertyName` attributes and sets corresponding Ref properties. +Relative path prefixes: `@self` (same/descendant), `./` (sibling), `../` (parent's sibling), `@game/` (cross-service). + +Forward sync resolves both relative and absolute formats via `resolve_ref_path_to_absolute()` + `get_instance_by_path()`. + +**Placeholder system:** During the syncback walk, before dedup suffixes are finalized, ref paths are written as `__ROJO_REF__TO___` placeholders (encoding both source and target refs). After the walk assigns final filesystem names, `fix_ref_paths()` substitutes placeholders with actual relative paths. Substitutions are sorted by length descending to prevent partial prefix matches. + +**Legacy: `Rojo_Target_*` / `$id` system (still supported for reading)** + +```json5 +{ + "id": "some-unique-id", + "attributes": { "Rojo_Target_Value": "some-unique-id" } +} +``` + +Forward sync also reads `Rojo_Target_*` attributes and resolves via `get_specified_id()`. Priority: `Rojo_Ref_*` takes precedence over `Rojo_Target_*`. + +**Key functions:** +- `collect_all_paths(dom)` - Maps all Refs to filesystem-name-compatible paths (called pre-prune for external refs) +- `collect_referents(dom, paths, final_paths)` - Finds all Ref properties, stores as (source, target_path) pairs +- `tentative_fs_path(dom, ref)` - Computes pre-dedup path: slugified name + extension based on class/children +- `tentative_fs_path_public()` - Public wrapper used in post-processing +- `link_referents(dom, referents)` - Writes `Rojo_Ref_*` attributes to instances + +**Ref path post-processing (in `mod.rs`):** + +During the syncback walk, `record_ref_path()` records each instance's final dedup'd filesystem path in `ref_path_map`. After the walk, `fix_ref_paths()` compares tentative paths to final paths and substitutes in all written meta/model files. Substitutions are sorted by length descending to prevent partial prefix matches. + +## Meta File Helpers (`meta.rs`) + +Shared helpers for reading/writing `.meta.json5` and `.model.json5` files: + +- `upsert_meta_name(path, name)` - Upsert `name` field in `.meta.json5` +- `remove_meta_name(path)` - Remove `name` field; returns `RemoveNameOutcome` (NoOp, FieldRemoved, FileDeleted) +- `upsert_model_name(path, name)` - Upsert `name` field in `.model.json5` +- `update_ref_paths_in_file(path, old_prefix, new_prefix)` - Update `Rojo_Ref_*` attribute paths in a meta/model file ## Statistics Tracking (`stats.rs`) @@ -428,12 +528,9 @@ Filesystem: ## Integration with Plugin -**Plugin does NOT handle syncback.** It only handles: -1. Downloading place file from Studio -2. Sending file to server -3. Server runs syncback command +**Live syncback (plugin-initiated):** The plugin encodes the Studio state via `encodeService()` + `SerializationService` and POSTs to `/api/syncback`. The server reconstructs the DOM, runs `syncback_loop(incremental=false)`, writes files, and restarts. Available from the NotConnected page -- no active WebSocket session required. See `atlas-api.mdc` for wire format. -**Two-way sync (experimental)** is different - see `atlas-api.mdc`. +**Two-way sync** is a separate system for incremental bidirectional changes -- see `atlas-api.mdc`. ## Performance Considerations @@ -441,7 +538,9 @@ Filesystem: **Incremental benefit:** On large projects, incremental mode with hashing can be 10-100x faster than rewriting everything. -**Parallelization:** Not currently parallelized - filesystem writes are sequential. +**Parallelization:** File writes can be parallelized via `write_to_vfs_parallel()`: +- 4-phase approach: create dirs (sequential) → write files (parallel via rayon) → remove files (parallel) → remove dirs (sequential via VFS) +- Windows retry logic: exponential backoff (10ms initial, doubles each retry, max 3 retries) for OS errors 5 (Access denied) and 32 (Sharing violation) ## Debugging @@ -473,14 +572,24 @@ diff place.rbxl rebuilt.rbxl **From code comments:** - Better detection of when root is directory vs file - More sophisticated middleware selection -- Parallel file writes - Better handling of Roblox format changes - Support for partial syncback (only changed instances) +- Globally optimal matching (Hungarian algorithm) for large ambiguous groups +- ~~Matching cache (parent_ref -> match_assignments) for avoiding redundant scoring~~ (session-level caching now implemented via `MatchingSession`; per-parent invalidation cache still deferred) ## Key Files to Reference -- `src/syncback/mod.rs` - Main orchestration loop +- `src/syncback/mod.rs` - Main orchestration loop, ref path post-processing - `src/syncback/snapshot.rs` - SyncbackSnapshot implementation +- `src/syncback/matching.rs` - Recursive change-count matching algorithm +- `src/syncback/file_names.rs` - slugify_name(), deduplicate_name_with_ext() +- `src/syncback/dedup_suffix.rs` - Dedup cleanup rules +- `src/syncback/meta.rs` - Meta file read/write helpers +- `src/syncback/ref_properties.rs` - Ref property path linking - `src/syncback/hash/` - Hashing algorithm +- `src/syncback/fs_snapshot.rs` - Filesystem operations, parallel writes, ref path post-processing - `src/syncback/property_filter.rs` - Property filtering logic - `src/cli/syncback.rs` - CLI command implementation +- `src/cli/serve.rs` - Live syncback handler (`run_live_syncback`, `build_dom_from_chunks`) +- `src/web/interface.rs` - `SyncbackRequest`, `ServiceChunk`, `SyncbackPayload` types +- `src/rojo_ref.rs` - Luau-style relative ref paths (`compute_relative_ref_path`, `resolve_ref_path_to_absolute`) diff --git a/.cursor/rules/atlas.mdc b/.cursor/rules/atlas.mdc index 7f718c2d..bb00e2b9 100644 --- a/.cursor/rules/atlas.mdc +++ b/.cursor/rules/atlas.mdc @@ -1,5 +1,4 @@ --- -description: "Essential Atlas context: architecture, contribution workflow, and code conventions for this project." alwaysApply: true --- @@ -33,6 +32,7 @@ Atlas bridges filesystem and Roblox Studio, enabling professional development to - Build/upload rbxl/rbxm files from command line - Syncback: pull instances from Roblox → filesystem (7.7.0+) - Two-way sync (plugin-based): live bidirectional sync during `atlas serve` sessions +- Live syncback: one-click full sync from Studio to filesystem during `atlas serve` (no WebSocket connection required) ### Two-Way Sync Strategy @@ -41,7 +41,7 @@ Atlas bridges filesystem and Roblox Studio, enabling professional development to **Current state:** - `atlas syncback` (CLI) is the **ground truth** for Roblox → filesystem syncing. It is battle-tested and handles edge cases reliably. - Plugin-based two-way sync (via `atlas serve` + the Studio plugin) is under active development to reach feature parity and match the resilience of CLI syncback. -- The bar for replacing syncback is high: plugin two-way sync must handle all the same scenarios (encoded names, nested projects, property conflicts, large trees, concurrent edits) just as reliably before syncback can be deprecated. +- The bar for replacing syncback is high: plugin two-way sync must handle all the same scenarios (slugified names, dedup suffixes, nested projects, property conflicts, large trees, concurrent edits) just as reliably before syncback can be deprecated. **When working on two-way sync:** Plugin-based sync must produce **exactly the same filesystem output** that `atlas syncback` would give for the same input. Byte-for-byte identical files, identical directory structures, identical naming. If the plugin path produces different results, treat syncback as the correct reference unless there's an explicit, documented reason to diverge. Any divergence is a bug until proven otherwise. @@ -207,7 +207,7 @@ pub struct InstanceContext { } ``` -**Filename handling:** Instance names with forbidden filesystem chars are slugified (forbidden chars → `_`), deduplicated (`~1`, `~2`, etc. on collision), and the real name is stored in `.meta.json5`/`.model.json5` `name` fields. See `src/syncback/file_names.rs`. +**Filename handling:** Instance names with forbidden filesystem chars are slugified (forbidden chars → `_`), deduplicated (`~2`, `~3`, etc. on collision), and the real name is stored in `.meta.json5`/`.model.json5` `name` fields. See `src/syncback/file_names.rs`. **VFS passed separately to snapshot functions:** ```rust @@ -316,16 +316,30 @@ Reconciler.applyPatch(instanceMap, patch) src/ - Rust server/CLI cli/ - Command implementations snapshot/ - Core snapshot system + matching.rs - Forward sync matching algorithm snapshot_middleware/ - File type handlers syncback/ - Roblox → filesystem + matching.rs - Syncback matching algorithm + dedup_suffix.rs - Dedup cleanup rules + meta.rs - Meta file read/write helpers web/ - HTTP/WebSocket server + interface.rs - Wire types (patches, syncback requests) change_processor.rs - Two-way sync filesystem writer + git.rs - Git integration (changed files, SHA1 hashing, auto-staging) + rojo_ref.rs - Ref path system (Rojo_Ref_*, RefPathIndex) + variant_eq.rs - Property value comparison plugin/src/ - Lua Studio plugin App/ - Roact UI components Reconciler/ - Patch application logic + matching.lua - Lua matching algorithm + trueEquals.lua - Shared value equality module ChangeBatcher/ - Two-way sync patch creation + encodeService.lua - Service-level property encoding for live syncback Config.lua - Version and protocol config + PatchTree.lua - Patch visualization tree (git-based default selections) + SHA1.luau - SHA1 hash for git blob format comparison + XXH32.luau - Hash function for plugin rbx-dom/ - Git submodule (separate repo) rbx_binary/ - Binary format (rbxm/rbxl) diff --git a/.cursor/rules/luau/CNAME b/.cursor/rules/luau-rfcs/CNAME similarity index 100% rename from .cursor/rules/luau/CNAME rename to .cursor/rules/luau-rfcs/CNAME diff --git a/.cursor/rules/luau/_config.yml b/.cursor/rules/luau-rfcs/_config.yml similarity index 100% rename from .cursor/rules/luau/_config.yml rename to .cursor/rules/luau-rfcs/_config.yml diff --git a/.cursor/rules/luau/abstract-module-paths-and-init-dot-luau.md b/.cursor/rules/luau-rfcs/abstract-module-paths-and-init-dot-luau.md similarity index 100% rename from .cursor/rules/luau/abstract-module-paths-and-init-dot-luau.md rename to .cursor/rules/luau-rfcs/abstract-module-paths-and-init-dot-luau.md diff --git a/.cursor/rules/luau/amended-require-resolution.md b/.cursor/rules/luau-rfcs/amended-require-resolution.md similarity index 100% rename from .cursor/rules/luau/amended-require-resolution.md rename to .cursor/rules/luau-rfcs/amended-require-resolution.md diff --git a/.cursor/rules/luau/behavior-eq-metamethod.md b/.cursor/rules/luau-rfcs/behavior-eq-metamethod.md similarity index 100% rename from .cursor/rules/luau/behavior-eq-metamethod.md rename to .cursor/rules/luau-rfcs/behavior-eq-metamethod.md diff --git a/.cursor/rules/luau/behavior-stricter-utf8-library.md b/.cursor/rules/luau-rfcs/behavior-stricter-utf8-library.md similarity index 100% rename from .cursor/rules/luau/behavior-stricter-utf8-library.md rename to .cursor/rules/luau-rfcs/behavior-stricter-utf8-library.md diff --git a/.cursor/rules/luau/change-global-version.md b/.cursor/rules/luau-rfcs/change-global-version.md similarity index 100% rename from .cursor/rules/luau/change-global-version.md rename to .cursor/rules/luau-rfcs/change-global-version.md diff --git a/.cursor/rules/luau/config-luauconfig.md b/.cursor/rules/luau-rfcs/config-luauconfig.md similarity index 100% rename from .cursor/rules/luau/config-luauconfig.md rename to .cursor/rules/luau-rfcs/config-luauconfig.md diff --git a/.cursor/rules/luau/config-luaurc.md b/.cursor/rules/luau-rfcs/config-luaurc.md similarity index 100% rename from .cursor/rules/luau/config-luaurc.md rename to .cursor/rules/luau-rfcs/config-luaurc.md diff --git a/.cursor/rules/luau/deprecate-getfenv-setfenv.md b/.cursor/rules/luau-rfcs/deprecate-getfenv-setfenv.md similarity index 100% rename from .cursor/rules/luau/deprecate-getfenv-setfenv.md rename to .cursor/rules/luau-rfcs/deprecate-getfenv-setfenv.md diff --git a/.cursor/rules/luau/deprecate-table-getn-foreach.md b/.cursor/rules/luau-rfcs/deprecate-table-getn-foreach.md similarity index 100% rename from .cursor/rules/luau/deprecate-table-getn-foreach.md rename to .cursor/rules/luau-rfcs/deprecate-table-getn-foreach.md diff --git a/.cursor/rules/luau/disallow-proposals-leading-to-ambiguity-in-grammar.md b/.cursor/rules/luau-rfcs/disallow-proposals-leading-to-ambiguity-in-grammar.md similarity index 100% rename from .cursor/rules/luau/disallow-proposals-leading-to-ambiguity-in-grammar.md rename to .cursor/rules/luau-rfcs/disallow-proposals-leading-to-ambiguity-in-grammar.md diff --git a/.cursor/rules/luau/explicit-type-parameter-instantiation.md b/.cursor/rules/luau-rfcs/explicit-type-parameter-instantiation.md similarity index 100% rename from .cursor/rules/luau/explicit-type-parameter-instantiation.md rename to .cursor/rules/luau-rfcs/explicit-type-parameter-instantiation.md diff --git a/.cursor/rules/luau/function-bit32-byteswap.md b/.cursor/rules/luau-rfcs/function-bit32-byteswap.md similarity index 100% rename from .cursor/rules/luau/function-bit32-byteswap.md rename to .cursor/rules/luau-rfcs/function-bit32-byteswap.md diff --git a/.cursor/rules/luau/function-bit32-countlz-countrz.md b/.cursor/rules/luau-rfcs/function-bit32-countlz-countrz.md similarity index 100% rename from .cursor/rules/luau/function-bit32-countlz-countrz.md rename to .cursor/rules/luau-rfcs/function-bit32-countlz-countrz.md diff --git a/.cursor/rules/luau/function-buffer-bits.md b/.cursor/rules/luau-rfcs/function-buffer-bits.md similarity index 100% rename from .cursor/rules/luau/function-buffer-bits.md rename to .cursor/rules/luau-rfcs/function-buffer-bits.md diff --git a/.cursor/rules/luau/function-coroutine-close.md b/.cursor/rules/luau-rfcs/function-coroutine-close.md similarity index 100% rename from .cursor/rules/luau/function-coroutine-close.md rename to .cursor/rules/luau-rfcs/function-coroutine-close.md diff --git a/.cursor/rules/luau/function-debug-info.md b/.cursor/rules/luau-rfcs/function-debug-info.md similarity index 100% rename from .cursor/rules/luau/function-debug-info.md rename to .cursor/rules/luau-rfcs/function-debug-info.md diff --git a/.cursor/rules/luau/function-inlining.md b/.cursor/rules/luau-rfcs/function-inlining.md similarity index 100% rename from .cursor/rules/luau/function-inlining.md rename to .cursor/rules/luau-rfcs/function-inlining.md diff --git a/.cursor/rules/luau/function-math-lerp.md b/.cursor/rules/luau-rfcs/function-math-lerp.md similarity index 100% rename from .cursor/rules/luau/function-math-lerp.md rename to .cursor/rules/luau-rfcs/function-math-lerp.md diff --git a/.cursor/rules/luau/function-math-map.md b/.cursor/rules/luau-rfcs/function-math-map.md similarity index 100% rename from .cursor/rules/luau/function-math-map.md rename to .cursor/rules/luau-rfcs/function-math-map.md diff --git a/.cursor/rules/luau/function-string-pack-unpack.md b/.cursor/rules/luau-rfcs/function-string-pack-unpack.md similarity index 100% rename from .cursor/rules/luau/function-string-pack-unpack.md rename to .cursor/rules/luau-rfcs/function-string-pack-unpack.md diff --git a/.cursor/rules/luau/function-table-clear.md b/.cursor/rules/luau-rfcs/function-table-clear.md similarity index 100% rename from .cursor/rules/luau/function-table-clear.md rename to .cursor/rules/luau-rfcs/function-table-clear.md diff --git a/.cursor/rules/luau/function-table-clone.md b/.cursor/rules/luau-rfcs/function-table-clone.md similarity index 100% rename from .cursor/rules/luau/function-table-clone.md rename to .cursor/rules/luau-rfcs/function-table-clone.md diff --git a/.cursor/rules/luau/function-table-create-find.md b/.cursor/rules/luau-rfcs/function-table-create-find.md similarity index 100% rename from .cursor/rules/luau/function-table-create-find.md rename to .cursor/rules/luau-rfcs/function-table-create-find.md diff --git a/.cursor/rules/luau/function-table-freeze.md b/.cursor/rules/luau-rfcs/function-table-freeze.md similarity index 100% rename from .cursor/rules/luau/function-table-freeze.md rename to .cursor/rules/luau-rfcs/function-table-freeze.md diff --git a/.cursor/rules/luau/function-vector-lerp.md b/.cursor/rules/luau-rfcs/function-vector-lerp.md similarity index 100% rename from .cursor/rules/luau/function-vector-lerp.md rename to .cursor/rules/luau-rfcs/function-vector-lerp.md diff --git a/.cursor/rules/luau/generalized-iteration.md b/.cursor/rules/luau-rfcs/generalized-iteration.md similarity index 100% rename from .cursor/rules/luau/generalized-iteration.md rename to .cursor/rules/luau-rfcs/generalized-iteration.md diff --git a/.cursor/rules/luau/generic-function-subtyping.md b/.cursor/rules/luau-rfcs/generic-function-subtyping.md similarity index 100% rename from .cursor/rules/luau/generic-function-subtyping.md rename to .cursor/rules/luau-rfcs/generic-function-subtyping.md diff --git a/.cursor/rules/luau/generic-functions.md b/.cursor/rules/luau-rfcs/generic-functions.md similarity index 100% rename from .cursor/rules/luau/generic-functions.md rename to .cursor/rules/luau-rfcs/generic-functions.md diff --git a/.cursor/rules/luau/index-type-operator.md b/.cursor/rules/luau-rfcs/index-type-operator.md similarity index 100% rename from .cursor/rules/luau/index-type-operator.md rename to .cursor/rules/luau-rfcs/index-type-operator.md diff --git a/.cursor/rules/luau/index.md b/.cursor/rules/luau-rfcs/index.md similarity index 100% rename from .cursor/rules/luau/index.md rename to .cursor/rules/luau-rfcs/index.md diff --git a/.cursor/rules/luau/keyof-type-operator.md b/.cursor/rules/luau-rfcs/keyof-type-operator.md similarity index 100% rename from .cursor/rules/luau/keyof-type-operator.md rename to .cursor/rules/luau-rfcs/keyof-type-operator.md diff --git a/.cursor/rules/luau/len-metamethod-rawlen.md b/.cursor/rules/luau-rfcs/len-metamethod-rawlen.md similarity index 100% rename from .cursor/rules/luau/len-metamethod-rawlen.md rename to .cursor/rules/luau-rfcs/len-metamethod-rawlen.md diff --git a/.cursor/rules/luau/local-type-inference.md b/.cursor/rules/luau-rfcs/local-type-inference.md similarity index 100% rename from .cursor/rules/luau/local-type-inference.md rename to .cursor/rules/luau-rfcs/local-type-inference.md diff --git a/.cursor/rules/luau/lower-bounds-calculation.md b/.cursor/rules/luau-rfcs/lower-bounds-calculation.md similarity index 100% rename from .cursor/rules/luau/lower-bounds-calculation.md rename to .cursor/rules/luau-rfcs/lower-bounds-calculation.md diff --git a/.cursor/rules/luau/luau-mascot.md b/.cursor/rules/luau-rfcs/luau-mascot.md similarity index 100% rename from .cursor/rules/luau/luau-mascot.md rename to .cursor/rules/luau-rfcs/luau-mascot.md diff --git a/.cursor/rules/luau/mascot.png b/.cursor/rules/luau-rfcs/mascot.png similarity index 100% rename from .cursor/rules/luau/mascot.png rename to .cursor/rules/luau-rfcs/mascot.png diff --git a/.cursor/rules/luau-rfcs/math-constants.md b/.cursor/rules/luau-rfcs/math-constants.md new file mode 100644 index 00000000..897b43fb --- /dev/null +++ b/.cursor/rules/luau-rfcs/math-constants.md @@ -0,0 +1,56 @@ +# Math constants + +## Summary + +Expand the constants provided in the `math` library. + +## Motivation + +With the introduction of [`math.isnan`](https://rfcs.luau.org/math-isnan-isfinite-isinf.html), Luau has the opportunity to add a new constant representing `NaN` (Not a Number). +As linting support for Luau continues becoming more robust, we can discourage patterns like `0/0` to generate `NaN` and instead provide the constant directly through the math library. + +Without the `math.isnan` API, this could have created confusion in the past: +```luau +if maybeNan == math.nan do + print("This line will never be hit: comparing to NaN always returns false.") +end +``` + +With the new API, however, this is now easier to use: +```luau +local maybeNan = math.nan +if math.isnan(maybeNan) do + print("This line will be hit!") +end +``` + +While adding `NaN`, we also have an opportunity to modernize the set of constants available through the `math` library. +Some history: +- Lua 5.0 (2003) introduced `math.pi`, representing the mathematical constant $\pi$. +- Lua 5.1 (2006) introduced `math.huge`, mapping directly to infinity as defined by the [IEEE 754 standard](https://en.wikipedia.org/wiki/IEEE_754). +- Lua 5.3 (2015) introduced integers and the `math.maxinteger` and `math.mininteger` constants alongside them. + +Luau currently only supports `math.pi` and `math.huge`. +Integer types are not supported in Luau, so `math.maxinteger` and `math.mininteger` wouldn't make much sense, but adding other commonly used constants would be an easy value-add for users of the language. + +## Design + +In addition to `math.pi` and `math.huge`, we propose adding the following constants. + +- `math.nan`: some `NaN` value as defined by IEEE 754. + - The exact representation (payload and sign) is unspecified. + - Comparing to `math.nan` will always return `false`; lint rules recommend using `math.isnan`. +- `math.e`: the mathematical constant $e = 2.71828182845904523536$. +- `math.phi`: the golden ratio, defined as $\phi = 1.61803398874989484820$. +- `math.sqrt2`: the square root of `2`, defined as equal to `math.sqrt(2)`. +- `math.tau`: the mathematical constant $\tau$, defined as equal to `2 * math.pi`. +## Drawbacks + +- The semantics of `math.nan` are unique and somewhat unexpected for users not already familiar with IEEE 754's floating-point arithmetic rules. +- Adding new constants increases the API surface of the `math` library and diverges further from Lua 5.1. + +## Alternatives + +- Do nothing: continue requiring users to define their own constants (e.g., `local nan = 0/0`, `local e = math.exp(1)`). +- Only add `math.nan` without the additional constants, deferring modernization for a future RFC. +- Rely on these constants being provided through external libraries. diff --git a/.cursor/rules/luau/math-isnan-isfinite-isinf.md b/.cursor/rules/luau-rfcs/math-isnan-isfinite-isinf.md similarity index 100% rename from .cursor/rules/luau/math-isnan-isfinite-isinf.md rename to .cursor/rules/luau-rfcs/math-isnan-isfinite-isinf.md diff --git a/.cursor/rules/luau/metatable-type-functions.md b/.cursor/rules/luau-rfcs/metatable-type-functions.md similarity index 100% rename from .cursor/rules/luau/metatable-type-functions.md rename to .cursor/rules/luau-rfcs/metatable-type-functions.md diff --git a/.cursor/rules/luau/method-type-issubtypeof.md b/.cursor/rules/luau-rfcs/method-type-issubtypeof.md similarity index 100% rename from .cursor/rules/luau/method-type-issubtypeof.md rename to .cursor/rules/luau-rfcs/method-type-issubtypeof.md diff --git a/.cursor/rules/luau/negation-types.md b/.cursor/rules/luau-rfcs/negation-types.md similarity index 100% rename from .cursor/rules/luau/negation-types.md rename to .cursor/rules/luau-rfcs/negation-types.md diff --git a/.cursor/rules/luau/never-and-unknown-types.md b/.cursor/rules/luau-rfcs/never-and-unknown-types.md similarity index 100% rename from .cursor/rules/luau/never-and-unknown-types.md rename to .cursor/rules/luau-rfcs/never-and-unknown-types.md diff --git a/.cursor/rules/luau/new-nonstrict.md b/.cursor/rules/luau-rfcs/new-nonstrict.md similarity index 100% rename from .cursor/rules/luau/new-nonstrict.md rename to .cursor/rules/luau-rfcs/new-nonstrict.md diff --git a/.cursor/rules/luau/new-require-by-string-semantics.md b/.cursor/rules/luau-rfcs/new-require-by-string-semantics.md similarity index 100% rename from .cursor/rules/luau/new-require-by-string-semantics.md rename to .cursor/rules/luau-rfcs/new-require-by-string-semantics.md diff --git a/.cursor/rules/luau/property-readonly.md b/.cursor/rules/luau-rfcs/property-readonly.md similarity index 100% rename from .cursor/rules/luau/property-readonly.md rename to .cursor/rules/luau-rfcs/property-readonly.md diff --git a/.cursor/rules/luau/property-writeonly.md b/.cursor/rules/luau-rfcs/property-writeonly.md similarity index 100% rename from .cursor/rules/luau/property-writeonly.md rename to .cursor/rules/luau-rfcs/property-writeonly.md diff --git a/.cursor/rules/luau/rawget-type-operator.md b/.cursor/rules/luau-rfcs/rawget-type-operator.md similarity index 100% rename from .cursor/rules/luau/rawget-type-operator.md rename to .cursor/rules/luau-rfcs/rawget-type-operator.md diff --git a/.cursor/rules/luau/recursive-type-restriction.md b/.cursor/rules/luau-rfcs/recursive-type-restriction.md similarity index 100% rename from .cursor/rules/luau/recursive-type-restriction.md rename to .cursor/rules/luau-rfcs/recursive-type-restriction.md diff --git a/.cursor/rules/luau/relax-recursive-type-restriction.md b/.cursor/rules/luau-rfcs/relax-recursive-type-restriction.md similarity index 100% rename from .cursor/rules/luau/relax-recursive-type-restriction.md rename to .cursor/rules/luau-rfcs/relax-recursive-type-restriction.md diff --git a/.cursor/rules/luau/require-by-string-aliases.md b/.cursor/rules/luau-rfcs/require-by-string-aliases.md similarity index 100% rename from .cursor/rules/luau/require-by-string-aliases.md rename to .cursor/rules/luau-rfcs/require-by-string-aliases.md diff --git a/.cursor/rules/luau/reserve-dollar-sign.md b/.cursor/rules/luau-rfcs/reserve-dollar-sign.md similarity index 100% rename from .cursor/rules/luau/reserve-dollar-sign.md rename to .cursor/rules/luau-rfcs/reserve-dollar-sign.md diff --git a/.cursor/rules/luau/sealed-table-subtyping.md b/.cursor/rules/luau-rfcs/sealed-table-subtyping.md similarity index 100% rename from .cursor/rules/luau/sealed-table-subtyping.md rename to .cursor/rules/luau-rfcs/sealed-table-subtyping.md diff --git a/.cursor/rules/luau/shared-self-types.md b/.cursor/rules/luau-rfcs/shared-self-types.md similarity index 100% rename from .cursor/rules/luau/shared-self-types.md rename to .cursor/rules/luau-rfcs/shared-self-types.md diff --git a/.cursor/rules/luau/support-for-generic-function-types-in-user-defined-type-functions.md b/.cursor/rules/luau-rfcs/support-for-generic-function-types-in-user-defined-type-functions.md similarity index 100% rename from .cursor/rules/luau/support-for-generic-function-types-in-user-defined-type-functions.md rename to .cursor/rules/luau-rfcs/support-for-generic-function-types-in-user-defined-type-functions.md diff --git a/.cursor/rules/luau/support-for-thread-and-buffer-types-in-user-defined-type-functions.md b/.cursor/rules/luau-rfcs/support-for-thread-and-buffer-types-in-user-defined-type-functions.md similarity index 100% rename from .cursor/rules/luau/support-for-thread-and-buffer-types-in-user-defined-type-functions.md rename to .cursor/rules/luau-rfcs/support-for-thread-and-buffer-types-in-user-defined-type-functions.md diff --git a/.cursor/rules/luau/syntax-array-like-table-types.md b/.cursor/rules/luau-rfcs/syntax-array-like-table-types.md similarity index 100% rename from .cursor/rules/luau/syntax-array-like-table-types.md rename to .cursor/rules/luau-rfcs/syntax-array-like-table-types.md diff --git a/.cursor/rules/luau/syntax-attribute-functions-deprecated.md b/.cursor/rules/luau-rfcs/syntax-attribute-functions-deprecated.md similarity index 100% rename from .cursor/rules/luau/syntax-attribute-functions-deprecated.md rename to .cursor/rules/luau-rfcs/syntax-attribute-functions-deprecated.md diff --git a/.cursor/rules/luau/syntax-attribute-functions-native.md b/.cursor/rules/luau-rfcs/syntax-attribute-functions-native.md similarity index 100% rename from .cursor/rules/luau/syntax-attribute-functions-native.md rename to .cursor/rules/luau-rfcs/syntax-attribute-functions-native.md diff --git a/.cursor/rules/luau/syntax-attributes-functions-parameters.md b/.cursor/rules/luau-rfcs/syntax-attributes-functions-parameters.md similarity index 100% rename from .cursor/rules/luau/syntax-attributes-functions-parameters.md rename to .cursor/rules/luau-rfcs/syntax-attributes-functions-parameters.md diff --git a/.cursor/rules/luau/syntax-attributes-functions.md b/.cursor/rules/luau-rfcs/syntax-attributes-functions.md similarity index 100% rename from .cursor/rules/luau/syntax-attributes-functions.md rename to .cursor/rules/luau-rfcs/syntax-attributes-functions.md diff --git a/.cursor/rules/luau/syntax-compound-assignment.md b/.cursor/rules/luau-rfcs/syntax-compound-assignment.md similarity index 100% rename from .cursor/rules/luau/syntax-compound-assignment.md rename to .cursor/rules/luau-rfcs/syntax-compound-assignment.md diff --git a/.cursor/rules/luau/syntax-continue-statement.md b/.cursor/rules/luau-rfcs/syntax-continue-statement.md similarity index 100% rename from .cursor/rules/luau/syntax-continue-statement.md rename to .cursor/rules/luau-rfcs/syntax-continue-statement.md diff --git a/.cursor/rules/luau/syntax-default-type-alias-type-parameters.md b/.cursor/rules/luau-rfcs/syntax-default-type-alias-type-parameters.md similarity index 100% rename from .cursor/rules/luau/syntax-default-type-alias-type-parameters.md rename to .cursor/rules/luau-rfcs/syntax-default-type-alias-type-parameters.md diff --git a/.cursor/rules/luau/syntax-floor-division-operator.md b/.cursor/rules/luau-rfcs/syntax-floor-division-operator.md similarity index 100% rename from .cursor/rules/luau/syntax-floor-division-operator.md rename to .cursor/rules/luau-rfcs/syntax-floor-division-operator.md diff --git a/.cursor/rules/luau/syntax-if-expression.md b/.cursor/rules/luau-rfcs/syntax-if-expression.md similarity index 100% rename from .cursor/rules/luau/syntax-if-expression.md rename to .cursor/rules/luau-rfcs/syntax-if-expression.md diff --git a/.cursor/rules/luau/syntax-leading-bar-and-ampersand.md b/.cursor/rules/luau-rfcs/syntax-leading-bar-and-ampersand.md similarity index 100% rename from .cursor/rules/luau/syntax-leading-bar-and-ampersand.md rename to .cursor/rules/luau-rfcs/syntax-leading-bar-and-ampersand.md diff --git a/.cursor/rules/luau/syntax-named-function-type-args.md b/.cursor/rules/luau-rfcs/syntax-named-function-type-args.md similarity index 100% rename from .cursor/rules/luau/syntax-named-function-type-args.md rename to .cursor/rules/luau-rfcs/syntax-named-function-type-args.md diff --git a/.cursor/rules/luau/syntax-number-literals.md b/.cursor/rules/luau-rfcs/syntax-number-literals.md similarity index 100% rename from .cursor/rules/luau/syntax-number-literals.md rename to .cursor/rules/luau-rfcs/syntax-number-literals.md diff --git a/.cursor/rules/luau/syntax-property-access-modifiers.md b/.cursor/rules/luau-rfcs/syntax-property-access-modifiers.md similarity index 100% rename from .cursor/rules/luau/syntax-property-access-modifiers.md rename to .cursor/rules/luau-rfcs/syntax-property-access-modifiers.md diff --git a/.cursor/rules/luau/syntax-singleton-types.md b/.cursor/rules/luau-rfcs/syntax-singleton-types.md similarity index 100% rename from .cursor/rules/luau/syntax-singleton-types.md rename to .cursor/rules/luau-rfcs/syntax-singleton-types.md diff --git a/.cursor/rules/luau/syntax-string-interpolation.md b/.cursor/rules/luau-rfcs/syntax-string-interpolation.md similarity index 100% rename from .cursor/rules/luau/syntax-string-interpolation.md rename to .cursor/rules/luau-rfcs/syntax-string-interpolation.md diff --git a/.cursor/rules/luau/syntax-type-alias-type-packs.md b/.cursor/rules/luau-rfcs/syntax-type-alias-type-packs.md similarity index 100% rename from .cursor/rules/luau/syntax-type-alias-type-packs.md rename to .cursor/rules/luau-rfcs/syntax-type-alias-type-packs.md diff --git a/.cursor/rules/luau/syntax-type-ascription-bidi.md b/.cursor/rules/luau-rfcs/syntax-type-ascription-bidi.md similarity index 100% rename from .cursor/rules/luau/syntax-type-ascription-bidi.md rename to .cursor/rules/luau-rfcs/syntax-type-ascription-bidi.md diff --git a/.cursor/rules/luau/syntax-type-ascription.md b/.cursor/rules/luau-rfcs/syntax-type-ascription.md similarity index 100% rename from .cursor/rules/luau/syntax-type-ascription.md rename to .cursor/rules/luau-rfcs/syntax-type-ascription.md diff --git a/.cursor/rules/luau/syntax-typed-variadics.md b/.cursor/rules/luau-rfcs/syntax-typed-variadics.md similarity index 100% rename from .cursor/rules/luau/syntax-typed-variadics.md rename to .cursor/rules/luau-rfcs/syntax-typed-variadics.md diff --git a/.cursor/rules/luau/type-ascription-by-inhabitance.md b/.cursor/rules/luau-rfcs/type-ascription-by-inhabitance.md similarity index 100% rename from .cursor/rules/luau/type-ascription-by-inhabitance.md rename to .cursor/rules/luau-rfcs/type-ascription-by-inhabitance.md diff --git a/.cursor/rules/luau/type-byte-buffer.md b/.cursor/rules/luau-rfcs/type-byte-buffer.md similarity index 100% rename from .cursor/rules/luau/type-byte-buffer.md rename to .cursor/rules/luau-rfcs/type-byte-buffer.md diff --git a/.cursor/rules/luau/type-error-suppression.md b/.cursor/rules/luau-rfcs/type-error-suppression.md similarity index 100% rename from .cursor/rules/luau/type-error-suppression.md rename to .cursor/rules/luau-rfcs/type-error-suppression.md diff --git a/.cursor/rules/luau-rfcs/type-long-integer.md b/.cursor/rules/luau-rfcs/type-long-integer.md new file mode 100644 index 00000000..de64d62c --- /dev/null +++ b/.cursor/rules/luau-rfcs/type-long-integer.md @@ -0,0 +1,393 @@ +# 64-bit Integer Type + +## Summary + +A builtin 'integer' type to represent 64 bit integer numbers. + +## Motivation + +Current methods of representing 64-bit integers in Luau with the default environment are done with high-low pairs or by splitting them among vectors which leads to poor ergonomics and may not be sufficient for cases where performance is important. + +In the case of a high-low pair implementation, each individual number takes 16 bytes (or 24) and needs to be handled together which can be confusing. + +In the case of vector implementations, ergonomics are improved by storing the entire integer in one value. However you have the same issues of implementation complexity and lack of integration with the type system. + +In both cases, performance is lacking compared to what could be provided by a native implementation. + +As Luau grows, the restriction to doubles will be an increasing pain point for any use case which involves a 64-bit integer. +System support for 64-bit integers is ubiquitous and their use is widespread. + +Besides the above solutions in userland, runtimes are also able to define their own userdata for 64-bit integers. However, this requires each runtime to reimplement 64-bit integer support themselves, and carries with it the potential of significant differences in their semantics. Such an outcome would harm the ecosystem as a whole, requiring libraries to find a common API across all of their supported runtimes, or otherwise program around differences in their APIs. + +Further, representing 64-bit numbers as heap-allocated userdata is sufficient for correctness, but also adds a layer of indirection for something that could fit within the existing value size. In domains where the use of these numbers is more common, the performance characteristics of these varying implementation techniques could be problematic or even prohibitive. + +Userdata remains the right mechanism for complex or embedder-specific numeric types, but 64-bit integers have a sufficiently large problem domain to justify their inclusion as a core value type, and bring significant benefits for performance and interoperability in the overall Luau ecosystem. + +## Design + +This will be implemented with a new type called `integer`. + +An additional character may be specified at the end of numeric literals `i` which will signify an 64 bit integer literal. +64 bit integer literals will support separators, hex, and binary values: + +```luau +local a = 123i +local b = 1_000i +local c = 0xABABi +local d = 0b1000_1000i +``` + +Integer literal numbers have to be exactly representable, overflow is a parsing error. + +Operations performing a string formatting of an integer should format the number as signed by default. + +Integer values will have a built-in equality comparison, but will not have any other operators or metamethods defined. + +Integers are never automatically converted to numbers or strings, and vice-versa. +Passing an integer to a function expecting a number (or string) will result in a type error. + +While there are no arithmentic operators defined for integers in this RFC, it is still worth noting that operations between `number` and `integer` are also not supported. + +Functions for creating and manipulating this type will exist in a new library called 'integer`. + +### Library + +Unless otherwise specified, all operations interpret integer values as signed two's complement 64 bit numbers. +Operations that treat the value as unsigned are explicitly prefixed with 'u'. + +`function integer.create(n: number): integer?` + +Converts a number to an integer. + +Returns `nil` If the double number cannot be represented as an integer exactly, i.e. it has a fractional part, is out of range or is NaN. + +This behavior is chosen to closely match `math.tointeger` from Lua 5.4 and can have a relatively efficient implementation. + +`function integer.fromstring(str: string, base: number?): integer?` + +Converts a string representation of a number into an integer. + +Number inside the string has to be an integer. +String is allowed to have leading and trailing spaces and number can be preceded by a sign. +If base is not specified, number can have a `0x` or `0X` prefix to be converted in base 16, otherwise base 10 is used. + +When base is specified, it has to be in range between 2 and 36 inclusive. +In base 10 and base 16, numbers are allowed to have a `0x` or `0X` prefix. + +Returns `nil` if the string doesn't contain a number. + +This behavior is chosen to match `tonumber`, but excludes floating-point numbers. + +`function integer.tostring(n: integer): string` + +Converts an integer to a string representation. + +`function integer.tonumber(n: integer): number` + +Converts an integer to a double. + +If the value cannot be represented as a double exactly, round to nearest, tie to even rounding mode is used. + +`function integer.neg(a: integer): integer` + +Negates the value. +Overflow wraps around according to rules of two-complement arithmetic. + +`function integer.add(a: integer, b: integer): integer` + +Adds `a` to `b`. +Overflow wraps around according to rules of two-complement arithmetic. + +`function integer.sub(a: integer, b: integer): integer` + +Subtracts `b` from `a`. +Overflow wraps around according to rules of two-complement arithmetic. + +`function integer.mul(a: integer, b: integer): integer` + +Multiplies `a` and `b`. +Overflow wraps around according to rules of two-complement arithmetic. + +`function integer.div(a: integer, b: integer): integer` + +Performs signed truncated division of `a` by `b`. + +If `b` is 0, throws a division by zero error. + +If `a` is -2^63 and `b` is -1, throws an overflow error. + +`function integer.rem(a: integer, b: integer): integer` + +Computes remainder of the signed truncated division of `a` by `b`. + +If `b` is 0, throws a division by zero error. + +If `a` is -2^63 and `b` is -1, result is 0. + +`function integer.idiv(a: integer, b: integer): integer` + +Performs signed floored division of `a` by `b`. + +If `b` is 0, throws a division by zero error. + +If `a` is -2^63 and `b` is -1, throws an overflow error. + +`function integer.mod(a: integer, b: integer): integer` + +Performs signed floored modulus division of `a` by `b`. + +If `b` is 0, throws a division by zero error. + +If `a` is -2^63 and `b` is -1, result is 0. + +`function integer.udiv(a: integer, b: integer): integer` + +Performs unsigned division of `a` by `b`. + +If `b` is 0, throws an error. + +`function integer.urem(a: integer, b: integer): integer` + +Computes remainder of the unsigned division of `a` by `b`. + +If `b` is 0, throws an error. + +`function integer.min(a: integer, b: integer): integer` + +Returns the smallest of two integer numbers. + +`function integer.max(a: integer, b: integer): integer` + +Returns the largest of two integer numbers. + +`function integer.clamp(a: integer, min: integer, max: integer): integer` + +Returns `a` if the number is in `[min, max]` range; otherwise, returns `min` when `a < min`, and `max` otherwise. + +The function errors if `min > max`, consistent with `math.clamp`. + +`function integer.band(a: integer, b: integer): integer` + +Performs a bitwise and of `a` and `b`. + +`function integer.bor(a: integer, b: integer): integer` + +Performs a bitwise or of `a` and `b`. + +`function integer.bnot(a: integer): integer` + +Returns a bitwise negation of the input number. + +`function integer.bxor(a: integer, b: integer): integer` + +Performs a bitwise xor (exclusive or) of `a` and `b`. + +`function integer.lt(a: integer, b: integer): boolean` + +Compares signed less than (<) comparison of `a` and `b`. + +`function integer.le(a: integer, b: integer): boolean` + +Compares signed less than or equal (<=) comparison of `a` and `b`. + +`function integer.ult(a: integer, b: integer): boolean` + +Compares unsigned less than (<) comparison of `a` and `b`. + +`function integer.ule(a: integer, b: integer): boolean` + +Compares unsigned less than or equal (<=) comparison of `a` and `b`. + +`function integer.lshift(n: integer, i: integer): integer` + +Shifts `n` to the left by `i` bits (if `i` is negative, a right shift is performed instead). + +When `i` is outside of `[-63..63]` range, returns 0. + +`function integer.rshift(n: integer, i: integer): integer` + +Shifts `n` to the right by `i` bits (if `i` is negative, a left shift is performed instead). + +When `i` is outside of `[-63..63]` range, returns 0. + +`function integer.arshift(n: integer, i: integer): integer` + +Shifts `n` by `i` bits to the right (if `i` is negative, a left shift is performed instead). + +The most significant bit of `n` is propagated during the shift. + +When `i` is larger than 63, returns an integer with all bits set to the sign bit of `n`. + +When `i` is smaller than -63, 0 is returned. + +`function integer.lrotate(n: integer, i: integer): integer` + +Rotates `n` to the left by `i` bits (if `i` is negative, a right rotate is performed instead); the bits that are shifted past the bit width are shifted back from the right. + +`i` is interpreted modulo 64. + +`function integer.rrotate(n: integer, i: integer): integer` + +Rotates `n` to the right by `i` bits (if `i` is negative, a left rotate is performed instead); the bits that are shifted past the bit width are shifted back from the left. + +`i` is interpreted modulo 64. + +`function integer.extract(n: integer, f: integer, w: integer?): integer` + +Extracts bits of `n` at position `f` with a width of `w`. + +`w` defaults to 1, so a two-argument version of extract returns the bit value at position `f`. + +Bits are indexed starting at 0. +`f` cannot be negative. +`w` has to be positive (> 0). +`f + w` must be less than or equal to 64. + +`function integer.replace(n: integer, r: integer, f: integer, w: integer?): integer` + +Replaces bits of `n` at position `f` and width `w` with `w` least significant bits of `r`. + +`w` defaults to 1, so a three-argument version of replace changes one bit at position `f` to `r` and returns the result. + +Bits are indexed starting at 0. +`f` cannot be negative. +`w` has to be positive (> 0). +`f + w` must be less than or equal to 64. + +`function integer.btest(a: integer, b: integer): boolean` + +Perform a bitwise and `a` and `b` and returns true iff the result is not 0. + +`function integer.countrz(n: integer): integer` + +Returns the number of consecutive zero bits starting from the right-most (least significant) bit. +Returns 64 if `n` is zero. + +`function integer.countlz(n: integer): integer` + +Returns the number of consecutive zero bits starting from the left-most (most significant) bit. +Returns 64 if `n` is zero. + +`function integer.bswap(n: integer): integer` + +Returns `n` with the order of the bytes swapped. + +### Extensions to the buffer library + +Functions to read and write integer values are added. +Functions use `integer` in the name as opposed to `i64` to signify the difference of the result/argument type from that of `number`. + +`function buffer.readinteger(b: buffer, offset: number): integer` + +Reads an integer out of a buffer at the given offset. + +`function buffer.writeinteger(b: buffer, offset: number, value: integer): ()` + +Writes an integer to a buffer at the given offset. + +### Extensions to the math library + +For parity with Lua 5.3+, two additional constants are added to the math library. + +`math.maxinteger` + +Integer value representing 2^63-1 (`9_223_372_036_854_775_807i`) + +`math.mininteger` + +Integer value representing -2^63 (`-9_223_372_036_854_775_808i`) + +### Extensions to the string library + +`string.format` function is updated to support integer arguments. + +'d', 'i' and '*' format specifiers will format an integer as a signed 64 bit integer number. + +'o', 'u', 'x' and 'X' format specifiers will format an integer as an unsigned 64 bit integer number. + +### C API + +`int64_t lua_tointeger64(lua_State *L, int idx, int* isinteger)` + +Returns a value of an integer at `idx` or 0 on failure. + +If `isinteger` is not a null pointer, writes 1 if the value at `idx` was an integer value and 0 on failure. + +`void lua_pushinteger64(lua_State *L, int64_t n)` + +Pushes an integer value on top of the stack. + +`int lua_isinteger64(lua_State *L, int idx)` + +Determines if a value at `idx` is an integer. + +This function is implemented as a C macro. + +`int64_t luaL_checkinteger64(lua_State *L, int narg)` + +Returns a value of an integer at `narg` or throws an error on failure. + +`int64_t luaL_optinteger64(lua_State *L, int narg, int64_t def)` + +Returns a value of an integer at `narg` or `def` if there is no value. + +Throws an error if there is a value but it is not an integer. + +`luaopen_integer(lua_State *L)` + +Registers the `integer` library. +Included in `luaL_openlibs`. + +## Drawbacks + +This introduces a library with a large set of functions which will likely take a large number of fastcall slots. + +The lack of operator overloads causes code with integers to be more verbose, decreasing readability in complex expressions. + +Introduction of a new type will require updates to libraries (serialization, input/output) which try to handle all potential Luau values. + +The suffix 'i' is standard mathematical notation for imaginary numbers. +Adopting it for integers will likely block proposals for a built-in `complex` number type in the future. + +## Alternatives + +Do nothing, the workarounds exist and are in use but this will restrict the areas Luau can be used without implementation burden or reliance on an externally maintained implementation of 64-bit integers. + +### Operator overloads + +This proposal does not introduce overloaded operators aside from equality checks. +This omission is deliberate. + +While the lack of operators makes code more verbose, Luau's primary numeric type remains `number`. +The `integer` type is intended to cover specialized use cases, many of which will specifically prefer explicit library functions over operators for performance. + +Polymorphic overloaded operators present many challenges and can cause unexpected drops of performance. +This is already true when working with `vector` types without type annotations and given the focus of this proposal, we do not want to provide additional pitfalls. +Monomorphic library functions allow compiler and runtime to optimize more aggressively. + +Heavy uses of 64-bit integers often rely on bitwise operations. +Since Luau does not have bitwise operators in its syntax, even with a subset of overloaded operators in place, code will still end up using library functions. + +Integer division has multiple conflicting definitions (truncated vs. floored). +Operators like `/` and `%` would force a specific default behavior that might not match the specific rounding that the user intended. +Default behavior might also be less performant, causing developers to still rely on library methods. + +Having different library methods allows the developers to choose the specific operation semantics they require. + +### Separate signed and unsigned types + +It has been suggested to create two separate types for signed and unsigned integers or have it be a property of the value. +Having distinct types would provide the typechecker with the ability to highlight mixed operations and a runtime check could throw an error. + +As mentioned above, integers are already not expected to be the default value type to be chosen by developers to work with. +Unsigned integer use is even rarer and other programming languages followed the similar path of only supporting auxiliary operations on them. +Other languages with unsigned integer support like C++ recommend limiting their use in their guidelines. +Given this, we have rejected the idea of distinct signed and unsigned integer types. + +Having two distinct integer types increases the API and library surface area and implementation complexity significantly for little gain. + +In two's complement representation, a single data type with default signed interpretation covers many signed and unsigned operations in an identical way. +Only a limited subset of operations interpreting data as unsigned is added in this proposal to cover most needs for unsigned integer numbers. + +Having library methods instead of operators allows the developer to make an explicit choice of the interpretation required (e.g. `lt` vs `ult`). +Reinterpretation of a negative value as a positive unsigned value can be made deliberately by the choice of function, and is not as error-prone as using an operator. diff --git a/.cursor/rules/luau/types-library-optional.md b/.cursor/rules/luau-rfcs/types-library-optional.md similarity index 100% rename from .cursor/rules/luau/types-library-optional.md rename to .cursor/rules/luau-rfcs/types-library-optional.md diff --git a/.cursor/rules/luau/udtf-is-extern.md b/.cursor/rules/luau-rfcs/udtf-is-extern.md similarity index 100% rename from .cursor/rules/luau/udtf-is-extern.md rename to .cursor/rules/luau-rfcs/udtf-is-extern.md diff --git a/.cursor/rules/luau/unsealed-table-assign-optional-property.md b/.cursor/rules/luau-rfcs/unsealed-table-assign-optional-property.md similarity index 100% rename from .cursor/rules/luau/unsealed-table-assign-optional-property.md rename to .cursor/rules/luau-rfcs/unsealed-table-assign-optional-property.md diff --git a/.cursor/rules/luau/unsealed-table-literals.md b/.cursor/rules/luau-rfcs/unsealed-table-literals.md similarity index 100% rename from .cursor/rules/luau/unsealed-table-literals.md rename to .cursor/rules/luau-rfcs/unsealed-table-literals.md diff --git a/.cursor/rules/luau/unsealed-table-subtyping-strips-optional-properties.md b/.cursor/rules/luau-rfcs/unsealed-table-subtyping-strips-optional-properties.md similarity index 100% rename from .cursor/rules/luau/unsealed-table-subtyping-strips-optional-properties.md rename to .cursor/rules/luau-rfcs/unsealed-table-subtyping-strips-optional-properties.md diff --git a/.cursor/rules/luau/user-defined-type-functions.md b/.cursor/rules/luau-rfcs/user-defined-type-functions.md similarity index 100% rename from .cursor/rules/luau/user-defined-type-functions.md rename to .cursor/rules/luau-rfcs/user-defined-type-functions.md diff --git a/.cursor/rules/luau/vector-library-vector2-constructor.md b/.cursor/rules/luau-rfcs/vector-library-vector2-constructor.md similarity index 100% rename from .cursor/rules/luau/vector-library-vector2-constructor.md rename to .cursor/rules/luau-rfcs/vector-library-vector2-constructor.md diff --git a/.cursor/rules/luau/vector-library.md b/.cursor/rules/luau-rfcs/vector-library.md similarity index 100% rename from .cursor/rules/luau/vector-library.md rename to .cursor/rules/luau-rfcs/vector-library.md diff --git a/.cursor/rules/luau.mdc b/.cursor/rules/luau.mdc index 8c939c4c..e6712788 100644 --- a/.cursor/rules/luau.mdc +++ b/.cursor/rules/luau.mdc @@ -1,12 +1,15 @@ --- -description: Luau language syntax, type system, single-threaded model, and common gotchas alwaysApply: true --- # Luau Language Rules +**Default context: Roblox Luau.** Unless otherwise stated, assume we are programming in Roblox Studio. Standard Luau differences are noted where relevant. + ## Finding Documentation -Search `.cursor/rules/roblox/luau/` for detailed Luau documentation files. +- `.cursor/rules/roblox/en-us/luau/` - Roblox-specific Luau documentation (types, Roblox classes/enums, Studio integration) +- `.cursor/rules/luau/` - Official Luau language documentation (type system, standard library, syntax, guides) +- `.cursor/rules/luau-rfcs/` - RFC documents detailing every language feature and design decision ## Single-Threaded Execution Model @@ -84,6 +87,19 @@ print(`Result: {1 + 2}`) -- Result: 3 print(`Literal braces: \{not interpolated}`) ``` +### String Literal Escapes + +Beyond standard escapes, Luau supports: + +```luau +"\xAB" -- Hex escape: inserts byte 0xAB +"\u{1F600}" -- Unicode escape: inserts UTF-8 for U+1F600 (braces required) +"long\z + string" -- \z skips following whitespace/newlines: "longstring" +``` + +Backtick strings (interpolated) use `\` to escape `` ` ``, `{`, `\`, and newlines. `{{` is a parse error (use `\{` to get a literal brace). + ### If Expressions (Ternary) ```luau @@ -114,6 +130,18 @@ str ..= "suffix" local half = width // 2 -- More efficient than math.floor(width / 2) ``` +### Number Literals + +Luau has a single number type (64-bit IEEE 754 double, integers exact up to 2^53). + +```luau +local hex = 0xFF -- Hexadecimal +local bin = 0b10101010 -- Binary +local big = 1_048_576 -- Decimal separators for readability +local hexSep = 0xFFFF_FFFF +local binSep = 0b_0101_0101 +``` + ## Type System Essentials ### Type Modes @@ -122,32 +150,70 @@ local half = width // 2 -- More efficient than math.floor(width / 2) - `--!nonstrict`: Reduced checking - `--!nocheck`: Disabled -### Core Types +### Primitive Types + +The Luau VM has 10 primitive types: `nil`, `string`, `number`, `boolean`, `table`, `function`, `thread`, `userdata`, `vector`, `buffer`. ```luau --- Primitives local x: number local s: string local b: boolean -local arr: {string} -- Array -local map: {[string]: number} -- Dictionary +local co: thread = coroutine.running() +local buf: buffer = buffer.create(256) +``` --- Optional -local maybe: string? -- string | nil +Note: `table` and `function` use dedicated syntax (not written by name). `vector` is not representable by name in type annotations. `userdata` is represented by concrete host types. --- Union / Intersection +### `unknown`, `any`, and `never` + +```luau +local u: unknown = anything -- Top type: accepts all values, but cannot be used without refinement +local a: any = anything -- Escape hatch: opts out of type checking entirely +local n: never -- Bottom type: no value inhabits this type +``` + +- `unknown` requires type refinement (e.g., `typeof(x) == "string"`) before use +- `any` silently allows all operations -- use sparingly +- `never` is useful for exhaustiveness checking and impossible branches + +### Core Type Syntax + +```luau +local arr: {string} -- Array shorthand (equivalent to {[number]: string}) +local map: {[string]: number} -- Dictionary with indexer +local obj: { x: number, y: number } -- Table with named properties + +local maybe: string? -- Optional: string | nil + +-- Union / Intersection (leading | and & are allowed for formatting) type Result = "ok" | "error" +type Action + = { type: "add", value: number } + | { type: "remove", id: string } type Combined = TypeA & TypeB --- Negation -type NotNil = ~nil +type NotNil = ~nil -- Negation type --- Singleton (literal types) +-- Singleton (literal) types -- strings and booleans only, not numbers type Direction = "up" | "down" | "left" | "right" +type Truthy = true --- Function types +-- Function types with optional named parameters type Callback = (number, string) -> boolean +type Named = (errorCode: number, errorText: string) -> () type Generic = (T) -> T +type Multi = (number, string) -> (boolean, string) -- Multiple returns +``` + +### Generics with Defaults + +Type aliases support default type parameters (functions do not): + +```luau +type Pair = { first: T, second: T } +type PairWithDefault = Pair + +local p: PairWithDefault = { first = "hello", second = "world" } -- T defaults to string ``` ### Read-Only Properties @@ -183,8 +249,10 @@ end ### Built-in Type Functions ```luau -type Keys = keyof -type PropType = index +type Keys = keyof -- Union of all property names +type RawKeys = rawkeyof -- Keys without __index +type PropType = index -- Property type lookup +type RawProp = rawget -- Property lookup without __index ``` ### Variadic Type Packs @@ -242,18 +310,164 @@ type Vector = typeof(setmetatable({}::{ })) ``` +### Type Refinements + +The type checker narrows types based on control flow: + +```luau +local x: string | number = getValue() + +-- Type guard with type() or typeof() +if type(x) == "string" then + local s: string = x -- ok, narrowed to string +end + +-- Truthy test +local maybe: string? = getMaybe() +if maybe then + local s: string = maybe -- ok, nil ruled out +end + +-- Equality narrowing to singleton +local s: string = getStr() +if s == "hello" then + local h: "hello" = s -- ok, narrowed to literal +end + +-- Compose with and/or/not +if (type(x) == "string" or type(x) == "number") and x ~= nil then + -- x: string | number +end + +-- assert() also refines +assert(type(x) == "string") +local s: string = x -- ok after assert +``` + +### Sealed vs Unsealed Tables + +Tables have different mutability rules depending on how they are created: + +**Unsealed** (table literals without annotation) -- can add new properties until scope exits: + +```luau +local t = { x = 1 } -- unsealed: {x: number} +t.y = 2 -- ok, adds y: {x: number, y: number} +t.z = 3 -- ok + +local u: { x: number, y: number? } = t -- ok, y is optional +``` + +**Sealed** (annotated or returned from function) -- locked, supports width subtyping: + +```luau +local t: { x: number } = { x = 1 } +t.y = 2 -- NOT ok, table is sealed + +type Point2D = { x: number, y: number } +type Point1D = { x: number } +local p: Point2D = { x = 5, y = 37 } +local q: Point1D = p -- ok, Point2D is a subtype (has extra properties) +``` + +Once an unsealed table leaves its defining scope (e.g., returned from a function), it becomes sealed. + +### User-Defined Type Functions + +Type functions run at analysis time and operate on types using the `types` library: + +```luau +type function simple_keyof(ty) + if not ty:is("table") then + error("Can only call keyof on tables.") + end + + local union = nil + for property in ty:properties() do + union = if union then types.unionof(union, property) else property + end + return if union then union else types.singleton(nil) +end + +type Person = { name: string, age: number } +type Keys = simple_keyof -- "age" | "name" +``` + +Available in type function environment: `types` library, `assert`, `error`, `print`, `next`, `pairs`, `ipairs`, `select`, `unpack`, `tostring`, `tonumber`, `type`, `typeof`, `math`, `table`, `string`, `bit32`, `utf8`, `buffer` libraries. + ## Roblox-Specific +For full Roblox Luau documentation, see `.cursor/rules/roblox/en-us/luau/`. + +### Roblox Types in Annotations + +All Roblox classes, data types, and enums are valid type annotations: + +```luau +local part: Part = Instance.new("Part") +local color: BrickColor = part.BrickColor +local material: Enum.Material = part.Material +local pos: Vector3 = part.Position +local cf: CFrame = part.CFrame +``` + ### Use Vector3, Not vector -The `vector` primitive exists but is poorly integrated with Roblox APIs. Always use `Vector3`: +The `vector` primitive exists but is not compatible with Roblox APIs. Always use `Vector3`: ```luau --- Correct local pos = Vector3.new(1, 2, 3) +-- NOT: vector.create(1, 2, 3) +``` --- Avoid (not compatible with most Roblox APIs) --- local pos = vector.create(1, 2, 3) +### Native Code Generation (Server Only) + +In Roblox, `--!native` compiles **server-side scripts** to machine code. There is a limit on total natively compiled code per experience. Use `@native` on individual hot functions to stay within budget. + +## Module Require Semantics + +### Roblox Luau (Default) + +In Roblox, use instance-path requires. This is the preferred style: + +```luau +local ReplicatedStorage = game:GetService("ReplicatedStorage") +local MyModule = require(ReplicatedStorage.Modules.MyModule) +local Types = require(ReplicatedStorage.Shared.Types) + +-- Common pattern: services at top of file +local Players = game:GetService("Players") +local RunService = game:GetService("RunService") +local ServerStorage = game:GetService("ServerStorage") +``` + +### Standalone Luau (String Requires) + +Outside Roblox (e.g., Lune, standalone tooling), requires use explicit string paths with mandatory prefixes: + +```luau +local sibling = require("./sibling") -- Relative to current file +local parent = require("../parent") -- Parent directory +local lib = require("@lune/net") -- Alias (configured in .luaurc) +``` + +All paths must start with `./`, `../`, or `@` -- bare names like `require("foo")` are not allowed. Aliases are configured in `.luaurc`: + +```json +{ + "aliases": { + "lune": "packages/lune", + "shared": "src/shared" + } +} +``` + +### Cyclic Dependencies + +Require paths must be statically resolvable. To break cyclic module dependencies, cast to `any`: + +```luau +local myModule = require(MyModule) :: any ``` ## Deprecated Functions @@ -340,21 +554,43 @@ Note: `@inline` is NOT supported. Compiler handles inlining automatically. ## OOP Pattern -```luau -type Point = { x: number, y: number } +Separate the data type from the class type using `typeof(setmetatable(...))`. Annotate `self` explicitly on methods -- this is **required** for the type checker to share `self` types across methods. +```luau local Point = {} Point.__index = Point -function Point.new(): Point - return setmetatable({ x = 0, y = 0 }, Point) +type PointData = { + x: number, + y: number, +} + +export type Point = typeof(setmetatable({} :: PointData, Point)) + +function Point.new(x: number, y: number): Point + return setmetatable({ x = x, y = y }, Point) end -function Point:getX(): number - return self.x +function Point.move(self: Point, dx: number, dy: number) + self.x += dx + self.y += dy end + +function Point.magnitude(self: Point): number + return math.sqrt(self.x * self.x + self.y * self.y) +end + +local p = Point.new(3, 4) +p:move(1, 2) -- Callers still use : syntax +print(p:magnitude()) -- 5.385... ``` +Key points: +- Define methods with `.` syntax and explicit `self: ClassName` parameter +- Callers still invoke with `:` syntax +- Return type annotation on the constructor ensures compatibility with methods +- The `typeof(setmetatable(...))` trick gives full autocomplete on `self` + ## Performance Tips 1. **Preallocate tables**: `table.create(count)` for known sizes @@ -434,6 +670,52 @@ function handle(x: "a" | "b") end ``` +## Standard Library Additions + +Luau extends Lua 5.1's standard library with many functions. Below are the most important additions not covered elsewhere. + +### Math Library Extensions + +```luau +math.clamp(n, min, max) -- Clamp n to [min, max] range +math.sign(n) -- Returns -1, 0, or 1 +math.round(n) -- Round to nearest integer (halfway rounds away from 0) +math.lerp(a, b, t) -- Linear interpolation: a + (b - a) * t, exact at t=1 +math.map(x, inmin, inmax, outmin, outmax) -- Map x from input range to output range +math.noise(x, y?, z?) -- 3D Perlin noise, returns [-1, 1] +``` + +Also available: `math.log(n, base?)` (optional base), `math.random()` uses PCG RNG. + +### String Library Extensions + +```luau +-- string.split: splits string into array (default separator: ",") +local parts = string.split("a,b,c", ",") -- {"a", "b", "c"} +local chars = string.split("hello", "") -- {"h", "e", "l", "l", "o"} + +-- string.pack/unpack/packsize: binary data encoding (Lua 5.3 format strings) +-- Fixed sizes: short=16bit, long=64bit, int=32bit, size_t=32bit +local packed = string.pack("i4f", 42, 3.14) +local int, float = string.unpack("i4f", packed) +local size = string.packsize("i4f") -- 8 bytes +``` + +### bit32 Library Extensions + +```luau +bit32.countlz(n) -- Count leading zeros (returns 32 if n is 0) +bit32.countrz(n) -- Count trailing zeros (returns 32 if n is 0) +bit32.byteswap(n) -- Swap byte order (endianness) +``` + +### coroutine.close + +```luau +coroutine.close(co) -- Puts suspended/dead coroutine into dead state, frees stack +-- Returns (true) on success, (false, error) if coroutine was in error state +``` + ## Immutable Tables ```luau @@ -446,3 +728,82 @@ local config = table.freeze({ local newConfig = table.clone(config) newConfig.maxRetries = 5 ``` + +## Buffer Library + +Buffers are fixed-size mutable byte arrays. All offsets are 0-based. Reads/writes outside bounds throw errors. Uses little-endian byte order. + +```luau +local buf = buffer.create(1024) -- All bytes initialized to 0 +local buf2 = buffer.fromstring("hello") -- From string contents +local str = buffer.tostring(buf2) -- Back to string +local len = buffer.len(buf) -- Size in bytes + +-- Typed reads/writes (i=signed, u=unsigned, f=float) +buffer.writei32(buf, 0, 42) -- Write signed 32-bit int at offset 0 +local n = buffer.readi32(buf, 0) -- Read it back +buffer.writef64(buf, 4, 3.14159) -- Write 64-bit float at offset 4 +local f = buffer.readf64(buf, 4) + +-- Available: readi8/u8/i16/u16/i32/u32/f32/f64 and matching write variants + +-- String read/write +buffer.writestring(buf, 0, "hello", 5) -- Write 5 bytes from string +local s = buffer.readstring(buf, 0, 5) -- Read 5 bytes as string + +-- Bit-level access +buffer.writebits(buf, 0, 4, 0b1010) -- Write 4 bits at bit offset 0 +local bits = buffer.readbits(buf, 0, 4) -- Read 4 bits (returns unsigned int) + +-- Bulk operations +buffer.copy(target, tOff, source, sOff?, count?) -- Copy between buffers (or within same) +buffer.fill(buf, offset, value, count?) -- Fill bytes with value +``` + +## Vector Library + +The `vector` primitive is a 3-component (or 4-component in 4-wide mode) immutable value type. Components are accessed via `.x`/`.y`/`.z` (and `.w` in 4-wide mode) but cannot be written individually. + +```luau +-- Construction and constants +local v = vector.create(1, 2, 3) +local zero = vector.zero -- (0, 0, 0) +local one = vector.one -- (1, 1, 1) + +-- Arithmetic: +, -, *, /, //, unary - all work component-wise +local sum = v + vector.create(4, 5, 6) + +-- Geometric operations +vector.magnitude(v) -- Length of vector +vector.normalize(v) -- Unit vector +vector.dot(v1, v2) -- Dot product (returns number) +vector.cross(v1, v2) -- Cross product (3D only) +vector.angle(v1, v2, axis?) -- Angle in radians (axis for signed angle) +vector.lerp(v1, v2, t) -- Linear interpolation + +-- Component-wise math +vector.abs(v) -- math.abs per component +vector.sign(v) -- math.sign per component +vector.floor(v) / vector.ceil(v) -- Rounding per component +vector.clamp(v, min, max) -- Clamp per component +vector.max(v1, v2, ...) / vector.min(v1, v2, ...) +``` + +**Note:** In Roblox, prefer `Vector3` over `vector` -- `Vector3` integrates with all Roblox APIs. The `vector` library is for standalone Luau or performance-critical native code. + +## Differences from Lua + +Luau is based on Lua 5.1 with selective features from 5.2-5.4 and its own additions: + +- `_VERSION` is `"Luau"` (not `"Lua 5.1"`) +- Single number type: 64-bit double (no separate integer type) +- No tail call optimization (simplifies debugging and stack traces) +- `__eq` metamethod is always called, even when objects are rawequal +- No `goto` statement +- No bitwise operators (`<<`, `>>`, `&`, `|`, `~`) -- use `bit32` library +- No `__gc` metamethod (sandboxing constraint) +- No `io`, `os.execute`, `package`, or full `debug` library (sandboxing) +- `loadstring` cannot load bytecode, `string.dump` not available +- Table assignment order in literals follows program order (Lua 5.x may reorder) +- `function()` expressions may reuse closures when all captured upvalues are identical +- `os.time` with table argument returns UTC (not local time) diff --git a/.cursor/rules/luau/getting-started/compatibility.md b/.cursor/rules/luau/getting-started/compatibility.md new file mode 100644 index 00000000..b48489bc --- /dev/null +++ b/.cursor/rules/luau/getting-started/compatibility.md @@ -0,0 +1,133 @@ +--- +slug: compatibility +title: Compatibility with Lua +toc: true +--- + +Luau is based on Lua 5.1, and as such incorporates all features of 5.1, except for ones that had to be taken out due to sandboxing limitations. Because of backwards compatibility constraints, we don't remove features deprecated by later versions (e.g. we still support `getfenv`/`setfenv`). Later Lua versions introduce new features into the language and new libraries/functions. + +Our overall goal is to incorporate features from the later versions of Lua when it makes sense for us to do so - the motivations behind some newer features are unclear or don't apply to the domain Luau is used in, and many features carry costs that don't always make sense to pay. The rest of this document describes the status of all features of Lua 5.2 and beyond, with the following classification: + +- ✔️ - the feature is available in Luau +- ❌ - the feature is not available in Luau because we don't believe it makes sense to include it +- 😞 - the feature is not available in Luau because of compatibility/sandboxing concerns +- 🔜 - the feature is not available in Luau yet but we'd like to include it and are possibly working on it +- 🤷‍♀️ - the feature is not available in Luau yet; we don't have strong opinions on it so it might make it at some point + +Please note that all of these decisions are not final, they are just our current stance. In some cases evolution of our VM may make a feature that was previously impractical to support due to performance complications feasible. In some cases a feature that didn't have a strong use case gains one, so we can implement it. + +## Implementation limits + +Luau has certain limitations around the number of local variables, registers, upvalues, constants and instructions. These limits are often different from the limits imposed by various versions of Lua, and are documented here without promising that future versions will adhere to these. Note that writing code that is close to any of these limits is dangerous because this code may become invalid as our codegen evolves. + +- Local variables: 200 per function (same as all versions of Lua, this includes function arguments) +- Upvalues: 200 per function (up from 60 in Lua 5.1) +- Registers: 255 per function (same as all versions of Lua, this includes local variables and function arguments) +- Constants: 2^23 per function (up from 2^18 in Lua 5.1) +- Instructions: 2^23 per function (up from 2^17 in Lua 5.1, although in both cases the limit only applies to control flow) +- Nested functions: 2^15 per function (down from 2^18 in Lua 5.1) +- Stack depth: 20000 Lua calls per Lua thread, 200 C calls per C thread (e.g. `coroutine.resume`/`pcall` nesting is limited to 200) + +Note that Lua 5.3 has a larger upvalue limit (255) and a larger constant limit (2^26); existing Luau limits are likely sufficient for reasonable use cases. + +## Lua 5.1 + +Since several features were removed from Lua 5.1 for sandboxing reasons, this table lists them for completeness. + +| feature | notes | +|---------|------| +| `io`, `os`, `package` and `debug` library | note that some functions in `os`/`debug` are still present | +| `loadfile`, `dofile` | removed for sandboxing, no direct file access | +| `loadstring` bytecode and `string.dump` | exposing bytecode is dangerous for sandboxing reasons | +| `newproxy` can only be called with nil or boolean | extra flexibility removed for sandboxing | + +Sandboxing challenges are [covered in the dedicated section](../sandbox). + +## Lua 5.2 + +| feature | status | notes | +|---------|--------|------| +| yieldable pcall/xpcall | ✔️ | | +| yieldable xpcall error handler | ❌ | no strong use cases, VM complexity and performance implications | +| yieldable metamethods | ❌ | significant performance implications | +| ephemeron tables | ❌ | this complicates and slows down the garbage collector esp. for large weak tables | +| emergency garbage collector | 🤷‍ | Luau runs in environments where handling memory exhaustion in emergency situations is not tenable | +| goto statement | ❌ | this complicates the compiler, makes control flow unstructured and doesn't address a significant need | +| finalizers for tables | ❌ | no `__gc` support due to sandboxing and performance/complexity | +| no more fenv for threads or functions | 😞 | we love this, but it breaks compatibility | +| tables honor the `__len` metamethod | ✔️ | | +| hex and `\z` escapes in strings | ✔️ | | +| support for hexadecimal floats | 🤷‍♀️ | no strong use cases | +| order metamethods (`__lt`/`__le`) are called for unrelated metatables | ❌ | no strong use cases and more complicated semantics, compatibility and performance implications | +| empty statement | 🤷‍♀️ | less useful in Lua than in JS/C#/C/C++ | +| `break` statement may appear in the middle of a block | 🤷‍♀️ | we'd like to do it consistently for `break`/`return`/`continue` but there be dragons | +| arguments for function called through `xpcall` | ✔️ | | +| optional base in `math.log` | ✔️ | | +| optional separator in `string.rep` | 🤷‍♀️ | no strong use cases | +| new metamethods `__pairs` and `__ipairs` | ❌ | superseded by `__iter` | +| frontier patterns | ✔️ | | +| `%g` in patterns | ✔️ | | +| `\0` in patterns | ✔️ | | +| `bit32` library | ✔️ | | +| `string.gsub` is stricter about using `%` on special characters only | ✔️ | | +| light C functions | 😞 | this changes semantics of fenv on C functions and has complex implications wrt runtime performance | +| NaN keys are supported for tables with `__newindex` | ✔️ | | + +Two things that are important to call out here are various new metamethods for tables and yielding in metamethods. In both cases, there are performance implications to supporting this - our implementation is *very* highly tuned for performance, so any changes that affect the core fundamentals of how Lua works have a price. To support yielding in metamethods we'd need to make the core of the VM more involved, since almost every single "interesting" opcode would need to learn how to be resumable - which also complicates future JIT/AOT story. Metamethods in general are important for extensibility, but very challenging to deal with in implementation, so we err on the side of not supporting any new metamethods unless a strong need arises. + +For `__pairs`/`__ipairs`, we felt that extending library functions to enable custom containers wasn't the right choice. Instead we revisited iteration design to allow for self-iterating objects via `__iter` metamethod, which results in a cleaner iteration design that also makes it easier to iterate over tables. As such, we have no plans to support `__pairs`/`__ipairs` as all use cases for it can now be solved by `__iter`. + +Ephemeron tables may be implemented at some point since they do have valid uses and they make weak tables semantically cleaner, however the cleanup mechanism for these is expensive and complicated, and as such this can only be considered after the pending GC rework is complete. + +## Lua 5.3 + +| feature | status | notes | +|---------|--------|------| +| `\u` escapes in strings | ✔️ | | +| integers (64-bit by default) | ❌ | backwards compatibility and performance implications | +| bitwise operators | ❌ | `bit32` library covers this in absence of 64-bit integers | +| basic utf-8 support | ✔️ | we include `utf8` library and other UTF8 features | +| functions for packing and unpacking values (string.pack/unpack/packsize) | ✔️ | | +| floor division | ✔️ | | +| `ipairs` and the `table` library respect metamethods | ❌ | no strong use cases, performance implications | +| new function `table.move` | ✔️ | | +| `collectgarbage("count")` now returns only one result | ✔️ | | +| `coroutine.isyieldable` | ✔️ | | +| stricter error checking for `table.insert`/`table.remove` | 😞 | we love this, but it breaks compatibility | +| `__eq` metamethod is called for unrelated metatables | ❌ | backwards compatibility and typechecking implications | + +It's important to highlight integer support and bitwise operators. For Luau, it's rare that a full 64-bit integer type is necessary - double-precision types support integers up to 2^53 (in Lua which is used in embedded space, integers may be more appealing in environments without a native 64-bit FPU). However, there's a *lot* of value in having a single number type, both from performance perspective and for consistency. Notably, Lua doesn't handle integer overflow properly, so using integers also carries compatibility implications. + +If integers are taken out of the equation, bitwise operators make less sense, as integers aren't a first class feature; additionally, `bit32` library is more fully featured (includes commonly used operations such as rotates and arithmetic shift; bit extraction/replacement is also more readable). Adding operators along with metamethods for all of them increases complexity, which means this feature isn't worth it on the balance. Common arguments for this include a more familiar syntax, which, while true, gets more nuanced as `^` isn't available as a xor operator, and arithmetic right shift isn't expressible without yet another operator, and performance, which in Luau is substantially better than in Lua because `bit32` library uses VM builtins instead of expensive function calls. + +## Lua 5.4 + +| feature | status | notes | +|--|--|--| +| new generational mode for garbage collection | 🔜 | we're working on gc optimizations and generational mode is on our radar +| to-be-closed variables | ❌ | the syntax is inconsistent with how we'd like to do attributes long-term; no strong use cases in our domain | +| const variables | ❌ | while there's some demand for const variables, we'd never adopt this syntax | +| new implementation for math.random | ✔️ | our RNG is based on PCG, unlike Lua 5.4 which uses Xoroshiro | +| optional `init` argument to `string.gmatch` | 🤷‍♀️ | no strong use cases | +| new functions `lua_resetthread` and `coroutine.close` | ✔️ || +| coercions string-to-number moved to the string library | 😞 | we love this, but it breaks compatibility | +| new format `%p` in `string.format` | 🤷‍♀️ | no strong use cases | +| `utf8` library accepts codepoints up to 2^31 | 🤷‍♀️ | no strong use cases | +| The use of the `__lt` metamethod to emulate `__le` has been removed | ❌ | breaks compatibility and complicates comparison overloading story | +| When finalizing objects, Lua will call `__gc` metamethods that are not functions | ❌ | no `__gc` support due to sandboxing and performance/complexity | +| function `print` calls `__tostring` instead of `tostring` to format its arguments | ✔️ | | +| decoding functions in the utf8 library do not accept surrogates | ✔️ | | + +Taking syntax aside (which doesn't feel idiomatic or beautiful), `` isn't very useful in Luau - its dominant use case is for code that works with external resources like files or sockets, but we don't provide such APIs - and has a very large complexity cost, evidences by a lot of bug fixes since the initial implementation in 5.4 work versions. `` in Luau doesn't matter for performance - our multi-pass compiler is already able to analyze the usage of the variable to know if it's modified or not and extract all performance gains from it - so the only use here is for code readability, where the `` syntax is... suboptimal. + +If we do end up introducing const variables, it would be through a `const var = value` syntax, which is backwards compatible through a context-sensitive keyword similar to `type`. That said, there's ambiguity wrt whether `const` should simply behave like a read-only variable, ala JavaScript, or if it should represent a stronger contract, for example by limiting the expressions on the right hand side to ones compiler can evaluate ahead of time, or by freezing table values and thus guaranteeing immutability. + +## Differences from Lua + +We have a few behavior deviations from Lua 5.x that come from either a different implementation, or our desire to clean up small inconsistencies in the language/libraries: + +* Tail calls are not supported to simplify implementation, make debugging/stack traces more predictable and allow deep validation of caller identity for security +* Order of table assignment in table literals follows program order in mixed tables (Lua 5.x assigns array elements first in some cases) +* Equality comparisons call `__eq` metamethod even when objects are rawequal (which matches other metamethods like `<=` and facilitates NaN checking) +* `function()` expressions may reuse a previously created closure in certain scenarios (when all upvalues captured are the same) for efficiency, which changes object identity but doesn't change call semantics -- this is different from Lua 5.1 but similar to Lua 5.2/5.3 +* `os.time` returns UTC timestamp when called with a table for consistency diff --git a/.cursor/rules/luau/getting-started/intro.md b/.cursor/rules/luau/getting-started/intro.md new file mode 100644 index 00000000..6e713cb0 --- /dev/null +++ b/.cursor/rules/luau/getting-started/intro.md @@ -0,0 +1,147 @@ +--- +slug: getting-started +title: An introduction to Luau +sidebar: + order: 1 +--- + +Luau is a fast, small, safe, gradually typed embeddable scripting language derived from Lua 5.1. Luau ships as a command line tool for running, analyzing, and linting your Luau scripts, and is also integrated with Roblox Studio. Roblox developers should also visit our [Creator Docs Luau Section](https://create.roblox.com/docs/luau). + +To get started with Luau, you can use the `luau` command line binary to run your code and `luau-analyze` to run static analysis (including type checking and linting). You can download these from [a recent release](https://github.com/luau-lang/luau/releases). + +## Creating a script + +To create your own testing script, create a new file with `.luau` as the extension: + +```luau +--!hidden mode=nonstrict +function ispositive(x) + return x > 0 +end + +print(ispositive(1)) +print(ispositive("2")) + +function isfoo(a) + return a == "foo" +end + +print(isfoo("bar")) +print(isfoo(1)) +``` + +You can now run the file using `luau test.luau` and analyze it using `luau-analyze test.luau`. + +Note that there are no warnings about calling ``ispositive()`` with a string, or calling ``isfoo()`` with a number. This is because Luau's type checking uses non-strict mode by default, which only reports errors if it's certain a program will error at runtime. + +## Type checking + +Now modify the script to include ``--!strict`` at the top: + +```luau +--!strict + +function ispositive(x) + return x > 0 +end + +print(ispositive(1)) +print(ispositive("2")) +``` + +We just mentioned ``nonstrict`` mode, where we only report errors if we can prove that a program will error at runtime. We also have ``strict`` mode, which will report errors if a program might error at runtime. + +In this case, Luau will use the ``return x > 0`` statement to infer that ``ispositive()`` is a function taking a number and returning a boolean. Note that in this case, we were able to determine the type of `ispositive` without the presence of any explicit type annotations. + +Based on Luau's type inference, the analysis tool will now flag the incorrect call to ``ispositive()``: + +``` +$ luau-analyze test.luau +test.luau(7,18): TypeError: Type 'string' could not be converted into 'number' +``` + +## Annotations + +You can add annotations to locals, arguments, and function return types. Among other things, annotations can help enforce that you don't accidentally do something silly. Here's how we would add annotations to ``ispositive()``: + +```luau +--!strict + +function ispositive(x : number) : boolean + return x > 0 +end + +local result : boolean +result = ispositive(1) + +``` + +Now we've explicitly told Luau that ``ispositive()`` accepts a number and returns a boolean. This wasn't strictly (pun intended) necessary in this case, because Luau's inference was able to deduce this already. But even in this case, there are advantages to explicit annotations. Imagine that later we decide to change ``ispositive()`` to return a string value: + +```luau +--!strict + +function ispositive(x : number) : boolean + if x > 0 then + return "yes" + else + return "no" + end +end + +local result : boolean +result = ispositive(1) +``` + +Oops -- we're returning string values, but we forgot to update the function return type. Since we've told Luau that ``ispositive()`` returns a boolean (and that's how we're using it), the call site isn't flagged as an error. But because the annotation doesn't match our code, we get a warning in the function body itself: + +``` +$ luau-analyze test.luau +test.luau(5,9): TypeError: Type 'string' could not be converted into 'boolean' +test.luau(7,9): TypeError: Type 'string' could not be converted into 'boolean' +``` + +The fix is simple; just change the annotation to declare the return type as a string: + +```luau +--!strict + +function ispositive(x : number) : string + if x > 0 then + return "yes" + else + return "no" + end +end + +local result : boolean +result = ispositive(1) +``` + +Well, almost - since we declared ``result`` as a boolean, the call site is now flagged: + +``` +$ luau-analyze test.luau +test.luau(12,10): TypeError: Type 'string' could not be converted into 'boolean' +``` + +If we update the type of the local variable, everything is good. Note that we could also just let Luau infer the type of ``result`` by changing it to the single line version ``local result = ispositive(1)``. + +```luau +--!strict + +function ispositive(x : number) : string + if x > 0 then + return "yes" + else + return "no" + end +end + +local result : string +result = ispositive(1) +``` + +## Conclusions + +This has been a brief tour of the basic functionality of Luau, but there's lots more to explore. If you're interested in reading more, check out our main reference pages for [syntax](/syntax) and [typechecking](/types). diff --git a/.cursor/rules/luau/getting-started/lint.md b/.cursor/rules/luau/getting-started/lint.md new file mode 100644 index 00000000..c9cfcbc8 --- /dev/null +++ b/.cursor/rules/luau/getting-started/lint.md @@ -0,0 +1,387 @@ +--- +slug: lint +title: Luau's linter +sidebar: + order: 4 +--- + +Luau comes with a set of linting passes, that help make sure that the code is correct and consistent. Unlike the type checker, that models the behavior of the code thoroughly and points toward type mismatches that are likely to result in runtime errors, the linter is more opinionated and produces warnings that can often be safely ignored, although it's recommended to keep the code clean of the warnings. + +Linter produces many different types of warnings; many of these are enabled by default, and can be suppressed by declaring `--!nolint NAME` at the top of the file. In dire situations `--!nolint` at the top of the file can be used to completely disable all warnings (note that the type checker is still active, and requires a separate `--!nocheck` declaration). + +The rest of this page documents all warnings produced by the linter; each warning has a name and a numeric code, the latter is used when displaying warnings. Note that the in-browser version of Luau on this website does not currently use the linter. + +## UnknownGlobal (1) + +By default, variables in Luau are global (this is inherited from Lua 5.x and can't be changed because of backwards compatibility). This means that typos in identifiers are invisible to the parser, and often break at runtime. For this reason, the linter considers all globals that aren't part of the builtin global table and aren't explicitly defined in the script "unknown": + +```luau +--!hidden mode=nocheck +local displayName = "Roblox" + +-- Unknown global 'displaName' +print(displaName) +``` + +Note that injecting globals via `setfenv` can produce this warning in correct code; global injection is incompatible with type checking and has performance implications so we recommend against it and in favor of using `require` with correctly scoped identifiers. + +## DeprecatedGlobal (2) + +Some global names exist for compatibility but their use is discouraged. This mostly affects globals introduced by Roblox, and since they can have problematic behavior or can break in the future, this warning highlights their uses: + +```luau +--!hidden mode=nocheck +-- Global 'ypcall' is deprecated, use 'pcall' instead +ypcall(function() + print("hello") +end) +``` + +## GlobalUsedAsLocal (3) + +The UnknownGlobal lint can catch typos in globals that are read, but can't catch them in globals that are assigned to. Because of this, and to discourage the use of globals in general, linter detects cases when a global is only used in one function and can be safely converted to a local variable. Note that in some cases this requires declaring the local variable in the beginning of the function instead of where it's being assigned to. + +```luau +--!hidden mode=nocheck +local function testFunc(a) + if a < 5 then + -- Global 'b' is only used in the enclosing function; consider changing it to local + b = 1 + else + b = 2 + end + print(b) +end +``` + +## LocalShadow (4) + +In Luau, it is valid to shadow locals and globals with a local variable, including doing it in the same function. This can result in subtle bugs, since the shadowing may not be obvious to the reader. This warning detects cases where local variables shadow other local variables in the same function, or global variables used in the script. + +```luau +--!hidden mode=nocheck +local function foo() + for i=1,10 do + -- Variable 'i' shadows previous declaration at line 2 + for i=1,10 do + print(i) + end + end +end +``` + +## SameLineStatement (5) + +Luau doesn't require the use of semicolons and doesn't automatically insert them at line breaks. When used wisely this results in code that is easy to read and understand, however it can cause subtle issues and hard to understand code when abused by using many different statements on the same line. This warning highlights cases where code should either be broken into multiple lines, or use `;` as a visual guide. + +```luau +--!hidden mode=nocheck +-- A new statement is on the same line; add semi-colon on previous statement to silence +if b < 0 then local a = b + 1 print(a, b) end +``` + +## MultiLineStatement (6) + +An opposite problem is having statements that span multiple lines. This is good for readability when the code is indented properly, but when it's not it results in code that's hard to understand, as its easy to confuse the next line for a separate statement. + +```luau +--!hidden mode=nocheck +-- Statement spans multiple lines; use indentation to silence +print(math.max(1, +math.min(2, 3))) +``` + +## LocalUnused (7) + +This warning is one of the few warnings that highlight unused variables. Local variable declarations that aren't used may indicate a bug in the code (for example, there could be a typo in the code that uses the wrong variable) or redundant code that is no longer necessary (for example, calling a function to get its result and never using this result). This warning warns about locals that aren't used; if the locals are not used intentionally they can be prefixed with `_` to silence the warning: + +```luau +--!hidden mode=nocheck +local x = 1 +-- Variable 'y' is never used; prefix with '_' to silence +local y = 2 +print(x, x) +``` + +## FunctionUnused (8) + +While unused local variables could be useful for debugging, unused functions usually contain dead code that was meant to be removed. Unused functions clutter code, can be a result of typos similar to local variables, and can mislead the reader into thinking that some functionality is supported. + +```luau +--!hidden mode=nocheck +-- Function 'bar' is never used; prefix with '_' to silence +local function bar() +end + +local function foo() +end + +return foo() +``` + +## ImportUnused (9) + +In Luau, there's no first-class module system that's part of the syntax, but `require` function acts as an import statement. When a local is initialized with a `require` result, and the local is unused, this is classified as "unused import". Removing unused imports improves code quality because it makes it obvious what the dependencies of the code are: + +```luau +--!hidden mode=nocheck +-- Import 'Roact' is never used; prefix with '_' to silence +local Roact = require("@game/Packages/Roact") +``` + +## BuiltinGlobalWrite (10) + +While the sandboxing model of Luau prevents overwriting built-in globals such as `table` for the entire program, it's still valid to reassign these globals - this results in "global shadowing", where the script's global table contains a custom version of `table` after writing to it. This is problematic because it disables some optimizations, and can result in misleading code. When shadowing built-in globals, use locals instead. + +```luau +-- Built-in global 'math' is overwritten here; consider using a local or changing the name +math = {} +``` + +## PlaceholderRead (11) + +`_` variable name is commonly used as a placeholder to discard function results. The linter follows this convention and doesn't warn about the use of `_` in various cases where a different name would cause a warning otherwise. To make sure the placeholder is only used to write values to it, this warning detects the cases where it's read instead: + +```luau +local _ = 5 +-- Placeholder value '_' is read here; consider using a named variable +return _ +``` + +## UnreachableCode (12) + +In some cases the linter can detect code that is never executed, because all execution paths through the function exit the function or the loop before reaching it. Such code is misleading because it's not always obvious to the reader that it never runs, and as such it should be removed. + +```luau +function cbrt(v) + if v >= 0 then + return v ^ (1/3) + else + error('cbrt expects a non-negative argument') + end + -- Unreachable code (previous statement always returns) + return 0 +end +``` + +## UnknownType (13) + +Luau provides several functions to get the value type as a string (`type`, `typeof`), and some Roblox APIs expose class names through string arguments (`Instance.new`). This warning detects incorrect use of the type names by checking the string literals used in type comparisons and function calls. + +```luau +--!hidden mode=nocheck +-- Unknown type 'String' (expected primitive type) +if type(v) == "String" then + print("v is a string") +end +``` + +## ForRange (14) + +When using a numeric for, it's possible to make various mistakes when specifying the for bounds. For example, to iterate through the table backwards, it's important to specify the negative step size. This warning detects several cases where the numeric for only runs for 0 or 1 iterations, or when the step doesn't divide the size of the range evenly. + +```luau +--!hidden mode=nocheck +-- For loop should iterate backwards; did you forget to specify -1 as step? +for i=#t,1 do +end +``` + +## UnbalancedAssignment (15) + +Assignment statements and local variable declarations in Luau support multiple variables on the left side and multiple values on the right side. The number of values doesn't need to match; when the right side has more values, the extra values are discarded, and then the left side has more variables the extra variables are set to `nil`. However, this can result in subtle bugs where a value is omitted mistakenly. This warning warns about cases like this; if the last expression on the right hand side returns multiple values, the warning is not emitted. + +```luau +-- Assigning 2 values to 3 variables initializes extra variables with nil; add 'nil' to value list to silence +local x, y, z = 1, 2 +``` + +## ImplicitReturn (16) + +In Luau, there's a subtle difference between returning no values from a function and returning `nil`. In many contexts these are equivalent, but when the results are passed to a variadic function (perhaps implicitly), the difference can be observed - for example, `print(foo())` prints nothing if `foo` returns no values, and `nil` if it returns `nil`. + +To help write code that has consistent behavior, linter warns about cases when a function implicitly returns no values, if there are cases when it explicitly returns a result. For code like this it's recommended to use explicit `return` or `return nil` at the end of the function (these have different semantics, so the correct version to use depends on the function): + +```luau +--!hidden mode=nocheck +local function find(t, expected) + for k,v in pairs(t) do + if k == expected then + return v + end + end + -- Function 'find' can implicitly return no values even though there's an explicit return at line 4; add explicit return to silence +end +``` + +## DuplicateLocal (17) + +Luau syntax allows to use the same name for different parameters of a function as well as different local variables declared in the same statement. This is error prone, even if it's occasionally useful, so a warning is emitted in cases like this, unless the duplicate name is the placeholder `_`: + +```luau +--!hidden mode=nocheck +function foo(a, b, a) -- Function parameter 'a' already defined on column 14 +end + +function obj:method(self) -- Function parameter 'self' already defined implicitly +end + +local x, y, x = v:GetComponents() -- Variable 'x' already defined on column 7 +``` + +## FormatString (18) + +Luau has several library functions that expect a format string that specifies the behavior for the function. These format strings follow a specific syntax that depends on the question; mistakes in these strings can lead to runtime errors or unexpected behavior of the code. + +To help make sure that the strings used for these functions are correct, linter checks calls to `string.format`, `string.pack`, `string.packsize`, `string.unpack`, `string.match`, `string.gmatch`, `string.find`, `string.gsub` and `os.date` and issues warnings when the call uses a literal string with an incorrect format: + +```luau +--!hidden mode=nocheck +-- Invalid match pattern: invalid capture reference, must refer to a closed capture +local cap = string.match(s, "(%d)%2") + +-- Invalid format string: unfinished format specifier +local str = ("%d %"):format(1, 2) +``` + + Note that with the exception of `string.format` this only works when the function is called via the library, not via the method call (so prefer `string.match(s, "pattern")` to `s:match("pattern")`). + +## TableLiteral (19) + +Table literals are often used in Luau to create objects or specify data. The syntax for table literals is rich but invites mistakes, for example it allows duplicate keys or redundant index specification for values already present in the list form. This warning is emitted for cases where some entries in the table literal are ignored at runtime as they're duplicate: + +```luau +print({ + first = 1, + second = 2, + first = 3, -- Table field 'first' is a duplicate; previously defined at line 2 +}) +``` + +## UninitializedLocal (20) + +It's easy to forget to initialize a local variable and then proceed to use it regardless. This can happen due to a typo, or sometimes it can happen because the original variable definition is shadowed. When a local variable doesn't have an initial value and is used without ever being assigned to, this warning is emitted: + +```luau +local foo + +if foo then -- Variable 'foo' defined at line 1 is never initialized or assigned; initialize with 'nil' to silence + print(foo) +end +``` + +## DuplicateFunction (21) + +This warning is emitted when a script defines two functions with the same name in the same scope. + +The warning is not produced when the functions are defined in different scopes because this is much more likely to be intentional. + +```luau +--!hidden mode=nocheck +function foo() end +function foo() end -- Duplicate function definition: 'foo' also defined on line 1 + +-- OK: the functions are not defined in the same scope. +if x then + function bar() end +else + function bar() end +end +``` + +## DeprecatedApi (22) + +This warning is emitted when a script accesses a method or field that is marked as deprecated. Use of deprecated APIs is discouraged since they may have performance or correctness issues, may result in unexpected behavior, and could be removed in the future. + +```luau +--!hidden mode=nocheck +function getSize(i: Instance) + return i.DataCost -- Member 'Instance.DataCost' is deprecated +end +``` + +## TableOperations (23) + +To manipulate array-like tables, Luau provides a set of functions as part of the standard `table` library. To help use these functions correctly, this warning is emitted when one of these functions is used incorrectly or can be adjusted to be more performant. + +```luau +local t = {} + +table.insert(t, 0, 42) -- table.insert uses index 0 but arrays are 1-based; did you mean 1 instead? +table.insert(t, #t+1, 42) -- table.insert will append the value to the table; consider removing the second argument for efficiency +``` + +In addition, when type information is present, this warning will be emitted when `#` or `ipairs` is used on a table that has no numeric keys or indexers. This helps avoid common bugs like using `#t == 0` to check if a dictionary is empty. + +```luau +local message = { data = { 1, 2, 3 } } + +if #message == 0 then -- Using '#' on a table without an array part is likely a bug +end +``` + +## DuplicateCondition (24) + +When checking multiple conditions via `and/or` or `if/elseif`, a copy & paste error may result in checking the same condition redundantly. This almost always indicates a bug, so a warning is emitted when use of a duplicate condition is detected. + +```luau +--!hidden mode=nocheck +assert(self._adorns[normID1] and self._adorns[normID1]) -- Condition has already been checked on column 8 +``` + +## MisleadingAndOr (25) + +In Lua, there is no first-class ternary operator but it can be emulated via `a and b or c` pattern. However, due to how boolean evaluation works, if `b` is `false` or `nil`, the resulting expression evaluates to `c` regardless of the value of `a`. Luau solves this problem with the `if a then b else c` expression; a warning is emitted for and-or expressions where the first alternative is `false` or `nil` because it's almost always a bug. + +```luau +--!hidden mode=nocheck +-- The and-or expression always evaluates to the second alternative because the first alternative is false; consider using if-then-else expression instead +local x = flag and false or true +``` + +The code above can be rewritten as follows to avoid the warning and the associated bug: + +```luau +--!hidden mode=nocheck +local x = if flag then false else true +``` + +## CommentDirective (26) + +Luau uses comments that start from `!` to control certain aspects of analysis, for example setting type checking mode via `--!strict` or disabling individual lints with `--!nolint`. Unknown directives are ignored, for example `--!nostrict` doesn't have any effect on the type checking process as the correct spelling is `--!nonstrict`. This warning flags comment directives that are ignored during processing: + +```luau +--!nostrict +-- Unknown comment directive 'nostrict'; did you mean 'nonstrict'?" +``` + +## IntegerParsing (27) + +Luau parses hexadecimal and binary literals as 64-bit integers before converting them to Luau numbers. As a result, numbers that exceed 2^64 are silently truncated to 2^64, which can result in unexpected program behavior. This warning flags literals that are truncated: + +```luau +-- Hexadecimal number literal exceeded available precision and was truncated to 2^64 +local x = 0x1111111111111111111111111111111111111 +``` + +Luau numbers are represented as double precision IEEE754 floating point numbers; they can represent integers up to 2^53 exactly, but larger integers may lose precision. This warning also flags literals that are parsed with a precision loss: + +```luau +-- Number literal exceeded available precision and was truncated to closest representable number +local x = 9007199254740993 +``` + +## ComparisonPrecedence (28) + +Because of operator precedence rules, not X == Y parses as (not X) == Y; however, often the intent was to invert the result of the comparison. This warning flags erroneous conditions like that, as well as flagging cases where two comparisons happen in a row without any parentheses: + +```luau +--!hidden mode=nocheck +-- not X == Y is equivalent to (not X) == Y; consider using X ~= Y, or wrap one of the expressions in parentheses to silence +if not x == 5 then +end + +-- X <= Y <= Z is equivalent to (X <= Y) <= Z; wrap one of the expressions in parentheses to silence +if 1 <= x <= 3 then +end +``` diff --git a/.cursor/rules/luau/getting-started/syntax.md b/.cursor/rules/luau/getting-started/syntax.md new file mode 100644 index 00000000..75019cac --- /dev/null +++ b/.cursor/rules/luau/getting-started/syntax.md @@ -0,0 +1,318 @@ +--- +slug: syntax +title: Luau syntax by example +description: An informal reference for Luau's syntax by example. +sidebar: + order: 3 +--- + +Luau uses the baseline [syntax of Lua 5.1](https://www.lua.org/manual/5.1/manual.html#2). For detailed documentation, please refer to the Lua manual, this is an example: + +```luau +local function tree_insert(tree, x) + local lower, equal, greater = split(tree.root, x) + if not equal then + equal = { + x = x, + y = math.random(0, 2^31-1), + left = nil, + right = nil + } + end + tree.root = merge3(lower, equal, greater) +end +``` + +Note that future versions of Lua extend the Lua 5.1 syntax with more features; Luau does support string literal extensions but does not support other 5.x additions; for details please refer to [compatibility section](../compatibility). + +The rest of this document documents additional syntax used in Luau. + +## String literals + +Luau implements support for hexadecimal (`\x`), Unicode (`\u`) and `\z` escapes for string literals. This syntax follows [Lua 5.3 syntax](https://www.lua.org/manual/5.3/manual.html#3.1): + +- `\xAB` inserts a character with the code 0xAB into the string +- `\u{ABC}` inserts a UTF8 byte sequence that encodes U+0ABC character into the string (note that braces are mandatory) +- `\z` at the end of the line inside a string literal ignores all following whitespace including newlines, which can be helpful for breaking long literals into multiple lines. + +## Number literals + +In addition to basic integer and floating-point decimal numbers, Luau supports: + +- Hexadecimal integer literals, `0xABC` or `0XABC` +- Binary integer literals, `0b01010101` or `0B01010101` +- Decimal separators in all integer literals, using `_` for readability: `1_048_576`, `0xFFFF_FFFF`, `0b_0101_0101` + +Note that Luau only has a single number type, a 64-bit IEEE754 double precision number (which can represent integers up to 2^53 exactly), and larger integer literals are stored with precision loss. + +## Continue statement + +In addition to `break` in all loops, Luau supports `continue` statement. Similar to `break`, `continue` must be the last statement in the block. + +Note that unlike `break`, `continue` is not a keyword. This is required to preserve backwards compatibility with existing code; so this is a `continue` statement: + +```luau +--!hidden mode=nocheck +if x < 0 then + continue +end +``` + +Whereas this is a function call: + +```luau +--!hidden mode=nocheck +if x < 0 then + continue() +end +``` + +When used in `repeat..until` loops, `continue` can not skip the declaration of a local variable if that local variable is used in the loop condition; code like this is invalid and won't compile: + +```luau +repeat + do continue end + local a = 5 +until a > 0 +``` + +## Compound assignments + +Luau supports compound assignments with the following operators: `+=`, `-=`, `*=`, `/=`, `//=`, `%=`, `^=`, `..=`. Just like regular assignments, compound assignments are statements, not expressions: + +```luau +local a = 5 + +-- this works +a += 1 + +-- this doesn't work +print(a += 1) +``` + +Compound assignments only support a single value on the left and right hand side; additionally, the function calls on the left hand side are only evaluated once: + +```luau +function foo() return 2 end +local a = { 1, 2, 3 } + +-- calls foo() twice +a[foo()] = a[foo()] + 1 + +-- calls foo() once +a[foo()] += 1 +``` + +Compound assignments call the arithmetic metamethods (`__add` et al) and table indexing metamethods (`__index` and `__newindex`) as needed - for custom types no extra effort is necessary to support them. + +## Type annotations + +To support gradual typing, Luau supports optional type annotations for variables and functions, as well as declaring type aliases. + +Types can be declared for local variables, function arguments and function return types using `:` as a separator: + +```luau +function foo(x: number, y: string): boolean + local k: string = y:rep(x) + return k == "a" +end +``` + +In addition, the type of any expression can be overridden using a type cast `::`: + +```luau +function foo(x: number, y: unknown): boolean + local k = (y :: string):rep(x) + return k == "a" +end +``` + +There are several simple builtin types: `any` (represents inability of the type checker to reason about the type), `nil`, `boolean`, `number`, `string` and `thread`. + +Function types are specified using the arguments and return types, separated with `->`: + +```luau +local foo: (number, string) -> boolean +``` + +To return no values or more than one, you need to wrap the return type position with parentheses, and then list your types there. + +```luau +local no_returns: (number, string) -> () +local returns_boolean_and_string: (number, string) -> (boolean, string) + +function foo(x: number, y: number): (number, string) + return x + y, tostring(x) .. tostring(y) +end +``` + +Note that function types are specified without the argument names in the examples above, but it's also possible to specify the names (that are not semantically significant but can show up in documentation and autocomplete): + +```luau +local callback: (errorCode: number, errorText: string) -> () +``` + +Table types are specified using the table literal syntax, using `:` to separate keys from values: + +```luau +local array: { [number] : string } +local object: { x: number, y: string } +``` + +When the table consists of values keyed by numbers, it's called an array-like table and has a special short-hand syntax, `{T}` (e.g. `{string}`). + +Additionally, the type syntax supports type intersections (`((number) -> string) & ((boolean) -> string)`) and unions (`(number | boolean) -> string`). An intersection represents a type with values that conform to both sides at the same time, which is useful for overloaded functions; a union represents a type that can store values of either type - `any` is technically a union of all possible types. + +It's common in Lua for function arguments or other values to store either a value of a given type or `nil`; this is represented as a union (`number | nil`), but can be specified using `?` as a shorthand syntax (`number?`). + +In addition to declaring types for a given value, Luau supports declaring type aliases via `type` syntax: + +```luau +type Point = { x: number, y: number } +type Array = { [number]: T } +type Something = typeof(string.gmatch("", "%d")) +``` + +The right hand side of the type alias can be a type definition or a `typeof` expression; `typeof` expression doesn't evaluate its argument at runtime. + +By default type aliases are local to the file they are declared in. To be able to use type aliases in other modules using `require`, they need to be exported: + +```luau +export type Point = { x: number, y: number } +``` + +An exported type can be used in another module by prefixing its name with the require alias that you used to import the module. + +```luau +--!file main.luau +local M = require("./other") + +local a: M.Point = {x=5, y=6} + +--!file other.luau +export type Point = { x: number, y: number } + +return {} +``` + +For more information please refer to [typechecking documentation](../types). + +## If-then-else expressions + +In addition to supporting standard if *statements*, Luau adds support for if *expressions*. Syntactically, `if-then-else` expressions look very similar to if statements. However instead of conditionally executing blocks of code, if expressions conditionally evaluate expressions and return the value produced as a result. Also, unlike if statements, if expressions do not terminate with the `end` keyword. + +Here is a simple example of an `if-then-else` expression: +```luau +local a, b = 6, 7 +local maxValue = if a > b then a else b +``` + +`if-then-else` expressions may occur in any place a regular expression is used. The `if-then-else` expression must match `if then else `; it can also contain an arbitrary number of `elseif` clauses, like `if then elseif then else `. Note that in either case, `else` is mandatory. + +Here's is an example demonstrating `elseif`: +```luau +local x = -21 +local sign = if x < 0 then -1 elseif x > 0 then 1 else 0 +``` + +**Note:** In Luau, the `if-then-else` expression is preferred vs the standard Lua idiom of writing `a and b or c` (which roughly simulates a ternary operator). However, the Lua idiom may return an unexpected result if `b` evaluates to false. The `if-then-else` expression will behave as expected in all situations. + +## Generalized iteration + +Luau uses the standard Lua syntax for iterating through containers, `for vars in values`, but extends the semantics with support for generalized iteration. In Lua, to iterate over a table you need to use an iterator like `next` or a function that returns one like `pairs` or `ipairs`. In Luau, you can simply iterate over a table: + +```luau +for k, v in {1, 4, 9} do + assert(k * k == v) +end +``` + +Further, iteration can be extended for tables or userdata by implementing the `__iter` metamethod which is called before the iteration begins, and should return an iterator function like `next` (or a custom one): + +```luau +local obj = { items = {1, 4, 9} } +setmetatable(obj, { __iter = function(o) return next, o.items end }) + +for k, v in obj do + assert(k * k == v) +end +``` + +The default iteration order for tables is specified to be consecutive for elements `1..#t` and unordered after that, visiting every element; similarly to iteration using `pairs`, modifying the table entries for keys other than the current one results in unspecified behavior. + +## String interpolation + +Luau adds an additional way to define string values that allows you to place runtime expressions directly inside specific spots of the literal. + +This is a more ergonomic alternative over using `string.format` or `("literal"):format`. + +To use string interpolation, use a backtick string literal: + +```luau +local count = 3 +print(`Bob has {count} apple(s)!`) +--> Bob has 3 apple(s)! +``` + +Any expression can be used inside `{}`: + +```luau +local combos = {2, 7, 1, 8, 5} +print(`The lock combination is {table.concat(combos)}.`) +--> The lock combination is 27185. +``` + +Inside backtick string literal, `\` is used to escape `` ` ``, `{`, `\` itself and a newline: + +```luau +print(`Some example escaping the braces \{like so}`) +--> Some example escaping the braces {like so} + +print(`Backslash \ that escapes the space is not a part of the string...`) +--> Backslash that escapes the space is not a part of the string... + +print(`Backslash \\ will escape the second backslash...`) +--> Backslash \ will escape the second backslash... + +print(`Some text that also includes \`...`) +--> Some text that also includes `... + +local name = "Luau" + +print(`Welcome to { + name +}!`) +--> Welcome to Luau! +``` + +### Restrictions and limitations + +The sequence of two opening braces {{"`{{`"}} is rejected with a parse error. +This restriction is made to prevent developers using other programming languages with a similar feature from trying to attempt that as a way to escape a single `{` and getting unexpected results in Luau. + +Luau currently does not support backtick string literals in type annotations, therefore `` type Foo = `Foo` `` is invalid syntax. + +Unlike single and double-quoted string literals, backtick string literals must always be wrapped in parentheses for function calls: +```luau +print "hello" -- valid +print`hello` -- invalid syntax +print(`hello`) -- valid +``` + +## Floor division (`//`) + +Luau supports floor division, including its operator (`//`), its compound assignment operator (`//=`), and overloading metamethod (`__idiv`), as an ergonomic alternative to `math.floor`. + +For numbers, `a // b` is equal to `math.floor(a / b)`: +```luau +local n = 6.72 +print(n // 2) --> 3 +n //= 3 +print(n) --> 2 +``` + +Note that it's possible to get `inf`, `-inf`, or `NaN` with floor division; when `b` is `0`, `a // b` results in positive or negative infinity, and when both `a` and `b` are `0`, `a // b` results in NaN. + +For native vectors, `c // d` applies `math.floor` to each component of the vector `c`. Therefore `c // d` is equivalent to `vector.create(math.floor(c.x / d), math.floor(c.y / b), math.floor(c.z / b))`. + +Floor division syntax and semantics follow from [Lua 5.3](https://www.lua.org/manual/5.3/manual.html#3.4.1) where applicable. diff --git a/.cursor/rules/luau/getting-started/why.md b/.cursor/rules/luau/getting-started/why.md new file mode 100644 index 00000000..752d7828 --- /dev/null +++ b/.cursor/rules/luau/getting-started/why.md @@ -0,0 +1,28 @@ +--- +slug: why +title: Why Luau? +sidebar: + order: 2 +--- + +Around 2006, [Roblox](https://www.roblox.com) started using Lua 5.1 as a scripting language for games. Over the years the runtime had to be tweaked to provide a safe, secure sandboxed environment; we gradually started accumulating small library changes and tweaks. + +Over the course of the last few years, instead of using Web-based stack for our player-facing application, Lua-based in-game UI and Qt-based editor UI, we've started consolidating a lot of the efforts and developing all of these using Roblox engine and Lua as a scripting language. + +Having grown a substantial internal codebase that needed to be correct and performant, and with the focus shifting a bit from novice game developers to professional studios building games on Roblox and our own teams of engineers building applications, there was a need to improve performance and quality of the code we were writing. + +Unlike mainline Lua, we also could not afford to do major breaking changes to the language (hence the 5.1 language baseline that remained unchanged for more than a decade). While faster implementations of Lua 5.1 like LuaJIT were available, they didn't meet our needs in terms of portability, ease of change and they didn't address the problem of developing robust code at scale. + +All of these motivated us to start reshaping Lua 5.1 that we started from into a new, derivative language that we call Luau. Our focus is on making the language more performant and feature-rich, and make it easier to write robust code through a combination of linting and type checking using a gradual type system. + +## Complete rewrite? + +A very large part of Luau codebase is written from scratch. We needed a set of tools to be able to write language analysis tools; Lua has a parser that is integrated with the bytecode compiler, which makes it unsuitable for complex semantic analysis. For bytecode compilation, while a single pass compiler can deliver better compilation throughput and be simpler than a full frontend/backend, it significantly limits the optimizations that can be done at the bytecode level. + +Luau compiler and analysis tools are thus written from scratch, closely following the syntax and semantics of Lua. Our compiler is not single-pass, and instead relies on a set of analysis passes that run over the AST to produce efficient bytecode, followed by some post-process optimizations. + +As for the runtime, we had to rewrite the interpreter from scratch to get substantially faster performance; using a combination of techniques pioneered by LuaJIT and custom optimizations that are able to improve performance by taking control over the entire stack (language, compiler, interpreter, virtual machine), we're able to get close to LuaJIT interpreter performance while using C as an implementation language. + +The garbage collector and the core libraries represent more of an incremental change, where we used Lua 5.1 as a baseline but we're continuing to rewrite these as well with performance in mind. + +While Luau initially didn't include a JIT/AOT, native code generation has since been implemented for x64/arm64 platform as an optional component. diff --git a/.cursor/rules/luau/guides/performance.md b/.cursor/rules/luau/guides/performance.md new file mode 100644 index 00000000..76c86a48 --- /dev/null +++ b/.cursor/rules/luau/guides/performance.md @@ -0,0 +1,212 @@ +--- +slug: performance +title: How we make Luau fast +sidebar: + order: 1 +--- + +One of main goals of Luau is to enable high performance code, with gameplay code being the main use case. This can be viewed as two separate goals: + +- Make idiomatic code that wasn't tuned faster +- Enable even higher performance through careful tuning + +Both of these goals are important - it's insufficient to just focus on the highly tuned code, and all things being equal we prefer to raise all boats by implementing general optimizations. However, in some cases it's important to be aware of optimizations that Luau does and doesn't do. + +Worth noting is that Luau is focused on, first and foremost, stable high performance code in interpreted context. This is because JIT compilation is not available on many platforms Luau runs on, and AOT compilation would only work for code that Roblox ships (and even that does not always work). This is in stark contrast with LuaJIT that, while providing an excellent interpreter as well, focuses a lot of the attention on JIT (with many optimizations unavailable in the interpreter). + +Having said that, Luau has been updated to include an optional JIT component for x64 and arm64 platforms. This component can compile a selected set of functions, including limiting compilation to functions or modules marked explicitly by the user. While functions can be compiled at any time, automated JIT compilation decisions based on statistics/tracing are not performed. Luau JIT takes into account the type annotations present in the source code to specialize code paths and at this time, doesn't include runtime analysis of the types/values flowing through the program. + +The rest of this document goes into some optimizations that Luau employs and how to best leverage them when writing code. The document is not complete - a lot of optimizations are transparent to the user and involve detailed low-level tuning of various parts that is not described here - and all of this is subject to change without notice, as it doesn't affect the semantics of valid code. + +## Fast bytecode interpreter + +Luau features a very highly tuned portable bytecode interpreter. It's similar to Lua interpreter in that it's written in C, but it's highly tuned to yield efficient assembly when compiled with Clang and latest versions of MSVC. On some workloads it can match the performance of LuaJIT interpreter which is written in highly specialized assembly. We are continuing to tune the interpreter and the bytecode format over time; while extra performance can be extracted by rewriting the interpreter in assembly, we're unlikely to ever do that as the extra gains at this point are marginal, and we gain a lot from C in terms of portability and being able to quickly implement new optimizations. + +Of course the interpreter isn't typical C code - it uses many tricks to achieve extreme levels of performance and to coerce the compiler to produce efficient assembly. Due to a better bytecode design and more efficient dispatch loop it's noticeably faster than Lua 5.x (including Lua 5.4 which made some of the changes similar to Luau, but doesn't come close). The bytecode design was partially inspired by excellent LuaJIT interpreter. Most computationally intensive scripts only use the interpreter core loop and builtins, which on x64 compiles into \~16 KB, thus leaving half of the instruction cache for other infrequently called code. + +## Optimizing compiler + +Unlike Lua and LuaJIT, Luau uses a multi-pass compiler with a frontend that parses source into an AST and a backend that generates bytecode from it. This carries a small penalty in terms of compilation time, but results in more flexible code and, crucially, makes it easier to optimize the generated bytecode. + +> Note: Compilation throughput isn't the main focus in Luau, but our compiler is reasonably fast; with all currently implemented optimizations enabled, it compiles 950K lines of Luau code in 1 second on a single core of a desktop Ryzen 5900X CPU, producing bytecode and debug information. + +While bytecode optimizations are limited due to the flexibility of Luau code (e.g. `a * 1` may not be equivalent to `a` if `*` is overloaded through metatables), even in absence of type information Luau compiler can perform some optimizations such as "deep" constant folding across functions and local variables, perform upvalue optimizations for upvalues that aren't mutated, do analysis of builtin function usage, optimize the instruction sequences for multiple variable assignments, and some peephole optimizations on the resulting bytecode. The compiler can also be instructed to use more aggressive optimizations by enabling optimization level 2 (`-O2` in CLI tools), some of which are documented further on this page. + +Most bytecode optimizations are performed on individual statements or functions, however the compiler also does a limited amount of interprocedural optimizations; notably, calls to local functions can be optimized with the knowledge of the argument count or number of return values involved. Interprocedural optimizations are limited to a single module due to the compilation model. + +Luau compiler is also able to use type information to do further optimizations. Because we control the entire stack (unlike e.g. TypeScript where the type information is discarded completely before reaching the VM), we have more flexibility there and can make some tradeoffs during codegen even if the type system isn't completely sound. For example, it might be reasonable to assume that in presence of known types, we can infer absence of side effects for arithmetic operations and builtins - if the runtime types mismatch due to intentional violation of the type safety through global injection, the code will still be safely sandboxed. Type information is currently limited to small peephole optimizations, but it has a potential to unlock optimizations such as common subexpression elimination and allocation hoisting in the future, without having to rely on a JIT. These future optimizations opportunities are speculative pending further research. + +## Epsilon-overhead debugger + +It's important for Luau to have stable and predictable performance. Something that comes up in Lua-based environments often is the use of line hooks to implement debugging (both for breakpoints and for stepping). This is problematic because the support for hooks is typically not free in general, but importantly once the hook is enabled, calling the hook has a considerable overhead, and the hook itself may be very costly to evaluate since it will need to associate the script:line pair with the breakpoint information. + +Luau does not support hooks at all, and relies on first-class support for breakpoints (using bytecode patching) and single-stepping (using a custom interpreter loop) to implement debugging. As a result, the presence of breakpoints doesn't slow the script execution down - the only noticeable discrepancy between running code under a debugger and without a debugger should be in cases where breakpoints are evaluated and skipped based on breakpoint conditions, or when stepping over long-running fragments of code. + +## Inline caching for table and global access + +Table access for field lookup is optimized in Luau using a mechanism that blends inline caching (classically used in Java/JavaScript VMs) and HREFs (implemented in LuaJIT). Compiler can predict the hash slot used by field lookup, and the VM can correct this prediction dynamically. + +As a result, field access can be very fast in Luau, provided that: + +- The field name is known at compile time. To make sure this is the case, `table.field` notation is recommended, although the compiler will also optimize `table["field"]` when the expression is known to be a constant string. +- The field access doesn't use metatables. The fastest way to work with tables in Luau is to store fields directly inside the table, and store methods in the metatable (see below); access to "static" fields in classic OOP designs is best done through `Class.StaticField` instead of `object.StaticField`. +- The object structure is usually uniform. While it's possible to use the same function to access tables of different shape - e.g. `function getX(obj) return obj.x end` can be used on any table that has a field `"x"` - it's best to not vary the keys used in the tables too much, as it defeats this optimization. + +The same optimization is applied to the custom globals declared in the script, although it's best to avoid these altogether by using locals instead. Still, this means that the difference between `function` and `local function` is less pronounced in Luau. + +## Importing global access chains + +While global access for library functions can be optimized in a similar way, this optimization breaks down when the global table is using sandboxing through metatables, and even when globals aren't sandboxed, `math.max` still requires two table accesses. + +It's always possible to "localize" the global accesses by using `local max = math.max`, but this is cumbersome - in practice it's easy to forget to apply this optimization. To avoid relying on programmers remembering to do this, Luau implements a special optimization called "imports", where most global chains such as `math.max` are resolved when the script is loaded instead of when the script is executed. + +This optimization relies on being able to predict the shape of the environment table for a given function; this is possible due to global sandboxing, however this optimization is invalid in some cases: + +- `loadstring` can load additional code that runs in context of the caller's environment +- `getfenv`/`setfenv` can directly modify the environment of any function + +The use of any of these functions performs a dynamic deoptimization, marking the affected environment as "impure". The optimizations are only in effect on functions with "pure" environments - because of this, the use of `loadstring`/`getfenv`/`setfenv` is not recommended. Note that `getfenv` deoptimizes the environment even if it's only used to read values from the environment. + +> Note: Luau still supports these functions as part of our backwards compatibility promise, although we'd love to switch to Lua 5.2's `_ENV` as that mechanism is cleaner and doesn't require costly dynamic deoptimization. + +## Fast method calls + +Luau specializes method calls to improve their performance through a combination of compiler, VM and binding optimizations. Compiler emits a specialized instruction sequence when methods are called through `obj:Method` syntax (while this isn't idiomatic anyway, you should avoid `obj.Method(obj)`). When the object in question is a Lua table, VM performs some voodoo magic based on inline caching to try to quickly discover the implementation of this method through the metatable. + +For this to be effective, it's crucial that `__index` in a metatable points to a table directly. For performance reasons it's strongly recommended to avoid `__index` functions as well as deep `__index` chains; an ideal object in Luau is a table with a metatable that points to itself through `__index`. + +When the object in question is a reflected userdata, a special mechanism called "namecall" is used to minimize the interop cost. In classical Lua binding model, `obj:Method` is called in two steps, retrieving the function object (`obj.Method`) and calling it; both steps are often implemented in C++, and the method retrieval needs to use a method object cache - all of this makes method calls slow. + +Luau can directly call the method by name using the "namecall" extension, and an optimized reflection layer can retrieve the correct method quickly through more voodoo magic based on string interning and custom Luau features that aren't exposed through Luau scripts. + +As a result of both optimizations, common Lua tricks of caching the method in a local variable aren't very productive in Luau and aren't recommended either. + +## Specialized builtin function calls + +Due to global sandboxing and the ability to dynamically deoptimize code running in impure environments, in pure environments we go beyond optimizing the interpreter and optimize many built-in functions through a "fastcall" mechanism. + +For this mechanism to work, function call must be "obvious" to the compiler - it needs to call a builtin function directly, e.g. `math.max(x, 1)`, although it also works if the function is "localized" (`local max = math.max`); this mechanism doesn't work for indirect function calls unless they were inlined during compilation, and doesn't work for method calls (so calling `string.byte` is more efficient than `s:byte`). + +The mechanism works by directly invoking a highly specialized and optimized implementation of a builtin function from the interpreter core loop without setting up a stack frame and omitting other work; additionally, some fastcall specializations are partial in that they don't support all types of arguments, for example all `math` library builtins are only specialized for numeric arguments, so calling `math.abs` with a string argument will fall back to the slower implementation that will do string->number coercion. + +As a result, builtin calls are very fast in Luau - they are still slightly slower than core instructions such as arithmetic operations, but only slightly so. The set of fastcall builtins is slowly expanding over time and as of this writing contains `assert`, `type`, `typeof`, `rawget`/`rawset`/`rawequal`, `getmetatable`/`setmetatable`, `tonumber`/`tostring`, all functions from `math` (except `noise` and `random`/`randomseed`) and `bit32`, and some functions from `string` and `table` library. + +Some builtin functions have partial specializations that reduce the cost of the common case further. Notably: + +- `assert` is specialized for cases when the assertion return value is not used and the condition is truthy; this helps reduce the runtime cost of assertions to the extent possible +- `bit32.extract` is optimized further when field and width selectors are constant +- `select` is optimized when the second argument is `...`; in particular, `select(x, ...)` is O(1) when using the builtin dispatch mechanism even though it's normally O(N) in variadic argument count. + +Some functions from `math` library like `math.floor` can additionally take advantage of advanced SIMD instruction sets like SSE4.1 when available. + +In addition to runtime optimizations for builtin calls, many builtin calls, as well as constants like `math.pi`/`math.huge`, can also be constant-folded by the bytecode compiler when using aggressive optimizations (level 2); this currently applies to most builtin calls with constant arguments and a single return value. For builtin calls that can not be constant folded, compiler assumes knowledge of argument/return count (level 2) to produce more efficient bytecode instructions. + +## Optimized table iteration + +Luau implements a fully generic iteration protocol; however, for iteration through tables in addition to generalized iteration (`for .. in t`) it recognizes three common idioms (`for .. in ipairs(t)`, `for .. in pairs(t)` and `for .. in next, t`) and emits specialized bytecode that is carefully optimized using custom internal iterators. + +As a result, iteration through tables typically doesn't result in function calls for every iteration; the performance of iteration using generalized iteration, `pairs` and `ipairs` is comparable, so generalized iteration (without the use of `pairs`/`ipairs`) is recommended unless the code needs to be compatible with vanilla Lua or the specific semantics of `ipairs` (which stops at the first `nil` element) is required. Additionally, using generalized iteration avoids calling `pairs` when the loop starts which can be noticeable when the table is very short. + +Iterating through array-like tables using `for i=1,#t` tends to be slightly slower because of extra cost incurred when reading elements from the table. + +## Optimized table length + +Luau tables use a hybrid array/hash storage, like in Lua; in some sense "arrays" don't truly exist and are an internal optimization, but some operations, notably `#t` and functions that depend on it, like `table.insert`, are defined by the Luau/Lua language to allow internal optimizations. Luau takes advantage of that fact. + +Unlike Lua, Luau guarantees that the element at index `#t` is stored in the array part of the table. This can accelerate various table operations that use indices limited by `#t`, and this makes `#t` worst-case complexity O(logN), unlike Lua where the worst case complexity is O(N). This also accelerates computation of this value for small tables like `{ [1] = 1 }` since we never need to look at the hash part. + +The "default" implementation of `#t` in both Lua and Luau is a binary search. Luau uses a special branch-free (depending on the compiler...) implementation of the binary search which results in 50+% faster computation of table length when it needs to be computed from scratch. + +Additionally, Luau can cache the length of the table and adjust it following operations like `table.insert`/`table.remove`; this means that in practice, `#t` is almost always a constant time operation. + +## Creating and modifying tables + +Luau implements several optimizations for table creation. When creating object-like tables, it's recommended to use table literals (`{ ... }`) and to specify all table fields in the literal in one go instead of assigning fields later; this triggers an optimization inspired by LuaJIT's "table templates" and results in higher performance when creating objects. When creating array-like tables, if the maximum size of the table is known up front, it's recommended to use `table.create` function which can create an empty table with preallocated storage, and optionally fill it with a given value. + +When the exact table shape isn't known, Luau compiler can still predict the table capacity required in case the table is initialized with an empty literal (`{}`) and filled with fields subsequently. For example, the following code creates a correctly sized table implicitly: + +```luau +local v = {} +v.x = 1 +v.y = 2 +v.z = 3 +return v +``` + +When appending elements to tables, it's recommended to use `table.insert` (which is the fastest method to append an element to a table if the table size is not known). In cases when a table is filled sequentially, however, it can be more efficient to use a known index for insertion - together with preallocating tables using `table.create` this can result in much faster code, for example this is the fastest way to build a table of squares: + +```luau +local t = table.create(N) + +for i=1,N do + t[i] = i * i +end +``` + +## Native vector math + +Luau uses tagged value storage - each value contains a type tag and the data that represents the value of a given type. Because of the need to store 64-bit double precision numbers *and* 64-bit pointers, we don't use NaN tagging and have to pay the cost of 16 bytes per value. + +We take advantage of this to provide a native value type that can store a 32-bit floating point vector with 3 components. This type is fundamental to game computations and as such it's important to optimize the storage and the operations with that type - our VM implements first class support for all math operations and component manipulation, which essentially means we have native 3-wide SIMD support. For code that uses many vector values this results in significantly smaller GC pressure and significantly faster execution, and gives programmers a mechanism to hand-vectorize numeric code if need be. + +## Optimized upvalue storage + +Lua implements upvalues as garbage collected objects that can point directly at the thread's stack or, when the value leaves the stack frame (and is "closed"), store the value inside the object. This representation is necessary when upvalues are mutated, but inefficient when they aren't - and 90% or more of upvalues aren't mutated in typical Lua code. Luau takes advantage of this by reworking upvalue storage to prioritize immutable upvalues - capturing upvalues that don't change doesn't require extra allocations or upvalue closing, resulting in faster closure allocation, faster execution, faster garbage collection and faster upvalue access due to better memory locality. + +Note that "immutable" in this case only refers to the variable itself - if the variable isn't assigned to it can be captured by value, even if it's a table that has its contents change. + +When upvalues are mutable, they do require an extra allocated object; we carefully optimize the memory consumption and access cost for mutable upvalues to reduce the associated overhead. + +## Closure caching + +With optimized upvalue storage, creating new closures (function objects) is more efficient but still requires allocating a new object every time. This can be problematic for cases when functions are passed to algorithms like `table.sort` or functions like `pcall`, as it results in excessive allocation traffic which then leads to more work for garbage collector. + +To make closure creation cheaper, Luau compiler implements closure caching - when multiple executions of the same function expression are guaranteed to result in the function object that is semantically identical, the compiler may cache the closure and always return the same object. This changes the function identity which may affect code that uses function objects as table keys, but preserves the calling semantics - compiler will only do this if calling the original (cached) function behaves the same way as a newly created function would. The heuristics used for this optimization are subject to change; currently, the compiler will cache closures that have no upvalues, or all upvalues are immutable (see previous section) and are declared at the module scope, as the module scope is (almost always) evaluated only once. + +## Fast memory allocator + +Similarly to LuaJIT, but unlike vanilla Lua, Luau implements a custom allocator that is highly specialized and tuned to the common allocation workloads we see. The allocator design is inspired by classic pool allocators as well as the excellent `mimalloc`, but through careful domain-specific tuning it beats all general purpose allocators we've tested, including `rpmalloc`, `mimalloc`, `jemalloc`, `ptmalloc` and `tcmalloc`. + +This doesn't mean that memory allocation in Luau is free - it's carefully optimized, but it still carries a cost, and a high rate of allocations requires more work from the garbage collector. The garbage collector is incremental, so short of some edge cases this rarely results in visible GC pauses, but can impact the throughput since scripts will interrupt to perform "GC assists" (helping clean up the garbage). Thus for high performance Luau code it's recommended to avoid allocating memory in tight loops, by avoiding temporary table and userdata creation. + +In addition to a fast allocator, all frequently used structures in Luau have been optimized for memory consumption, especially on 64-bit platforms, compared to Lua 5.1 baseline. This helps to reduce heap memory footprint and improve performance in some cases by reducing the memory bandwidth impact of garbage collection. + +## Optimized libraries + +While the best performing code in Luau spends most of the time in the interpreter, performance of the standard library functions is critical to some applications. In addition to specializing many small and simple functions using the builtin call mechanism, we spend extra care on optimizing all library functions and providing additional functions beyond the Lua standard library that help achieve good performance with idiomatic code. + +Functions from the `table` library like `insert`, `remove` and `move` have been tuned for performance on array-like tables, achieving 3x and more performance compared to un-tuned versions, and Luau provides additional functions like `table.create` and `table.find` to achieve further speedup when applicable. Our implementation of `table.sort` is using `introsort` algorithm which results in guaranteed worst case `NlogN` complexity regardless of the input, and, together with the array-like specializations, helps achieve ~4x speedup on average. + +For `string` library, we use a carefully tuned dynamic string buffer implementation; it is optimized for smaller strings to reduce garbage created during string manipulation, and for larger strings it allows to produce a large string without extra copies, especially in cases where the resulting size is known ahead of time. Additionally, functions like `format` have been tuned to avoid the overhead of `sprintf` where possible, resulting in further speedups. + +## Improved garbage collector pacing + +Luau uses an incremental garbage collector which does a little bit of work every so often, and at no point does it stop the world to traverse the entire heap. The runtime will make sure that the collector runs interspersed with the program execution as the program allocates additional memory, which is known as "garbage collection assists", and can also run in response to explicit garbage collection invocation via `lua_gc`. In interactive environments such as video game engines it's possible, and even desirable, to request garbage collection every frame to make sure assists are minimized, since that allows scheduling the garbage collection to run concurrently with other engine processing that doesn't involve script execution. + +Inspired by excellent work by Austin Clements on Go's garbage collector pacer, we've implemented a pacing algorithm that uses a proportional–integral–derivative controller to estimate internal garbage collector tunables to reach a target heap size, defined as a percentage of the live heap data (which is more intuitive and actionable than Lua 5.x "GC pause" setting). Luau runtime also estimates the allocation rate making it easy (given uniform allocation rates) to adjust the per-frame garbage collection requests to do most of the required GC work outside of script execution. + +## Reduced garbage collector pauses + +While Luau uses an incremental garbage collector, once per each collector cycle it runs a so-called "atomic" step. While all other GC steps can do very little work by only looking at a few objects at a given time, which means that the collector can have arbitrarily short pauses, the "atomic" step needs to traverse some amount of data that, in some cases, may scale with the application heap. Since atomic step is indivisible, it can result in occasional pauses on the order of tens of milliseconds, which is problematic for interactive applications. We've implemented a series of optimizations to help reduce the atomic step. + +Normally objects that have been modified after the GC marked them in an incremental mark phase need to be rescanned during atomic phase, so frequent modifications of existing tables may result in a slow atomic step. To address this, we run a "remark" step where we traverse objects that have been modified after being marked once more (incrementally); additionally, the write barrier that triggers for object modifications changes the transition logic during remark phase to reduce the probability that the object will need to be rescanned. + +Another source of scalability challenges is coroutines. Writes to coroutine stacks don't use a write barrier, since that's prohibitively expensive as they are too frequent. This means that coroutine stacks need to be traversed during atomic step, so applications with many coroutines suffer large atomic pauses. To address this, we implement incremental marking of coroutines: marking a coroutine makes it "inactive" and resuming a coroutine (or pushing extra objects on the coroutine stack via C API) makes it "active". Atomic step only needs to traverse active coroutines again, which reduces the cost of atomic step by effectively making coroutine collection incremental as well. + +While large tables can be a problem for incremental GC in general since currently marking a single object is indivisible, large weak tables are a unique challenge because they also need to be processed during atomic phase, and the main use case for weak tables - object caches - may result in tables with large capacity but few live objects in long-running applications that exhibit bursts of activity. To address this, weak tables in Luau can be marked as "shrinkable" by including `s` as part of `__mode` string, which results in weak tables being resized to the optimal capacity during GC. This option may result in missing keys during table iteration if the table is resized while iteration is in progress and as such is only recommended for use in specific circumstances. + +## Optimized garbage collector sweeping + +The incremental garbage collector in Luau runs three phases for each cycle: mark, atomic and sweep. Mark incrementally traverses all live objects, atomic finishes various operations that need to happen without mutator intervention (see previous section), and sweep traverses all objects in the heap, reclaiming memory used by dead objects and performing minor fixup for live objects. While objects allocated during the mark phase are traversed in the same cycle and thus may get reclaimed, objects allocated during the sweep phase are considered live. Because of this, the faster the sweep phase completes, the less garbage will accumulate; and, of course, the less time sweeping takes the less overhead there is from this phase of garbage collection on the process. + +Since sweeping traverses the whole heap, we maximize the efficiency of this traversal by allocating garbage-collected objects of the same size in 16 KB pages, and traversing each page at a time, which is otherwise known as a paged sweeper. This ensures good locality of reference as consecutively swept objects are contiguous in memory, and allows us to spend no memory for each object on sweep-related data or allocation metadata, since paged sweeper doesn't need to be able to free objects without knowing which page they are in. Compared to linked list based sweeping that Lua/LuaJIT implement, paged sweeper is 2-3x faster, and saves 16 bytes per object on 64-bit platforms. + +## Function inlining and loop unrolling + +By default, the bytecode compiler performs a series of optimizations that result in faster execution of the code, but they preserve both execution semantics and debuggability. For example, a function call is compiled as a function call, which may be observable via `debug.traceback`; a loop is compiled as a loop, which may be observable via `lua_getlocal`. To help improve performance in cases where these restrictions can be relaxed, the bytecode compiler implements additional optimizations when optimization level 2 is enabled (which requires using `-O2` switch when using Luau CLI), namely function inlining and loop unrolling. + +Only loops with loop bounds known at compile time, such as `for i=1,4 do`, can be unrolled. The loop body must be simple enough for the optimization to be profitable; compiler uses heuristics to estimate the performance benefit and automatically decide if unrolling should be performed. + +Only local functions (defined either as `local function foo` or `local foo = function`) can be inlined. The function body must be simple enough for the optimization to be profitable; compiler uses heuristics to estimate the performance benefit and automatically decide if each call to the function should be inlined instead. Additionally recursive invocations of a function can’t be inlined at this time, and inlining is completely disabled for modules that use `getfenv`/`setfenv` functions. + +In both cases, in addition to removing the overhead associated with function calls or loop iteration, these optimizations can additionally benefit by enabling additional optimizations, such as constant folding of expressions dependent on loop iteration variable or constant function arguments, or using more efficient instructions for certain expressions when the inputs to these instructions are constants. diff --git a/.cursor/rules/luau/guides/profile.md b/.cursor/rules/luau/guides/profile.md new file mode 100644 index 00000000..3d69ecdb --- /dev/null +++ b/.cursor/rules/luau/guides/profile.md @@ -0,0 +1,60 @@ +--- +permalink: /profile +title: Profiling your Luau code +sidebar: + order: 3 +--- + +One of main goals of Luau is to enable high performance code. To help with that goal, we are relentlessly optimizing the compiler and runtime - but ultimately, performance of their +code is in developers' hands, and is a combination of good algorithm design and implementation that adheres to the strengths of the language. To help write efficient code, Luau +provides a built-in profiler that samples the execution of the program and outputs a profiler dump that can be converted to an interactive flamegraph. + +To run the profiler, make sure you have an optimized build of the interpreter (otherwise profiling results are going to be very skewed) and run it with `--profile` argument: + +``` +$ luau --profile tests/chess.lua +OK 8902 rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1 +OK 2039 r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 0 +OK 2812 8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 0 +OK 9467 r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1 +OK 1486 rnbq1k1r/pp1Pbppp/2p5/8/2B5/8/PPP1NnPP/RNBQK2R w KQ - 1 8 +OK 2079 r4rk1/1pp1qppp/p1np1n2/2b1p1B1/2B1P1b1/P1NP1N2/1PP1QPPP/R4RK1 w - - 0 10 +Profiler dump written to profile.out (total runtime 2.034 seconds, 20344 samples, 374 stacks) +GC: 0.378 seconds (18.58%), mark 46.80%, remark 3.33%, atomic 1.93%, sweepstring 6.77%, sweep 41.16% +``` + +The resulting `profile.out` file can be converted to an SVG file by running `perfgraph.py` script that is part of Luau repository: + +``` +$ python tools/perfgraph.py profile.out >profile.svg +``` + +This produces an SVG file that can be opened in a browser (the image below is clickable): + +[![profile.svg](/images/chess-profile.svg)](/images/chess-profile.svg) + +In a flame graph visualization, the individual bars represent function calls, the width represents how much of the total program runtime they execute, and the nesting matches the call stack encountered during program execution. This is a fantastic visualization technique that allows you to hone in on the specific bottlenecks affecting +your program performance, optimize those exact bottlenecks, and then re-generate the profile data and visualizer, and look for the next set of true bottlenecks (if any). + +Hovering your mouse cursor over individual sections will display detailed function information in the status bar and in a tooltip. If you want to Search for a specific named +function, use the Search field in the upper right, or press Ctrl+F. + +Notice that some of the bars in the screenshot don't have any text. In some cases, there isn't enough room in the size of the bar to display the name. +You can hover your mouse over those bars to see the name and source location of the function in the tool tip, or double-click to zoom in on that part of the flame graph. + +Some tooltips will have a source location for the function you're hovering over, but no name. Those are anonymous functions, or functions that were not declared in a way that +allows Luau compiler to track the name. To fill in more names, you may want to make these changes to your code: + +`local myFunc = function() --[[ work ]] end` -> `local function myFunc() --[[ work ]] end` + +Even without these changes, you can hover over a given bar with no visible name and see it's source location. + +As any sampling profiler, this profiler relies on gathering enough information for the resulting output to be statistically meaningful. It may miss short functions if they +aren't called often enough. By default the profiler runs at 10 kHz, this can be customized by passing a different parameter to `--profile=`. Note that higher +frequencies result in higher profiling overhead and longer program execution, potentially skewing the results. + +This profiler doesn't track leaf C functions and instead attributes the time spent there to calling Luau functions. As a result, when thinking about why a given function is +slow, consider not just the work it does immediately but also the library functions it calls. + +This profiler tracks time consumed by Luau thread stacks; when a thread calls another thread via `coroutine.resume`, the time spent is not attributed to the parent thread that's +waiting for resume results. This limitation will be removed in the future. diff --git a/.cursor/rules/luau/guides/sandbox.md b/.cursor/rules/luau/guides/sandbox.md new file mode 100644 index 00000000..86734050 --- /dev/null +++ b/.cursor/rules/luau/guides/sandbox.md @@ -0,0 +1,77 @@ +--- +slug: sandbox +title: Embedding a sandboxed Luau virtual machine +sidebar: + order: 2 +--- + +Luau is safe to embed. Broadly speaking, this means that even in the face of untrusted (and in Roblox case, actively malicious) code, the language and the standard library don't allow unsafe access to the underlying system, and don't have known bugs that allow escaping out of the sandbox (e.g. to gain native code execution through ROP gadgets et al). Additionally, the VM provides extra features to implement isolation of privileged code from unprivileged code and protect one from the other; this is important if the embedding environment decides to expose some APIs that may not be safe to call from untrusted code, for example because they do provide controlled access to the underlying system or risk PII exposure through fingerprinting etc. + +This safety is achieved through a combination of removing features from the standard library that are unsafe, adding features to the VM that make it possible to implement sandboxing and isolation, and making sure the implementation is safe from memory safety issues using fuzzing. + +Of course, since the entire stack is implemented in C++, the sandboxing isn't formally proven - in theory, compiler or the standard library can have exploitable vulnerabilities. In practice these are very rare and usually found and fixed quickly. While implementing the stack in a safer language such as Rust would make it easier to provide these guarantees, to our knowledge (based on prior art) this would make it difficult to reach the level of performance required. + +## Library + +Parts of the Lua 5.x standard library are unsafe. Some of the functions provide access to the host operating system, including process execution and file reads. Some functions lack sufficient memory safety checks. Some functions are safe if all code is untrusted, but can break the isolation barrier between trusted and untrusted code. + +The following libraries and global functions have been removed as a result: + +- `io.` library has been removed entirely, as it gives access to files and allows running processes +- `package.` library has been removed entirely, as it gives access to files and allows loading native modules +- `os.` library has been cleaned up from file and environment access functions (`execute`, `exit`, etc.). The only supported functions in the library are `clock`, `date`, `difftime` and `time`. +- `debug.` library has been removed to a large extent, as it has functions that aren't memory safe and other functions break isolation; the only supported functions are `traceback` and `info` (which is similar to `debug.getinfo` but has a slightly different interface). +- `dofile` and `loadfile` allowed access to file system and have been removed. + +To achieve memory safety, access to function bytecode has been removed. Bytecode is hard to validate and using untrusted bytecode may lead to exploits. Thus, `loadstring` doesn't work with bytecode inputs, and `string.dump`/`load` have been removed as they aren't necessary anymore. When embedding Luau, bytecode should be encrypted/signed to prevent MITM attacks as well, as the VM assumes that the bytecode was generated by the Luau compiler (which never produces invalid/unsafe bytecode). + +Finally, to make isolation possible within the same VM, the following global functions have reduced functionality: + +- `collectgarbage` only works with `"count"` argument, as modifying the state of GC can interfere with the expectations of other code running in the process. As such, `collectgarbage()` became an inferior version of `gcinfo()` and is deprecated. +- `newproxy` only works with `true`/`false`/`nil` arguments. +- `module` allowed overriding global packages and was removed as a result. + +> Note: `getfenv`/`setfenv` result in additional isolation challenges, as they allow injecting globals into scripts on the call stack. Ideally, these should be disabled as well, but unfortunately Roblox community relies on these for various reasons. This can be mitigated by limiting interaction between trusted and untrusted code, and/or using separate VMs. + +## Environment + +The modification to the library functions are sufficient to make embedding safe, but aren't sufficient to provide isolation within the same VM. It should be noted that to achieve guaranteed isolation, it's advisable to load trusted and untrusted code into separate VMs; however, even within the same VM Luau provides additional safety features to make isolation cheaper. + +When initializing the default globals table, the tables are protected from modification: + +- All libraries (`string`, `math`, etc.) are marked as readonly +- The string metatable is marked as readonly +- The global table itself is marked as readonly + +This is using the VM feature that is not accessible from scripts, that prevents all writes to the table, including assignments, `rawset` and `setmetatable`. This makes sure that globals can't be monkey-patched in place, and can only be substituted through `setfenv`. + +By itself this would mean that code that runs in Luau can't use globals at all, since assigning globals would fail. While this is feasible, in Roblox we solve this by creating a new global table for each script, that uses `__index` to point to the builtin global table. This safely sandboxes the builtin globals while still allowing writing globals from each script. This also means that short of exposing special shared globals from the host, all scripts are isolated from each other. + +## `__gc` + +Lua 5.1 exposes a `__gc` metamethod for userdata, which can be used on proxies (`newproxy`) to hook into garbage collector. Later versions of Lua extend this mechanism to work on tables. + +This mechanism is bad for performance, memory safety and isolation: + +- In Lua 5.1, `__gc` support requires traversing userdata lists redundantly during garbage collection to filter out finalizable objects +- In later versions of Lua, userdata that implement `__gc` are split into separate lists; however, finalization prolongs the lifetime of the finalized objects which results in less prompt memory reclamation, and two-step destruction results in extra cache misses for userdata +- `__gc` runs during garbage collection in context of an arbitrary thread which makes the thread identity mechanism used in Roblox to support trusted Luau code invalid +- Objects can be removed from weak tables *after* being finalized, which means that accessing these objects can result in memory safety bugs, unless all exposed userdata methods guard against use-after-gc. +- If `__gc` method ever leaks to scripts, they can call it directly on an object and use any method exposed by that object after that. This means that `__gc` and all other exposed methods must support memory safety when called on a destroyed object. + +Because of these issues, Luau does not support `__gc`. Instead it uses tag-based destructors that can perform additional memory cleanup during userdata destruction; crucially, these are only available to the host (so they can never be invoked manually), and they run right before freeing the userdata memory block which is both optimal for performance, and guaranteed to be memory safe. + +For monitoring garbage collector behavior the recommendation is to use weak tables instead. + +## Interrupts + +In addition to preventing API access, it can be important for isolation to limit the memory and CPU usage of code that runs inside the VM. + +By default, no memory limits are imposed on the running code, so it's possible to exhaust the address space of the host; this is easy to configure from the host for Luau allocations, but of course with a rich API surface exposed by the host it's hard to eliminate this as a possibility. Memory exhaustion doesn't result in memory safety issues or any particular risk to the system that's running the host process, other than the host process getting terminated by the OS. + +Limiting CPU usage can be equally challenging with a rich API. However, Luau does provide a VM-level feature to try to contain runaway scripts which makes it possible to terminate any script externally. This works through a global interrupt mechanism, where the host can setup an interrupt handler at any point, and any Luau code is guaranteed to call this handler "eventually" (in practice this can happen at any function call or at any loop iteration). This still leaves the possibility of a very long running script open if the script manages to find a way to call a single C function that takes a lot of time, but short of that the interruption is very prompt. + +Roblox sets up the interrupt handler using a watchdog that: + +- Limits the runtime of any script in Studio to 10 seconds (configurable through Studio settings) +- Upon client shutdown, interrupts execution of every running script 1 second after shutdown diff --git a/.cursor/rules/luau/index.mdx b/.cursor/rules/luau/index.mdx new file mode 100644 index 00000000..a1da377d --- /dev/null +++ b/.cursor/rules/luau/index.mdx @@ -0,0 +1,62 @@ +--- +title: Luau +description: Luau is a small, fast, and embeddable programming language based on Lua with a gradual type system. +template: splash +hero: + tagline: > + Luau (lowercase u, /ˈlu.aʊ/) is a small, fast, and embeddable programming language based on Lua with a gradual type system. + image: + file: ../../../public/images/mascot.png + alt: Hina, a Hawaiian monk seal and the official mascot of the Luau language + actions: + - text: Get Started + link: /getting-started/ + - text: News + link: /news/ + variant: secondary + - text: Try Luau online + link: https://play.luau.org + icon: right-caret + variant: minimal +--- + +import { Card, CardGrid } from '@astrojs/starlight/components'; +import ColoredCard from '../../components/ColoredCard.astro'; +import LuauPlayground from '../../components/LuauPlayground.astro'; + +# Try Luau! + + + +# Why Luau? + + + + Luau is a high-performance managed language with a fast bytecode compiler and interpreter, and an optimized, incremental garbage collector. Its optional JIT compiler runs on x64 and arm64, and supports plugins that extend native code generation to any embedder-defined types. + + + Luau has a state-of-the-art gradual type system with features like [type refinements](./types/type-refinements) and [type functions](./types/type-functions). This system is powered by an ambitious and powerful type inference engine that aims to both rule out program errors, _and_ provide autocomplete with minimal annotations. + + + Luau is designed to be easy-to-learn, and easy-to-use over time. The syntax of the language is small and easy to pick up, and the semantics of the language should be broadly unsurprising to most programmers, with recursive functions, conventional loops and branching, and so forth. + + + Luau aims to be pleasant and productive to use, and to make programming as fun as it can be! Besides the language itself, we're building a full suite of developer tools: package management, linting, code transformation, documentation generation, and servers for editing and debugging code. + + + Luau, like Lua before it, is designed with embedding in mind. The language features an extended version of Lua's C API with better support for coroutines and yielding, code coverage, and more performant, tagged userdata. We've also carefully sandboxed Luau, limiting the exposed API surface by default and enabling embedders to safely support executing untrusted code from their users. + + + Whenever possible, Luau aims to be backwards-compatible with Lua 5.1 and at the same time to incorporate features from later revisions of Lua. However, Luau is not a full superset of later versions of Lua - we do not always agree with Lua design decisions, and have different use cases and constraints. All post-5.1 Lua features, along with their support status in Luau, [are documented here](compatibility). + + diff --git a/.cursor/rules/luau/reference/grammar.md b/.cursor/rules/luau/reference/grammar.md new file mode 100644 index 00000000..c3a783fe --- /dev/null +++ b/.cursor/rules/luau/reference/grammar.md @@ -0,0 +1,109 @@ +--- +slug: grammar +title: Syntax Grammar +sidebar: + order: 3 +--- + +This is the complete syntax grammar for Luau in EBNF. More information about the terminal nodes STRING and NUMBER +is available in the [syntax section](../syntax). + +```ebnf +chunk ::= block +block ::= {stat [';']} [laststat [';']] +stat ::= varlist '=' explist | + var compoundop exp | + functioncall | + 'do' block 'end' | + 'while' exp 'do' block 'end' | + 'repeat' block 'until' exp | + 'if' exp 'then' block {'elseif' exp 'then' block} ['else' block] 'end' | + 'for' binding '=' exp ',' exp [',' exp] 'do' block 'end' | + 'for' bindinglist 'in' explist 'do' block 'end' | + attributes 'function' funcname funcbody | + attributes 'local' 'function' NAME funcbody | + 'local' bindinglist ['=' explist] | + ['export'] 'type' NAME ['<' GenericTypeListWithDefaults '>'] '=' Type | + ['export'] 'type' 'function' NAME funcbody + +laststat ::= 'return' [explist] | 'break' | 'continue' + +funcname ::= NAME {'.' NAME} [':' NAME] +funcbody ::= ['<' GenericTypeList '>'] '(' [parlist] ')' [':' ReturnType] block 'end' +parlist ::= bindinglist [',' '...' [':' (GenericTypePack | Type)]] | '...' [':' (GenericTypePack | Type)] + +explist ::= {exp ','} exp + +binding ::= NAME [':' Type] +bindinglist ::= binding [',' bindinglist] (* equivalent of Lua 5.1 'namelist', except with optional type annotations *) + +var ::= NAME | prefixexp '[' exp ']' | prefixexp '.' NAME +varlist ::= var {',' var} +prefixexp ::= var | functioncall | '(' exp ')' +functioncall ::= prefixexp funcargs | prefixexp ':' NAME funcargs + +exp ::= asexp { binop exp } | unop exp { binop exp } +ifelseexp ::= 'if' exp 'then' exp {'elseif' exp 'then' exp} 'else' exp +asexp ::= simpleexp ['::' Type] +stringinterp ::= INTERP_BEGIN exp { INTERP_MID exp } INTERP_END +simpleexp ::= NUMBER | STRING | 'nil' | 'true' | 'false' | '...' | tableconstructor | attributes 'function' funcbody | prefixexp | ifelseexp | stringinterp +funcargs ::= '(' [explist] ')' | tableconstructor | STRING + +tableconstructor ::= '{' [fieldlist] '}' +fieldlist ::= field {fieldsep field} [fieldsep] +field ::= '[' exp ']' '=' exp | NAME '=' exp | exp +fieldsep ::= ',' | ';' + +compoundop ::= '+=' | '-=' | '*=' | '/=' | '//=' | '%=' | '^=' | '..=' +binop ::= '+' | '-' | '*' | '/' | '//' | '^' | '%' | '..' | '<' | '<=' | '>' | '>=' | '==' | '~=' | 'and' | 'or' +unop ::= '-' | 'not' | '#' + +littable ::= '{' [litfieldlist] '}' +litfieldlist ::= litfield {fieldsep litfield} [fieldsep] +litfield ::= [NAME '='] literal + +literal ::= 'nil' | 'false' | 'true' | NUMBER | STRING | littable +litlist ::= literal {',' literal} + +pars ::= '(' [litlist] ')' | littable | STRING +parattr ::= NAME [pars] +attribute ::= '@' NAME | '@[' parattr {',' parattr} ']' +attributes ::= {attribute} + +SimpleType ::= + 'nil' | + SingletonType | + NAME ['.' NAME] [ '<' [TypeParams] '>' ] | + 'typeof' '(' exp ')' | + TableType | + FunctionType | + '(' Type ')' + +SingletonType ::= STRING | 'true' | 'false' + +Union ::= [SimpleType {'?'}] {'|' SimpleType {'?'}} +Intersection ::= [SimpleType] {'&' SimpleType} +Type ::= Union | Intersection + +GenericTypePackParameter ::= NAME '...' +GenericTypeList ::= NAME [',' GenericTypeList] | GenericTypePackParameter {',' GenericTypePackParameter} + +GenericTypePackParameterWithDefault ::= NAME '...' '=' (TypePack | VariadicTypePack | GenericTypePack) +GenericTypeListWithDefaults ::= + NAME ['=' Type] [',' GenericTypeListWithDefaults] | + GenericTypePackParameterWithDefault {',' GenericTypePackParameterWithDefault} + +TypeList ::= Type [',' TypeList] | '...' Type +BoundTypeList ::= [NAME ':'] Type [',' BoundTypeList] | GenericTypePack | VariadicTypePack +TypeParams ::= (Type | TypePack | VariadicTypePack | GenericTypePack) [',' TypeParams] +TypePack ::= '(' [TypeList] ')' +GenericTypePack ::= NAME '...' +VariadicTypePack ::= '...' Type +ReturnType ::= Type | TypePack | GenericTypePack | VariadicTypePack +TableIndexer ::= ['read' | 'write'] '[' Type ']' ':' Type +TableProp ::= ['read' | 'write'] NAME ':' Type +PropList ::= TableProp [fieldsep PropList] | TableIndexer {fieldsep TableProp} + +TableType ::= '{' Type '}' | '{' [PropList] '}' +FunctionType ::= ['<' GenericTypeList '>'] '(' [BoundTypeList] ')' '->' ReturnType +``` diff --git a/.cursor/rules/luau/reference/library.md b/.cursor/rules/luau/reference/library.md new file mode 100644 index 00000000..36d61af9 --- /dev/null +++ b/.cursor/rules/luau/reference/library.md @@ -0,0 +1,1097 @@ +--- +slug: library +title: Standard Library +description: The official reference for Luau's standard library. +sidebar: + order: 1 +--- + +Luau comes equipped with a standard library of functions designed to manipulate the built-in data types. Note that the library is relatively minimal and doesn't expose ways for +scripts to interact with the host environment - it's expected that embedding applications provide extra functionality on top of this and limit or sandbox the system access +appropriately, if necessary. For example, Roblox provides [a rich API to interact with the 3D environment and limited APIs to interact with external services](https://developer.roblox.com/en-us/api-reference). + +This page documents the available builtin libraries and functions. All of these are accessible by default by any script, assuming the host environment exposes them (which is usually a safe assumption outside of extremely constrained environments). + +## Global functions + +While most library functions are provided as part of a library like `table`, a few global functions are exposed without extra namespacing. + +``` +function assert(value: T, message: string?): T +``` + +`assert` checks if the value is truthy; if it's not (which means it's `false` or `nil`), it raises an error. The error message can be customized with an optional parameter. +Upon success the function returns the `value` argument. + +``` +function error(obj: any, level: number?) +``` + +`error` raises an error with the specified object. Note that errors don't have to be strings, although they often are by convention; various error handling mechanisms like `pcall` +preserve the error type. When `level` is specified, the error raised is turned into a string that contains call frame information for the caller at level `level`, where `1` refers +to the function that called `error`. This can be useful to attribute the errors to callers, for example `error("Expected a valid object", 2)` highlights the caller of the function +that called `error` instead of the function itself in the callstack. + +``` +function gcinfo(): number +``` + +`gcinfo` returns the total heap size in kilobytes, which includes bytecode objects, global tables as well as the script-allocated objects. Note that Luau uses an incremental +garbage collector, and as such at any given point in time the heap may contain both reachable and unreachable objects. The number returned by `gcinfo` reflects the current heap +consumption from the operating system perspective and can fluctuate over time as garbage collector frees objects. + +``` +function getfenv(target: (function | number)?): table +``` + +Returns the environment table for target function; when `target` is not a function, it must be a number corresponding to the caller stack index, where 1 means the function that calls `getfenv`, and the environment table is returned for the corresponding function from the call stack. When `target` is omitted it defaults to `1`, so `getfenv()` returns the environment table for the calling function. + +``` +function getmetatable(obj: any): table? +``` + +Returns the metatable for the specified object; when object is not a table or a userdata, the returned metatable is shared between all objects of the same type. Note that when metatable is protected (has a `__metatable` key), the value corresponding to that key is returned instead and may not be a table. + +``` +function next(t: { [K]: V }, i: K?): (K, V)? +``` + +Given the table `t`, returns the next key-value pair after `i` in the table traversal order, or nothing if `i` is the last key. When `i` is `nil`, returns the first key-value pair instead. + +``` +function newproxy(mt: boolean?): userdata +``` + +Creates a new untyped userdata object; when `mt` is true, the new object has an empty metatable that can be modified using `getmetatable`. + +``` +function print(args: ...any) +``` + +Prints all arguments to the standard output, using Tab as a separator. + +``` +function rawequal(a: any, b: any): boolean +``` + +Returns true iff `a` and `b` have the same type and point to the same object (for garbage collected types) or are equal (for value types). + +``` +function rawget(t: { [K]: V }, k: K): V? +``` + +Performs a table lookup with index `k` and returns the resulting value, if present in the table, or nil. This operation bypasses metatables/`__index`. + +``` +function rawlen(t: { [K]: V } | string): number +``` + +Returns the raw length of the table or string. If it is a string, this operation is identical to `#str` or `string.len(str)`. This operation bypasses metatables/`__len`. + +``` +function rawset(t: { [K] : V }, k: K, v: V) +``` + +Assigns table field `k` to the value `v`. This operation bypasses metatables/`__newindex`. + +``` +function select(i: string, args: ...T): number +function select(i: number, args: ...T): ...T +``` + +When called with `'#'` as the first argument, returns the number of remaining parameters passed. Otherwise, returns the subset of parameters starting with the specified index. +Index can be specified from the start of the arguments (using 1 as the first argument), or from the end (using -1 as the last argument). + +``` +function setfenv(target: function | number, env: table) +``` + +Changes the environment table for target function to `env`; when `target` is not a function, it must be a number corresponding to the caller stack index, where 1 means the function that calls `setfenv`, and the environment table is returned for the corresponding function from the call stack. + +``` +function setmetatable(t: table, mt: table?) +``` + +Changes metatable for the given table. Note that unlike `getmetatable`, this function only works on tables. If the table already has a protected metatable (has a `__metatable` field), this function errors. + +``` +function tonumber(s: string, base: number?): number? +``` + +Converts the input string to the number in base `base` (default 10) and returns the resulting number. If the conversion fails (that is, if the input string doesn't represent a valid number in the specified base), returns `nil` instead. + +``` +function tostring(obj: any): string +``` + +Converts the input object to string and returns the result. If the object has a metatable with `__tostring` field, that method is called to perform the conversion. + +``` +function type(obj: any): string +``` + +Returns the type of the object, which is one of `"nil"`, `"boolean"`, `"number"`, `"vector"`, `"string"`, `"table"`, `"function"`, `"userdata"`, `"thread"`, or `"buffer"`. + +``` +function typeof(obj: any): string +``` + +Returns the type of the object; for userdata objects that have a metatable with the `__type` field *and* are defined by the host (not `newproxy`), returns the value for that key. +For custom userdata objects, such as ones returned by `newproxy`, this function returns `"userdata"` to make sure host-defined types can not be spoofed. + +``` +function ipairs(t: table): +``` + +Returns the triple (generator, state, nil) that can be used to traverse the table using a `for` loop. The traversal results in key-value pairs for the numeric portion of the table; key starts from 1 and increases by 1 on each iteration. The traversal terminates when reaching the first `nil` value (so `ipairs` can't be used to traverse array-like tables with holes). + +``` +function pairs(t: table): +``` + +Returns the triple (generator, state, nil) that can be used to traverse the table using a `for` loop. The traversal results in key-value pairs for all keys in the table, numeric and otherwise, but doesn't have a defined order. + +``` +function pcall(f: function, args: ...any): (boolean, ...any) +``` + +Calls function `f` with parameters `args`. If the function succeeds, returns `true` followed by all return values of `f`. If the function raises an error, returns `false` followed by the error object. +Note that `f` can yield, which results in the entire coroutine yielding as well. + +``` +function xpcall(f: function, e: function, args: ...any): (boolean, ...any) +``` + +Calls function `f` with parameters `args`. If the function succeeds, returns `true` followed by all return values of `f`. If the function raises an error, calls `e` with the error object as an argument, and returns `false` followed by the first return value of `e`. +Note that `f` can yield, which results in the entire coroutine yielding as well. +`e` can neither yield nor error - if it does raise an error, `xpcall` returns with `false` followed by a special error message. + +``` +function unpack(a: {V}, f: number?, t: number?): ...V +``` + +Returns all values of `a` with indices in `[f..t]` range. `f` defaults to 1 and `t` defaults to `#a`. Note that this is equivalent to `table.unpack`. + +## math library + +``` +function math.abs(n: number): number +``` + +Returns the absolute value of `n`. Returns NaN if the input is NaN. + +``` +function math.acos(n: number): number +``` + +Returns the arc cosine of `n`, expressed in radians. Returns a value in `[0, pi]` range. Returns NaN if the input is not in `[-1, +1]` range. + +``` +function math.asin(n: number): number +``` + +Returns the arc sine of `n`, expressed in radians. Returns a value in `[-pi/2, +pi/2]` range. Returns NaN if the input is not in `[-1, +1]` range. + +``` +function math.atan2(y: number, x: number): number +``` + +Returns the arc tangent of `y/x`, expressed in radians. The function takes into account the sign of both arguments in order to determine the quadrant. Returns a value in `[-pi, pi]` range. + +``` +function math.atan(n: number): number +``` + +Returns the arc tangent of `n`, expressed in radians. Returns a value in `[-pi/2, pi-2]` range. + +``` +function math.ceil(n: number): number +``` + +Rounds `n` upwards to the next integer boundary. + +``` +function math.cosh(n: number): number +``` + +Returns the hyperbolic cosine of `n`. + +``` +function math.cos(n: number): number +``` + +Returns the cosine of `n`, which is an angle in radians. Returns a value in `[0, 1]` range. + +``` +function math.deg(n: number): number +``` + +Converts `n` from radians to degrees and returns the result. + +``` +function math.exp(n: number): number +``` + +Returns the base-e exponent of `n`, that is `e^n`. + +``` +function math.floor(n: number): number +``` + +Rounds `n` downwards to previous integer boundary. + +``` +function math.fmod(x: number, y: number): number +``` + +Returns the remainder of `x` modulo `y`, rounded towards zero. Returns NaN if `y` is zero. + +``` +function math.frexp(n: number): (number, number) +``` + +Splits the number into a significand (a number in `[-1, +1]` range) and binary exponent such that `n = s * 2^e`, and returns `s, e`. + +``` +function math.ldexp(s: number, e: number): number +``` + +Given the significand and a binary exponent, returns a number `s * 2^e`. + +``` +function math.lerp(a: number, b: number, t: number): number +``` + +Linearly interpolated between number value `a` and `b` using factor `t`, generally returning the result of `a + (b - a) * t`. +When `t` is exactly `1`, the value of `b` will be returned instead to ensure that when `t` is on the interval `[0, 1]`, the result of `lerp` will be on the interval `[a, b]`. + +``` +math.map(x: number, inmin: number, inmax: number, outmin: number, outmax: number): number +``` + +Returns a value that represents `x` mapped linearly from the input range (`inmin` to `inmax`) to the output range (`outmin` to `outmax`). + +``` +function math.log10(n: number): number +``` + +Returns base-10 logarithm of the input number. Returns NaN if the input is negative, and negative infinity if the input is 0. +Equivalent to `math.log(n, 10)`. + +``` +function math.log(n: number, base: number?): number +``` + +Returns logarithm of the input number in the specified base; base defaults to `e`. Returns NaN if the input is negative, and negative infinity if the input is 0. + +``` +function math.max(list: ...number): number +``` + +Returns the maximum number of the input arguments. The function requires at least one input and will error if zero parameters are passed. If one of the inputs is a NaN, the result may or may not be a NaN. + +``` +function math.min(list: ...number): number +``` + +Returns the minimum number of the input arguments. The function requires at least one input and will error if zero parameters are passed. If one of the inputs is a NaN, the result may or may not be a NaN. + +``` +function math.modf(n: number): (number, number) +``` + +Returns the integer and fractional part of the input number. Both the integer and fractional part have the same sign as the input number, e.g. `math.modf(-1.5)` returns `-1, -0.5`. + +``` +function math.pow(x: number, y: number): number +``` + +Returns `x` raised to the power of `y`. + +``` +function math.rad(n: number): number +``` + +Converts `n` from degrees to radians and returns the result. + +``` +function math.random(): number +function math.random(n: number): number +function math.random(min: number, max: number): number +``` + +Returns a random number using the global random number generator. A zero-argument version returns a number in `[0, 1]` range. A one-argument version returns a number in `[1, n]` range. A two-argument version returns a number in `[min, max]` range. The input arguments are truncated to integers, so `math.random(1.5)` always returns 1. + +``` +function math.randomseed(seed: number) +``` + +Reseeds the global random number generator; subsequent calls to `math.random` will generate a deterministic sequence of numbers that only depends on `seed`. + +``` +function math.sinh(n: number): number +``` + +Returns a hyperbolic sine of `n`. + +``` +function math.sin(n: number): number +``` + +Returns the sine of `n`, which is an angle in radians. Returns a value in `[0, 1]` range. + +``` +function math.sqrt(n: number): number +``` + +Returns the square root of `n`. Returns NaN if the input is negative. + +``` +function math.tanh(n: number): number +``` + +Returns the hyperbolic tangent of `n`. + +``` +function math.tan(n: number): number +``` + +Returns the tangent of `n`, which is an angle in radians. + +``` +function math.noise(x: number, y: number?, z: number?): number +``` + +Returns 3D Perlin noise value for the point `(x, y, z)` (`y` and `z` default to zero if absent). Returns a value in `[-1, 1]` range. + +``` +function math.clamp(n: number, min: number, max: number): number +``` + +Returns `n` if the number is in `[min, max]` range; otherwise, returns `min` when `n < min`, and `max` otherwise. If `n` is NaN, may or may not return NaN. +The function errors if `min > max`. + +``` +function math.sign(n: number): number +``` + +Returns `-1` if `n` is negative, `1` if `n` is positive, and `0` if `n` is zero or NaN. + +``` +function math.round(n: number): number +``` + +Rounds `n` to the nearest integer boundary. If `n` is exactly halfway between two integers, rounds `n` away from 0. + +## table library + +``` +function table.concat(a: {string}, sep: string?, f: number?, t: number?): string +``` + +Concatenate all elements of `a` with indices in range `[f..t]` together, using `sep` as a separator if present. `f` defaults to 1 and `t` defaults to `#a`. + +``` +function table.foreach(t: { [K]: V }, f: (K, V) -> R?): R? +``` + +Iterates over all elements of the table in unspecified order; for each key-value pair, calls `f` and returns the result of `f` if it's non-nil. If all invocations of `f` returned `nil`, returns no values. This function has been deprecated and is not recommended for use in new code; use `for` loop instead. + +``` +function table.foreachi(t: {V}, f: (number, V) -> R?): R? +``` + +Iterates over numeric keys of the table in `[1..#t]` range in order; for each key-value pair, calls `f` and returns the result of `f` if it's non-nil. If all invocations of `f` returned `nil`, returns no values. This function has been deprecated and is not recommended for use in new code; use `for` loop instead. + +``` +function table.getn(t: {V}): number +``` + +Returns the length of table `t`. This function has been deprecated and is not recommended for use in new code; use `#t` instead. + +``` +function table.maxn(t: {V}): number +``` + +Returns the maximum numeric key of table `t`, or zero if the table doesn't have numeric keys. + +``` +function table.insert(t: {V}, v: V) +function table.insert(t: {V}, i: number, v: V) +``` + +When using a two-argument version, appends the value to the array portion of the table (equivalent to `t[#t+1] = v`). +When using a three-argument version, inserts the value at index `i` and shifts values at indices after that by 1. `i` should be in `[1..#t]` range. + +``` +function table.remove(t: {V}, i: number?): V? +``` + +Removes element `i` from the table and shifts values at indices after that by 1. If `i` is not specified, removes the last element of the table. +`i` should be in `[1..#t]` range. +Returns the value of the removed element, or `nil` if no element was removed (e.g. table was empty). + +``` +function table.sort(t: {V}, f: ((V, V) -> boolean)?) +``` + +Sorts the table `t` in ascending order, using `f` as a comparison predicate: `f` should return `true` iff the first parameter should be before the second parameter in the resulting table. When `f` is not specified, builtin less-than comparison is used instead. +The comparison predicate must establish a strict weak ordering - sort results are undefined otherwise. + +``` +function table.pack(args: ...V): { [number]: V, n: number } +``` + +Returns a table that consists of all input arguments as array elements, and `n` field that is set to the number of inputs. + +``` +function table.unpack(a: {V}, f: number?, t: number?): ...V +``` + +Returns all values of `a` with indices in `[f..t]` range. `f` defaults to 1 and `t` defaults to `#a`. +Note that if you want to unpack varargs packed with `table.pack` you have to specify the index fields because `table.unpack` doesn't automatically use the `n` field that `table.pack` creates. Example usage for packed varargs: `table.unpack(args, 1, args.n)` + +``` +function table.move(a: {V}, f: number, t: number, d: number, tt: {V}?) +``` + +Copies elements in range `[f..t]` from table `a` to table `tt` if specified and `a` otherwise, starting from the index `d`. + +``` +function table.create(n: number, v: V?): {V} +``` + +Creates a table with `n` elements; all of them (range `[1..n]`) are set to `v`. When `v` is nil or omitted, the returned table is empty but has preallocated space for `n` elements which can make subsequent insertions faster. +Note that preallocation is only performed for the array portion of the table - using `table.create` on dictionaries is counter-productive. + +``` +function table.find(t: {V}, v: V, init: number?): number? +``` + +Find the first element in the table that is equal to `v` and returns its index; the traversal stops at the first `nil`. If the element is not found, `nil` is returned instead. The traversal starts at index `init` if specified, otherwise 1. + +``` +function table.clear(t: table) +``` + +Removes all elements from the table while preserving the table capacity, so future assignments don't need to reallocate space. + +``` +function table.freeze(t: table): table +``` + +Given a non-frozen table, freezes it such that all subsequent attempts to modify the table or assign its metatable raise an error. If the input table is already frozen or has a protected metatable, the function raises an error; otherwise it returns the input table. +Note that the table is frozen in-place and is not being copied. Additionally, only `t` is frozen, and keys/values/metatable of `t` don't change their state and need to be frozen separately if desired. + +``` +function table.isfrozen(t: table): boolean +``` + +Returns `true` iff the input table is frozen. + +``` +function table.clone(t: table): table +``` + +Returns a copy of the input table that has the same metatable, same keys and values, and is not frozen even if `t` was. +The copy is shallow: implementing a deep recursive copy automatically is challenging, and often only certain keys need to be cloned recursively which can be done after the initial clone by modifying the resulting table. + +## string library + +``` +function string.byte(s: string, f: number?, t: number?): ...number +``` + +Returns the numeric code of every byte in the input string with indices in range `[f..t]`. `f` defaults to 1 and `t` defaults to `f`, so a two-argument version of this function returns a single number. If the function is called with a single argument and the argument is out of range, the function returns no values. + +``` +function string.char(args: ...number): string +``` + +Returns the string that contains a byte for every input number; all inputs must be integers in `[0..255]` range. + +``` +function string.find(s: string, p: string, init: number?, plain: boolean?): (number?, number?, ...string) +``` + +Tries to find an instance of pattern `p` in the string `s`, starting from position `init` (defaults to 1). When `plain` is true, the search is using raw (case-sensitive) string equality, otherwise `p` should be a [string pattern](https://www.lua.org/manual/5.3/manual.html#6.4.1). If a match is found, returns the position of the match and the length of the match, followed by the pattern captures; otherwise returns `nil`. + +``` +function string.format(s: string, args: ...any): string +``` + +Returns a formatted version of the input arguments using a [printf-style format string](https://en.cppreference.com/w/c/io/fprintf) `s`. The following format characters are supported: + +- `c`: expects an integer number and produces a character with the corresponding character code +- `d`, `i`, `u`: expects an integer number and produces the decimal representation of that number +- `o`: expects an integer number and produces the octal representation of that number +- `x`, `X`: expects an integer number and produces the hexadecimal representation of that number, using lower case or upper case hexadecimal characters +- `e`, `E`, `f`, `g`, `G`: expects a number and produces the floating point representation of that number, using scientific or decimal representation +- `q`: expects a string and produces the same string quoted using double quotation marks, with escaped special characters if necessary +- `s`: expects a string and produces the same string verbatim + +The formats support modifiers `-`, `+`, space, `#` and `0`, as well as field width and precision modifiers - with the exception of `*`. + +``` +function string.gmatch(s: string, p: string): +``` + +Produces an iterator function that, when called repeatedly explicitly or via `for` loop, produces matches of string `s` with [string pattern](https://www.lua.org/manual/5.3/manual.html#6.4.1) `p`. For every match, the captures within the pattern are returned if present (if a pattern has no captures, the entire matching substring is returned instead). + +``` +function string.gsub(s: string, p: string, f: function | table | string, maxs: number?): (string, number) +``` + +For every match of [string pattern](https://www.lua.org/manual/5.3/manual.html#6.4.1) `p` in `s`, replace the match according to `f`. The substitutions stop after the limit of `maxs`, and the function returns the resulting string followed by the number of substitutions. + +When `f` is a string, the substitution uses the string as a replacement. When `f` is a table, the substitution uses the table element with key corresponding to the first pattern capture, if present, and entire match otherwise. Finally, when `f` is a function, the substitution uses the result of calling `f` with call pattern captures, or entire matching substring if no captures are present. + +``` +function string.len(s: string): number +``` + +Returns the number of bytes in the string (equivalent to `#s`). + +``` +function string.lower(s: string): string +``` + +Returns a string where each byte corresponds to the lower-case ASCII version of the input byte in the source string. + +``` +function string.match(s: string, p: string, init: number?): ...string? +``` + +Tries to find an instance of pattern `p` in the string `s`, starting from position `init` (defaults to 1). `p` should be a [string pattern](https://www.lua.org/manual/5.3/manual.html#6.4.1). If a match is found, returns all pattern captures, or entire matching substring if no captures are present, otherwise returns `nil`. + +``` +function string.rep(s: string, n: number): string +``` + +Returns the input string `s` repeated `n` times. Returns an empty string if `n` is zero or negative. + +``` +function string.reverse(s: string): string +``` + +Returns the string with the order of bytes reversed compared to the original. Note that this only works if the input is a binary or ASCII string. + +``` +function string.sub(s: string, f: number, t: number?): string +``` + +Returns a substring of the input string with the byte range `[f..t]`; `t` defaults to `#s`, so a two-argument version returns a string suffix. + +``` +function string.upper(s: string): string +``` + +Returns a string where each byte corresponds to the upper-case ASCII version of the input byte in the source string. + +``` +function string.split(s: string, sep: string?): {string} +``` + +Splits the input string using `sep` as a separator (defaults to `","`) and returns the resulting substrings. If separator is empty, the input string is split into separate one-byte strings. + +``` +function string.pack(f: string, args: ...any): string +``` + +Given a [pack format string](https://www.lua.org/manual/5.3/manual.html#6.4.2), encodes all input parameters according to the packing format and returns the resulting string. Note that Luau uses fixed sizes for all types that have platform-dependent size in Lua 5.x: short is 16 bit, long is 64 bit, integer is 32-bit and size_t is 32 bit for the purpose of string packing. + +``` +function string.packsize(f: string): number +``` + +Given a [pack format string](https://www.lua.org/manual/5.3/manual.html#6.4.2), returns the size of the resulting packed representation. The pack format can't use variable-length format specifiers. Note that Luau uses fixed sizes for all types that have platform-dependent size in Lua 5.x: short is 16 bit, long is 64 bit, integer is 32-bit and size_t is 32 bit for the purpose of string packing. + +``` +function string.unpack(f: string, s: string): ...any +``` + +Given a [pack format string](https://www.lua.org/manual/5.3/manual.html#6.4.2), decodes the input string according to the packing format and returns all resulting values. Note that Luau uses fixed sizes for all types that have platform-dependent size in Lua 5.x: short is 16 bit, long is 64 bit, integer is 32-bit and size_t is 32 bit for the purpose of string packing. + +## coroutine library + +``` +function coroutine.create(f: function): thread +``` + +Returns a new coroutine that, when resumed, will run function `f`. + +``` +function coroutine.running(): thread? +``` + +Returns the currently running coroutine, or `nil` if the code is running in the main coroutine (depending on the host environment setup, main coroutine may never be used for running code). + +``` +function coroutine.status(co: thread): string +``` + +Returns the status of the coroutine, which can be `"running"`, `"suspended"`, `"normal"` or `"dead"`. Dead coroutines have finished their execution and can not be resumed, but their state can still be inspected as they are not dead from the garbage collector point of view. + +``` +function coroutine.wrap(f: function): function +``` + +Creates a new coroutine and returns a function that, when called, resumes the coroutine and passes all arguments along to the suspension point. When the coroutine yields or finishes, the wrapped function returns with all values returned at the suspension point. + +``` +function coroutine.yield(args: ...any): ...any +``` + +Yields the currently running coroutine and passes all arguments along to the code that resumed the coroutine. The coroutine becomes suspended; when the coroutine is resumed again, the resumption arguments will be forwarded to `yield` which will behave as if it returned all of them. + +``` +function coroutine.isyieldable(): boolean +``` + +Returns `true` iff the currently running coroutine can yield. Yielding is prohibited when running inside metamethods like `__index` or C functions like `table.foreach` callback, with the exception of `pcall`/`xpcall`. + +``` +function coroutine.resume(co: thread, args: ...any): (boolean, ...any) +``` + +Resumes the coroutine and passes the arguments along to the suspension point. When the coroutine yields or finishes, returns `true` and all values returned at the suspension point. If an error is raised during coroutine resumption, this function returns `false` and the error object, similarly to `pcall`. + +``` +function coroutine.close(co: thread): (boolean, any?) +``` + +Closes the coroutine which puts coroutine in the dead state. The coroutine must be dead or suspended - in particular it can't be currently running. If the coroutine that's being closed was in an error state, returns `false` along with an error object; otherwise returns `true`. After closing, the coroutine can't be resumed and the coroutine stack becomes empty. + +## bit32 library + +All functions in the `bit32` library treat input numbers as 32-bit unsigned integers in `[0..4294967295]` range. The bit positions start at 0 where 0 corresponds to the least significant bit. + +``` +function bit32.arshift(n: number, i: number): number +``` + +Shifts `n` by `i` bits to the right (if `i` is negative, a left shift is performed instead). The most significant bit of `n` is propagated during the shift. When `i` is larger than 31, returns an integer with all bits set to the sign bit of `n`. When `i` is smaller than `-31`, 0 is returned. + +``` +function bit32.band(args: ...number): number +``` + +Performs a bitwise `and` of all input numbers and returns the result. If the function is called with no arguments, an integer with all bits set to 1 is returned. + +``` +function bit32.bnot(n: number): number +``` + +Returns a bitwise negation of the input number. + +``` +function bit32.bor(args: ...number): number +``` + +Performs a bitwise `or` of all input numbers and returns the result. If the function is called with no arguments, zero is returned. + +``` +function bit32.bxor(args: ...number): number +``` + +Performs a bitwise `xor` (exclusive or) of all input numbers and returns the result. If the function is called with no arguments, zero is returned. + +``` +function bit32.btest(args: ...number): boolean +``` + +Perform a bitwise `and` of all input numbers, and return `true` iff the result is not 0. If the function is called with no arguments, `true` is returned. + +``` +function bit32.extract(n: number, f: number, w: number?): number +``` + +Extracts bits of `n` at position `f` with a width of `w`, and returns the resulting integer. `w` defaults to `1`, so a two-argument version of `extract` returns the bit value at position `f`. Bits are indexed starting at 0. Errors if `f` and `f+w-1` are not between 0 and 31. + +``` +function bit32.lrotate(n: number, i: number): number +``` + +Rotates `n` to the left by `i` bits (if `i` is negative, a right rotate is performed instead); the bits that are shifted past the bit width are shifted back from the right. + +``` +function bit32.lshift(n: number, i: number): number +``` + +Shifts `n` to the left by `i` bits (if `i` is negative, a right shift is performed instead). When `i` is outside of `[-31..31]` range, returns 0. + +``` +function bit32.replace(n: number, r: number, f: number, w: number?): number +``` + +Replaces bits of `n` at position `f` and width `w` with `r`, and returns the resulting integer. `w` defaults to `1`, so a three-argument version of `replace` changes one bit at position `f` to `r` (which should be 0 or 1) and returns the result. Bits are indexed starting at 0. Errors if `f` and `f+w-1` are not between 0 and 31. + +``` +function bit32.rrotate(n: number, i: number): number +``` + +Rotates `n` to the right by `i` bits (if `i` is negative, a left rotate is performed instead); the bits that are shifted past the bit width are shifted back from the left. + +``` +function bit32.rshift(n: number, i: number): number +``` + +Shifts `n` to the right by `i` bits (if `i` is negative, a left shift is performed instead). When `i` is outside of `[-31..31]` range, returns 0. + +``` +function bit32.countlz(n: number): number +``` + +Returns the number of consecutive zero bits in the 32-bit representation of `n` starting from the left-most (most significant) bit. Returns 32 if `n` is zero. + +``` +function bit32.countrz(n: number): number +``` + +Returns the number of consecutive zero bits in the 32-bit representation of `n` starting from the right-most (least significant) bit. Returns 32 if `n` is zero. + +``` +function bit32.byteswap(n: number): number +``` + +Returns `n` with the order of the bytes swapped. + +## utf8 library + +Strings in Luau can contain arbitrary bytes; however, in many applications strings representing text contain UTF8 encoded data by convention, that can be inspected and manipulated using `utf8` library. + +``` +function utf8.offset(s: string, n: number, i: number?): number? +``` + +Returns the byte offset of the Unicode codepoint number `n` in the string, starting from the byte position `i`. When the character is not found, returns `nil` instead. + +``` +function utf8.codepoint(s: string, i: number?, j: number?): ...number +``` + +Returns a number for each Unicode codepoint in the string with the starting byte offset in `[i..j]` range. `i` defaults to 1 and `j` defaults to `i`, so a two-argument version of this function returns the Unicode codepoint that starts at byte offset `i`. + +``` +function utf8.char(args: ...number): string +``` + +Creates a string by concatenating Unicode codepoints for each input number. + +``` +function utf8.len(s: string, i: number?, j: number?): number? +``` + +Returns the number of Unicode codepoints with the starting byte offset in `[i..j]` range, or `nil` followed by the first invalid byte position if the input string is malformed. +`i` defaults to 1 and `j` defaults to `#s`, so `utf8.len(s)` returns the number of Unicode codepoints in string `s` or `nil` if the string is malformed. + +``` +function utf8.codes(s: string): +``` + +Returns an iterator that, when used in `for` loop, produces the byte offset and the codepoint for each Unicode codepoints that `s` consists of. + +## os library + +``` +function os.clock(): number +``` + +Returns a high-precision timestamp (in seconds) that doesn't have a defined baseline, but can be used to measure duration with sub-microsecond precision. + +``` +function os.date(s: string?, t: number?): table | string +``` + +Returns the table or string representation of the time specified as `t` (defaults to current time) according to `s` format string. + +When `s` starts with `!`, the result uses UTC, otherwise it uses the current timezone. + +If `s` is equal to `*t` (or `!*t`), a table representation of the date is returned, with keys `sec`/`min`/`hour` for the time (using 24-hour clock), `day`/`month`/`year` for the date, `wday` for week day (1..7), `yday` for year day (1..366) and `isdst` indicating whether the timezone is currently using daylight savings. + +Otherwise, `s` is interpreted as a [date format string](https://www.cplusplus.com/reference/ctime/strftime/), with the valid specifiers including any of `aAbBcdHIjmMpSUwWxXyYzZ` or `%`. `s` defaults to `"%c"` so `os.date()` returns the human-readable representation of the current date in local timezone. + +``` +function os.difftime(a: number, b: number): number +``` + +Calculates the difference in seconds between `a` and `b`; provided for compatibility only. Please use `a - b` instead. + +``` +function os.time(t: table?): number +``` + +When called without arguments, returns the current date/time as a Unix timestamp. When called with an argument, expects it to be a table that contains `sec`/`min`/`hour`/`day`/`month`/`year` keys and returns the Unix timestamp of the specified date/time in UTC. + +## debug library + +``` +function debug.info(co: thread, level: number, s: string): ...any +function debug.info(level: number, s: string): ...any +function debug.info(f: function, s: string): ...any +``` + +Given a stack frame or a function, and a string that specifies the requested information, returns the information about the stack frame or function. + +Each character of `s` results in additional values being returned in the same order as the characters appear in the string: + +- `s` returns source path for the function +- `l` returns the line number for the stack frame or the line where the function is defined when inspecting a function object +- `n` returns the name of the function, or an empty string if the name is not known +- `f` returns the function object itself +- `a` returns the number of arguments that the function expects followed by a boolean indicating whether the function is variadic or not + +For example, `debug.info(2, "sln")` returns source file, current line and function name for the caller of the current function. + +``` +function debug.traceback(co: thread, msg: string?, level: number?): string +function debug.traceback(msg: string?, level: number?): string +``` + +Produces a stringified callstack of the given thread, or the current thread, starting with level `level`. If `msg` is specified, then the resulting callstack includes the string before the callstack output, separated with a newline. The format of the callstack is human-readable and subject to change. + +## buffer library + +Buffer is an object that represents a fixed-size mutable block of memory. + +All operations on a buffer are provided using the 'buffer' library functions. + +Many of the functions accept an offset in bytes from the start of the buffer. +Offset of 0 from the start of the buffer memory block accesses the first byte. + +All offsets, counts and sizes should be non-negative integer numbers. + +If the bytes that are accessed by any read or write operation are outside the buffer memory, an error is thrown. + +``` +function buffer.create(size: number): buffer +``` + +Creates a buffer of the requested size with all bytes initialized to 0. + +Size limit is 1GB or 1,073,741,824 bytes. + +``` +function buffer.fromstring(str: string): buffer +``` + +Creates a buffer initialized to the contents of the string. + +The size of the buffer equals to the length of the string. + + +``` +function buffer.tostring(b: buffer): string +``` + +Returns the buffer data as a string. + +``` +function buffer.len(b: buffer): number +``` + +Returns the size of the buffer in bytes. + +``` +function buffer.readi8(b: buffer, offset: number): number +function buffer.readu8(b: buffer, offset: number): number +function buffer.readi16(b: buffer, offset: number): number +function buffer.readu16(b: buffer, offset: number): number +function buffer.readi32(b: buffer, offset: number): number +function buffer.readu32(b: buffer, offset: number): number +function buffer.readf32(b: buffer, offset: number): number +function buffer.readf64(b: buffer, offset: number): number +``` + +Used to read the data from the buffer by reinterpreting bytes at the offset as the type in the argument and converting it into a number. + +Available types: + +Function | Type | Range | +---------|------|-------| +readi8 | signed 8-bit integer | [-128, 127] +readu8 | unsigned 8-bit integer | [0, 255] +readi16 | signed 16-bit integer | [-32,768, 32,767] +readu16 | unsigned 16-bit integer | [0, 65,535] +readi32 | signed 32-bit integer | [-2,147,483,648, 2,147,483,647] +readu32 | unsigned 32-bit integer | [0, 4,294,967,295] +readf32 | 32-bit floating-point number | Single-precision IEEE 754 number +readf64 | 64-bit floating-point number | Double-precision IEEE 754 number + +Floating-point numbers are read and written using a format specified by IEEE 754. + +If a floating-point value matches any of bit patterns that represent a NaN (not a number), returned value might be converted to a different quiet NaN representation. + +Read and write operations use the little endian byte order. + +Integer numbers are read and written using two's complement representation. + +``` +function buffer.writei8(b: buffer, offset: number, value: number): () +function buffer.writeu8(b: buffer, offset: number, value: number): () +function buffer.writei16(b: buffer, offset: number, value: number): () +function buffer.writeu16(b: buffer, offset: number, value: number): () +function buffer.writei32(b: buffer, offset: number, value: number): () +function buffer.writeu32(b: buffer, offset: number, value: number): () +function buffer.writef32(b: buffer, offset: number, value: number): () +function buffer.writef64(b: buffer, offset: number, value: number): () +``` + +Used to write data to the buffer by converting the number into the type in the argument and reinterpreting it as individual bytes. + +Ranges of acceptable values can be seen in the table above. + +When writing integers, the number is converted using `bit32` library rules. + +Values that are out-of-range will take less significant bits of the full number. +For example, writing 43,981 (0xabcd) using writei8 function will take 0xcd and interpret it as an 8-bit signed number -51. +It is still recommended to keep all numbers in range of the target type. + +Results of converting special number values (inf/nan) to integers are platform-specific. + +``` +function buffer.readstring(b: buffer, offset: number, count: number): string +``` + +Used to read a string of length 'count' from the buffer at specified offset. + +``` +function buffer.writestring(b: buffer, offset: number, value: string, count: number?): () +``` + +Used to write data from a string into the buffer at a specified offset. + +If an optional 'count' is specified, only 'count' bytes are taken from the string. + +Count cannot be larger than the string length. + +``` +buffer.readbits(b: buffer, bitOffset: number, bitCount: number): number +``` + +Used to read a range of `bitCount` bits from the buffer, at specified offset `bitOffset`, into an unsigned integer. + +`bitCount` must be in `[0, 32]` range. + +``` +buffer.writebits(b: buffer, bitOffset: number, bitCount: number, value: number): () +``` + +Used to write `bitCount` bits from `value` into the buffer at specified offset `bitOffset`. + +`bitCount` must be in `[0, 32]` range. + +``` +function buffer.copy(target: buffer, targetOffset: number, source: buffer, sourceOffset: number?, count: number?): () +``` + +Copy 'count' bytes from 'source' starting at offset 'sourceOffset' into the 'target' at 'targetOffset'. + +It is possible for 'source' and 'target' to be the same. Copying an overlapping region inside the same buffer acts as if the source region is copied into a temporary buffer and then that buffer is copied over to the target. + +If 'sourceOffset' is nil or is omitted, it defaults to 0. + +If 'count' is 'nil' or is omitted, the whole 'source' data starting from 'sourceOffset' is copied. + +``` +function buffer.fill(b: buffer, offset: number, value: number, count: number?): () +``` + +Sets the 'count' bytes in the buffer starting at the specified 'offset' to the 'value'. + +If 'count' is 'nil' or is omitted, all bytes from the specified offset until the end of the buffer are set. + +## vector library + +This library implements functionality for the vector type in addition to the built-in primitive operator support. +Default configuration uses vectors with 3 components (`x`, `y`, and `z`). +If the _4-wide mode_ is enabled by setting the `LUA_VECTOR_SIZE` VM configuration to 4, vectors get an additional `w` component. + +Individual vector components can be accessed using the fields `x` or `X`, `y` or `Y`, `z` or `Z`, and `w` or `W` in 4-wide mode. +Since vector values are immutable, writes to individual components are not supported. + +``` +vector.zero +vector.one +``` + +Constant vectors with all components set to 0 and 1 respectively. Includes the fourth component in _4-wide mode_. + +``` +vector.create(x: number, y: number, z: number): vector +vector.create(x: number, y: number, z: number, w: number): vector +``` + +Creates a new vector with the given component values. The first constructor sets the fourth (`w`) component to 0.0 in _4-wide mode_. + +``` +vector.magnitude(vec: vector): number +``` + +Calculates the magnitude of a given vector. Includes the fourth component in _4-wide mode_. + +``` +vector.normalize(vec: vector): vector +``` + +Computes the normalized version (unit vector) of a given vector. Includes the fourth component in _4-wide mode_. + +``` +vector.cross(vec1: vector, vec2: vector): vector +``` + +Computes the cross product of two vectors. Ignores the fourth component in _4-wide mode_ and returns the 3-dimensional cross product. + +``` +vector.dot(vec1: vector, vec2: vector): number +``` + +Computes the dot product of two vectors. Includes the fourth component in _4-wide mode_. + +``` +vector.angle(vec1: vector, vec2: vector, axis: vector?): number +``` + +Computes the angle between two vectors in radians. The axis, if specified, is used to determine the sign of the angle. Ignores the fourth component in _4-wide mode_ and returns the 3-dimensional angle. + +``` +vector.floor(vec: vector): vector +``` + +Applies `math.floor` to every component of the input vector. Includes the fourth component in _4-wide mode_. + +``` +vector.ceil(vec: vector): vector +``` + +Applies `math.ceil` to every component of the input vector. Includes the fourth component in _4-wide mode_. + +``` +vector.abs(vec: vector): vector +``` + +Applies `math.abs` to every component of the input vector. Includes the fourth component in _4-wide mode_. + +``` +vector.sign(vec: vector): vector +``` + +Applies `math.sign` to every component of the input vector. Includes the fourth component in _4-wide mode_. + +``` +vector.clamp(vec: vector, min: vector, max: vector): vector +``` + +Applies `math.clamp` to every component of the input vector. Includes the fourth component in _4-wide mode_. + +``` +vector.max(...: vector): vector +``` + +Applies `math.max` to the corresponding components of the input vectors. Includes the fourth component in _4-wide mode_. Equivalent to `vector.create(math.max((...).x), math.max((...).y), math.max((...).z), math.max((...).w))`. + +``` +vector.min(...: vector): vector +``` + +Applies `math.min` to the corresponding components of the input vectors. Includes the fourth component in _4-wide mode_. Equivalent to `vector.create(math.min((...).x), math.min((...).y), math.min((...).z), math.min((...).w))`. diff --git a/.cursor/rules/luau/reference/types-library.md b/.cursor/rules/luau/reference/types-library.md new file mode 100644 index 00000000..b3dcacd8 --- /dev/null +++ b/.cursor/rules/luau/reference/types-library.md @@ -0,0 +1,368 @@ +--- +slug: types-library +title: Type Function Library +sidebar: + order: 2 +--- + +The `types` library is used to create and transform types, and can only be used within [type functions](../types/type-functions). + +### `types` library properties + +``` +types.any +``` + +The [any](../types/basic-types#any-type) `type`. + +``` +types.unknown +``` + +The [unknown](../types/basic-types#unknown-type) `type`. + +``` +types.never +``` + +The [never](../types/basic-types#never-type) `type`. + +``` +types.boolean +``` + +The boolean `type`. + +``` +types.buffer +``` + +The [buffer](../library#buffer-library) `type`. + +``` +types.number +``` + +The number `type`. + +``` +types.string +``` + +The string `type`. + +``` +types.thread +``` + +The thread `type`. + +## `types` library functions + +``` +types.singleton(arg: string | boolean | nil): type +``` + +Returns the [singleton](../types/basic-types#singleton-types-aka-literal-types) type of the argument. + +``` +types.negationof(arg: type): type +``` + +Returns an immutable negation of the argument type. + +``` +types.optional(arg: type): type +``` + +Returns a version of the given type that is now optional. + +- If the given type is a [union type](../types/unions-and-intersections#union-types), `nil` will be added unconditionally as a component. +- Otherwise, the result will be a union of the given type and the `nil` type. + +``` +types.unionof(first: type, second: type, ...: type): type +``` + +Returns an immutable [union](../types/unions-and-intersections#union-types) of two or more arguments. + +``` +types.intersectionof(first: type, second: type, ...: type): type +``` + +Returns an immutable [intersection](../types/unions-and-intersections#intersection-types) of two or more arguments. + +``` +types.newtable(props: { [type]: type | { read: type?, write: type? } }?, indexer: { index: type, readresult: type, writeresult: type? }?, metatable: type?): type +``` + +Returns a fresh, mutable table `type`. Property keys must be string singleton `type`s. The table's metatable is set if one is provided. + +``` +types.newfunction(parameters: { head: {type}?, tail: type? }, returns: { head: {type}?, tail: type? }?, generics: {type}?): type +``` + +Returns a fresh, mutable function `type`, using the ordered parameters of `head` and the variadic tail of `tail`. + +``` +types.copy(arg: type): type +``` + +Returns a deep copy of the argument type. + +``` +types.generic(name: string?, ispack: boolean?): type +``` + +Creates a [generic](../types/generics#generic-functions) named `name`. If `ispack` is `true`, the result is a [generic pack](../types/basic-types#type-packs). + +### `type` instance + +`type` instances can have extra properties and methods described in subsections depending on its tag. + +``` +type.tag: "nil" | "unknown" | "never" | "any" | "boolean" | "number" | "string" | "singleton" | "negation" | "union" | "intersection" | "table" | "function" | "class" | "thread" | "buffer" +``` + +An immutable property holding the type's tag. + +``` +__eq(arg: type): boolean +``` + +Overrides the `==` operator to return `true` if `self` is syntactically equal to `arg`. This excludes semantically equivalent types, `true | false` is unequal to `boolean`. + +``` +type:is(arg: "nil" | "unknown" | "never" | "any" | "boolean" | "number" | "string" | "singleton" | "negation" | "union" | "intersection" | "table" | "function" | "class" | "thread" | "buffer") +``` + +Returns `true` if `self` has the argument as its tag. + +### Singleton `type` instance + +``` +singletontype:value(): boolean | nil | "string" +``` + +Returns the singleton's actual value, like `true` for `types.singleton(true)`. + +### Generic `type` instance + +``` +generictype:name(): string? +``` + +Returns the name of the [generic](../types/generics#generic-functions) or `nil` if it has no name. + +``` +generictype:ispack(): boolean +``` + +Returns `true` if the [generic](../types/generics#generic-functions) is a [pack](../types/basic-types#type-packs), or `false` otherwise. + +### Table `type` instance + +``` +tabletype:setproperty(key: type, value: type?) +``` + +Sets the type of the property for the given `key`, using the same type for both reading from and writing to the table. + +- `key` is expected to be a string singleton type, naming the property. +- `value` will be set as both the `read type` and `write type` of the property. +- If `value` is `nil`, the property is removed. + +``` +tabletype:setreadproperty(key: type, value: type?) +``` + +Sets the type for reading from the property named by `key`, leaving the type for writing this property as-is. + +- `key` is expected to be a string singleton type, naming the property. +- `value` will be set as the `read type`, the `write type` will be unchanged. +- If `key` is not already present, only a `read type` will be set, making the property read-only. +- If `value` is `nil`, the property is removed. + +``` +tabletype:setwriteproperty(key: type, value: type?) +``` + + +Sets the type for writing to the property named by `key`, leaving the type for reading this property as-is. + +- `key` is expected to be a string singleton type, naming the property. +- `value` will be set as the `write type`, the `read type` will be unchanged. +- If `key` is not already present, only a `write type` will be set, making the property write-only. +- If `value` is `nil`, the property is removed. + +``` +tabletype:readproperty(key: type): type? +``` + +Returns the type used for reading values from this property, or `nil` if the property doesn't exist. + +``` +tabletype:writeproperty(key: type): type? +``` + +Returns the type used for writing values to this property, or `nil` if the property doesn't exist. + +``` +tabletype:properties(): { [type]: { read: type?, write: type? } } +``` + +Returns a table mapping property keys to their read and write types. + +``` +tabletype:setindexer(index: type, result: type) +``` + +Sets the table's indexer, using the same type for reads and writes. + +``` +tabletype:setreadindexer(index: type, result: type) +``` + +Sets the type resulting from reading from this table via indexing. + +``` +tabletype:setwriteindexer(index: type, result: type) +``` + +Sets the type for writing to this table via indexing. + +``` +tabletype:indexer(): { index: type, readresult: type, writeresult: type } +``` + +Returns the table's indexer as a table, or `nil` if it doesn't exist. + +``` +tabletype:readindexer(): { index: type, result: type }? +``` + +Returns the table's indexer using the result's read type, or `nil` if it doesn't exist. + +``` +tabletype:writeindexer() +``` + +Returns the table's indexer using the result's write type, or `nil` if it doesn't exist. + +``` +tabletype:setmetatable(arg: type) +``` + +Sets the table's metatable. + +``` +tabletype:metatable(): type? +``` + +Gets the table's metatable, or `nil` if it doesn't exist. + +### Function `type` instance + +``` +functiontype:setparameters(head: {type}?, tail: type?) +``` + +Sets the function's parameters, with the ordered parameters in `head` and the variadic tail in `tail`. + +``` +functiontype:parameters(): { head: {type}?, tail: type? } +``` + +Returns the function's parameters, with the ordered parameters in `head` and the variadic tail in `tail`. + +``` +functiontype:setreturns(head: {type}?, tail: type?) +``` + +Sets the function's return types, with the ordered parameters in `head` and the variadic tail in `tail`. + +``` +functiontype:returns(): { head: {type}?, tail: type? } +``` + +Returns the function's return types, with the ordered parameters in `head` and the variadic tail in `tail`. + +``` +functiontype:generics(): {type} +``` + +Returns an array of the function's [generic](../types/generics#generic-functions) `type`s. + +``` +functiontype:setgenerics(generics: {type}?) +``` + +Sets the function's [generic](../types/generics#generic-functions) `type`s. + +### Negation `type` instance + +``` +type:inner(): type +``` + +Returns the `type` being negated. + +### Union `type` instance + +``` +uniontype:components(): {type} +``` + +Returns an array of the [unioned](../types/unions-and-intersections#union-types) types. + +### Intersection `type` instance + +``` +intersectiontype:components() +``` + +Returns an array of the [intersected](../types/unions-and-intersections#intersection-types) types. + +### Class `type` instance + +``` +classtype:properties(): { [type]: { read: type?, write: type? } } +``` + +Returns the properties of the class with their respective `read` and `write` types. + +``` +classtype:readparent(): type? +``` + +Returns the type of reading this class' parent, or returns `nil` if the parent class doesn't exist. + +``` +classtype:writeparent(): type? +``` + +Returns the type for writing to this class' parent, or returns `nil` if the parent class doesn't exist. + +``` +classtype:metatable(): type? +``` + +Returns the class' metatable, or `nil` if it doesn't exist. + +``` +classtype:indexer(): { index: type, readresult: type, writeresult: type }? +``` + +Returns the class' indexer, or `nil` if it doesn't exist. + +``` +classtype:readindexer(): { index: type, result: type }? +``` + +Returns result type of reading from the class via indexing, or `nil` if it doesn't exist. + +``` +classtype:writeindexer(): { index: type, result: type }? +``` + +Returns the type for writing to the class via indexing, or `nil` if it doesn't exist. diff --git a/.cursor/rules/luau/types/basic-types.md b/.cursor/rules/luau/types/basic-types.md new file mode 100644 index 00000000..2e65c804 --- /dev/null +++ b/.cursor/rules/luau/types/basic-types.md @@ -0,0 +1,240 @@ +--- +slug: types/basic-types +title: Primitives and Simple Types +sidebar: + order: 2 +--- + +## Builtin types + +The Luau VM supports 10 primitive types: + +1. `nil` +2. `string` +3. `number` +4. `boolean` +5. `table` +6. `function` +7. `thread` +8. `userdata` +9. `vector` +10. `buffer` + +Most of these can be specified by their name and written directly in type annotations: + +```luau +local message: string = "Hello" +local count: number = 42 +local isReady: boolean = true +local co: thread = coroutine.running() +``` + +Some types have special syntax: +* `table` and `function` are not represented by name, but have their dedicated syntax as covered in this [syntax document](../../syntax) +* `userdata` is represented by [concrete types](../roblox-types), and +* `vector` is not representable by name at all + +The type checker also provides the builtin types [`unknown`](#unknown-type), [`never`](#never-type), and [`any`](#any-type). + +```luau +local s = "foo" +local n = 1 +local b = true +local t = coroutine.running() + +local a: any = 1 +print(a.x) -- Type checker believes this to be ok, but crashes at runtime. +``` + +#### Special behavior with `nil` + +There's a special case where we intentionally avoid inferring `nil` for local variables. This allows you to declare variables first and assign values to them later — if we inferred `nil`, you wouldn't be able to assign other types to these variables. + +```luau +local a +local b = nil +``` + +### `unknown` type + +`unknown` is also said to be the _top_ type, that is it's a union of all types. + +```luau +local a: unknown = "hello world!" +local b: unknown = 5 +local c: unknown = function() return 5 end +``` + +Unlike `any`, `unknown` will not allow itself to be used as a different type! + +```luau +local function unknown(): unknown + return if math.random() > 0.5 then "hello world!" else 5 +end + +local a: string = unknown() -- not ok +local b: number = unknown() -- not ok +local c: string | number = unknown() -- not ok +``` + +In order to turn a variable of type `unknown` into a different type, you must apply [type refinements](../types/refinements.md) on that variable. + +```luau +local function unknown(): unknown return 5 end +local x = unknown() +if typeof(x) == "number" then + print(x) -- x : number +end +``` + +### `never` type + +`never` is also said to be the _bottom_ type, meaning there doesn't exist a value that inhabits the type `never`. In fact, it is the _dual_ of `unknown`. `never` is useful in many scenarios, and one such use case is when type refinements proves it impossible: + +```luau +local function unknown(): unknown return 5 end +local x = unknown() +if typeof(x) == "number" and typeof(x) == "string" then + print(x) -- x : never +end +``` + +### `any` type + +`any` is just like `unknown`, except that it allows itself to be used as an arbitrary type without further checks or annotations. Essentially, it's an opt-out from the type system entirely. + +```luau +local x: any = 5 +local y: string = x -- no type errors here! +``` + +## Function types + +Let's start with something simple. + +```luau +local function f(x) return x end + +local a: number = f(1) -- ok +local b: string = f("foo") -- ok +local c: string = f(true) -- not ok +``` + +In strict mode, the inferred type of this function `f` is `(A) -> A` (take a look at [generics](../generics)), whereas in nonstrict we infer `(any) -> any`. We know this is true because `f` can take anything and then return that. If we used `x` with another concrete type, then we would end up inferring that. + +Similarly, we can infer the types of the parameters with ease. By passing a parameter into *anything* that also has a type, we are saying "this and that has the same type." + +```luau +local function greetingsHelper(name: string) + return "Hello, " .. name +end + +local function greetings(name) + return greetingsHelper(name) +end + +print(greetings("Alexander")) -- ok +print(greetings({name = "Alexander"})) -- not ok +``` + +## Variadic types + +Luau permits assigning a type to the `...` variadic symbol like any other parameter: + +```luau +local function f(...: number) +end + +f(1, 2, 3) -- ok +f(1, "string") -- not ok +``` + +`f` accepts any number of `number` values. + +In type annotations, this is written as `...T`: + +```luau +type F = (...number) -> ...string +``` + +## Type packs + +Multiple function return values as well as the function variadic parameter use a type pack to represent a list of types. + +When a type alias is defined, generic type pack parameters can be used after the type parameters: + +```luau +type Signal = { f: (T, U...) -> (), data: T } +``` + +> Keep in mind that `...T` is a variadic type pack (many elements of the same type `T`), while `U...` is a generic type pack that can contain zero or more types and they don't have to be the same. + +It is also possible for a generic function to reference a generic type pack from the generics list: + +```luau +type Signal = { f: (T, U...) -> (), data: T } + +local function call(s: Signal, ...: U...) + s.f(s.data, ...) +end +``` + +Generic types with type packs can be instantiated by providing a type pack: + +```luau +type Signal = { f: (T, U...) -> (), data: T } +local function call(s: Signal, ...: U...) end + +local signal: Signal = {} :: any + +call(signal, 1, 2, false) +``` + +There are also other ways to instantiate types with generic type pack parameters: + +```luau +type A = (T) -> U... + +type B = A -- with a variadic type pack +type C = A -- with a generic type pack +type D = A -- with an empty type pack +``` + +Trailing type pack argument can also be provided without parentheses by specifying variadic type arguments: + +```luau +type List = (Head, Rest...) -> () + +type B = List -- Rest... is () +type C = List -- Rest is (string, boolean) + +type Returns = () -> T... + +-- When there are no type parameters, the list can be left empty +type D = Returns<> -- T... is () +``` + +Type pack parameters are not limited to a single one, as many as required can be specified: + +```luau +type Callback = { f: (Args...) -> Rets... } + +type A = Callback<(number, string), ...number> +``` + +## Singleton types (aka literal types) + +Luau's type system also supports singleton types, which means it's a type that represents one single value at runtime. At this time, both string and booleans are representable in types. + +> We do not currently support numbers as types. For now, this is intentional. + +```luau +local foo: "Foo" = "Foo" -- ok +local bar: "Bar" = foo -- not ok +local baz: string = foo -- ok + +local t: true = true -- ok +local f: false = false -- ok +``` + +This happens all the time, especially through [type refinements](../type-refinements) and is also incredibly useful when you want to enforce program invariants in the type system! See [tagged unions](../unions-and-intersections/#tagged-unions) for more information. diff --git a/.cursor/rules/luau/types/considerations.md b/.cursor/rules/luau/types/considerations.md new file mode 100644 index 00000000..33c01b3d --- /dev/null +++ b/.cursor/rules/luau/types/considerations.md @@ -0,0 +1,42 @@ +--- +slug: types/considerations +title: Additional Considerations +sidebar: + order: 10 +--- + +## Module interactions + +Let's say that we have two modules, `Foo` and `Bar`. Luau will try to resolve the paths if it can find any `require` in any scripts. In this case, when you say `./bar`, Luau will resolve it as: relative to this script, go to my sibling script named Bar. + +```luau +--!file foo.luau +local Bar = require("./bar") + +local baz1: Bar.Baz = 1 -- not ok +local baz2: Bar.Baz = "foo" -- ok + +print(Bar.Quux) -- ok +print(Bar.FakeProperty) -- not ok + +Bar.NewProperty = true -- not ok + +--!file bar.luau +export type Baz = string + +local module = {} + +module.Quux = "Hello, world!" + +return module +``` + +There are some caveats here though. For instance, the require path must be resolvable statically, otherwise Luau cannot accurately type check it. + +### Cyclic module dependencies + +Cyclic module dependencies can cause problems for the type checker. In order to break a module dependency cycle a typecast of the module to `any` may be used: +```luau +--!hidden mode=nocheck +local myModule = require(MyModule) :: any +``` diff --git a/.cursor/rules/luau/types/generics.md b/.cursor/rules/luau/types/generics.md new file mode 100644 index 00000000..0adb651d --- /dev/null +++ b/.cursor/rules/luau/types/generics.md @@ -0,0 +1,71 @@ +--- +slug: types/generics +title: Generics and Polymorphism +sidebar: + order: 5 +--- + +The type inference engine was built from the ground up to recognize generics. A generic is simply a type parameter in which another type could be slotted in. It's extremely useful because it allows the type inference engine to remember what the type actually is, unlike `any`. + +```luau +type Pair = {first: T, second: T } +-- generics can also have defaults! +type PairWithDefault = Pair + +local strings: Pair = { first="Hello", second="World" } +local numbers: Pair = { first=1, second=2 } +-- can just treat PairWithDefault as a type that doesnt have generics because it has all of its generics assigned a default! +local more_strings: PairWithDefault = { first = "meow", second = "mrrp" } +``` + +## Generic functions + +As well as generic type aliases like `Pair`, Luau supports generic functions. These are functions that, as well as their regular data parameters, take type parameters. For example, a function which reverses an array is: +```luau +function reverse(a) + local result = {} + for i = #a, 1, -1 do + table.insert(result, a[i]) + end + return result +end +``` +The type of this function is that it can reverse an array, and return an array of the same type. Luau can infer this type, but if you want to be explicit, you can declare the type parameter `T`, for example: +```luau +function reverse(a: {T}): {T} + local result: {T} = {} + for i = #a, 1, -1 do + table.insert(result, a[i]) + end + return result +end +``` +When a generic function is called, Luau infers type arguments, for example +```luau +--!file main.luau +local reverse = require("./reverse") +local x: {number} = reverse({1, 2, 3}) +local y: {string} = reverse({"a", "b", "c"}) + +--!file reverse.luau +function reverse(a: {T}): {T} + local result: {T} = {} + for i = #a, 1, -1 do + table.insert(result, a[i]) + end + return result +end + +return reverse +``` +Generic types are used for built-in functions as well as user functions, +for example the type of two-argument `table.insert` is: +``` +({T}, T) -> () +``` +Note: Functions don't support having defaults assigned to generics, meaning the following is invalid +```luau +function meow(mrrp: T) + print(mrrp .. " :3") +end +``` diff --git a/.cursor/rules/luau/types/object-oriented-programs.md b/.cursor/rules/luau/types/object-oriented-programs.md new file mode 100644 index 00000000..d749c667 --- /dev/null +++ b/.cursor/rules/luau/types/object-oriented-programs.md @@ -0,0 +1,84 @@ +--- +slug: types/object-oriented-programs +title: Object-Oriented Programming +sidebar: + order: 7 +--- + +## Adding types for faux object oriented programs + +One common pattern we see with existing Lua/Luau code is the following object-oriented code. While Luau is capable of inferring a decent chunk of this code, it cannot pin down on the types of `self` when it spans multiple methods. + +```luau +local Account = {} +Account.__index = Account + +function Account.new(name, balance) + local self = {} + self.name = name + self.balance = balance + + return setmetatable(self, Account) +end + +-- The `self` type is different from the type returned by `Account.new` +function Account:deposit(credit) + self.balance += credit +end + +-- The `self` type is different from the type returned by `Account.new` +function Account:withdraw(debit) + self.balance -= debit +end + +local account = Account.new("Alexander", 500) +``` + +For example, the type of `Account.new` is `(name: a, balance: b) -> { ..., name: a, balance: b, ... }` (snipping out the metatable). For better or worse, this means you are allowed to call `Account.new(5, "hello")` as well as `Account.new({}, {})`. In this case, this is quite unfortunate, so your first attempt may be to add type annotations to the parameters `name` and `balance`. + +There's the next problem: the type of `self` is not shared across methods of `Account`, this is because you are allowed to explicitly opt for a different value to pass as `self` by writing `account.deposit(another_account, 50)`. As a result, the type of `Account:deposit` is `(self: { balance: a }, credit: b) -> ()`. Consequently, Luau cannot infer the result of the `+` operation from `a` and `b`, so a type error is reported. + +We can see there's a lot of problems happening here. This is a case where you'll have to provide some guidance to Luau in the form of annotations today, but the process is straightforward and without repetition. You first specify the type of _data_ you want your class to have, and then you define the class type separately with `setmetatable` (either via `typeof`, or in the New Type Solver, the `setmetatable` type function). +From then on, you can explicitly annotate the `self` type of each method with your class type! Note that while the definition is written e.g. `Account.deposit`, you can still call it as `account:deposit(...)`. + +```luau +local Account = {} +Account.__index = Account + +type AccountData = { + name: string, + balance: number, +} + +export type Account = typeof(setmetatable({} :: AccountData, Account)) +-- or alternatively, in the new type solver... +-- export type Account = setmetatable + + +-- this return annotation is not required, but ensures that you cannot +-- accidentally make the constructor incompatible with the methods +function Account.new(name, balance): Account + local self = {} + self.name = name + self.balance = balance + + return setmetatable(self, Account) +end + +-- this annotation on `self` is currently a _required_ annotation. +function Account.deposit(self: Account, credit: number) + -- autocomplete on `self` works here! + self.balance += credit +end + +-- this annotation on `self` is currently a _required_ annotation. +function Account.withdraw(self: Account, debit: number) + -- autocomplete on `self` works here! + self.balance -= debit +end + +local account = Account.new("Hina", 500) +account:deposit(20) -- this still works, and we had autocomplete after hitting `:`! +``` + +Based on feedback, we plan to restrict the types of all functions defined with `:` syntax to [share their self types](https://rfcs.luau.org/shared-self-types.html). This will enable future versions of this code to work without any explicit `self` annotations because it amounts to having type inference make precisely the assumptions we are encoding with annotations here --- namely, that the type of the constructors and the method definitions is intended by the developer to be the same. diff --git a/.cursor/rules/luau/types/overview.md b/.cursor/rules/luau/types/overview.md new file mode 100644 index 00000000..3d4226ba --- /dev/null +++ b/.cursor/rules/luau/types/overview.md @@ -0,0 +1,160 @@ +--- +slug: types +title: An introduction to Luau types +sidebar: + order: 1 +--- + +Luau supports a gradual type system through the use of type annotations and type inference. These types are used to provide warnings, errors, and suggestions for our developers. Type checking helps you find bugs early - while you're writing code - rather than discovering when your program crashes at runtime. + +## Type inference modes + +Luau offers three different modes that control how strictly it checks your types. You can set the mode by adding one of the following to the top of your file: + +* `--!nocheck`, +* `--!nonstrict` (default), and +* `--!strict` + +##### `--!nocheck` + +`nocheck` mode completely disables the type inference engine for the file. + +With this mode enabled, we don't provide any feedback on the types, so scripts with errors (like following example that tries to add a `string` and a `number`) will not raise a typecheck error, even though the program errors when executed. + +Try pressing the `Check` and `Run` buttons to try out this snippet: + +```luau +--!nocheck + +local foo = 1 +local x = "hello " + foo -- No error, type checking is disabled +print(foo) +``` + +If, instead, you want to catch these kinds of errors before running your code, you can enable type checking with the `--!nonstrict` or `--!strict` modes. Both of these modes analyze your types and provide helpful feedback, but they differ slightly in how strictly they enforce type safety. + +##### `--!nonstrict` + +In `nonstrict` mode, the type checker is more forgiving: if we can't figure out what type something is early on, we infer the type could be anything (the `any` type) and let you proceed without errors. + +This means that if we define a variable without initializing it or specifying its type: + +```luau +--!nonstrict + +local foo +foo = 1 +foo = "hello " + foo -- Still no error, foo is 'any' +``` + +The type checker can't tell what type `foo` should be when it's first declared, so in `nonstrict` mode it infers `any` - allowing the addition to proceed without warnings, even though it will error at runtime. + +##### `--!strict` + +In `strict` mode, Luau is smarter about tracking types across statements. Given the previous example now in `strict` mode: + +```luau +--!strict + +local foo +foo = 1 +foo = "hello " + foo +``` + +We see an error! The type checker sees `foo = 1` on line 4 and infers that `foo` must be a `number`. Then, when you try to add `"hello "` and `foo`, `strict` mode catches our error of trying to add a `string` and a `number`, which isn't allowed. (tip: hover over the erroneous lines to see the error message!) + +## Structural type system + +Luau's type system is structural by default, which is to say that we inspect the shape of two tables to see if they are similar enough. This was chosen because Lua 5.1 is inherently structural. + +Take, for example, these two tables `A` and `B`: + +```luau +--!hidden FIXME(LUAU): there's a bug in the new type solver unfortunately +--!hidden solver=old +type A = {x: number, y: number, z: number?} +type B = {x: number, y: number, z: number} + +local a1: A = {x = 1, y = 2} -- ok +local b1: B = {x = 1, y = 2, z = 3} -- ok + +local a2: A = b1 -- ok +local b2: B = a1 -- not ok +``` + +## Type annotations + +While Luau's type system can infer many types, you can also annotate types using a colon (`:`) followed by the type: + +```luau +-- Variable annotations +local age: number = 25 +local names: {string} = {"Alice", "Bob"} + +-- Function annotations +function greet(name: string): string + return "Hello, " .. name +end + +-- Table type annotations +type Person = { + name: string, + age: number, + email: string? +} + +local user: Person = { + name = "Alice", + age = 30 +} + +print(greet(user.name)) +``` + +## Type casts + +Sometimes, you might want to specify the type of an expression when the automatically inferred type is too generic. In these cases, you can use a type cast with the `::` operator to tell Luau what type something should be. + +For example, consider the following table constructor where the intent is to store a table of names: + +```luau +local myTable = {names = {}} +table.insert(myTable.names, 42) +``` + +Inserting a number into `names` ought to cause a type error, but doesn't. + +In order to specify the type of the `names` table, we can use a typecast: + +```luau +local myTable = {names = {} :: {string}} +table.insert(myTable.names, 42) +``` + +Now, inserting a number raises a type error, informing the user that there is an `invalid 'number' to 'string' conversion`. + +### Type cast rules + +Type casts themselves are also type checked to ensure that one of the conversion operands is the subtype of the other or `any`: + +```luau +local numericValue = 1 +local value = numericValue :: any -- ok, all expressions may be cast to 'any' +local flag = numericValue :: boolean -- not ok, invalid 'number' to 'boolean' conversion +``` + +This prevents unsafe casts between incompatible types. + +#### Type casts with multiple return values + +When typecasting a variadic or the result of a function with multiple returns, only the first value will be preserved, and the rest discarded: + +```luau +function returnsMultiple(...): (number, number, number) + print(... :: string) -- "x" + return 1, 2, 3 +end + +print(returnsMultiple("x", "y", "z")) -- 1, 2, 3 +print(returnsMultiple("x", "y", "z") :: number) -- 1 +``` diff --git a/.cursor/rules/luau/types/refinements.md b/.cursor/rules/luau/types/refinements.md new file mode 100644 index 00000000..66e755e0 --- /dev/null +++ b/.cursor/rules/luau/types/refinements.md @@ -0,0 +1,82 @@ +--- +slug: types/type-refinements +title: Type Refinements +sidebar: + order: 6 +--- + +When we check the type of any lvalue (a global, a local, or a property), what we're doing is we're refining the type, hence "type refinement." The support for this is arbitrarily complex, so go at it! + +Here are all the ways you can refine: +1. Truthy test: `if x then` will refine `x` to be truthy. +2. Type guards: `if type(x) == "number" then` will refine `x` to be `number`. +3. Equality: `if x == "hello" then` will refine `x` to be a singleton type `"hello"`. + +And they can be composed with many of `and`/`or`/`not`. `not`, just like `~=`, will flip the resulting refinements, that is `not x` will refine `x` to be falsy. + +The `assert(..)` function may also be used to refine types instead of `if/then`. + +Using truthy test: +```luau +local maybeString: string? = nil + +if maybeString then + local onlyString: string = maybeString -- ok + local onlyNil: nil = maybeString -- not ok +end + +if not maybeString then + local onlyString: string = maybeString -- not ok + local onlyNil: nil = maybeString -- ok +end +``` + +Using `type` test: +```luau +local stringOrNumber: string | number = "foo" + +if type(stringOrNumber) == "string" then + local onlyString: string = stringOrNumber -- ok + local onlyNumber: number = stringOrNumber -- not ok +end + +if type(stringOrNumber) ~= "string" then + local onlyString: string = stringOrNumber -- not ok + local onlyNumber: number = stringOrNumber -- ok +end +``` + +Using equality test: +```luau +function f() return "meow" end +local myString: string = f() + +if myString == "hello" then + local hello: "hello" = myString -- ok because it is absolutely "hello"! + local copy: string = myString -- ok +end +``` + +And as said earlier, we can compose as many of `and`/`or`/`not` as we wish with these refinements: +```luau +local function f(x: any, y: any) + if (x == "hello" or x == "bye") and type(y) == "string" then + -- x is of type "hello" | "bye" + -- y is of type string + end + + if not (x ~= "hi") then + -- x is of type "hi" + end +end +``` + +`assert` can also be used to refine in all the same ways: +```luau +local stringOrNumber: string | number = "foo" + +assert(type(stringOrNumber) == "string") + +local onlyString: string = stringOrNumber -- ok +local onlyNumber: number = stringOrNumber -- not ok +``` diff --git a/.cursor/rules/luau/types/roblox-types.md b/.cursor/rules/luau/types/roblox-types.md new file mode 100644 index 00000000..3430d618 --- /dev/null +++ b/.cursor/rules/luau/types/roblox-types.md @@ -0,0 +1,34 @@ +--- +slug: types/roblox-types +title: Foreign Types from the Embedder +sidebar: + order: 9 +--- + +Roblox supports a rich set of classes and data types, [documented here](https://developer.roblox.com/en-us/api-reference). All of them are readily available for the type checker to use by their name (e.g. `Part` or `RaycastResult`). + +When one type inherits from another type, the type checker models this relationship and allows to cast a subclass to the parent class implicitly, so you can pass a `Part` to a function that expects an `Instance`. + +All enums are also available to use by their name as part of the `Enum` type library, e.g. `local m: Enum.Material = part.Material`. + +We can automatically deduce what calls like `Instance.new` and `game:GetService` are supposed to return: + +```luau +--!hidden mode=nocheck +local part = Instance.new("Part") +local basePart: BasePart = part +``` + +Finally, Roblox types can be refined using `IsA`: + +```luau +--!hidden mode=nocheck +local function getText(x : Instance) : string + if x:IsA("TextLabel") or x:IsA("TextButton") or x:IsA("TextBox") then + return child.Text + end + return "" +end +``` + +Note that many of these types provide some properties and methods in both lowerCase and UpperCase; the lowerCase variants are deprecated, and the type system will ask you to use the UpperCase variants instead. diff --git a/.cursor/rules/luau/types/tables.md b/.cursor/rules/luau/types/tables.md new file mode 100644 index 00000000..11aa8b36 --- /dev/null +++ b/.cursor/rules/luau/types/tables.md @@ -0,0 +1,87 @@ +--- +slug: types/tables +title: Table Types +sidebar: + order: 3 +--- + +From the type checker perspective, each table can be in one of three states. They are: `unsealed table`, `sealed table`, and `generic table`. This is intended to represent how the table's type is allowed to change. + +### Unsealed tables + +An unsealed table is a table which supports adding new properties, which updates the tables type. Unsealed tables are created using table literals. This is one way to accumulate knowledge of the shape of this table. + +```luau +local t = {x = 1} -- {x: number} +t.y = 2 -- {x: number, y: number} +t.z = 3 -- {x: number, y: number, z: number} +``` + +However, if this local were written as `local t: { x: number } = { x = 1 }`, it ends up sealing the table, so the two assignments henceforth will not be ok. + +Furthermore, once we exit the scope where this unsealed table was created in, we seal it. + +```luau +local function vec2(x, y) + local t = {} + t.x = x + t.y = y + return t +end + +local v2 = vec2(1, 2) +v2.z = 3 -- not ok +``` + +Unsealed tables are *exact* in that any property of the table must be named by the type. Since Luau treats missing properties as having value `nil`, this means that we can treat an unsealed table which does not mention a property as if it mentioned the property, as long as that property is optional. + +```luau +local t = {x = 1} +local u : { x : number, y : number? } = t -- ok because y is optional +local v : { x : number, z : number } = t -- not ok because z is not optional +``` + +### Sealed tables + +A sealed table is a table that is now locked down. This occurs when the table type is spelled out explicitly via a type annotation, or if it is returned from a function. + +```luau +local t : { x: number } = {x = 1} +t.y = 2 -- not ok +``` + +Sealed tables are *inexact* in that the table may have properties which are not mentioned in the type. +As a result, sealed tables support *width subtyping*, which allows a table with more properties to be used as a table with fewer properties. + +```luau +type Point1D = { x : number } +type Point2D = { x : number, y : number } +local p : Point2D = { x = 5, y = 37 } +local q : Point1D = p -- ok because Point2D has more properties than Point1D +``` + +### Generic tables + +This typically occurs when the symbol does not have any annotated types or were not inferred anything concrete. In this case, when you index on a parameter, you're requesting that there is a table with a matching interface. + +```luau +local function f(t) + return t.x + t.y + --^ --^ {x: _, y: _} +end + +f({x = 1, y = 2}) -- ok +f({x = 1, y = 2, z = 3}) -- ok +f({x = 1}) -- not ok +``` + +## Table indexers + +These are particularly useful for when your table is used similarly to an array. + +```luau +local t = {"Hello", "world!"} -- {[number]: string} +print(table.concat(t, ", ")) +``` + +Luau supports a concise declaration for array-like tables, `{T}` (for example, `{string}` is equivalent to `{[number]: string}`); the more explicit definition of an indexer is still useful when the key isn't a number, or when the table has other fields like `{ [number]: string, n: number }`. diff --git a/.cursor/rules/luau/types/type-functions.md b/.cursor/rules/luau/types/type-functions.md new file mode 100644 index 00000000..3da01d23 --- /dev/null +++ b/.cursor/rules/luau/types/type-functions.md @@ -0,0 +1,53 @@ +--- +slug: types/type-functions +title: Type Functions +sidebar: + order: 8 +--- + +Type functions are functions that run during analysis time and operate on types, instead of runtime values. They can use the [types](../../types-library) library to transform existing types or create new ones. + + +Here's a simplified implementation of the builtin type function `keyof`. It takes a table type and returns its property names as a [union](../unions-and-intersections#union-types) of [singletons](../basic-types#singleton-types-aka-literal-types). + +```luau +type function simple_keyof(ty) + -- Ignoring unions or intersections of tables for simplicity. + if not ty:is("table") then + error("Can only call keyof on tables.") + end + + local union = nil + + for property in ty:properties() do + union = if union then types.unionof(union, property) else property + end + + return if union then union else types.singleton(nil) +end + +type person = { + name: string, + age: number, +} +--- keys = "age" | "name" +type keys = simple_keyof +``` + +### Type function environment + +In addition to the [types](../../types-library) library, type functions have access to: + +* `assert`, `error`, `print` +* `next`, `ipairs`, `pairs` +* `select`, `unpack` +* `getmetatable`, `setmetatable` +* `rawget`, `rawset`, `rawlen`, `raweq` +* `tonumber`, `tostring` +* `type`, `typeof` +* `math` library +* `table` library +* `string` library +* `bit32` library +* `utf8` library +* `buffer` library diff --git a/.cursor/rules/luau/types/unions-and-intersections.md b/.cursor/rules/luau/types/unions-and-intersections.md new file mode 100644 index 00000000..22bef3c5 --- /dev/null +++ b/.cursor/rules/luau/types/unions-and-intersections.md @@ -0,0 +1,92 @@ +--- +slug: types/unions-and-intersections +title: Union and Intersection Types +sidebar: + order: 4 +--- + +## Union types + +A union type represents *one of* the types in this set. If you try to pass a union onto another thing that expects a *more specific* type, it will fail. + +For example, what if this `string | number` was passed into something that expects `number`, but the passed in value was actually a `string`? + +```luau +local stringOrNumber: string | number = "foo" + +local onlyString: string = stringOrNumber -- not ok +local onlyNumber: number = stringOrNumber -- not ok +``` + +Note: it's impossible to be able to call a function if there are two or more function types in this union. + +### Tagged unions + +Tagged unions are just union types! In particular, they're union types of tables where they have at least _some_ common properties but the structure of the tables are different enough. Here's one example: + +```luau +type Ok = { type: "ok", value: T } +type Err = { type: "err", error: E } +type Result = Ok | Err +``` + +This `Result` type can be discriminated by using type refinements on the property `type`, like so: + +```luau +--!file main.luau +local res = require("./result") +local result: res.Result = res.something() + +if result.type == "ok" then + -- result is known to be Ok + -- and attempting to index for error here will fail + print(result.value) +elseif result.type == "err" then + -- result is known to be Err + -- and attempting to index for value here will fail + print(result.error) +end + +--!file result.luau +export type Ok = { type: "ok", value: T } +export type Err = { type: "err", error: E } +export type Result = Ok | Err + +return { + something = function(): any return {} end +} +``` + +Which works out because `value: T` exists only when `type` is in actual fact `"ok"`, and `error: E` exists only when `type` is in actual fact `"err"`. + + +## Intersection types + +An intersection type represents *all of* the types in this set. It's useful for two main things: to join multiple tables together, or to specify overloadable functions. + +```luau +type XCoord = {x: number} +type YCoord = {y: number} +type ZCoord = {z: number} + +type Vector2 = XCoord & YCoord +type Vector3 = XCoord & YCoord & ZCoord + +local vec2: Vector2 = {x = 1, y = 2} -- ok +local vec3: Vector3 = {x = 1, y = 2, z = 3} -- ok +``` + +```luau +type SimpleOverloadedFunction = ((string) -> number) & ((number) -> string) + +local f: SimpleOverloadedFunction + +local r1: number = f("foo") -- ok +local r2: number = f(12345) -- not ok +local r3: string = f("foo") -- not ok +local r4: string = f(12345) -- ok +``` + +Note: it's impossible to create an intersection type of some primitive types, e.g. `string & number`, or `string & boolean`, or other variations thereof. + +Note: Luau still does not support user-defined overloaded functions. Some of Roblox and Lua 5.1 functions have different function signature, so inherently requires overloaded functions. diff --git a/.cursor/rules/persona.mdc b/.cursor/rules/persona.mdc index 8faceef0..f667ceee 100644 --- a/.cursor/rules/persona.mdc +++ b/.cursor/rules/persona.mdc @@ -1,5 +1,4 @@ --- -description: Agent behavior constraints and communication style alwaysApply: true --- diff --git a/.cursor/rules/plugin.mdc b/.cursor/rules/plugin.mdc deleted file mode 100644 index b2e7efdd..00000000 --- a/.cursor/rules/plugin.mdc +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: "Rojo Studio plugin specifics: Roact patterns, reconciler logic, WebSocket protocol, and plugin testing." -globs: - - "plugin/**/*.lua" - - "plugin/**/*.luau" - - "plugin/**/*.json" ---- - -# Rojo Studio Plugin - -Built with Roact (React-like), handles WebSocket sync with server, patch application to Studio, and UI. - -**Entry:** `plugin/src/init.server.lua` | **Protocol:** WebSocket v5 | **Must match server MAJOR.MINOR** - -## Architecture - -### Key Components - -**1. UI Layer** (`App/`) -- **App.lua** - Root component, manages connection state -- **Components/** - Reusable UI components - - `StatusPages/` - Connected, Connecting, Error, Settings screens - - `PatchVisualizer/` - Shows changes before applying - - `StringDiffVisualizer/` - Side-by-side script diffs - - `TableDiffVisualizer/` - Attributes and Tags diffs - - `Notifications/` - Toast notifications -- **Theme.lua** - Studio-matched color palette -- Uses Roact for declarative UI - -**2. Communication** (`ServeSession.lua`) -- WebSocket connection to server (previously long-polling) -- Handles patches, errors, reconnection -- Protocol version checking - -**3. Reconciler** (`Reconciler/`) -- **applyPatch.lua** - Applies server patches to Studio tree -- **reify.lua** - Creates instances from snapshots -- **hydrate.lua** - Updates existing instances -- **diff.lua** - Generates patches (for two-way sync) -- **decodeValue.lua** - Converts encoded values to Roblox types -- **setProperty.lua** - Sets properties safely -- **getProperty.lua** - Gets properties with fallbacks - -**4. State Management** -- **Settings.lua** - Persistent plugin settings -- **InstanceMap.lua** - Tracks server ID → Studio instance mapping -- **PatchSet.lua** / **PatchTree.lua** - Patch representation - -**5. Testing** -- Unit tests: `.spec.lua` files (TestEZ) -- Integration tests: `integration/` directory -- Stress tests: `.stress.spec.lua` files -- Test runner: `runTests.lua` - -## Key Configuration (`Config.lua`) - -```lua -{ - version = {7, 7, 0, "-rc.1"}, - protocolVersion = 5, -- MUST match server - ignoredClassNames = { TouchTransmitter = true, Camera = true } -} -``` - -**When changing protocol:** Bump `protocolVersion` in both plugin AND server. - -## WebSocket Protocol (v5) - -1. Connect to `ws://host:port/api/rojo` -2. Server sends: protocol version, project name -3. Plugin sends: `{"type": "Subscribe"}` -4. Server sends patches on filesystem changes - -**Patch structure:** -```lua -{ - type = "ApplyPatch", - sessionId = "uuid", - patch = { - removed = {"id1", "id2"}, -- Array of IDs (Vec) - added = { ["id3"] = {...} }, -- Map: ID → Instance data - updated = { -- Array of updates (Vec) - { id = "id4", changedName = "...", changedProperties = {...} } - } - } -} -``` - -## Reconciler (`Reconciler/`) - -**applyPatch.lua** - Main patch application: -1. Remove instances in `removed` -2. Create instances in `added` via `reify` -3. Update instances in `updated` via `hydrate` -4. Collect errors, show in UI - -**reify.lua** - Create instances (parent last - Roblox best practice) -**hydrate.lua** - Update existing (only changed properties) -**setProperty.lua** / **getProperty.lua** - Handle unscriptable properties - -## UI Components (`App/`) - -**StatusPages:** NotConnected, Connecting, Connected, Error, Settings -**PatchVisualizer:** Tree view of changes with property diffs -**StringDiffVisualizer:** Side-by-side script diffs -**TableDiffVisualizer:** Attributes and Tags diffs -**Notifications:** Toast messages -**Theme.lua:** Studio-matched colors (auto light/dark) - -## InstanceMap - -Bidirectional Server ID ↔ Studio Instance mapping: - -```lua -instanceMap:insert(serverId, studioInstance) -local instance = instanceMap:fromId(serverId) -local serverId = instanceMap:fromInstance(instance) -``` - -Critical for patch application and avoiding duplicates. - -## Two-Way Sync (Experimental) - -⚠️ Watches Studio changes, sends to server. Has known issues - use with caution. - -## Dependencies (`plugin/Packages/`) - -Roact, Flipper, Promise, TestEZ, t, Highlighter, msgpack-luau. **Don't edit vendored packages.** - -## Common Issues - -**Plugin not reloading:** Enable Studio setting "Reload plugins on file change" -**Connection fails:** Check server running, host/port match, protocol versions match -**Properties not syncing:** Verify scriptable in rbx_reflection_database -**Duplicates:** InstanceMap corruption - disconnect, restart Studio - -## Debugging - -Settings → Log Level → "Trace" for verbose output. `ROJO_DEV_BUILD` instance enables auto-tests. diff --git a/.cursor/rules/python.mdc b/.cursor/rules/python.mdc index 4330332e..d35b54e7 100644 --- a/.cursor/rules/python.mdc +++ b/.cursor/rules/python.mdc @@ -1,6 +1,5 @@ --- -description: Python code style and conventions -globs: ["**/*.py"] +globs: **/*.py alwaysApply: false --- diff --git a/.cursor/rules/roblox.mdc b/.cursor/rules/roblox.mdc index a0a26b91..476efa66 100644 --- a/.cursor/rules/roblox.mdc +++ b/.cursor/rules/roblox.mdc @@ -1,5 +1,4 @@ --- -description: Index for Roblox API documentation in .cursor/rules/roblox/ alwaysApply: true --- diff --git a/.cursor/rules/roblox/common/navigation/documentation.yaml b/.cursor/rules/roblox/common/navigation/documentation.yaml index 483f6cd1..e3853add 100644 --- a/.cursor/rules/roblox/common/navigation/documentation.yaml +++ b/.cursor/rules/roblox/common/navigation/documentation.yaml @@ -27,7 +27,7 @@ columns: path: /production/analytics/ navigation: analytics.yaml - title: Platform - path: /platform + path: /get-started/ navigation: /platform.yaml - title: Tutorials navigation: /engine/tutorials.yaml diff --git a/.cursor/rules/roblox/common/navigation/engine/guides.yaml b/.cursor/rules/roblox/common/navigation/engine/guides.yaml index 5508291c..a71ca53e 100644 --- a/.cursor/rules/roblox/common/navigation/engine/guides.yaml +++ b/.cursor/rules/roblox/common/navigation/engine/guides.yaml @@ -144,6 +144,8 @@ navigation: path: /physics/constraints/torque - title: LineForce path: /physics/constraints/line-force + - title: Animation + path: /physics/constraints/animation - title: Sleep system path: /physics/sleep-system - title: Adaptive timestepping diff --git a/.cursor/rules/roblox/common/navigation/platform.yaml b/.cursor/rules/roblox/common/navigation/platform.yaml index 03a7f638..faaa7de4 100644 --- a/.cursor/rules/roblox/common/navigation/platform.yaml +++ b/.cursor/rules/roblox/common/navigation/platform.yaml @@ -1,23 +1,36 @@ navigation: - heading: Platform - - - title: Get started - path: /get-started + - title: Get Started + path: /get-started/ + section: + - title: Why should I develop on Roblox? + path: /get-started/why-build-on-roblox + - title: How is Roblox different from other platforms? + path: /get-started/how-is-roblox-different + - title: What tools does Roblox provide? + path: /get-started/tools + - title: How do I make money? + path: /get-started/monetization + - title: Recommended strategies + path: /get-started/strategies + - title: Start creating + path: /get-started/creating + - title: Resources + path: /get-started/resources - title: Create path: /platform - title: Scale path: /scale - title: Monetize path: /monetize - section: - - title: Earn on Roblox - path: /production/earn-on-roblox - title: Guidelines and policies section: - title: Safety on Roblox path: /safety - title: Content maturity path: /production/promotion/content-maturity + - title: Regional content availability + path: /production/promotion/regional-content-availability - title: GDPR and CCPA path: /production/publishing/about-GDPR-and-CCPA - title: DMCA diff --git a/.cursor/rules/roblox/en-us/art/accessories/clothing-specifications.md b/.cursor/rules/roblox/en-us/art/accessories/clothing-specifications.md index f3fc717b..5f9abc09 100644 --- a/.cursor/rules/roblox/en-us/art/accessories/clothing-specifications.md +++ b/.cursor/rules/roblox/en-us/art/accessories/clothing-specifications.md @@ -142,7 +142,8 @@ Face accessories, such as hair, eyebrows, and eyelashes are unique accessories t ## Textures -Textures created for accessories must meet Roblox's [texture specifications](../../art/modeling/texture-specifications.md). Keep in mind that smaller assets, such as accessories, can use smaller texture map sizes, such as **256×256px**, to optimize performance with virtually no loss in detail. +- Textures for Marketplace assets can't exceed 2048x2048 resolution. +- Textures created for accessories must meet Roblox's [texture specifications](../../art/modeling/texture-specifications.md). High resolution textures are automatically converted to lower-resolution textures to optimize performance. ## Layerable Properties diff --git a/.cursor/rules/roblox/en-us/art/accessories/specifications.md b/.cursor/rules/roblox/en-us/art/accessories/specifications.md index 8cb5da65..56edecc7 100644 --- a/.cursor/rules/roblox/en-us/art/accessories/specifications.md +++ b/.cursor/rules/roblox/en-us/art/accessories/specifications.md @@ -277,9 +277,8 @@ You can use tools like the [Accessory Fitting Tool](../../art/accessories/access ## Textures -Textures created for accessories must meet Roblox's [texture specifications](../../art/modeling/texture-specifications.md). - -- Non-albedo maps (RGH, MTL, NOR) for rigid accessories cannot exceed 256x256. +- Textures for Marketplace assets cannot exceed 2048x2048 resolution. +- Textures created for accessories must meet Roblox's [texture specifications](../../art/modeling/texture-specifications.md). ## Attachment points diff --git a/.cursor/rules/roblox/en-us/art/characters/specifications.md b/.cursor/rules/roblox/en-us/art/characters/specifications.md index 95167a1a..6ec82822 100644 --- a/.cursor/rules/roblox/en-us/art/characters/specifications.md +++ b/.cursor/rules/roblox/en-us/art/characters/specifications.md @@ -652,7 +652,9 @@ For information on required facial animation support, see [head specifications]( ## Textures -Avatar characters must follow Roblox's [texture specifications](../../art/modeling/texture-specifications.md). Characters created for the Marketplace can take advantage of [custom skin tones](#custom-skin-tone) which use alpha layers allow users to select their own base color. +- Textures for Marketplace assets can't exceed 2048x2048 resolution. +- Textures created for accessories must meet Roblox's [texture specifications](../../art/modeling/texture-specifications.md). High resolution textures are automatically converted to lower-resolution textures to optimize performance. +- Avatar characters must follow Roblox's [texture specifications](../../art/modeling/texture-specifications.md). Characters created for the Marketplace can take advantage of [custom skin tones](#custom-skin-tone) which use alpha layers allow users to select their own base color. ### Custom skin tone diff --git a/.cursor/rules/roblox/en-us/art/modeling/surface-appearance.md b/.cursor/rules/roblox/en-us/art/modeling/surface-appearance.md index ba90c20e..19b3eb2e 100644 --- a/.cursor/rules/roblox/en-us/art/modeling/surface-appearance.md +++ b/.cursor/rules/roblox/en-us/art/modeling/surface-appearance.md @@ -3,6 +3,8 @@ title: PBR textures description: PBR textures are advanced textures using multiple texture maps. --- +import BetaAlert from '../../includes/beta-features/beta-alert.md' + **Physically-Based Rendering** (PBR) textures allow you to represent realistic shading and lighting by using multiple types of texture images, or **maps**, on a single object. Combining multiple texture maps can more accurately simulate color, roughness, and reflectivity in any lighting environment and can enhance the visual elements of your assets and environment. @@ -140,15 +142,39 @@ For objects that require partial or complete sections of transparency, such as g You can apply transparency in two different behaviors by setting the following `Class.SurfaceAppearance.AlphaMode|AlphaMode` values: -- [Overlay](#overlay) — Overlays the `Class.SurfaceAppearance.ColorMap|ColorMap` over the underlying mesh's `Class.MeshPart.Color`. Color maps using **Overlay** reveal the base color of the mesh anywhere transparency is present. This is the default setting. +- [Opaque](#opaque) — Ignores the alpha channel of the `Class.SurfaceAppearance.ColorMap|ColorMap` and uses the color value directly from the color map. +- [Overlay](#overlay) — Overlays the `Class.SurfaceAppearance.ColorMap|ColorMap` over the underlying mesh's `Class.MeshPart.Color`. Color maps using overlay mode reveal the base color of the mesh anywhere transparency is present. This is the default setting. - [Transparency](#alpha-modes) — Removes the visible mesh based on transparency in the `ColorMap`. This renders the mesh see-through and does not reveal the original mesh color whenever transparency is present. - [TintMask](#tintmask) — Blends the tinted `Class.SurfaceAppearance.ColorMap|ColorMap` over the un-tinted `Class.SurfaceAppearance.ColorMap|ColorMap`. +##### Opaque + + + +You can use `Enum.AlphaMode.Opaque` mode to ignore the alpha channel of the `Class.SurfaceAppearance.ColorMap|ColorMap` altogether. In this case the alpha value is assumed to be 1 (fully opaque). + +The following example demonstrates how the `Enum.AlphaMode.Opaque|Opaque` mode works using a white object as reference: + + +
+ A polkadot texture with transparency. The background is completely transparent. +
Example color/albedo map with transparency
+
+
+ A white torus on a plane. +
Example mesh object (white)
+
+
+ The torus has the polkadot patterns scattered over its surface. Transparency value is ignored. +
`Class.SurfaceAppearance.AlphaMode|AlphaMode` set to `Enum.AlphaMode.Opaque|Opaque` using reference map and material
+
+ + ##### Overlay -You can use **Overlay** to reveal sections of the mesh's original color. Since the transparent areas of the color map expose the underlying color, you can design a unique texture map that partially or fully reveals the mesh's `Class.MeshPart.Color|Color` property for custom skin tones or other situations with unique colors. +You can use `Enum.AlphaMode.Overlay` to reveal sections of the mesh's original color. Since the transparent areas of the color map expose the underlying color, you can design a unique texture map that partially or fully reveals the mesh's `Class.MeshPart.Color|Color` property for custom skin tones or other situations with unique colors. -The following example demonstrates how the **Overlay** mode works using a white sphere reference: +The following example demonstrates how the `Enum.AlphaMode.Overlay|Overlay` mode works using a white sphere reference:
@@ -161,11 +187,11 @@ The following example demonstrates how the **Overlay** mode works using a white
A white sphere with small blue squares scattered over the surface. -
`Class.SurfaceAppearance.AlphaMode|AlphaMode` set to **Overlay** using reference map and material
+
`Class.SurfaceAppearance.AlphaMode|AlphaMode` set to `Enum.AlphaMode.Overlay|Overlay` using reference map and material
 -The following example uses the **Overlay** mode for custom characters, revealing the character's original skin-tone: +The following example uses the `Enum.AlphaMode.Overlay|Overlay` mode for custom characters, revealing the character's original skin-tone:
@@ -174,7 +200,7 @@ The following example uses the **Overlay** mode for custom characters, revealing
Four of the same humanoid models with different skin tones: light brown, black, white, dark green. -
Several variations of characters with a single color map using **Overlay** mode
+
Several variations of characters with a single color map using `Enum.AlphaMode.Overlay|Overlay` mode
 @@ -182,7 +208,7 @@ See [custom skin tone](../../art/characters/specifications.md#custom-skin-tone) ##### Transparency -You can use the **Transparency** mode to create complex or extremely fine objects, such as lace or netting, by removing visible parts of the mesh as an alternative to sculpting the mesh geometry. Since this does not affect the geometry of the mesh object, this can allow you to create detailed objects without the performance impact of an intricate mesh model. +You can use the `Enum.AlphaMode.Transparency` mode to create complex or extremely fine objects, such as lace or netting, by removing visible parts of the mesh as an alternative to sculpting the mesh geometry. Since this does not affect the geometry of the mesh object, this can allow you to create detailed objects without the performance impact of an intricate mesh model. The following example demonstrates how a partial and full transparency in this mode visually removes sections of the mesh: @@ -221,7 +247,7 @@ The following example demonstrates how a partial and full transparency in this m -There are two outcomes of `Enum.AlphaMode|Transparency` mode, depending on whether the `Class.MeshPart.Transparency` is set to `0` or **at least** `0.02`, although numerical precision may result in small values such as `0.01` triggering the opaque transparency. +There are two outcomes of `Enum.AlphaMode.Transparency|Transparency` mode, depending on whether the `Class.MeshPart.Transparency` is set to `0` or **at least** `0.02`, although numerical precision may result in small values such as `0.01` triggering the opaque transparency. - When alpha is used to cut out the shape of mostly opaque objects with hard edges such as foliage, lace fabric, and netting, set the `Class.MeshPart.Transparency` to `0`. When parts of the surface are fully opaque, the Roblox engine can render them with proper depth‑based occlusion. Opaque surfaces also generally work better with depth‑based effects like `Class.DepthOfFieldEffect`, glass and water refraction, and water reflection. - When alpha is used to add detail to semi‑transparent objects or objects with smooth gradients of transparency, set the `Class.MeshPart.Transparency` to **at least** `0.02`. This improves the quality of blending for objects such as dirty windows and soft‑edged feathers, but it will not work with all effects. diff --git a/.cursor/rules/roblox/en-us/assets/audio/audio-objects/French-Female-Voice.wav b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/French-Female-Voice.wav new file mode 100644 index 00000000..c4507997 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/French-Female-Voice.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6e0921ee71090a803beb7eabe18adca534ff0dc92b0ba2cb2568cc404a36800e +size 175148 diff --git a/.cursor/rules/roblox/en-us/assets/audio/audio-objects/French-Male-Voice.wav b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/French-Male-Voice.wav new file mode 100644 index 00000000..75139d96 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/French-Male-Voice.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f1b38f276f16df08f3422d976a76359e0cab2d78fa6483df161051c593cf21ea +size 175148 diff --git a/.cursor/rules/roblox/en-us/assets/audio/audio-objects/German-Female-Voice.wav b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/German-Female-Voice.wav new file mode 100644 index 00000000..c95fbb5f --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/German-Female-Voice.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6ac6e95ad2327743f7637c2a4bc1cfbfb7a166084b6f9cd303342bba2fd54bf8 +size 144940 diff --git a/.cursor/rules/roblox/en-us/assets/audio/audio-objects/German-Male-Voice.wav b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/German-Male-Voice.wav new file mode 100644 index 00000000..c7482bb2 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/German-Male-Voice.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9a0b87d69c69545a1a2ef7cd657429f4831a3e5a5d1cd86b881659856fce792e +size 121900 diff --git a/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Italian-Female-Voice.wav b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Italian-Female-Voice.wav new file mode 100644 index 00000000..cf2c0d07 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Italian-Female-Voice.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2ea8e0ffa988b70b9b61fa631dfd528794f83478c2691e1295ea4c042173db6 +size 118828 diff --git a/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Italian-Male-Voice.wav b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Italian-Male-Voice.wav new file mode 100644 index 00000000..472dac1b --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Italian-Male-Voice.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aa9a65c96fa51876e32adb2a87d2a209f4fd51929402fccd56cc531fc7180eca +size 94252 diff --git a/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Narrator-Voice.wav b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Narrator-Voice.wav new file mode 100644 index 00000000..300c9e2b --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Narrator-Voice.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:02205bdcf3da43616fec1ae4e4d584110f7671dbfac524336a878154ed391210 +size 182316 diff --git a/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Spanish-Female-Voice.wav b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Spanish-Female-Voice.wav new file mode 100644 index 00000000..64da9cf7 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Spanish-Female-Voice.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9d56a65949dbb7dc2be7b10a945076ac64f47f9d645d42a5d62056251fe61910 +size 186412 diff --git a/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Spanish-Male-Voice.wav b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Spanish-Male-Voice.wav new file mode 100644 index 00000000..efcadcea --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/audio/audio-objects/Spanish-Male-Voice.wav @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5fb305e6b4bd54763aecc43b79abf20b9679bbd02a485bc169e1c7174bdc8ebd +size 143404 diff --git a/.cursor/rules/roblox/en-us/assets/avatar/2k-Texture-Renders.zip b/.cursor/rules/roblox/en-us/assets/avatar/2k-Texture-Renders.zip new file mode 100644 index 00000000..d96df080 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/avatar/2k-Texture-Renders.zip @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5e744bcca5cdbd072dcd60d7ac7ea57f5120535b8e0dc6c61a340bc8a48dd5f4 +size 353059568 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image1.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image1.png new file mode 100644 index 00000000..05a8d6ec --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image1.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:76c9082a4d6c2a018367281b9d334921db46ea941b169f13ac2bc11abf0dc797 +size 2750997 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image10.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image10.png new file mode 100644 index 00000000..9657a6d9 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image10.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:70810ab723611ddfaa3e5eb4c083957305705fe0a3bccf0172e6ff74b4a61317 +size 145292 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image11.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image11.png new file mode 100644 index 00000000..693d151b --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image11.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ad871abf054b942f18d6b9706a5809538379cc7cfe510644d1f68d3fe3425633 +size 129402 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image12.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image12.png new file mode 100644 index 00000000..560f9fab --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image12.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:30cf0564de491feab8d562b24a16591965cfa97b59e3a132c113aa6b3eb7d319 +size 70941 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image13.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image13.png new file mode 100644 index 00000000..80b9854e --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image13.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a81ba2a1a2e2ae0fc885058c915e331893ceed912cac1254c4915826b436028f +size 46340 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image14.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image14.png new file mode 100644 index 00000000..8cf0fcdb --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image14.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7117ccf49dc072e89b0e69a90ef1e2d83ea22ace7d7ae65ae436bab6ee13a966 +size 1117027 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image15.jpg b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image15.jpg new file mode 100644 index 00000000..f3c2e66e --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image15.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:918668cbcd41fbc173b2e35aaccd399ec033c00d30d074e8bf891a645d136376 +size 265039 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image16.jpg b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image16.jpg new file mode 100644 index 00000000..4284f9f4 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image16.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:955470e94499f57a24c36af154443329edcb2bf8f71f0c761866a4c21911b3dd +size 240939 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image17.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image17.png new file mode 100644 index 00000000..7835a223 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image17.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eb8622fb891b867d642ba45e843a643e43653ebdd8cd2ea36b3402c98a048a51 +size 315796 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image18.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image18.png new file mode 100644 index 00000000..4c6545b2 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image18.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8a3e9bc7a72e6e9a1cc62fbd4c1709b3af9360eb9688d7490638dc5d247019d5 +size 2013316 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image19.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image19.png new file mode 100644 index 00000000..0818cbf9 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image19.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bccbd83910720f1a5152a02b0d4441782df19453430a0345f71d1f658404de3d +size 187780 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image2.jpg b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image2.jpg new file mode 100644 index 00000000..ea178b60 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image2.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:41e4e5f7450f9e1be7500a2d7a8f14c6856fad50ef59d71e7ca572b58e135490 +size 1077476 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image20.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image20.png new file mode 100644 index 00000000..cedd71b8 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image20.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98057fe1a1cc8175a7f052267a3638d48a3e0f42270ed98ddf8e836c73e0d744 +size 267557 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image21.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image21.png new file mode 100644 index 00000000..53fad468 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image21.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a0786316db482ee25364af853eed51876bfb81ab4786db17323ef265af5d56eb +size 173660 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image22.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image22.png new file mode 100644 index 00000000..98e852c1 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image22.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c16bd7f4c9bc37a637ad009313c0c5a751f2fd29561408db9aba40845a7b57c5 +size 844696 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image23.gif b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image23.gif new file mode 100644 index 00000000..ce44ae45 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image23.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:85e1fd035de78a8ffc2bdc305c34b88bb3deebb4239f6c0f021d4d5750e92f60 +size 18031730 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image24.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image24.png new file mode 100644 index 00000000..224a706e --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image24.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aeb52ab29ae4c9a793a197451da5884dc2cb2a0baa9a31c971398c1b50b8ba38 +size 109093 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image25.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image25.png new file mode 100644 index 00000000..2c621c64 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image25.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5188fc9bcaedd345b0fbbe4e183c55f6bac3e72262a28d6b1ac14dbfb50f3c6c +size 216460 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image26.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image26.png new file mode 100644 index 00000000..d47ba542 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image26.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4f1198144379ee88204a500159dfb499b3ccee6a8e68f5cee5a9defa99286b20 +size 54884 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image27.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image27.png new file mode 100644 index 00000000..11b9a35e --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image27.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:36d924b74162a2e6421db3cc0c4223da87a5a99a4f119e80cf18b97eb17d65d7 +size 1066242 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image28.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image28.png new file mode 100644 index 00000000..091fee32 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image28.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e0b6f5c9d068ee3122a35c67c7628f0b1ab4493e23f6b68a0606150fc2b6c3cd +size 4925536 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image29.gif b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image29.gif new file mode 100644 index 00000000..d568d2a6 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image29.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1bf17a0b5e3ca1b7fa73def2a9bfd85df21708757cda49ce98c3ff860673f8d0 +size 40765240 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image3.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image3.png new file mode 100644 index 00000000..13fbbc7a --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image3.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e36b717e5562c5c9a66b2f00eaa9750b397e28959f7f505ec8324f96159a5e39 +size 178534 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image30.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image30.png new file mode 100644 index 00000000..e2c5f84b --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image30.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7150cf017752c682801bfdef709bf5e6638cee34e859f4cb4b36c9628c64ac38 +size 236647 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image31.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image31.png new file mode 100644 index 00000000..50800665 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image31.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d52a00914cfdce2fc0442eba7a54ded193247f2626a46a6bee5230d2bfbe585b +size 69756 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image32.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image32.png new file mode 100644 index 00000000..481f2e91 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image32.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fefd29af09fa54e9209f0ae93f8797892b15ba9819fac43e337f8496076ce1de +size 53332 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image33.gif b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image33.gif new file mode 100644 index 00000000..6ed3dec6 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image33.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:85374f4c89ae4ebfe67614c0f8c230db1cf8c372bd72ae4d7ed5ce4caee14f8e +size 18755456 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image34.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image34.png new file mode 100644 index 00000000..87b6a9dc --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image34.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd88fe5badff1d52f8028fe644b5f89176f4f67d0c375d0239835355c67a7ffb +size 262639 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image35.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image35.png new file mode 100644 index 00000000..9759b7bd --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image35.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d5581bfe016052d6e52c601022394d7a10e80682a14e6b8dc7e5c51c826f4525 +size 44798 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image36.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image36.png new file mode 100644 index 00000000..db2c4523 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image36.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:98e6bc16697cc21c923c0b35e213482eed92f47f7cd6325d4b87921cf1d4ee63 +size 167126 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image37.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image37.png new file mode 100644 index 00000000..97bf1aa7 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image37.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e480949344eb622abf5ce522690c9628f2a580e1797d33a83a5be0415dc13bf9 +size 3029750 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image38.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image38.png new file mode 100644 index 00000000..c2f6e0ec --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image38.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1fa29deefd40bd2d9483815031f5538cf703fe2648aff8834bcd680de78d633b +size 126616 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image39.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image39.png new file mode 100644 index 00000000..e92c7045 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image39.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b795366ded7792a4964e5c59645662031a48318954d63e2612e1fa0f964ef936 +size 450824 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image4.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image4.png new file mode 100644 index 00000000..42d824d5 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image4.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8c27305fada5e8a1aa1733246a5b87b8ea2acd866180f7dac34a1e61c3f0523 +size 165418 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image40.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image40.png new file mode 100644 index 00000000..a480b072 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image40.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:765565deb33074ea5f93660873936db067a762630332e6d1bd392ea90be20b02 +size 109376 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image41.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image41.png new file mode 100644 index 00000000..526444ae --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image41.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1a0ad3f6b427d71bab4256847590b2e692af1e035e332de9ae030254894262e0 +size 589826 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image42.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image42.png new file mode 100644 index 00000000..d1eea7ab --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image42.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dfd9f13a0f2ee429276c7455ce8d5dca41c328c1ca5b11fab9e2ee2243accb06 +size 3676154 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image43.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image43.png new file mode 100644 index 00000000..f7f04660 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image43.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1b005bc03997530b3aceaeb3df017e8d0869bba7e697e67e5c5702feee9ab88 +size 57886 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image44.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image44.png new file mode 100644 index 00000000..dda01af9 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image44.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af9f634df41ce54f01123fb11dd1af2b5e3e8093d4c7fb986255fbe5a0b99cd9 +size 5249243 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image45.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image45.png new file mode 100644 index 00000000..6369324a --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image45.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1309d258d20fd2dcad61a0b612be42a856c036ed54bd27d6b75c05fb3853bfe +size 187533 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image46.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image46.png new file mode 100644 index 00000000..4069dd7d --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image46.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:021edfda621770d17c7f861efbf5b4ad4ac1c51182dc3edc8b4a40c933c20420 +size 93610 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image47.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image47.png new file mode 100644 index 00000000..4e94fa64 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image47.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f9bb8633adb5593d6bd33111eb8ca7972b60b01851dac73e1ba82b6defa3f86 +size 1241629 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image48.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image48.png new file mode 100644 index 00000000..8e58b214 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image48.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c3779cfcb818207d45783f5431026138fdfe2e5617bbf139e3a272ce460e2e9b +size 189270 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image49.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image49.png new file mode 100644 index 00000000..cf6ed578 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image49.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ff2210aa55c0f7ad127ba18efb26995eeed0376dadf08420a1c81e94f132adcf +size 3877158 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image5.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image5.png new file mode 100644 index 00000000..6ddd215b --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image5.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:14c12a3464c1c970c31912f0444cf59b3a574d5ab97df815feee6ba60d06c9c7 +size 441294 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image50.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image50.png new file mode 100644 index 00000000..2e588682 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image50.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:df4f13605002ceb2702ee7fa6dcc31412746866f7c521b2b8dab808e28780cac +size 1360643 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image51.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image51.png new file mode 100644 index 00000000..b9548f3d --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image51.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:557c74590d011bf49a046f1061dc5155a2399766a02a21fcfd1b83632fddcb6a +size 89372 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image6.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image6.png new file mode 100644 index 00000000..8715330d --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image6.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b4d943dff04c4dce79616390951e8793ff2a857c00779b530be51b9684192b33 +size 390456 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image7.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image7.png new file mode 100644 index 00000000..a2cbe149 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image7.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:961d4deca1a44e6a48a263f7b1618b676d9f0e3e9c2944063f428dcf5a7b08a0 +size 816454 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image8.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image8.png new file mode 100644 index 00000000..d3ec2b35 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image8.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:af0eddb5a616851447e16211881c8c184da7104cfd39a45b4d628d33afefbd51 +size 2293973 diff --git a/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image9.png b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image9.png new file mode 100644 index 00000000..b812a434 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/developer-jumpstart/image9.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d63ea8c9322f391075b3730c8989bde2dc9f835c26c06d6cacf9565034f433ff +size 163176 diff --git a/.cursor/rules/roblox/en-us/assets/engine-api/datatypes/Font/Builder-Extended.png b/.cursor/rules/roblox/en-us/assets/engine-api/datatypes/Font/Builder-Extended.png new file mode 100644 index 00000000..bacbeebe --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/engine-api/datatypes/Font/Builder-Extended.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:19b1f77eddc0d4055712a37aeb0e7d41cbe388e8b979a4920b79943e736eb494 +size 89138 diff --git a/.cursor/rules/roblox/en-us/assets/engine-api/datatypes/Font/Builder-Mono.png b/.cursor/rules/roblox/en-us/assets/engine-api/datatypes/Font/Builder-Mono.png new file mode 100644 index 00000000..9c5df783 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/engine-api/datatypes/Font/Builder-Mono.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6275a0279861f5edecabd85d6f1017ac7b302ab3117028fe9781184b60b296ee +size 59182 diff --git a/.cursor/rules/roblox/en-us/assets/landing/monetization/hero.png b/.cursor/rules/roblox/en-us/assets/landing/monetization/hero.png new file mode 100644 index 00000000..cf6ed578 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/landing/monetization/hero.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ff2210aa55c0f7ad127ba18efb26995eeed0376dadf08420a1c81e94f132adcf +size 3877158 diff --git a/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial1.jpg b/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial1.jpg new file mode 100644 index 00000000..6aad4372 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial1.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8fd2373256bbfdc236ed4223cb47e524a74df36a422802a0574649c390575bcd +size 967928 diff --git a/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial2.jpg b/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial2.jpg new file mode 100644 index 00000000..73b3a30b --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial2.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b9acf01a291a3cece8ea76493c539c25fea1a52a0051d374836aa15b5c9e4d34 +size 6343420 diff --git a/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial3.png b/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial3.png new file mode 100644 index 00000000..8a826a52 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial3.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dbea877a77568ddb929d83f3319ab4c01dc5604456402395215551e062a70b99 +size 401951 diff --git a/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial4.png b/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial4.png new file mode 100644 index 00000000..fb951139 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/landing/monetization/testimonial4.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a79478787194f8a64dc7752c13dee1e2090c59fa5a06b26a54c3fa7096a27974 +size 5267580 diff --git a/.cursor/rules/roblox/en-us/assets/modeling/surface-appearance/SurfaceAppearance-Opaque-After.jpg b/.cursor/rules/roblox/en-us/assets/modeling/surface-appearance/SurfaceAppearance-Opaque-After.jpg new file mode 100644 index 00000000..9f0f432e --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/modeling/surface-appearance/SurfaceAppearance-Opaque-After.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:38d66b8622b26a10f3165b6376cede8bef4d642e295ab39e1317ba4fd1555e8f +size 207939 diff --git a/.cursor/rules/roblox/en-us/assets/modeling/surface-appearance/SurfaceAppearance-Opaque-Before.jpg b/.cursor/rules/roblox/en-us/assets/modeling/surface-appearance/SurfaceAppearance-Opaque-Before.jpg new file mode 100644 index 00000000..d741ff67 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/modeling/surface-appearance/SurfaceAppearance-Opaque-Before.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9c47b9a886f16e9a2fab955c09b162a9b834132ea51d539764b83545bd99a029 +size 176093 diff --git a/.cursor/rules/roblox/en-us/assets/modeling/surface-appearance/SurfaceAppearance-Opaque-ColorMap.jpg b/.cursor/rules/roblox/en-us/assets/modeling/surface-appearance/SurfaceAppearance-Opaque-ColorMap.jpg new file mode 100644 index 00000000..f9b2dc88 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/modeling/surface-appearance/SurfaceAppearance-Opaque-ColorMap.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9084b7c51326b59ea314aa6a7d05b1081f2cb3bd3a8bf146956e08af5e8ae19 +size 120982 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-AlignType-AllAxes.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-AlignType-AllAxes.mp4 deleted file mode 100644 index 1ff11435..00000000 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-AlignType-AllAxes.mp4 +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:7b18ede5dcd31e8940590903ea63d41eb9a488bf7dfd25c019a706c5856e5db6 -size 3107410 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-AlignType-PrimaryAxisParallel.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-AlignType-PrimaryAxisParallel.mp4 deleted file mode 100644 index b3ba1b3e..00000000 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-AlignType-PrimaryAxisParallel.mp4 +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:817a09c90efeb87129dd7ef5f244bf0af24cb60c9fa7d080f515e9906009fc6e -size 3164377 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-AlignType.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-AlignType.mp4 new file mode 100644 index 00000000..45bebcea --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-AlignType.mp4 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e513e39d96822e66c3fe244e007598748f93c97935be7e52042991fc08cccbee +size 2611787 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-Demo.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-Demo.mp4 index 385530b4..6fe5427b 100644 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-Demo.mp4 +++ b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-Demo.mp4 @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:cb424364a43fe2221737def0da9f369563a83721b6dd87a2514f5208246d0e5c -size 2207883 +oid sha256:330bd3b467d10e3e34d15f39d5998eac18b33c503dcaa4aa44ab259706129e35 +size 2182967 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-ReactionTorqueEnabled-False.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-ReactionTorqueEnabled-False.mp4 deleted file mode 100644 index b04bb9db..00000000 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-ReactionTorqueEnabled-False.mp4 +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:ab84f5f35233337f7197e4880b8f27248144dbe202c7dde0f3ea1cfbb7dbb21c -size 1787417 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-ReactionTorqueEnabled-True.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-ReactionTorqueEnabled-True.mp4 deleted file mode 100644 index 4b6e30df..00000000 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-ReactionTorqueEnabled-True.mp4 +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:ce8d7cdaba541e2184176e4fec21313a8a81f58a0656c1785e2a2c15acfcbe7f -size 1808094 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-RigidityEnabled.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-RigidityEnabled.mp4 new file mode 100644 index 00000000..c3e50631 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignOrientation-RigidityEnabled.mp4 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:174e9f04557d08cf029a1e37ef0458e299bb733aef2d9feb590a0b0ffa64d0be +size 2420977 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ApplyAtCenterOfMass-False.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ApplyAtCenterOfMass-False.mp4 deleted file mode 100644 index de349c5f..00000000 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ApplyAtCenterOfMass-False.mp4 +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:e03d8b657e4b522e87abd70286de2b8b55708425b34bcacc9d47646b9df5b187 -size 2576298 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ApplyAtCenterOfMass-True.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ApplyAtCenterOfMass-True.mp4 deleted file mode 100644 index 827489a1..00000000 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ApplyAtCenterOfMass-True.mp4 +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:7ae52275fa02a9f488a570a9e67e4639e60087288d09ce93d49cd9e00550f12c -size 2375207 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-Demo.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-Demo.mp4 index 9dc0096f..32e439ed 100644 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-Demo.mp4 +++ b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-Demo.mp4 @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:211fac7fb009995450a0a4b619face50489d13221349ab329b93d6220c4dc54e -size 2162434 +oid sha256:4b1dd6cd61d3235363abd42e675341e5b42f2bfa5fff22edffdbf4fa0052facf +size 2203263 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ReactionForceEnabled-False.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ReactionForceEnabled-False.mp4 deleted file mode 100644 index 9ac2d79e..00000000 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ReactionForceEnabled-False.mp4 +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:825e29647b74e80d23d72932a2f487369b5545f2cd1fdd1e4f4a595c4e53666c -size 1986224 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ReactionForceEnabled-True.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ReactionForceEnabled-True.mp4 deleted file mode 100644 index 8c96a834..00000000 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ReactionForceEnabled-True.mp4 +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:667504455207cf83b81bfd20fe3185b0bb819f27a74a73c2b10385820bf6a716 -size 2023964 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ReactionForceEnabled.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ReactionForceEnabled.mp4 new file mode 100644 index 00000000..fd785f23 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/physics/constraints/AlignPosition-ReactionForceEnabled.mp4 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ef0359b130a8526cd3cf392e26160afa2be69a2206f7238313ab90fa9b72f89 +size 2621444 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AnimationConstraint-Demo.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AnimationConstraint-Demo.mp4 new file mode 100644 index 00000000..505a41f9 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/physics/constraints/AnimationConstraint-Demo.mp4 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3eb2fe7964c8b5d07d055a4dcbf777c153020787623d604a00d96d578b9831f +size 2023529 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AnimationConstraint-IsKinematic.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AnimationConstraint-IsKinematic.mp4 new file mode 100644 index 00000000..838214d7 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/physics/constraints/AnimationConstraint-IsKinematic.mp4 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d108ca2ff0e99dd882551f519141c2150ac4859872779574a99aeec858af7001 +size 3278127 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/AnimationConstraint-Transform.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/AnimationConstraint-Transform.mp4 new file mode 100644 index 00000000..56ec7785 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/physics/constraints/AnimationConstraint-Transform.mp4 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:088dcca05844079c82a2166f27138659b6bc9059e3f227ee994936706ac7fef3 +size 1662204 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/BallInSocket-Demo.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/BallInSocket-Demo.mp4 deleted file mode 100644 index 0397de89..00000000 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/BallInSocket-Demo.mp4 +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:3439a9b306fe3baba52e6271f657273a7faaaf7b32d98a6a8f369a78ddc13bb6 -size 2993628 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/BallInSocket-Limits-Tilt.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/BallInSocket-Limits-Tilt.mp4 deleted file mode 100644 index 3d5399e6..00000000 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/BallInSocket-Limits-Tilt.mp4 +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:d49c29ad0f6685e8f34591928ecf0681fb4664d5b483a6f74d3addcbd552c9d7 -size 2344763 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/BallInSocket-Limits-Twist.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/BallInSocket-Limits-Twist.mp4 deleted file mode 100644 index 3ef1d9af..00000000 --- a/.cursor/rules/roblox/en-us/assets/physics/constraints/BallInSocket-Limits-Twist.mp4 +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:b466865f8b9e39310fe413f721cef3df389f650a5aa404dca734c2a59e48deb3 -size 1384988 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/BallSocket-Demo.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/BallSocket-Demo.mp4 new file mode 100644 index 00000000..18f18351 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/physics/constraints/BallSocket-Demo.mp4 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c8d8026eab69003f0c77ee09bc64fb0f346ce400b30663e6c0bba468ddca2ce5 +size 2386795 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/BallSocket-Limits-Tilt.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/BallSocket-Limits-Tilt.mp4 new file mode 100644 index 00000000..4b910788 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/physics/constraints/BallSocket-Limits-Tilt.mp4 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c27938037a8804590118d1e7702b0de9b0ecff322c23e7eb99bb3b5adcf0895 +size 2477896 diff --git a/.cursor/rules/roblox/en-us/assets/physics/constraints/BallSocket-Limits-Twist.mp4 b/.cursor/rules/roblox/en-us/assets/physics/constraints/BallSocket-Limits-Twist.mp4 new file mode 100644 index 00000000..a282d2e3 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/physics/constraints/BallSocket-Limits-Twist.mp4 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:48eb94241539bce1f9a9c1ccfd5b2f3b8ea6e297573aa25f48dd69797dffffd0 +size 2584327 diff --git a/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Bulk-A.png b/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Bulk-A.png new file mode 100644 index 00000000..e9256c6f --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Bulk-A.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b9fad20a7b321d98119fbb9ef7e1d026993d861274788407220f1e1ed33af345 +size 107640 diff --git a/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Bulk-B.png b/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Bulk-B.png new file mode 100644 index 00000000..1e46e00a --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Bulk-B.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:03509d1ddf8ae09e3aec8edfd5b385a09e1d5c5908f81da26582a1f2a1cedcfc +size 32339 diff --git a/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Mobile-Marketplace.png b/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Mobile-Marketplace.png new file mode 100644 index 00000000..72115cca --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Mobile-Marketplace.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a284a2d6ef727f788cc258ce965b09f3923a165cb74d5992267524551999847c +size 107638 diff --git a/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Per-Item.png b/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Per-Item.png new file mode 100644 index 00000000..4d2c8052 --- /dev/null +++ b/.cursor/rules/roblox/en-us/assets/publishing/marketplace/Timed-Options-Per-Item.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:46b4962e6aa9e61129fd9e5cad812d3165e23825063c5b38d493d70b55bf5203 +size 137385 diff --git a/.cursor/rules/roblox/en-us/audio/objects.md b/.cursor/rules/roblox/en-us/audio/objects.md index 1ffb2a40..8d7096cf 100644 --- a/.cursor/rules/roblox/en-us/audio/objects.md +++ b/.cursor/rules/roblox/en-us/audio/objects.md @@ -223,7 +223,52 @@ To play 2D text-to-speech audio: 10 Retro voice #2 - + + + 11 + Host voice + + + + 101 + Spanish male + + + + 102 + Spanish female + + + + 201 + German male + + + + 202 + German female + + + + 301 + Italian male + + + + 302 + Italian female + + + + 401 + French male + + + + 402 + French female + + diff --git a/.cursor/rules/roblox/en-us/cloud-services/extended-services.md b/.cursor/rules/roblox/en-us/cloud-services/extended-services.md index c151e9d4..35d0543f 100644 --- a/.cursor/rules/roblox/en-us/cloud-services/extended-services.md +++ b/.cursor/rules/roblox/en-us/cloud-services/extended-services.md @@ -91,98 +91,109 @@ For a detailed breakdown of your current pending balance costs, click **View Det ## Service pricing -See the following table for Extended Services pricing. +See the following table for Extended Services pricing: - - - + + + + + + - + + + + - - - + + + + + + + + + + + + + + + + + + + + + + + + + - - - - - - + + + + - - - - - - + + + + - - - + + + + + + + + + + + - - - - - - - - - - - - - + + + + - + + + + - - - + + + + + - - - - - - - - - - - - - - - - - - - - - + + + +
**Service****Default****Extended**
**Service****Default****Extended**
Memory storesMemory storesRequests1000 + (CCU x 120) requests per minute$0.003 / 1M requests
Request units1000 + (CCU x 120) request units per minute$0.003 / 1M request unitsStorage64KB + (CCU x 1.2KB)$0.10 / GB per hour¹
Text-to-speechRequests1 + (CCU x 6) requests per minute$5.00 / 1M requests
Speech-to-textRequests1 + (CCU x 5) requests per minute$1 / 1M requests
Standard data stores (coming soon)Get250 + (CCU * 40) requests per minute$0.08 / 1M requests
Storage64KB + (CCU x 1.2KB)$0.10 / GB per Hour¹
Text-to-speechSet250 + (CCU * 20) requests per minute$0.80 / 1M requests
Requests1 + (CCU x 6) requests per minute$5.00 / 1M requests
Access to standard data stores (coming soon)List10 + (CCU * 2) requests per minute$0.60 / 1M requests
Get250 + (CCU * 40) requests per minute$0.08 / 1M requestsRemove100 + (CCU * 40) requests per minute$0.80 / 1M requests
Ordered data stores (coming soon)Get250 + (CCU * 40) requests per minute$0.08 / 1M requests
Set250 + (CCU * 20) requests per minute$0.80 / 1M requests
List10 + (CCU * 2) requests per minute$0.60 / 1M requests
Remove100 + (CCU * 40) requests per minute$0.80 / 1M requestsSet250 + (CCU * 20) requests per minute$0.25 / 1M requests
Access to ordered data stores (coming soon)GetSorted100 + (CCU * 2) requests per minute$0.32 / 1M requests
Get250 + (CCU * 40) requests per minute$0.08 / 1M requestsRemove100 + (CCU * 40) requests per minute$0.25 / 1M requests
Set250 + (CCU * 20) requests per minute$0.25 / 1M requests
GetSorted100 + (CCU * 2) requests per minute$0.32 / 1M requests
Remove100 + (CCU * 40) requests per minute$0.25 / 1M requests
Data store storage (coming soon)
Storage100 MB + (1MB * Lifetime Players) GB per Month$0.12 / GB per MonthData store storage (coming soon)Storage100 MB + (1MB * Lifetime Players) GB per month$0.12 / GB per month
-
¹ GB per Hour = the storage consumption multiplied by the amount of time. For example, 100GB Hours might equate to "100GB x 1 hour" or "200GB x 0.5 hours".
+
¹ GB per hour = the storage consumption multiplied by the amount of time. For example, 100GB hours might equate to "100GB x 1 hour" or "200GB x 0.5 hours".
## Supported countries -Extended Services is available in the following countries. +Extended Services is available in the following countries: diff --git a/.cursor/rules/roblox/en-us/get-started.md b/.cursor/rules/roblox/en-us/get-started.md deleted file mode 100644 index ec2f46c5..00000000 --- a/.cursor/rules/roblox/en-us/get-started.md +++ /dev/null @@ -1,205 +0,0 @@ ---- -title: Get started on Roblox -description: Learn how to get started quickly on Roblox -hideInPageNavigation: true ---- - -export const statsByline = "* Figures are as of Q4 2024"; - -export const scaleStats = [ - { header: "85.3M", description: "Daily active users" }, - { header: "18.9M", description: "Monthly unique payers" }, - { header: "18.7B", description: "Quarterly hours engaged" }, - { header: "750M", description: "Yearly creator earnings" }, - { header: "18.9M", description: "DevX" }, - { header: "18.7B", description: "xyz" }, -]; - -export const sections = [ - - { - title: "", - description: "Getting started on Roblox is easy, and learning about the ecosystem only takes a few minutes. Better yet, you can start creating and sharing your creations in less than an hour with these resources.", - content: [ - { - title: "1. Learn about what you can do", - description: "This video goes over some of the things you can build on Roblox and the tools you need to do it.", - video: "https://www.youtube-nocookie.com/embed/vIiVbFiDbBE", - links: [ - {text: "Watch overview", href: "https://www.youtube.com/watch?v=vIiVbFiDbBE" }, - ] - }, - { - title: "2. Get a Roblox account and the app", - description: "The same Roblox account works for playing and creating. Sign up at roblox.com and download the app for your preferred device.", - image: "./assets/landing/app.jpg", - links: [ - {text: "Go to roblox.com", href: "https://www.roblox.com" }, - ] - }, - { - title: "3. Customize your avatar and play something", - description: "The best way to understand Roblox is to use it! Customize your avatar with free or paid items and play a few of the millions of experiences.", - image: "./assets/landing/avatar-play.jpg", - links: [ - {text: "Browse avatar items", href: "https://www.roblox.com/catalog" }, - {text: "Browse experiences", href: "https://www.roblox.com/charts" }, - ] - }, - { - title: "4. Get Studio and take the tour", - description: "Roblox Studio is your main creation tool and is supported on Windows and Mac. The onboarding tour gets you acclimated to basic workflows.", - image: "./assets/landing/get-started/studio.png", - links: [ - {text: "Roblox Studio setup", href: "./studio/setup" }, - ] - } - ] - }, - { - title: "Experiences", - description: "Making an experience can seem intimidating, but we promise it's not too bad! Walk through the tutorial or watch the following videos to understand a bit more about how experiences work.", - buttons: [ - { text: "See all docs", href: "./experiences" } - ], - content: [ - { - title: "Create your first experience", - description: "Create a simple 3D platformer while learning important Roblox building and scripting concepts.", - image: "./assets/landing/core-experience.png", - links: [ - { text: "Start creating", href: "./tutorials/curriculums/core/" }, - ] - }, - { - title: "Intro to world building", - description: "Sit back and watch one of our 3D artists show you how to construct 3D worlds in Studio.", - video: "https://www.youtube-nocookie.com/embed/SgPU84AqpkY", - links: [ - { text: "Watch livestream", href: "https://youtu.be/SgPU84AqpkY" } - - ] - }, - { - title: "Intro to physics", - description: "The Roblox engine simulates real-life materials, physics, collisions and more. Join one of our product managers for a overview of how the physics simulation works.", - video: "https://www.youtube-nocookie.com/embed/9mXvhYSv7fc", - links: [ - { text: "Watch livestream", href: "https://youtu.be/9mXvhYSv7fc" }, - - ] - }, - ] - }, - { - title: "Avatar items", - description: "Making avatar items can sometimes be easier than making an experience. You can make a simple 2D t-shirt with an image editor without having to open Studio at all. If you're interested in 3D modeling, use Blender and Studio together to make a basic 3D accessory.", - buttons: [ - { text: "See all docs", href: "./avatar" } - ], - content: [ - { - title: "Avatar items overview", - description: "Learn about all the different things you can create and allow users to equip for their avatars.", - video: "https://www.youtube-nocookie.com/embed/EUDSIUmLjxA", - links: [{ text: "Watch overview", href: "https://youtu.be/EUDSIUmLjxA" }] - }, - { - title: "Create a t-shirt", - description: "A simple t-shirt is the easiest example of how to create an avatar item.", - video: "https://www.youtube-nocookie.com/embed/r_unfGZT5Ps", - links: [ - { text: "Watch how to do it", href: "https://youtu.be/r_unfGZT5Ps" }, - { text: "Guides", href: "./art/classic-clothing" } - ] - }, - { - title: "Create a backpack accessory", - description: "Learn how to create a backpack, which is an example of a 3D accessory. These accessories are simple, non-animated items that attach to a single point on an avatar's body.", - video: "https://www.youtube-nocookie.com/embed/Eed29gV0hLA", - links: [ - { text: "Watch how to do it", href: "https://youtu.be/Eed29gV0hLA" }, - { text: "Guides", href: "./art/accessories" } - ] - } - ] - }, -]; - -{sections.map((section, index) => ( - - {section.title && ( - {section.title} - )} -
- {section.description && ( -

{section.description}

- )} - {section.buttons && section.buttons.map((button, btnIndex) => ( - - ))} -
- {section.content && ( - - {section.content.map((content, contentIndex) => { - const mdValue = 12 / section.content.length; - return ( - -
- {content.image && ( - - )} - {content.video && ( - - )} -

{content.title}

-

{content.description}

- {content.links && content.links.map((link, linkIndex) => ( - - {link.text}
- - ))} - {content.buttons && content.buttons.map((button, btnIndex) => ( - !button.useStudioButton && ( - - ) - - ))} -
- - ); - })} - - )} - -))} diff --git a/.cursor/rules/roblox/en-us/get-started/creating.md b/.cursor/rules/roblox/en-us/get-started/creating.md new file mode 100644 index 00000000..1479ac62 --- /dev/null +++ b/.cursor/rules/roblox/en-us/get-started/creating.md @@ -0,0 +1,194 @@ +--- +title: Get started creating on Roblox +description: Learn how to create experiences and avatar items quickly on Roblox +hideInPageNavigation: true +--- + +export const sections = [ + +{ +title: "", +description: "Getting started on Roblox is easy, and learning about the ecosystem only takes a few minutes. Better yet, you can start creating and sharing your creations in less than an hour with these resources.", +content: [ +{ +title: "1. Learn about what you can do", +description: "This video goes over some of the things you can build on Roblox and the tools you need to do it.", +video: "https://www.youtube-nocookie.com/embed/vIiVbFiDbBE", +links: [ +{text: "Watch overview", href: "https://www.youtube.com/watch?v=vIiVbFiDbBE" }, +] +}, +{ +title: "2. Get a Roblox account and the app", +description: "The same Roblox account works for playing and creating. Sign up at roblox.com and download the app for your preferred device.", +image: "../assets/landing/app.jpg", +links: [ +{text: "Go to roblox.com", href: "https://www.roblox.com" }, +] +}, +{ +title: "3. Customize your avatar and play something", +description: "The best way to understand Roblox is to use it! Customize your avatar with free or paid items and play a few of the millions of experiences.", +image: "../assets/landing/avatar-play.jpg", +links: [ +{text: "Browse avatar items", href: "https://www.roblox.com/catalog" }, +{text: "Browse experiences", href: "https://www.roblox.com/charts" }, +] +}, +{ +title: "4. Get Studio and take the tour", +description: "Roblox Studio is your main creation tool and is supported on Windows and Mac. The onboarding tour gets you acclimated to basic workflows.", +image: "../assets/landing/get-started/studio.png", +links: [ +{text: "Roblox Studio setup", href: "../studio/setup" }, +] +} +] +}, +{ +title: "Experiences", +description: "Making an experience can seem intimidating, but we promise it's not too bad! Walk through the tutorial or watch the following videos to understand a bit more about how experiences work.", +buttons: [ +{ text: "See all docs", href: "../experiences" } +], +content: [ +{ +title: "Create your first experience", +description: "Create a simple 3D platformer while learning important Roblox building and scripting concepts.", +image: ".../assets/landing/core-experience.png", +links: [ +{ text: "Start creating", href: "../tutorials/curriculums/core/" }, +] +}, +{ +title: "Intro to world building", +description: "Sit back and watch one of our 3D artists show you how to construct 3D worlds in Studio.", +video: "https://www.youtube-nocookie.com/embed/SgPU84AqpkY", +links: [ +{ text: "Watch livestream", href: "https://youtu.be/SgPU84AqpkY" } + + ] + }, + { + title: "Intro to physics", + description: "The Roblox engine simulates real-life materials, physics, collisions and more. Join one of our product managers for a overview of how the physics simulation works.", + video: "https://www.youtube-nocookie.com/embed/9mXvhYSv7fc", + links: [ + { text: "Watch livestream", href: "https://youtu.be/9mXvhYSv7fc" }, + + ] + }, + ] + +}, +{ +title: "Avatar items", +description: "Making avatar items can sometimes be easier than making an experience. You can make a simple 2D t-shirt with an image editor without having to open Studio at all. If you're interested in 3D modeling, use Blender and Studio together to make a basic 3D accessory.", +buttons: [ +{ text: "See all docs", href: "../avatar" } +], +content: [ +{ +title: "Avatar items overview", +description: "Learn about all the different things you can create and allow users to equip for their avatars.", +video: "https://www.youtube-nocookie.com/embed/EUDSIUmLjxA", +links: [{ text: "Watch overview", href: "https://youtu.be/EUDSIUmLjxA" }] +}, +{ +title: "Create a t-shirt", +description: "A simple t-shirt is the easiest example of how to create an avatar item.", +video: "https://www.youtube-nocookie.com/embed/r_unfGZT5Ps", +links: [ +{ text: "Watch how to do it", href: "https://youtu.be/r_unfGZT5Ps" }, +{ text: "Guides", href: "../art/classic-clothing" } +] +}, +{ +title: "Create a backpack accessory", +description: "Learn how to create a backpack, which is an example of a 3D accessory. These accessories are simple, non-animated items that attach to a single point on an avatar's body.", +video: "https://www.youtube-nocookie.com/embed/Eed29gV0hLA", +links: [ +{ text: "Watch how to do it", href: "https://youtu.be/Eed29gV0hLA" }, +{ text: "Guides", href: "../art/accessories" } +] +} +] +}, +]; + +{sections.map((section, index) => ( + +{section.title && ( +{section.title} +)} + +
+{section.description && ( +

{section.description}

+)} +{section.buttons && section.buttons.map((button, btnIndex) => ( + +))} +
+{section.content && ( + +{section.content.map((content, contentIndex) => { +const mdValue = 12 / section.content.length; +return ( + +
+{content.image && ( + +)} +{content.video && ( + +)} +

{content.title}

+

{content.description}

+{content.links && content.links.map((link, linkIndex) => ( + +{link.text}
+ +))} +{content.buttons && content.buttons.map((button, btnIndex) => ( +!button.useStudioButton && ( + +) + + ))} +
+ + ); + })} + + )} + + +))} diff --git a/.cursor/rules/roblox/en-us/get-started/how-is-roblox-different.md b/.cursor/rules/roblox/en-us/get-started/how-is-roblox-different.md new file mode 100644 index 00000000..58a65e4f --- /dev/null +++ b/.cursor/rules/roblox/en-us/get-started/how-is-roblox-different.md @@ -0,0 +1,87 @@ +--- +title: How is Roblox different from other platforms? +description: Frictionless play, community collaboration, and reach beyond Roblox. +hideInPageNavigation: true +fullWidth: true +--- + +

Frictionless play

+ + + + +

No complicated downloads

+ Roblox is built on the cloud, allowing users to join new experiences in seconds, without having to download massive files. + + + + +

Automatic matchmaking

+ Users are automatically placed in servers with friends and other users from their region, making it easier than ever to connect with friends and make new ones. + + + + +

Cross-platform support

+ Experiences can be published across all major devices, allowing players to play how they want, where they want, with whomever they want. + + + + + + + + +

Collaborate with your players

+ + + + +

Communities are part of the process

+ Due to the user-generated content (UGC) nature of Roblox, the community is much more involved, as many devs are also often community members. Cultivating a healthy community and involving users in development is key to success. + + + + +

Benefit from direct feedback

+ Since Roblox enables creators to develop and publish updates quickly, they can efficiently gather feedback and iterate -- resulting in a deeper understanding of their audience and a more focused experience for users. + + + + + + + + +

Reach beyond Roblox

+ + + + +

Massive creator ecosystem

+ Roblox fuels a thriving community of content creators on platforms like YouTube and TikTok, generating views and driving engagement. + + + + +

Engaged cross-platform audience

+ Roblox content resonates beyond the platform, with creators building loyal followings and extending reach to new players. + + + + + + + diff --git a/.cursor/rules/roblox/en-us/get-started/index.md b/.cursor/rules/roblox/en-us/get-started/index.md new file mode 100644 index 00000000..57d0abc4 --- /dev/null +++ b/.cursor/rules/roblox/en-us/get-started/index.md @@ -0,0 +1,104 @@ +--- +title: Get Started +description: A guide for new Roblox developers. Explains why to develop on Roblox the platform, and how to get started. +hideInPageNavigation: true +--- + + + +

Welcome to Roblox, we're glad youʼre here! Itʼs an exciting time to develop on Roblox. This guide is a crash course in all things Roblox, covering the most commonly asked questions by developers:

+
    +
  • Why should I develop on Roblox?
    • +
  • How is Roblox different from other platforms?
    • +
  • What tools does Roblox provide?
    • +
  • How do I make money?
    • +
  • Are there any recommended strategies?
    • +
  • What other resources are there to learn more?
    • +
+ + + + + + +
+ + + Choose a section to learn more about Roblox + + + + + + Why should I develop on Roblox?
+ + Reasons to build on the Roblox platform—audience, tools, and opportunity. + + + + + + + + How is Roblox different from other platforms?
+ + Frictionless play, community collaboration, and reach beyond Roblox. + + + + + + + + What tools does Roblox provide?
+ + Roblox Studio, Creator Hub, AI Assistant, and more. + + + + + + + + How do I make money?
+ + Creator Rewards, in-experience purchases, subscriptions, UGC, and more. + + + + + + + + Recommended strategies
+ + Set your game up for success, build and test your MVP, and iterate. + + + + + + + + Start creating
+ + Get started creating experiences and avatar items. + + + + + + + + Resources
+ + Find more useful resources for your creation journey. + + + + +
diff --git a/.cursor/rules/roblox/en-us/get-started/monetization.md b/.cursor/rules/roblox/en-us/get-started/monetization.md new file mode 100644 index 00000000..4dc8ef85 --- /dev/null +++ b/.cursor/rules/roblox/en-us/get-started/monetization.md @@ -0,0 +1,256 @@ +--- +title: How do I make money? +description: Creator Rewards, in-experience purchases, subscriptions, UGC, immersive ads, and more. +hideInPageNavigation: true +fullWidth: true +--- + +

Earn on Roblox

+ + + + +

Diverse earning opportunities

+ We're excited to provide creators with a wide variety of ways to earn from their experiences, including subscriptions, in-app purchases, engagement rewards, and more. We're always evolving and adding new features to help you earn and succeed. + + + + +

Developer Exchange Program (DevEx)

+ The DevEx program allows eligible creators to convert earned Robux into real-world currency. Once you've met the requirements, including having a minimum of 30,000 earned Robux, you can submit a request to cash out. + + + + + + + + +

In-app purchases

+ + + + +

Repeat purchases

+ Dev Products are a type of sale item that can be purchased multiple times, only with Robux. This is used for things like in-game purchases, boosts, etc. Dev Products generally monetize better, as they can be repeatedly purchased. + + + + +

One-time purchases

+ [Game passes](../production/monetization/passes.md) are a type of sale item that can only be purchased once, only with Robux. This is useful for things like VIP passes, content bundles, or battle passes. + + + + +

You receive 70% of Robux spent

+ The remaining 30% is Robloxʼs Marketplace Fee. To convert these Robux to a real currency, youʼll still need to withdraw it via DevEx. + + + + + + + + +

Subscriptions

+ + + + +

Subscriptions

+ [Subscriptions](../production/monetization/subscriptions.md) provide players with benefits for a recurring monthly fee. Common benefits include in-game currency, multipliers, and access to exclusive vehicles, weapons and clothing. Subscriptions auto-renew and are priced in local currency. + + + + +

Subscriber analytics

+ The Creator dashboard allows you to track metrics including subscriber breakdown and cancellations, estimated revenue, subscriptions by platform, and platform earnings. + + + + + + + + + +

Creator rewards

+ + + + +

Creator Rewards

+ Weʼre increasing our investment in the developer community by rewarding creators for producing engaging content and bringing new users to the platform. + + + + +

Daily engagement

+ Earn a fixed payout of 5 Robux when a user who has spent at least $9.99 on the platform in the last 60 days (an "Active Spender") plays your game for 10 minutes or more. This reward applies to the first three experiences they play per day. + + + + +

Audience expansion

+ Earn a 35% revenue share on the first $100 in Robux purchases made by a new or returning user who finds your experience through a direct link or search, and plays for at least 10 minutes.1



1 Less qualifying purchases they have made in the 60 days after each time (if any) they have previously reactivated. Additional restrictions apply. + + + + + + + + +

UGC avatar items

+ + + + +

UGC for everyone

+ Previously restricted to approved creators, selling user-generated avatar clothing is now available to all creators who are age-verified and subscribe to Premium. + + + + +

Creators receive 30%, sellers 40%

+ For items sold in the Avatar Marketplace, Roblox collects the 40% seller portion + a 30% market fee. If your item is sold from within another experience, the owner of that experience will collect the 40% seller portion instead. + + + + +

Sell in your own experience

+ If you sell your UGC item within an experience you own, you will earn 30% for being the creator and 40% for being the seller, bringing your return to 70% of Robux spent. + + + + + + + + +

Immersive ads

+ + + + +

Advertise

+ Programmatically-served ad content from brands appear directly within your experiences. These native ads blend into the 3D environment, offering a seamless way to monetize your experience. + + + + +

Ad formats

+ Integrate a [variety of formats](../production/promotion/advertise.md), including video ads that can autoplay or be click-to-play, image ads that are static billboards, and portal ads that can teleport users to an advertiser's experience. + + + + +

How to earn

+ Creators are paid through a revenue share based on engagement. You can earn from video views, ad impressions, and successful teleports that are generated by the ads in your experience. + + + + + + + + +

Brand integrations

+ + + + +

Limited-time activations

+ Limited-time activations bring brands into an experience for a brief time – usually 2–4 weeks – through content like quests and avatar items. For a more authentic integration that players will respond to, be sure to choose brand partners that fit your experienceʼs vibe, aesthetics, and goals. + + + + +

Make it meaningful

+ For a seamless and authentic brand integration, weave the brand into your game's [core loop](https://youtu.be/gkFKF9A-snY). This means the brand content should feel like a natural extension of the gameplay, enhancing the player's progression and overall experience. + + + + + + + + +

Digital + physical merchandising

+ + + + +

Sell physical goods

+ The new Roblox Commerce APIs enable creators and brands to sell physical products directly within their Roblox experiences, creating a virtual mall-like experience. Shopify is the first integrated partner. + + + + +

Bridge digital & physical

+ The Approved Merchandiser Program (AMP) links physical product purchases with digital items on Roblox. Physical products with the AMP badge will include a code unlocking a unique digital item in the experience. + + + + +

Empowering creators

+ By bridging the gap between virtual and real-world shopping, Roblox is opening up new avenues for revenue generation and brand building. + + + + + + + + +

Price optimization

+ + + + +

Automated price testing

+ Our [Price Optimization](../production/monetization/price-optimization.md) system intelligently experiments with different price points, showing them to subsets of your users to identify the sweet spot that maximizes your revenue. + + + + +

Dynamic pricing

+ Keep your prices competitive with dynamically-scripted pricing. Our tools make flexible pricing strategies simple, allowing you to respond to market trends and player behavior in real time. + + + + +

Data-driven decisions

+ Monitor both short-term and long-term impacts, and use our insights to help ensure that your pricing strategy drives sustainable growth and maximizes your overall earnings. + + + + + + + + +

Regional pricing

+ + + + +

Unlock international reach

+ Roblox's [Regional Pricing](https://corp.roblox.com/newsroom/2025/04/roblox-launches-regional-pricing-for-in-experience-items) automatically determines and sets optimal local prices for your in-experience items, making your content more accessible and appealing to players around the world. + + + + +

Help boost worldwide spending

+ Early testing demonstrates substantial increases in user spending in regions like Mexico, Brazil, and the Philippines, demonstrating the power of localized pricing strategies. + + + + + + + + diff --git a/.cursor/rules/roblox/en-us/get-started/resources.md b/.cursor/rules/roblox/en-us/get-started/resources.md new file mode 100644 index 00000000..74b4c136 --- /dev/null +++ b/.cursor/rules/roblox/en-us/get-started/resources.md @@ -0,0 +1,98 @@ +--- +title: Resources +description: Get started with these resources, community feedback, growth strategies, and RDC. +hideInPageNavigation: true +fullWidth: true +--- + +

Get started with these resources

+ + + + + + Roblox Creator Roadmap
+ + See what's coming to the platform and plan ahead. + + + + + + + + Tips for finding the right developer partners
+ + Guidance for brands on partnering with the right developers. + + + + + + + + More on building engaging experiences
+ + Best practices for creating compelling brand experiences on Roblox. + + + + + + + + Learn techniques for monetizing your experience
+ + Learn how to monetize your experience in the creator economy. + + + + + + + + Get pre-built Feature Packages
+ + Ready-to-use systems for common game features. + + + + + + + + Learn about Roblox's License Manager and Catalog
+ + Licensing and catalog tools for experiences and content. + + + + + +

Learn more at the Roblox Developers Conference

+ + + + +

Presentations

+ RDC offers creators a firsthand look into the future of Roblox with keynote presentations from our CEO David Baszucki and other Roblox leaders. + + + + +

Breakout sessions

+ There are in-depth breakout sessions showcasing the latest technology that will empower creator success and lots of great networking opportunities. + + + + +

Roblox Innovation Awards

+ This is where we also hold the Roblox Innovation Awards, an annual award show recognizing the yearʼs most innovative experiences and creators. + + + + +

+Read more about RDC 2025 + + diff --git a/.cursor/rules/roblox/en-us/get-started/strategies.md b/.cursor/rules/roblox/en-us/get-started/strategies.md new file mode 100644 index 00000000..f1cfbcfc --- /dev/null +++ b/.cursor/rules/roblox/en-us/get-started/strategies.md @@ -0,0 +1,554 @@ +--- +title: Recommended strategies +description: Set your game up for success, build and test your MVP, and iterate. +hideInPageNavigation: true +fullWidth: true +--- + +

Set your game up for success

+ + + + +

Assemble your Roblox dream team

+ Look for developers with expertise in game design, 3D modeling, scripting, and optimization from within the Roblox community. A seasoned developer can help navigate the technical complexities of Roblox and ensure your vision is executed effectively, leading to a higher quality and more engaging experience. + + + + +

Be aware of popular games & genres

+ Understanding the preferences of the Roblox user base is essential when selecting a genre. Popular genres on Roblox include role-playing games, simulators, obstacle courses (obbies), and survival games. Choosing a proven genre will increase the likelihood of attracting and retaining players. + + + + + + + + +

Understand your target audience

+ + + + +

Play similar experiences

+ Itʼs important to have a good knowledge of popular Roblox experiences, especially those that are similar to yours. Why are users drawn to this experience? What keeps them interested? + + + + +

Talk to players

+ Watch the chat or join an experienceʼs Discord server to see what players talk about. Feel free to join in and ask questions yourself. + + + + +

Watch YouTube videos

+ If you donʼt have the time to get deep into an experience, watch videos on YouTube. These are often made by the community and can give you a good idea of what theyʼre interested in. + + + + + + + + +

Define your KPIs

+ + + + +

Identify goals

+ To measure the success of your Roblox experience, define your Key Performance Indicators (KPIs) before development begins. These metrics (such as engagement, retention, and monetization) should guide your design from the start to ensure your game supports its target goals. + + + + +

Organic social growth KPIs

+ D1 Retention: Users who return to your game the day after their first session. Average Playtime Per User: The average amount of time that users spend in your game (Average Sessions/Day x Session Length). + + + + +

Sustained growth KPIs

+ D7 & D30 Retention: Users who return 7 and 30 days after their first session. Conversion: The percentage of users who spend Robux in your experience. ARPPU: Average Revenue Per Paying User. + + + + + + + + +

Design for multi-platform play

+ + + + +

Playable anywhere

+ Roblox is played on a variety of devices, including PCs, mobile phones, and consoles. It's essential to design a user interface (UI) and user experience (UX) that works seamlessly across all platforms. The UI should be clean, intuitive, and easy to navigate, regardless of the device being used. + + + + +

Massive mobile audience

+ Given Roblox's large mobile audience, ensure that your game is easily playable on a touchscreen with a well-designed mobile UI. Even for games with a higher percentage of PC players, such as shooters and RPGs, supporting mobile is still a best practice. + + + + +

Optimize for performance

+ Many Roblox users play on low-end devices, so performance is essential. Non-performant games may struggle to grow and maintain a user base.

+ Learn more about Performance Optimization + + + + + + + + +

Build a strong foundation

+ + + + +

Design a killer core loop

+ The [core loop](../production/game-design/core-loops.md) is the fundamental gameplay cycle that keeps players engaged and returning to your experience. It should consist of a series of actions and rewards that are satisfying, repeatable, and progressively challenging in order to motivate players to continue interacting with your experience. + + + + +

Create a world they'll love

+ Create a visually appealing and engaging environment that immerses players in your experience. The environment should be easy to navigate, fun to explore, and encourage positive interactions. Provide opportunities for players to connect with each other, fostering a sense of community. + + + + +

Think ahead

+ Plan long-term by designing with updates in mind. Implement reusable systems, such as quest, collection, or event frameworks that can be easily expanded. This approach will reduce development time and costs for future updates and ensure that your experience remains fresh and engaging. + + + + + + + + +

Create a social, multiplayer experience

+ + + + +

Friends play together

+ Users often play together as a group. Try to design your gameplay to support group play wherever possible and make it easy for friends to connect. + + + + +

Make users into content

+ Designing for social interaction in games is not only good for growing your game, but it also serves as a form of content, taking some of the burden off of your team. + + + + +

Design spaces for interaction

+ When youʼre building out your level, think about how players can find and interact with each other. If the map is huge, players may be spread too thin. + + + + + + + + +

Get to the fun quickly

+ + + + +

Get to the fun quickly

+ What makes your experience fun? What makes it different? Is it worth playing again? Users should be able to answer these questions within the first few minutes of playing your experience. + + + + +

Avoid long tutorials

+ Roblox players like to get to the action immediately and figure it out on their own (or with friends). If you must have a tutorial, keep it short, contextual, and interactive. + + + + +

Donʼt restrict socialization

+ Allow new players to socialize and learn from each other. Donʼt create barriers like individual server instances that prevent users from playing with friends. + + + + + + + + +

Keep players invested

+ + + + +

Give players goals

+ Keep players motivated by providing a mix of short-term, mid-term, and long-term goals that cater to different playstyles and encourage exploration. This variety will improve player retention and prevent them from losing interest. + + + + +

Add progression to make content last

+ Progression systems give players a clear path toward their goals and a sense of accomplishment while slowing down their content consumption, giving you time to produce more. + + + + +

Donʼt gate the fun

+ If you do add progression, make sure there is still enough content available upfront to ensure new users can have fun in their first play session. They should also have a clear understanding of what their next goal is and how to achieve it. + + + + + + Season Pass Feature Package + + + +

Use F2P monetization practices

+ + + + +

Give spenders options

+ Help maximize your monetization potential by providing a variety of virtual items at different price points. This strategy appeals to both casual spenders and dedicated fans. Consider adding a starter pack containing items that appeal to new players. + + + + +

Design for repeat spend

+ Why would a player want to return and continue to spend, over time? As opposed to one-time purchases, players will spend much more on repeatable purchases (like consumable items) as thereʼs no limit to how much they can spend. + + + + +

Optimize your storefront

+ Make your shop as transparent and frictionless as possible, so players understand what to buy, why, and how. Clearly communicate the value and availability of all items.

+ Learn more about merchandising best practices + + + + + + + + +

Build & test your MVP

+ + + + +

Define your MVP

+ What are your core features? If youʼre making a game, what is your core loop? Whatʼs the minimal version of your experience that can demonstrate these features? + + + + +

Get feedback early

+ While it may be tempting to build your fully realized experience, starting with MVP will allow you to get feedback and iterate early, resulting in a better final product. + + + + +

Test your MVP

+ As soon as itʼs ready, get your MVP in playersʼ hands. Their early feedback is the key to identifying issues, finding the fun, and setting your game up for success with real Roblox players. Donʼt wait until itʼs too late to pivot. + + + + + + + + +

Playtest early and often

+ + + + +

Small group playtests

+ Give access to specific users only, which allows you to run small playtests. This option is good for testing whether players understand how to play and find the experience fun. + + + + +

Limited-time public test

+ Release a test experience to the public, acquire a small amount of users, then remove the game after a set amount of time (a weekend, week, etc.) This option is great for getting organic growth metrics like D1 Retention and session length. + + + + + + +

Closed group beta

+ The experience is playable by any user that joins your Group. This option is great for gathering community feedback, long-term retention, and monetization. However, it requires some existing community, and may be more friendly than an open test. + + + + +

Open beta

+ The experience is open to everyone, but has a "Beta" tag in the title. This is very common on Roblox; however, there is no practical difference between open beta and a full release beyond setting player expectations. + + + + + +

Keep testing and iterating

+ + + + +

Identify soft metrics

+ Comparing your analytics to your target [KPIs](https://www.youtube.com/watch?v=RfpMZZFy8Zw) will help you figure out what areas need improvement, whether itʼs retention, engagement, or monetization. + + + + +

Gather community insight

+ Once you know what areas needs work, use your community to find out what is causing those problems. + + + + +

Implement & evaluate

+ Once you identify a potential cause, make the necessary changes and test again. Go through this cycle until your KPIs are in line with your target. + + + + + + + + + +

Get community feedback

+ + + + +

Start a community chat server

+ Be sure to set up your own social chat server, link it to your experience, and have moderation measures in place. This is where your dedicated community members will congregate. + + + + +

Talk to your users

+ Ask your users what they think. What do they like? What donʼt they like? What could make it better? Users will be happy to answer, plus it will make them feel like you care. + + + + +

Watch how users interact

+ Join your experience to observe what users do and what they talk about in the chat. Are they behaving how you would expect? Do they get confused at certain points? Are they interacting with other players? + + + + + + +

Helpful Channels to Have in Your Community Chat Server

+
#Rules or #FAQ
Explains how the server works and the expected behavior

+
#Announcements
Post any updates or announcements here

+
#General
Area for general conversation

+
#Bugs
For players to report bugs

+
#Suggestions
For players to submit ideas and feedback

+
#Creative
For players to submit content & fan art

+ + + + + +

Be creator-friendly

+ + + + +

Consider streamer needs

+ Design your game to be streamer-friendly. Ensure players can easily record videos without UI elements getting in the way, and create an experience that's engaging for viewers to watch on platforms like YouTube and Twitch. + + + + +

Fuel excitement & speculation

+ Many experiences have built-in hidden secrets, backstories, and "surprise and delight" moments that encourage players to share and discuss the content, leading to more social media engagement. + + + + +

Create shareable moments

+ In your experience, is there anything that users would want to share on TikTok or YouTube? Have they created something of their own? Are there achievements or "wow moments" worthy of posting on social media? + + + + + + + + +

Build awareness through social media

+ + + + +

Connect with creators

+ Many Roblox users learn about new experiences from their favorite YouTubers. Getting one or a few of these influencers to play your experience can help get some initial traction. + + + + +

Find the right influencer

+ Influencers are more likely to cover your experience if it fits their audience – so do your research. If engaging with an influencer, be sure to follow the FTC's [Endorsement and Testimonial Guides](https://www.ftc.gov/sites/default/files/attachments/press-releases/ftc-publishes-final-guides-governing-endorsements-testimonials/091005revisedendorsementguides.pdf) and consult with your own legal counsel for specific advice. + + + + +

Try targeted TikTok ads

+ TikTok has also proven to be a good way to raise awareness off platform. You can use their targeting tools to surface ads to users that are interested in similar content. Check out TikTokʼs [best practices for Roblox developers](https://activity-va.tiktok.com/magic/eco/runtime/release/689e44394a786f02edabc8a7). + + + + + + + + +

Unlock organic growth

+ + + + +

Most users come from Home

+ Beyond external sources like social media and word of mouth, Home is where a majority of discovery takes place and consists of recommendations that are algorithmically generated to optimize for engagement, long-term retention, and monetization on a per-user basis. + + + + +

Data-defined placement

+ We take many factors and signals into account when generating recommendations for someone. Beyond user-focused personalization (age, device, location, etc.) we also look at how well an experience retains players, how much Robux and time players spend, and how often users return to play again. + + + + +

Spread the word

+ The best way to maximize how much youʼre surfaced in recommendations is to create a strong experience with solid KPIs. Packaging (icon, thumbnail, media, description) can also help improve D1 retention and play through rate. + + + + + + + + +

Use paid UA to build momentum

+ + + + +

Use sponsored ads

+ Sponsored Ads appear in the "Sponsored" sort on the home page and within search results. Theyʼre great at getting users into experiences but arenʼt as good at targeting specific users.

+ Learn more about Roblox Ads Manager + + + + +

Use earned Robux

+ If you do run an ad campaign, try to use Robux youʼve earned rather than purchasing Robux – the exchange rate is higher for purchased Robux. + + + + + + + + +

Keep the content coming

+ + + + +

Update to stay relevant

+ New content updates keep experiences from becoming stale and give players another reason to come back. They also give content creators new material to cover. + + + + +

A consistent cadence forms habits

+ Establishing a consistent content cadence can encourage players to check in regularly. If itʼs sustainable for your team, aim for minor content updates every 1–2 weeks and major system updates every 2–3 months. + + + + +

Plan for LiveOps from the start

+ Itʼs important to plan for liveops ahead of time. What would those updates include? How do you plan on delivering these updates regularly? Scope updates to fit your teamʼs capacity. + + + + + + + + +

Generate buzz with limited-time content

+ + + + +

Limited-time events add variety

+ Running seasonal events keeps your experience feeling fresh. Events can include thematic map updates, battle/season passes, holiday content and limited-time rewards. + + + + +

Limited rewards create social value

+ Time-limited reward items create rarity. The rarer an item is, the more social value it has amongst players, driving more of them to seek out and earn those items. + + + + +

Let players flex

+ Players will notice when other players have a special item or ability and ask "Howʼd you get that?!" This makes the owner feel special and encourages information sharing, which further increases the value of rare or limited items. + + + + + + + + +

Plan & promote your updates

+ + + + +

Plan for the long haul

+ Develop a roadmap outlining your content plans for the next 6 to 12 months, outlining regular content updates and new systems or major features that keep gameplay fresh. A well-defined roadmap will help you stay organized, manage resources, and communicate your vision to both your development team and your players. + + + + +

Spread the word

+ Promote your updates across your social media channels and within the experience itself. Build anticipation by teasing new features, showcasing behind-the-scenes development, and engaging with your community. + + + + +

Use update announcements

+ The integrated events system lets you create time-based events for your experience. Players can discover your events on the experience's detail page and through an event details page, and they can opt into notifications that they'll receive when your event begins. + + + + + + + diff --git a/.cursor/rules/roblox/en-us/get-started/tools.md b/.cursor/rules/roblox/en-us/get-started/tools.md new file mode 100644 index 00000000..cbc0d6f1 --- /dev/null +++ b/.cursor/rules/roblox/en-us/get-started/tools.md @@ -0,0 +1,207 @@ +--- +title: What tools does Roblox provide? +description: Roblox Studio, Creator Hub, AI Assistant, matchmaking, discovery, and more. +hideInPageNavigation: true +fullWidth: true +--- + +

Dive into Roblox Studio

+ + + + +

Your one-stop shop

+ Create 3D environments, add animations, build physics simulations, and program game logic with Lua. + + + + +

Collaborate with Team Create

+ Multiple users can access and edit the same project simultaneously, making it easier to collaborate and communicate with teammates. + + + + +

Test and publish directly from Studio

+ Test your creations and quickly iterate, then when youʼre ready, publish to Roblox with the click of a button. + + + + + + + + +

Learn and grow with Creator Hub

+ + + + +

Creator Dashboard

+ Access your experienceʼs engagement, retention, and monetization KPIs, free of charge. + + + + +

Documentation

+ Hit the ground running with articles, tutorials and videos on topics ranging from design and implementation to analytics and LiveOps. + + + + +

Developer Forum

+ Get advice from the community, file a bug report, and learn about the latest Roblox updates. + + + + + + + + +

Creator Hub: Marketplace and Talent Hub

+ + + + +

Marketplace

+ Find all the assets you need, from royalty-free music to community-made models and helpful plugins. + + + + +

Talent Hub

+ Whether youʼre searching for artists, programmers or designers, you can find them and review their work on Talent Hub. + + + + + + + + +

Meet your AI assistant

+ + + + +

Supercharge your workflow

+ Our AI Assistant revolutionizes your development process by providing instant, context-aware support. Get immediate answers to your Roblox questions, generate and modify code snippets, and insert assets directly into your game, all within Studio. + + + + +

Automate the mundane

+ AI Assistant empowers you to automate common actions such as bulk property modifications. By handling these time-consuming chores, Assistant frees you to concentrate on the higher-level design and innovative gameplay elements that make your experiences truly unique. + + + + + + + + +

Imagine it, generate it

+ + + + +

Generate 3D worlds

+ Roblox Cube is a groundbreaking generative AI system that lets you create 3D models and environments directly from text prompts, reducing the time and effort required for 3D asset creation. + + + + +

Unlock player creativity

+ Cube empowers your players to personalize their gameplay in exciting new ways. By enabling the generation of unique 3D content within experiences, Cube opens the door for deeper player engagement and self-expression. + + + + + + + + +

Master matchmaking

+ + + + +

Fine-tune with signal control

+ Adjust the weight of [matchmaking](../matchmaking/customize-matchmaking.md) signals such as friends, occupancy, latency, and play history. Define up to two custom signals based on player attributes like skill level or game mode preference. + + + + +

Configure and preview

+ The intuitive interface in Creator Hub lets you configure your settings with ease, and the built-in preview and test feature allows you to see the impact of your changes before they go live. + + + + +

Optimize with integrated analytics

+ The dedicated analytics dashboard provides in-depth insights into signal metrics over time, allowing you to monitor the impact of your configurations on the player experience. + + + + + + + + + +

Unlock the power of discovery

+ + + + +

Targeted discovery

+ Roblox's "Recommended for You" algorithm intelligently connects each user with their ideal experiences, fostering deep engagement, social interaction, and repeat plays. + + + + +

Objective ranking

+ Experiences are ranked by key performance indicators like playtime and Robux spent. The Home Recommendations dashboard displays the relevant analytics for informed decision-making. + + + + +

Quality content

+ Roblox is committed to continuously improving search functionality to better understand user intent and showcase the highest-quality content, ensuring that exceptional experiences get the attention they deserve. + + + + + + + + +

Supercharge your growth

+ + + + +

Targeted user acquisition

+ Roblox Sponsored Ads empower you to reach the players most likely to engage with your experience. With comprehensive audience selection options, you can tailor your campaigns to specific demographics and interests, delivering maximum impact and driving high-quality traffic to your game. + + + + +

Managed promotions

+ Take control of your advertising strategy with flexible campaign scheduling and daily budget options. Roblox provides the tools you need to manage your ad spend effectively and track performance. + + + + + + + + + + + +Studio documentation
+First experience tutorial + + diff --git a/.cursor/rules/roblox/en-us/get-started/why-build-on-roblox.md b/.cursor/rules/roblox/en-us/get-started/why-build-on-roblox.md new file mode 100644 index 00000000..bb268944 --- /dev/null +++ b/.cursor/rules/roblox/en-us/get-started/why-build-on-roblox.md @@ -0,0 +1,98 @@ +--- +title: Why should I develop on Roblox? +description: Reasons to build on the Roblox platform—audience, tools, and opportunity. +hideInPageNavigation: true +fullWidth: true +--- + +

Robust tools with no upfront costs

+ + + + +

Hit the ground running

+ We provide services, developer support, and free tools such as Roblox Studio to help you get started immediately. + + + + +

Keep it simple

+ We cover all hosting, storage, customer support, localization, payment processing, and platform fees, allowing you to focus on developing your experience. + + + + + + + + +

Multiplayer made easy

+ + + + +

Out of the box multiplayer

+ All experiences support multiplayer out of the gate. No need to write complicated backend code, no need to set up or pay for your own servers. + + + + +

Servers support up to 100 players

+ You donʼt need a big team to create a big game. Creating large multiplayer experiences has never been easier. + + + + +

Scale as you grow

+ Weʼve significantly increased the limits on free data storage and now offer creators the option to purchase additional service usage to support even larger experiences. + + + + + + + + +

Publish, analyze, and iterate with ease

+ + + + +

No publishing approvals

+ Assets are automatically moderated, so you can modify and publish your experience immediately across console, desktop, and mobile devices. + + + + +

Get deep insights

+ Built-in Roblox analytics give you access to key retention, engagement, and monetization KPIs for free with no setup required. + + + + +

Track your traffic

+ Acquire users with our paid UA system and get insights into how users are finding your game to optimize your UA strategy. + + + + + + + + diff --git a/.cursor/rules/roblox/en-us/luau/comments.md b/.cursor/rules/roblox/en-us/luau/comments.md index 71272108..9038248d 100644 --- a/.cursor/rules/roblox/en-us/luau/comments.md +++ b/.cursor/rules/roblox/en-us/luau/comments.md @@ -6,7 +6,7 @@ description: Comments are text that the Luau parser ignores at runtime. A **comment** is text that the Luau parser ignores at runtime. -Roblox Studio has a similarly-named [comments](../projects/collaboration.md#studio-comments) feature that is unrelated to code comments in Luau. +Roblox Studio has a similarly-named [comments](../projects/collaboration.md#comments) feature that is unrelated to code comments in Luau. ## Single-line comments diff --git a/.cursor/rules/roblox/en-us/marketplace/marketplace-fees-and-commissions.md b/.cursor/rules/roblox/en-us/marketplace/marketplace-fees-and-commissions.md index 406ba904..b944c9f8 100644 --- a/.cursor/rules/roblox/en-us/marketplace/marketplace-fees-and-commissions.md +++ b/.cursor/rules/roblox/en-us/marketplace/marketplace-fees-and-commissions.md @@ -453,6 +453,41 @@ To protect against automated purchases, Roblox sets purchase limits for Limiteds - A maximum of 3 successful purchases per user per item. - For in-experience purchases, users must be in the experience for 60 seconds to successfully complete a purchase. +## Timed options + + +At this time, timed options are available for 3D shirts, pants, and sweater categories. + + +Timed options allows users to purchase your items for a period of time of either 3, 7, or 14 days at a cheaper cost. You can enable or disable timed options for your Marketplace assets [in bulk](#in-bulk) or [per item](#per-item). + +
+Example of timed options displaying for end user viewing marketplace item +
With timed options enabled, users gain access to multiple purchase options for each available time range.
+
+ +### In bulk + +To toggle timed options for multiple assets: + +1. Navigate to your [Creations](https://create.roblox.com/dashboard/creations) page > Avatar items. +2. Filter by avatar asset type, such as **Clothing**. +3. On the top right, select the **Timed option** button. + Timed options button on top right of Creations page + +4. Follow the prompts to confirm the bulk selection and timed option settings. + + Confirmation prompt for setting timed options for all assets + +### Per item + +To toggle timed options for individual assets: + +1. In your [Creations](https://create.roblox.com/dashboard/creations) tab, navigate to the individual asset's **Manage Item** page. +2. Toggle the timed option for your asset. + Individual timed option toggle on the specific asset sale setting page +3. When complete, click the **Save Changes** button. + ## Price ranges Depending on the 3D asset type, you must set the price of the item within the marketplace price range. If you are selling a [Limited](#limiteds) item, you can set your item price as **0**, or any value after the **Limiteds Price Floor**. diff --git a/.cursor/rules/roblox/en-us/parts/model-generation.md b/.cursor/rules/roblox/en-us/parts/model-generation.md index 77353e18..8d79664b 100644 --- a/.cursor/rules/roblox/en-us/parts/model-generation.md +++ b/.cursor/rules/roblox/en-us/parts/model-generation.md @@ -116,7 +116,7 @@ style FunctionalModel color:white, fill:SeaGreen To test retargetable scripts: -1. Download the `Behaviors.rbxm` file (right‑click and "save as" to your local system). +1. Download the `Behaviors.rbxm` file. 2. In Studio's [Explorer](../studio/explorer.md) hierarchy, right‑click `Class.ReplicatedStorage`, select **Insert** ⟩ **Import Roblox Model**, and choose the downloaded file. The model unpacks into several folders. diff --git a/.cursor/rules/roblox/en-us/parts/textures-decals.md b/.cursor/rules/roblox/en-us/parts/textures-decals.md index a46a5b33..849cf803 100644 --- a/.cursor/rules/roblox/en-us/parts/textures-decals.md +++ b/.cursor/rules/roblox/en-us/parts/textures-decals.md @@ -21,7 +21,7 @@ add a `Class.Texture` or `Class.Decal` object to a part or union, you can:
- The same blue texture repeated eight times on a block part. + The same texture repeated eight times on a block part.
Texture applied to a part (repeating)
 @@ -31,13 +31,13 @@ add a `Class.Texture` or `Class.Decal` object to a part or union, you can:
An example decal image of a light purple hexagon on top of a dark purple background. -
Texture image
+
Decal image
- The same blue texture repeated eight times on a block part. -
Texture applied to a part (repeating)
+ The same decal stretched on a block part. +
Decal applied to a part (stretched)
 diff --git a/.cursor/rules/roblox/en-us/physics/constraints/align-orientation.md b/.cursor/rules/roblox/en-us/physics/constraints/align-orientation.md index 42c678c2..da223b27 100644 --- a/.cursor/rules/roblox/en-us/physics/constraints/align-orientation.md +++ b/.cursor/rules/roblox/en-us/physics/constraints/align-orientation.md @@ -4,53 +4,39 @@ description: The AlignOrientation constraint applies torque to align two attachm --- -For an overview on creating, visualizing, and simulating mover constraints, including `Class.AlignOrientation`, see [Mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. +For an overview on creating, visualizing, and simulating mover constraints, including `Class.AlignOrientation`, see [mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. The `Class.AlignOrientation` constraint applies torque to align two attachments, or to align one attachment with a goal orientation. As indicated by the name, it only affects the **orientation** of the attachments, not their position (to align attachments **positionally**, see [AlignPosition](../../physics/constraints/align-position.md)). - + -By default, the constraint only applies torque to `Class.Constraint.Attachment0|Attachment0`, although this behavior can be controlled through `Class.AlignOrientation.ReactionTorqueEnabled|ReactionTorqueEnabled`. +By default, the constraint only applies torque to `Class.Constraint.Attachment0|Attachment0`, although this behavior can be controlled through the constraint's [reactionary torque](#reactionary-torque). ## Affected axes -The axes affected by torque are controlled through the constraint's `Class.AlignOrientation.AlignType|AlignType` property. When set to `Enum.AlignType|PrimaryAxisParallel`, `Enum.AlignType|PrimaryAxisPerpendicular` or `Enum.AlignType|PrimaryAxisLookAt`, torque will only occur when the primary axes become misaligned. Otherwise, the constraint will apply torque about all 3 axes to achieve alignment. +The axes affected by torque are controlled through the constraint's `Class.AlignOrientation.AlignType|AlignType` property. When set to `Enum.AlignType.AllAxes|AllAxes` (default), the constraint will apply torque about all 3 axes to achieve alignment. When set to `Enum.AlignType.PrimaryAxisParallel|PrimaryAxisParallel`, `Enum.AlignType.PrimaryAxisPerpendicular|PrimaryAxisPerpendicular`, or `Enum.AlignType.PrimaryAxisLookAt|PrimaryAxisLookAt`, torque will only occur when the **primary** axes become misaligned. - -
- -
AlignType = **AllAxes**
-
-
- -
AlignType = **PrimaryAxisParallel**
-
- - -## Reactionary torque - -By default, the constraint only applies torque to `Class.Constraint.Attachment0|Attachment0` while `Class.Constraint.Attachment1|Attachment1` remains unaffected. If desired, torque can be applied to both attachments in **equal and opposite directions** by enabling `Class.AlignOrientation.ReactionTorqueEnabled|ReactionTorqueEnabled`. - - -
- -
ReactionTorqueEnabled = **false**
-
-
- -
ReactionTorqueEnabled = **true**
-
- +
+ +
## Torque magnitude -You can configure the `Class.AlignOrientation` constraint to apply the maximum torque that constraints allow through the `Class.AlignOrientation.RigidityEnabled|RigidityEnabled` property. When true, the physics solver reacts as quickly as possible to complete the alignment. When false, the torque is determined by `Class.AlignOrientation.MaxTorque|MaxTorque`, `Class.AlignOrientation.MaxAngularVelocity|MaxAngularVelocity`, and `Class.AlignOrientation.Responsiveness|Responsiveness`. +You can configure the `Class.AlignOrientation` constraint to apply the maximum torque that constraints allow through the `Class.AlignOrientation.RigidityEnabled|RigidityEnabled` property. When `true`, the physics solver reacts as quickly as possible to complete the alignment. When `false`, the torque is determined by `Class.AlignOrientation.MaxTorque|MaxTorque`, `Class.AlignOrientation.MaxAngularVelocity|MaxAngularVelocity`, and `Class.AlignOrientation.Responsiveness|Responsiveness`. + +
+ +
+ +## Reactionary torque + +By default, the constraint only applies torque to `Class.Constraint.Attachment0|Attachment0` while `Class.Constraint.Attachment1|Attachment1` remains unaffected. If desired, torque can be applied to **both** attachments in **equal and opposite directions** by enabling `Class.AlignOrientation.ReactionTorqueEnabled|ReactionTorqueEnabled`. ## Attachment mode -The `Class.AlignOrientation.Mode|Mode` property specifies whether the constraint uses **one** or **two** attachments in calculating its goal. By default, this is **TwoAttachment**, meaning that the constraint attempts to match the orientation of `Class.Constraint.Attachment0|Attachment0` with the orientation of `Class.Constraint.Attachment1|Attachment1`, disregarding `Class.AlignOrientation.CFrame|CFrame`, `Class.AlignOrientation.PrimaryAxis|PrimaryAxis`, and `Class.AlignOrientation.SecondaryAxis|SecondaryAxis`. +The `Class.AlignOrientation.Mode|Mode` property specifies whether the constraint uses **one** or **two** attachments in calculating its goal. By default, this is `Enum.OrientationAlignmentMode.TwoAttachment|TwoAttachment`, meaning that the constraint attempts to match the orientation of `Class.Constraint.Attachment0|Attachment0` with the orientation of `Class.Constraint.Attachment1|Attachment1`, disregarding `Class.AlignOrientation.CFrame|CFrame`, `Class.AlignOrientation.PrimaryAxis|PrimaryAxis`, and `Class.AlignOrientation.SecondaryAxis|SecondaryAxis`. -If set to **OneAttachment**, the constraint disregards `Class.Constraint.Attachment1|Attachment1` and attempts to match the orientation of `Class.Constraint.Attachment0|Attachment0` with the orientation of `Class.AlignOrientation.CFrame|CFrame`, or match the attachment's `Class.Attachment.Axis|Axis` and `Class.Attachment.SecondaryAxis|SecondaryAxis` with the constraint's `Class.AlignOrientation.PrimaryAxis|PrimaryAxis` and `Class.AlignOrientation.SecondaryAxis|SecondaryAxis` properties respectively. +If set to `Enum.OrientationAlignmentMode.OneAttachment|OneAttachment`, the constraint disregards `Class.Constraint.Attachment1|Attachment1` and attempts to match the orientation of `Class.Constraint.Attachment0|Attachment0` with the orientation of `Class.AlignOrientation.CFrame|CFrame`, or match the attachment's `Class.Attachment.Axis|Axis` and `Class.Attachment.SecondaryAxis|SecondaryAxis` with the constraint's `Class.AlignOrientation.PrimaryAxis|PrimaryAxis` and `Class.AlignOrientation.SecondaryAxis|SecondaryAxis` properties respectively. diff --git a/.cursor/rules/roblox/en-us/physics/constraints/align-position.md b/.cursor/rules/roblox/en-us/physics/constraints/align-position.md index 6f48a1b3..59ed6833 100644 --- a/.cursor/rules/roblox/en-us/physics/constraints/align-position.md +++ b/.cursor/rules/roblox/en-us/physics/constraints/align-position.md @@ -4,51 +4,33 @@ description: The AlignPosition constraint applies force to move two attachments --- -For an overview on creating, visualizing, and simulating mover constraints, including `Class.AlignPosition`, see [Mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. +For an overview on creating, visualizing, and simulating mover constraints, including `Class.AlignPosition`, see [mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. An `Class.AlignPosition` constraint applies force to move two attachments together, or to move one attachment to a goal position. As indicated by the name, it only affects the **position** of the attachments, not their orientation (to align attachments by **orientation**, see [AlignOrientation](../../physics/constraints/align-orientation.md)). - + -By default, the constraint only applies force to `Class.Constraint.Attachment0|Attachment0`, although this behavior can be controlled through `Class.AlignPosition.ReactionForceEnabled|ReactionForceEnabled`. +By default, the constraint only applies force to `Class.Constraint.Attachment0|Attachment0`, although this behavior can be controlled through the constraint's [reactionary force](#reactionary-force). ## Force location By default, force is applied to the parent of `Class.Constraint.Attachment0|Attachment0` at that attachment's location, meaning that if the parent's center of mass is not aligned with the direction of the force, torque will be applied as well as force. Alternatively, force can be applied to the parents' center of mass by toggling on `Class.AlignPosition.ApplyAtCenterOfMass|ApplyAtCenterOfMass`. - -
- -
ApplyAtCenterOfMass = **false**
-
-
- -
ApplyAtCenterOfMass = **true**
-
- +## Force magnitude + +You can configure the `Class.AlignPosition` constraint to apply the maximum force that constraints allow through the `Class.AlignPosition.RigidityEnabled|RigidityEnabled` property. When `true`, the physics solver reacts as quickly as possible to complete the alignment. When `false`, the force is determined by `Class.AlignPosition.MaxForce|MaxForce`, `Class.AlignPosition.MaxVelocity|MaxVelocity`, and `Class.AlignPosition.Responsiveness|Responsiveness`. ## Reactionary force -By default, the constraint only applies force to `Class.Constraint.Attachment0|Attachment0` while `Class.Constraint.Attachment1|Attachment1` remains unaffected. If desired, force can be applied to both attachments in **equal and opposite directions** by enabling `Class.AlignPosition.ReactionForceEnabled|ReactionForceEnabled`. +By default, the constraint only applies force to `Class.Constraint.Attachment0|Attachment0` while `Class.Constraint.Attachment1|Attachment1` remains unaffected. If desired, force can be applied to **both** attachments in **equal and opposite directions** by enabling `Class.AlignPosition.ReactionForceEnabled|ReactionForceEnabled`. - -
- -
ReactionForceEnabled = **false**
-
-
- -
ReactionForceEnabled = **true**
-
- - -## Force magnitude - -You can configure the `Class.AlignPosition` constraint to apply the maximum force that constraints allow through the `Class.AlignPosition.RigidityEnabled|RigidityEnabled` property. When true, the physics solver reacts as quickly as possible to complete the alignment. When false, the force is determined by `Class.AlignPosition.MaxForce|MaxForce`, `Class.AlignPosition.MaxVelocity|MaxVelocity`, and `Class.AlignPosition.Responsiveness|Responsiveness`. +
+ +
## Attachment mode -The `Class.AlignPosition.Mode|Mode` property specifies whether the constraint uses **one** or **two** attachments in calculating its goal. By default, this is `Enum.PositionAlignmentMode|TwoAttachment`, meaning that the constraint disregards `Class.AlignPosition.Position|Position` and attempts to move `Class.Constraint.Attachment0|Attachment0` to the position of `Class.Constraint.Attachment1|Attachment1`. If set to `Enum.PositionAlignmentMode|OneAttachment`, the constraint disregards `Class.Constraint.Attachment1|Attachment1` and attempts to move `Class.Constraint.Attachment0|Attachment0` to `Class.AlignPosition.Position|Position`. +The `Class.AlignPosition.Mode|Mode` property specifies whether the constraint uses **one** or **two** attachments in calculating its goal. By default, this is `Enum.PositionAlignmentMode.TwoAttachment|TwoAttachment`, meaning that the constraint disregards `Class.AlignPosition.Position|Position` and attempts to move `Class.Constraint.Attachment0|Attachment0` to the position of `Class.Constraint.Attachment1|Attachment1`. If set to `Enum.PositionAlignmentMode.OneAttachment|OneAttachment`, the constraint disregards `Class.Constraint.Attachment1|Attachment1` and attempts to move `Class.Constraint.Attachment0|Attachment0` to `Class.AlignPosition.Position|Position`. diff --git a/.cursor/rules/roblox/en-us/physics/constraints/angular-velocity.md b/.cursor/rules/roblox/en-us/physics/constraints/angular-velocity.md index b577482e..0fec305b 100644 --- a/.cursor/rules/roblox/en-us/physics/constraints/angular-velocity.md +++ b/.cursor/rules/roblox/en-us/physics/constraints/angular-velocity.md @@ -4,7 +4,7 @@ description: The AngularVelocity constraint applies torque on an assembly to mai --- -For an overview on creating, visualizing, and simulating mover constraints, including `Class.AngularVelocity`, see [Mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. +For an overview on creating, visualizing, and simulating mover constraints, including `Class.AngularVelocity`, see [mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. An `Class.AngularVelocity` constraint applies torque on an assembly to maintain a constant angular velocity. diff --git a/.cursor/rules/roblox/en-us/physics/constraints/animation.md b/.cursor/rules/roblox/en-us/physics/constraints/animation.md new file mode 100644 index 00000000..b7de66fa --- /dev/null +++ b/.cursor/rules/roblox/en-us/physics/constraints/animation.md @@ -0,0 +1,32 @@ +--- +title: AnimationConstraint +description: AnimationConstraint aligns two base parts with an animate-able kinematic or force-based joint. +--- + + +For an overview on creating, visualizing, and simulating mover constraints, including `Class.AnimationConstraint`, see [mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. + + +An `Class.AnimationConstraint` constrains its `Class.Attachment|Attachments` by an offset [transform](#transform). It can be either [kinematic](#kinematic) for snap‑to behavior or it can follow the target trajectory using [force and torque](#force-and-torque). limited by `Class.AnimationConstraint.MaxForce|MaxForce` and `Class.AnimationConstraint.MaxTorque|MaxTorque`. + + + +## Transform + +The constraint's `Class.AnimationConstraint.Transform|Transform` is the internal `Datatype.CFrame` that's manipulated when the constraint is being animated. How this transform is reached depends on the constraint's [kinematic](#kinematic) setting. + +
+ +
+ +## Kinematic + +When the constraint's `Class.AnimationConstraint.IsKinematic|IsKinematic` property is `true`, the connected parts snap to the [transform](#transform) instantly and precisely without participating in physics simulation. When `false`, the connected parts follow the trajectory using [force and torque](#force-and-torque). + +
+ +
+ +## Force and torque + +If the constraint is **not** set to [kinematic](#kinematic), the connected parts follow a trajectory toward the [transform](#transform) using a maximum force/torque limited by `Class.AnimationConstraint.MaxForce|MaxForce` and `Class.AnimationConstraint.MaxTorque|MaxTorque` respectively. diff --git a/.cursor/rules/roblox/en-us/physics/constraints/ball-socket.md b/.cursor/rules/roblox/en-us/physics/constraints/ball-socket.md index e3469699..ae617512 100644 --- a/.cursor/rules/roblox/en-us/physics/constraints/ball-socket.md +++ b/.cursor/rules/roblox/en-us/physics/constraints/ball-socket.md @@ -9,25 +9,26 @@ For an overview on creating, visualizing, and simulating mechanical constraints, A `Class.BallSocketConstraint` forces its two attachments into the same position and allows them to freely rotate about all three axes. However, you can enable limits to restrict both tilt and twist. - + ## Limits You can set limits to restrict both **tilt** and **twist** of a ball socket, similar to how a human's head can tilt and turn within a limited axial range. Enabling the `Class.BallSocketConstraint.LimitsEnabled|LimitsEnabled` property exposes the `Class.BallSocketConstraint.UpperAngle|UpperAngle` value to restrict **tilt** within a cone; it also exposes the `Class.BallSocketConstraint.TwistLimitsEnabled|TwistLimitsEnabled` property which, when enabled, lets you restrict **twist** rotation through the `Class.BallSocketConstraint.TwistLowerAngle|TwistLowerAngle` and `Class.BallSocketConstraint.TwistUpperAngle|TwistUpperAngle` limits. - -
- -
UpperAngle = **30**
-
-
- -
TwistLimitsEnabled = **true**
-
- +
+ +
Constraint with `Class.BallSocketConstraint.LimitsEnabled|LimitsEnabled` set to `true` and `Class.BallSocketConstraint.UpperAngle|UpperAngle` set to `15`
+
-Enabling `Class.BallSocketConstraint.LimitsEnabled|LimitsEnabled` also exposes the `Class.BallSocketConstraint.Restitution|Restitution` value which defines the elasticity of the attachments at their tilt and/or twist limits. +
+ +
Constraint with `Class.BallSocketConstraint.LimitsEnabled|LimitsEnabled` and `Class.BallSocketConstraint.TwistLimitsEnabled|TwistLimitsEnabled` set to `true`, `Class.BallSocketConstraint.TwistLowerAngle|TwistLowerAngle` set to `-60`, and `Class.BallSocketConstraint.TwistUpperAngle|TwistUpperAngle` set to `0`
+
+Enabling `Class.BallSocketConstraint.LimitsEnabled|LimitsEnabled` also exposes the `Class.BallSocketConstraint.Restitution|Restitution` value which defines the elasticity of the attachments at their tilt and/or twist limits. + + + When `Class.BallSocketConstraint.LimitsEnabled|LimitsEnabled` is enabled, orientation of a ball socket's attachments affects how it rotates. To ensure rotation occurs around the desired axis, each attachment's `Class.Attachment.Axis|Axis`, visualized by the yellow arrow, should point in the same direction. diff --git a/.cursor/rules/roblox/en-us/physics/constraints/line-force.md b/.cursor/rules/roblox/en-us/physics/constraints/line-force.md index 6aade607..6c17703c 100644 --- a/.cursor/rules/roblox/en-us/physics/constraints/line-force.md +++ b/.cursor/rules/roblox/en-us/physics/constraints/line-force.md @@ -4,7 +4,7 @@ description: The LineForce constraint applies force along the theoretical line c --- -For an overview on creating, visualizing, and simulating mover constraints, including `Class.LineForce`, see [Mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. +For an overview on creating, visualizing, and simulating mover constraints, including `Class.LineForce`, see [mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. The `Class.LineForce` constraint applies force along the theoretical line connecting its two `Class.Attachment|Attachments`. As the end points (attachments) move, the direction of force will change accordingly. diff --git a/.cursor/rules/roblox/en-us/physics/constraints/linear-velocity.md b/.cursor/rules/roblox/en-us/physics/constraints/linear-velocity.md index 7d461f24..bddd3a32 100644 --- a/.cursor/rules/roblox/en-us/physics/constraints/linear-velocity.md +++ b/.cursor/rules/roblox/en-us/physics/constraints/linear-velocity.md @@ -4,7 +4,7 @@ description: The LinearVelocity constraint applies force on an assembly to maint --- -For an overview on creating, visualizing, and simulating mover constraints, including `Class.LinearVelocity`, see [Mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. +For an overview on creating, visualizing, and simulating mover constraints, including `Class.LinearVelocity`, see [mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. A `Class.LinearVelocity` constraint applies force on an assembly to maintain a constant velocity. It can be set to apply force along a `Datatype.Vector3`, line, or 2D plane. diff --git a/.cursor/rules/roblox/en-us/physics/constraints/torque.md b/.cursor/rules/roblox/en-us/physics/constraints/torque.md index e00f74fb..60a39e9f 100644 --- a/.cursor/rules/roblox/en-us/physics/constraints/torque.md +++ b/.cursor/rules/roblox/en-us/physics/constraints/torque.md @@ -4,7 +4,7 @@ description: The Torque constraint applies constant torque on an assembly from i --- -For an overview on creating, visualizing, and simulating mover constraints, including `Class.Torque`, see [Mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. +For an overview on creating, visualizing, and simulating mover constraints, including `Class.Torque`, see [mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. A `Class.Torque` constraint applies constant torque on an assembly from its center of mass. diff --git a/.cursor/rules/roblox/en-us/physics/constraints/vector-force.md b/.cursor/rules/roblox/en-us/physics/constraints/vector-force.md index 89f3e883..8c760639 100644 --- a/.cursor/rules/roblox/en-us/physics/constraints/vector-force.md +++ b/.cursor/rules/roblox/en-us/physics/constraints/vector-force.md @@ -4,7 +4,7 @@ description: The VectorForce constraint applies constant linear force on an asse --- -For an overview on creating, visualizing, and simulating mover constraints, including `Class.VectorForce`, see [Mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. +For an overview on creating, visualizing, and simulating mover constraints, including `Class.VectorForce`, see [mover constraints](../../physics/mover-constraints.md). Also see [Roblox units](../../physics/units.md) to understand how Roblox units compare to metric units. The `Class.VectorForce` constraint applies constant linear force on an assembly. The direction and strength of the force is determined by a `Datatype.Vector3` and can be relative to an attachment on the part, another attachment, or the world coordinate system. diff --git a/.cursor/rules/roblox/en-us/physics/mechanical-constraints.md b/.cursor/rules/roblox/en-us/physics/mechanical-constraints.md index 59c6977c..9aae151b 100644 --- a/.cursor/rules/roblox/en-us/physics/mechanical-constraints.md +++ b/.cursor/rules/roblox/en-us/physics/mechanical-constraints.md @@ -15,9 +15,9 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

- +

-
[BallSocketConstraint](../physics/constraints/ball-socket.md) forces its two attachments into the same position and allows them to freely rotate about all three axes, with optional limits to restrict both tilt and twist
+
`Class.BallSocketConstraint` forces its two attachments into the same position and allows them to freely rotate about all three axes, with optional limits to restrict both tilt and twist
 @@ -29,7 +29,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[HingeConstraint](../physics/constraints/hinge.md) allows its two attachments to rotate about one axis, with optional assigned power for motor or servo behavior
+
`Class.HingeConstraint` allows its two attachments to rotate about one axis, with optional assigned power for motor or servo behavior
@@ -41,7 +41,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[PrismaticConstraint](../physics/constraints/prismatic.md) allows two attachments to slide along one axis but not rotate, with optional assigned power for mechanisms like sliding doors and elevator platforms
+
`Class.PrismaticConstraint` allows two attachments to slide along one axis but not rotate, with optional assigned power for mechanisms like sliding doors and elevator platforms
@@ -53,7 +53,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[CylindricalConstraint](../physics/constraints/cylindrical.md) allows its attachments to slide along one axis and rotate about another axis, with optional assigned angular and/or linear power
+
`Class.CylindricalConstraint` allows its attachments to slide along one axis and rotate about another axis, with optional assigned angular and/or linear power
@@ -65,7 +65,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[SpringConstraint](../physics/constraints/spring.md) applies a force on its attachments based on spring and damper behavior, with an optional minimum/maximum length
+
`Class.SpringConstraint` applies a force on its attachments based on spring and damper behavior, with an optional minimum/maximum length
@@ -77,7 +77,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[TorsionSpringConstraint](../physics/constraints/torsion-spring.md) applies torque based on a relative angle and relative angular velocity, in an attempt to bring two axes from two parts together
+
`Class.TorsionSpringConstraint` applies torque based on a relative angle and relative angular velocity, in an attempt to bring two axes from two parts together
@@ -89,7 +89,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[UniversalConstraint](../physics/constraints/universal.md) ensures two axes on two assemblies remain perpendicular, useful for applications such as vehicle power transmission to rear drive shafts, robotics, and more
+
`Class.UniversalConstraint` ensures two axes on two assemblies remain perpendicular, useful for applications such as vehicle power transmission to rear drive shafts, robotics, and more
@@ -101,7 +101,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[RopeConstraint](../physics/constraints/rope.md) prevents two attachments from separating further than a defined length, with optional behavior as an extending or contracting winch
+
`Class.RopeConstraint` prevents two attachments from separating further than a defined length, with optional behavior as an extending or contracting winch
@@ -113,7 +113,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[RodConstraint](../physics/constraints/rod.md) keeps two attachments separated by a defined length, with optional limits on rotational tilt
+
`Class.RodConstraint` keeps two attachments separated by a defined length, with optional limits on rotational tilt
@@ -125,7 +125,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[PlaneConstraint](../physics/constraints/plane.md) moves two attachments into a position/orientation along a plane, and both attachments remain free to translate and rotate unless otherwise constrained
+
`Class.PlaneConstraint` moves two attachments into a position/orientation along a plane, and both attachments remain free to translate and rotate unless otherwise constrained
@@ -135,7 +135,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[WeldConstraint](../physics/constraints/weld.md) connects two `Class.BasePart|BaseParts` and ensures they stay in the same relative position and orientation to each other, even if they are not touching
+
`Class.WeldConstraint` connects two `Class.BasePart|BaseParts` and ensures they stay in the same relative position and orientation to each other, even if they are not touching
 @@ -145,7 +145,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[RigidConstraint](../physics/constraints/rigid.md) connects two `Class.Attachment|Attachments` or `Class.Bone|Bones` and ensures they stay in the same relative position/orientation to each other, even if they are not touching
+
`Class.RigidConstraint` connects two `Class.Attachment|Attachments` or `Class.Bone|Bones` and ensures they stay in the same relative position/orientation to each other, even if they are not touching
 @@ -155,7 +155,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that be

-
[NoCollisionConstraint](../physics/constraints/no-collision.md) prevents collisions between two specific parts, but those parts may still register collisions with the rest of the world
+
`Class.NoCollisionConstraint` prevents collisions between two specific parts, but those parts may still register collisions with the rest of the world
 diff --git a/.cursor/rules/roblox/en-us/physics/mover-constraints.md b/.cursor/rules/roblox/en-us/physics/mover-constraints.md index f86941bb..eba1df38 100644 --- a/.cursor/rules/roblox/en-us/physics/mover-constraints.md +++ b/.cursor/rules/roblox/en-us/physics/mover-constraints.md @@ -17,7 +17,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that ap

-
[LinearVelocity](../physics/constraints/linear-velocity.md) applies force on an assembly to maintain a constant velocity along a 3D vector, line, or 2D plane
+
`Class.LinearVelocity` applies force on an assembly to maintain a constant velocity along a 3D vector, line, or 2D plane
@@ -29,7 +29,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that ap

-
[AngularVelocity](../physics/constraints/angular-velocity.md) applies torque on an assembly to maintain a constant angular velocity
+
`Class.AngularVelocity` applies torque on an assembly to maintain a constant angular velocity
@@ -41,7 +41,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that ap

-
[AlignPosition](../physics/constraints/align-position.md) applies force to move two attachments together, or to move one attachment to a goal position
+
`Class.AlignPosition` applies force to move two attachments together, or to move one attachment to a goal position
@@ -53,7 +53,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that ap

-
[AlignOrientation](../physics/constraints/align-orientation.md) applies torque to align two attachments, or to align one attachment with a goal orientation
+
`Class.AlignOrientation` applies torque to align two attachments, or to align one attachment with a goal orientation
@@ -65,7 +65,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that ap

-
[VectorForce](../physics/constraints/vector-force.md) applies constant linear force on an assembly
+
`Class.VectorForce` applies constant linear force on an assembly
@@ -77,7 +77,7 @@ The physics engine includes the following `Class.Constraint|Constraints` that ap

-
[Torque](../physics/constraints/torque.md) applies constant torque on an assembly from its center of mass
+
`Class.Torque` applies constant torque on an assembly from its center of mass
@@ -89,7 +89,19 @@ The physics engine includes the following `Class.Constraint|Constraints` that ap

-
[LineForce](../physics/constraints/line-force.md) applies force along the theoretical line connecting its two attachments
+
`Class.LineForce` applies force along the theoretical line connecting its two attachments
+ + + + + + + + +

+ +

+
`Class.AnimationConstraint` constrains its `Class.Attachment|Attachments` by an offset transform `Datatype.CFrame`.
 diff --git a/.cursor/rules/roblox/en-us/production/creator-store.md b/.cursor/rules/roblox/en-us/production/creator-store.md index 754a7980..e0fd5710 100644 --- a/.cursor/rules/roblox/en-us/production/creator-store.md +++ b/.cursor/rules/roblox/en-us/production/creator-store.md @@ -50,6 +50,8 @@ In addition to being able to distribute more of each asset type, verifying your For information on how to verify your account either through a government-issued ID or through a phone number, see [account verification](../production/publishing/account-verification.md). +Verification status for group-owned assets is based on the user that last updated the asset. + **Phone number verification is not sufficient** to sell priced assets on the Creator Store. To sell priced assets, you need to verify with a [government ID](./publishing/account-verification.md#verify-through-government-id) and create a seller account. For more information, see [Sell on the Creator Store](./sell-on-creator-store.md). diff --git a/.cursor/rules/roblox/en-us/production/earn-on-roblox.md b/.cursor/rules/roblox/en-us/production/earn-on-roblox.md deleted file mode 100644 index 42c947f5..00000000 --- a/.cursor/rules/roblox/en-us/production/earn-on-roblox.md +++ /dev/null @@ -1,300 +0,0 @@ ---- -title: Earn on Roblox -description: Explains how Roblox's economy and virtual currency works. ---- - -Roblox's goal is to enable the creation of anything, anywhere, by anyone, and we are constantly -investing in new tools and services to empower our creator community. We give our creators, from an -individual to a large studio, diverse ways to earn on Roblox by creating content and experiences, -specifically: - -- [Develop an experience](#develop-an-experience) - - [In-experience purchases](#in-experience-purchases) - - [Immersive ads](#immersive-ads) -- [Creator Rewards](#creator-rewards) -- [Create, sell, and resell certain avatar items](#create-sell-and-resell-certain-avatar-items) -- [Create and sell Studio plugins and models](#create-and-sell-studio-plugins-and-models) - -This page provides information about different ways to earn on Roblox and how to [convert earned Robux to cash](#convert-earned-robux-to-cash) through the [Developer Exchange][Link_Developer_Exchange] program. - -
- -
†† 30,000 earned Robux required to cash out through the Developer Exchange program
-
- -## Economy powered by creators - -We are building one of the world's largest immersive platforms with 151.5M global daily active users (DAUs) in 17 languages across 180 countries, available across platforms including mobile, desktop, consoles, and VR headsets. In 2024, approximately 60% of our daily active users were 13 and over, and users aged 17 and over accounted for 44% of our daily active users. In 2024, our community visited over 14 million experiences. In the third quarter of 2025 alone, users spent 39.6 billion hours on our platform. - - -
-Roblox economy stats: Daily Active Users -
⑴
-
-
-Roblox economy stats: Creator Earnings -
⑵
-
- - -

We are one of the biggest virtual economies ⑶

-Roblox's GDP ⑷ has grown to become as large as that of some countries. Our success is directly tied to the success of our creators. Roblox earns money by selling Robux which users spend in‑experience and on items and assets in our marketplaces, all created by you. - -

We exist to serve our community

-We're always searching for new ways to increase creators' earnings. In 2024, our creator community earned \$923 million. We've paid our developer community \$3.3 billion since 2018. - -

As a result, creators can focus on creating

-We are seeing healthy growth across creators and studios of varying sizes. Since 2019, the average annual earnings of the top 10 and top 1000 creators has increased by ~5.5 and ~6.8 times, respectively. ⑸

- - - -
⑴ DAUs as of the quarter ended September 30, 2025.
-
⑵ Payouts for the year ended December 31, 2024.
-
⑶ Based on readily available public data for daily active users of virtual worlds.
-
⑷ Measured based on Roblox's annual bookings, which measures total economic activity on our platform.
-
⑸ Of the millions of creators monetizing on Roblox, over 24,500 were part of our DevEx program, with the median creator participating in DevEx receiving \$1,575 USD during the twelve months ended December 31, 2024.
- -## Develop an experience - -Building successful content takes time and effort, but there is low risk and low upfront cost to building an experience on Roblox. You pay **after** earning on Roblox and we provide the tools, services, and support to help you build the most immersive experiences. Roblox has a robust community and [developer forum](https://devforum.roblox.com/) to guide and support you along the way. - -### Low upfront costs; publish globally in seconds - -
Instant publishing to reach a global audience
-Unlike other platforms where you wait days or weeks for approvals, you can modify your Roblox experiences and publish immediately in multiple languages and on all popular platforms — iOS, Android, Mac, Windows, PlayStation, and Xbox. - -
Platform-driven discovery and rapid viral growth
-Our platform, with 151.5 million daily active users (DAUs) now able to see and join experiences with no additional downloads required, provides opportunity for rapid growth. Creators can take advantage of our organic channels, our rich social graph and network effect, seamless exploration between experiences and our cost effective [Roblox ads](../production/promotion/advertise-on-roblox.md). - -
Rapid iteration and scaling with no upfront costs
-With your content live and in front of lots of users, we provide a rapid and rich feedback loop on how users are engaging with your work. This feedback loop is facilitated both by our integrated [analytics](../production/analytics/index.md) system and by our monetization mechanisms. On the cost side, we handle infrastructure hosting, storage, customer support, localization, payment processing, moderation, and platform costs. You do not have to worry about standing up multiplayer servers or scaling them as your experience goes viral. Off Roblox, you often have to take care of these expenses and manage these operations yourself. - -
Pay after you earn on Roblox
-On other platforms, you can incur expenses such as storage, hosting, and infrastructure before you earn any money or even start building. On Roblox, developing an experience has minimal risk as you are charged for these expenses after you monetize. - -### Average cash payouts - -One of our goals is to drive as much money to our creators as possible while maintaining reasonable margins for our company to develop, maintain, and improve the tools and technology that support our community. - -On average, 67% of all spending in experiences supports or goes to developers. The below applies to Robux spending within experiences only; it excludes developer subscriptions which are a relatively small portion of the business today. - -- We pay out 40% to cover costs supporting experiences — like infrastructure hosting, storage, customer support, localization, payment processing, and moderation — which other platforms generally pass on to developers. -- This enables us to return 28%\* directly to the developers. -- That leaves 33% to cover Roblox's ongoing services and operating costs, future-looking investments in the platform, and margins. - -This chart illustrates the estimated utilization of each dollar spent in an experience on Roblox. Note that the chart doesn't reflect our expenses as disclosed in our GAAP financial statements. - -
-Pie chart illustrating the estimated utilization of each dollar spent in an experience on Roblox. -
- - - - - - - - - -
-

This percentage is the experience developer's earnings based on transactions in the experience, after cashing out through our [Developer Exchange][Link_Developer_Exchange] program (before any developer spending on the Roblox platform for services such as ads). Each developer's earnings share may differ. Earnings share may also differ by payment channel; users can now receive up to 25% more Robux when purchasing through gift cards, computer, or web. This extra Robux translates into higher revenue shares on Robux purchased on those channels.

-
- - - - - - - - - -
-

Developers also get extra Robux for simply building an engaging experience through the [Creator Rewards](../creator-rewards.md) program.

-

This percentage reflects our platform average, though the actual percent of earnings from Creator Rewards varies from experience to experience based on the engagement level in the experience.

-

We are a platform and therefore we do not make experiences that compete with our creators for these payouts.

-
- - - - - - - - - -
-

Roblox covers this on your behalf. This reflects the processing payment fees related to all purchases and refunds of Robux. These fees are paid directly to Apple (iOS), Google and Amazon (Android), Sony, Meta, and Microsoft (Xbox and Microsoft Store).

-

These costs also include processing fees associated with other payment methods, such as credit card or PayPal, as well as costs associated with the sale of our prepaid cards.

-

If you developed outside of Roblox, you may have to cover these fees on your own and manage chargebacks and fraud and process refunds.

-
- - - - - - - - - -
-

Roblox covers this on your behalf. These costs relate to Roblox's maintenance of the servers that host all Roblox experiences, as well as platform-wide customer support, user and experience moderation, user acquisition, translation, and local compliance. If you develop outside of Roblox, you may have to pay for hosting, servers, moderation, and customer service on your own. You also have to dedicate time to managing these services; on Roblox you can focus on building your experience.

-
- - - - - - - - - -
-

These costs relate to future-looking investments in the Roblox platform. This includes investments in the people building out new capabilities and tools for our platform which will allow you to be more innovative, build more engaging experiences, help bring in new users, and make creating on Roblox much easier.

-
- - - - - - - - - -
-

Roblox uses some of this to cover ongoing services and operating costs.

-
- -
-
\*28% returned directly to the developers of experiences per dollar spent in experience assumes the full amount of Robux earned are exchanged through the DevEx program, and not spent on the platform. Figures are as of December 31, 2024.
-
- -### Experience monetization - -Once you've developed an experience, you can earn through the following methods: - -#### In-experience purchases - -- [Subscriptions](./monetization/subscriptions.md) within experiences offer you the ability to earn from your experience through subscription-based offerings and business models. -- [Developer Products](../production/monetization/developer-products.md), which are items or abilities that a user can purchase, such as in-experience currency. -- [Passes](../production/monetization/passes.md) for access to special privileges within an experience, such as entry to a restricted area or a permanent power‑up. When you sell your own passes in your experience, you earn 70% of the Robux spent. You can also sell passes within your experience that were made by other creators; in this case, you earn a 10% affiliate fee on the Robux spent within your experience, and the original creator of the pass earns 60%. -- [Private servers](../production/monetization/private-servers.md) for users to play with just connections (subscription-based access fees). - -#### Creator Rewards - -Developers earn extra Robux for simply building an engaging experience. Roblox calculates -[payouts](../creator-rewards.md) based on how Active Spenders use your experience on a daily basis, and these payouts happen -automatically as a way to reward engaging experiences. - -#### Immersive ads - -[Immersive ads](../production/monetization/immersive-ads.md) allow developers to place high-quality, native ad units in their experience that will programmatically serve ads. Creators are paid through a revenue share for the video views, impressions, and teleports generated. - -## Create, sell, and resell certain avatar items - -Millions of users already visit the [Marketplace][Link_Marketplace] every day. [Eligible](../marketplace/marketplace-policy.md#creator-requirements) creators can [create](../marketplace/publish-to-marketplace.md) and sell [accessories](../art/accessories/index.md), [clothing](../art/accessories/index.md), and [avatar bodies and heads](../art//characters/index.md) in the Marketplace or in their own or other developers' experiences. We have a breadth of ways users can discover and buy your creations, whether through searching the Marketplace or enabling developers to promote and sell your items. Users can purchase your item within an experience using the [Avatar Inspect Menu](../players/avatar-inspect-menu.md) or in templates such as [UGC Homestore](../resources/templates.md#ugc-homestore). You can also increase the discoverability of your 3D user-generated content within the Marketplace through [sponsored item](../production/promotion/sponsor-items.md) ads. - -After you pay any applicable upload and publishing fees and submit a new asset for approval, the moderation team reviews your asset and, if approved, you'll be able to publish your asset to the Marketplace. - -You receive a commission every time a user purchases your item. When community items are sold in the marketplace, a portion of the sales is split between the item creator and Roblox. If the item is sold within an experience, the revenue share will additionally be split with the owner of the experience who we refer to as an "affiliate." An affiliate earns commission for marketing another user's creations. - -As follows is an overview of our commission structure. To learn more about fees and commissions for the Marketplace, see [Marketplace fees and commissions](../marketplace/marketplace-fees-and-commissions.md). - - - - - -
In-experience purchase
- -- Creator receives **30%** -- Affiliate (experience owner) receives **40%** -- Platform receives **30%** - -
Marketplace purchase
- -- Creator of the virtual item receives **30%** -- Affiliate (Roblox) receives **40%** -- Platform receives **30%** - -These commissions also apply to the original sale of [Limiteds](../marketplace/marketplace-fees-and-commissions.md#limiteds). When setting assets on sale, you can [configure](../marketplace/publish-to-marketplace.md#marketplace-settings) your creation as a Limited item to set an available quantity for that asset. Limiteds require a [per-unit fee](../marketplace/marketplace-fees-and-commissions.md#per-unit-fee) and can be [resold](../marketplace/marketplace-fees-and-commissions.md#reselling) by [Roblox Premium](https://www.roblox.com/premium/membership) members. At this time, only Roblox-created Limiteds are tradeable. - - -Pie chart illustrating the commission structure for 3D clothes and accessories - - - - - - -While anyone can buy a Limited item, only Roblox [Premium][Link_Premium_Membership] subscribers can resell them. Every time a creator's Limited item is resold, the creator benefits from a 10% original creator commission on the value of each resale. - -- Reseller receives **50%** -- Creator receives **10%** -- Affiliate receives **10%** -- Platform receives **30%** - -After purchasing a Limited, there is up to a 30‑day holding period when the item cannot be resold. Robux acquired from trading or selling items that you didn't create are not considered earned and are ineligible for the [Developer Exchange][Link_Developer_Exchange] program. - - -Pie chart illustrating the commission structure for community-created Limiteds - - - - - -## Create and sell Studio plugins and models - -You can create and sell Studio [plugins](../studio/plugins.md) and [models](../parts/models.md) to the creator community through the [Creator Store](../production/creator-store.md). A [plugin](../studio/plugins.md) is an extension that adds additional functionality to Studio, and a [model](../parts/models.md) is a reusable asset type. You can offer both to other creators on the [Creator Store](../production/creator-store.md) for free, or you can sell them for **United States Dollars** (the minimum price is \$4.99 for plugins and \$2.99 for models). Roblox offers a market-leading revenue share for these sales, as only taxes and payment processing fees are deducted. For more information, see [Sell on the Creator Store](../production/sell-on-creator-store.md). - -## Join the Creator Affiliate Program - - -To join the Creator Affiliate Program, you must be 13+ and reside in a country supported by the [Developer Exchange][Link_Developer_Exchange] program. - - -The [Creator Affiliate Program](https://create.roblox.com/affiliate) is a pilot program that lets you earn Robux using affiliate links. With it, you can: - -- Generate affiliate share links for any Roblox experience, including other developers' experiences, through the Creator Hub. -- Promote your affiliate links off-platform on social media channels. -- Earn revenue in Robux whenever new users create a Roblox account and purchase Robux using your affiliate link. A new user is someone who doesn't already have an alternative Roblox account. -- Exchange your Robux earnings for real-world money using the Developer Exchange program. - -When a user creates a Roblox account using your affiliate link and then buys Robux, you can receive up to 50% of the value of their Robux purchases during their first six months on the platform, for a maximum revenue of \$100 USD per new user. For example, if a new user joins Roblox and purchases \$100 USD of Robux, you can earn up to \$50 USD, minus certain fees like taxes and VAT. - -You receive your revenue in Robux, which you can exchange for real-world money through Developer Exchange. - -## Convert earned Robux to cash - -Once you earn Robux, you can continue to use it in our platform to buy developer assets, plugins, [advertising](../production/promotion/advertise-on-roblox.md), in-experience items, or virtual items. In addition, when you have at least 30,000 earned Robux in your account, you may be eligible to exchange your earned Robux for real currency using the [Developer Exchange][Link_Developer_Exchange] program. - -To learn more about strategies to earn Robux, see [Monetization](../production/monetization/index.md). - -## Frequently asked questions - - - -Who can earn on Roblox? - - -Any user can earn Robux on Roblox, however you must be a creator who is 13 years of age or older and have the requisite minimum amount of earned Robux in your account in order to convert your earned Robux to real currency through the [Developer Exchange][Link_Developer_Exchange] program. You must also meet certain other requirements such as having a verified email address and being in good standing within the community. Also, only those over the age of 18 may purchase Robux or engage in other real-money transactions on the platform per Roblox's [Terms of Use](https://www.roblox.com/info/terms). - -Users, unlike creators, cannot convert in-experience virtual currency for real-world currency on Roblox. Creators do have the ability to convert (if, for example, they earn Robux via an experience they create) but Robux acquired from trading or reselling virtual items is not eligible to be converted. - - - - - -Other platforms advertise giving a higher share to their creators, why is that? - - -Every platform reports their creator share a bit differently. When you compare creator shares across platforms, you should consider: - -- **The breadth of business models** — While some platforms may have a high creator share, the opportunities to earn and the ways to earn may be limited, making the absolute earning potential on the platform smaller. -- **The platform may also be competing for engagement and payouts** — Some platforms are actually also creators on their platform and therefore compete with their creators for engagement. Roblox does not. -- **Other platform benefits** — Creator shares can be high but consider other benefits that the platform provides such as costs that they are paying on your behalf, the scale and diversity of their audience, the devices/platforms they are available on, the types of creators that are earning, and more. -- **The total size of the payout pool** — Understand the total size of the platform's payout pool. Other platforms may invest less in growing their user base and making their products accessible in new markets across the globe. This may allow them to pay developers a higher share, but at the expense of slower growth and less total earnings being put in creators' pockets. - - - -[Link_Developer_Exchange]: https://create.roblox.com/dashboard/devex -[Link_Premium_Membership]: https://www.roblox.com/premium/membership -[Link_Marketplace]: https://www.roblox.com/catalog?Category=1&salesTypeFilter=1 diff --git a/.cursor/rules/roblox/en-us/production/monetization/immersive-ads.md b/.cursor/rules/roblox/en-us/production/monetization/immersive-ads.md index bf18358c..c1b4eb12 100644 --- a/.cursor/rules/roblox/en-us/production/monetization/immersive-ads.md +++ b/.cursor/rules/roblox/en-us/production/monetization/immersive-ads.md @@ -78,27 +78,28 @@ Roblox pays out earnings on the 25th of the following month from when you insert ## Publisher eligibility -While anyone can insert ad units into their experiences, Roblox only serves ads into ad units if the experience or group owner and experience is eligible to serve ads. Eligibility does not guarantee Roblox will serve ads in your experience and nothing obligates Roblox to serve ads into your experience. +While anyone can insert ad units into their experiences, Roblox only serves ads into ad units if the experience is eligible to serve ads. Eligibility does not guarantee Roblox will serve ads in your experience, and nothing obligates Roblox to serve ads into your experience. -Experiences must meet the following criteria to be eligible to be an ad publisher: +For your experience to be eligible to serve ads, your account and experience must meet the following criteria: -- Your account must have **2-factor authentication (2FA)** enabled. - - For group-owned experiences, this requirement applies to the owner of the group. - - If a publisher disables 2FA, they will lose eligibility. +- Your experience must be **public**. - You must be **13+ years of age**. - For group-owned experiences, this requirement applies to the owner of the group. -- Your account must be [ID-verified](../publishing/account-verification.md). +- Your account must be [ID-verified](../publishing/account-verification.md) and have **2-factor authentication (2FA)** enabled. - For group-owned experiences, this requirement applies to the owner of the group. - - This requirement is persistent. If a publisher disables identity verification, they lose eligibility. -- Your experience must be **Public**. If a publisher makes the experience Private, the experience loses eligibility. -- You must complete the **Maturity & Compliance Questionnaire** for your experience. + - This requirement is persistent. If you disable identity verification or 2FA, your experience loses eligibility. +- You must complete the **Maturity and Compliance Questionnaire** for your experience. + - The questionnaire must be reviewed and approved by Roblox before the experience is eligible to serve ads. If the review status is **Pending**, you can request a re-review from the experience's advertising eligibility page under **Monetization** ⟩ **Ads** ⟩ **Eligiblity**. You'll receive a response within 24 hours. - Your experience must maintain **2,000 unique visitors per month**. - - This is calculated and updated monthly, based on visitor data. - - Bots can't be included in visitor counts. -- You must comply with the [Roblox Terms of Use](https://en.help.roblox.com/hc/en-us/articles/115004647846-Roblox-Terms-of-Use), the [Community Standards](https://en.help.roblox.com/hc/en-us/articles/203313410-Roblox-Community-Standards), and the [Advertising Standards](https://en.help.roblox.com/hc/en-us/articles/13722260778260-Advertising-Standards#publisher-integrity). + - This is calculated and updated monthly, based on visitor data. Bots are not included in visitor counts. +- You must comply with the [Roblox terms of use](https://en.help.roblox.com/hc/en-us/articles/115004647846-Roblox-Terms-of-Use), [community standards](https://en.help.roblox.com/hc/en-us/articles/203313410-Roblox-Community-Standards), and [publisher integrity requirements](https://en.help.roblox.com/hc/en-us/articles/13722260778260-Advertising-Standards#publisher-integrity). + + + Resubmitting your **Maturity and Compliance Questionnaire** will temporarily remove your ads eligibility while the questionnaire is under review. To avoid repeated interruptions, only resubmit the questionnaire when you publish an experience update that changes your previous responses. + - Publishers who don't comply with these requirements might lose their eligibility temporarily or permanently. In some cases, they might lose some or all ad payouts and might have their content or account suspended. + If you don't comply with these requirements, you might lose your publisher eligibility temporarily or permanently. In some cases, you might lose some or all ad payouts and might have your content or account suspended. ## User eligibility diff --git a/.cursor/rules/roblox/en-us/production/promotion/regional-content-availability.md b/.cursor/rules/roblox/en-us/production/promotion/regional-content-availability.md new file mode 100644 index 00000000..be3a79a4 --- /dev/null +++ b/.cursor/rules/roblox/en-us/production/promotion/regional-content-availability.md @@ -0,0 +1,23 @@ +--- +title: Regional Content Availability +description: Explains why experiences or assets may not be available in every country. +--- + +## Our approach to global compliance + +As a global platform, we are committed to respecting the local requirements of the countries in which we operate. This commitment is a core part of our long-term strategy to maintain a presence in nearly every country, allowing creators to reach the widest possible audience. + +## Why is my content unavailable in certain regions? + +Content availability can change based on regional requirements. At times, we may receive requests to restrict certain types of content in a specific country, as well as requests to restrict a particular experience or avatar item in a specific country. + +## What does this mean for my content? + +When content is restricted in a specific country, the following changes occur: + +- **Experiences**: Impacted experiences will be neither discoverable nor playable by users in that country. +- **Avatar items**: Impacted avatar items will be undiscoverable and taken off-sale for users in that country. + +Your content’s status in other countries remains unchanged, and your global engagement and analytics in other regions will continue as usual. + +Our approach to content availability is not static; it is subject to the ongoing evolution of requirements worldwide. Content that was once restricted may become available, and new requirements may be introduced. diff --git a/.cursor/rules/roblox/en-us/production/promotion/rewarded-video-ads.md b/.cursor/rules/roblox/en-us/production/promotion/rewarded-video-ads.md index fe27a316..02a3eb88 100644 --- a/.cursor/rules/roblox/en-us/production/promotion/rewarded-video-ads.md +++ b/.cursor/rules/roblox/en-us/production/promotion/rewarded-video-ads.md @@ -334,7 +334,7 @@ To prevent abuse of the rewarded ad system and to provide the best user experien - + @@ -347,7 +347,13 @@ To prevent abuse of the rewarded ad system and to provide the best user experien
**Experiences**Experiences must:

  • Be public and unrestricted.
  • Offer no free-form user creation or AI interaction.
  • Have a complete [Maturity & Compliance questionnaire](../promotion/content-maturity.md).
  • Have an average of at least **2 thousand unique visitors per month**.
  • Comply with the Roblox [terms of use](https://en.help.roblox.com/hc/en-us/articles/115004647846-Roblox-Terms-of-Use), [community standards](https://en.help.roblox.com/hc/en-us/articles/203313410-Roblox-Community-Standards), and [publisher integrity requirements](https://en.help.roblox.com/hc/en-us/articles/13722260778260-Advertising-Standards#publisher-integrity).
Experiences must:

  • Be public and unrestricted.
  • Offer no free-form user creation.
  • Have a complete **Maturity and Compliance Questionnaire** that has been reviewed and approved by Roblox. If the review status is **Pending**, you can request a re-review from the experience's advertising eligibility page under **Monetization** ⟩ **Ads** ⟩ **Eligiblity**. You'll receive a response within 24 hours.
  • Have an average of at least **2,000 unique visitors per month**.
  • Comply with the [Roblox terms of use](https://en.help.roblox.com/hc/en-us/articles/115004647846-Roblox-Terms-of-Use), [community standards](https://en.help.roblox.com/hc/en-us/articles/203313410-Roblox-Community-Standards), and [publisher integrity requirements](https://en.help.roblox.com/hc/en-us/articles/13722260778260-Advertising-Standards#publisher-integrity).
**Publishers**
-Any violation of the eligilibity requirements can result in account suspension, removal of content, revocation of ad payouts, or loss of eligibility to earn from ads. + + Resubmitting your **Maturity and Compliance Questionnaire** will temporarily remove your ads eligibility while the questionnaire is under review. To avoid repeated interruptions, only resubmit the questionnaire when you publish an experience update that changes your previous responses. + + + + Any violation of the eligilibity requirements can result in account suspension, removal of content, revocation of ad payouts, or loss of eligibility to earn from ads. + ## Best practices diff --git a/.cursor/rules/roblox/en-us/production/publishing/adaptive-design.md b/.cursor/rules/roblox/en-us/production/publishing/adaptive-design.md index 783afa1f..849eb30c 100644 --- a/.cursor/rules/roblox/en-us/production/publishing/adaptive-design.md +++ b/.cursor/rules/roblox/en-us/production/publishing/adaptive-design.md @@ -29,13 +29,19 @@ Ensure your experience feels natural on various input devices, including gamepad +
+
 +
+
 +
+
 @@ -48,10 +54,14 @@ The layout of your experience UI should adapt to different screen sizes and orie +
+
 +
+
 @@ -64,13 +74,19 @@ This is about the **size** of your UI elements, not just their position. Your UI +
+
 - +
+ +
 +
+
 diff --git a/.cursor/rules/roblox/en-us/reference/cloud/cloud.docs.json b/.cursor/rules/roblox/en-us/reference/cloud/cloud.docs.json index bd18f2ae..a6c97200 100644 --- a/.cursor/rules/roblox/en-us/reference/cloud/cloud.docs.json +++ b/.cursor/rules/roblox/en-us/reference/cloud/cloud.docs.json @@ -308,6 +308,344 @@ ] } }, + "/cloud/v2/groups/{group_id}/forum-categories": { + "get": { + "tags": ["Cloud"], + "summary": "List Group Forum Categories", + "description": "Lists forum categories in the group.", + "operationId": "Cloud_ListGroupForumCategories", + "parameters": [ + { + "name": "group_id", + "in": "path", + "description": "The group ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "maxPageSize", + "in": "query", + "description": "The maximum number of group forum categories to return. The service might\nreturn fewer than this value. If unspecified, at most 10 group forum\ncategories are returned. The maximum value is 10 and higher values are set\nto 10.", + "schema": { + "type": "integer", + "format": "int32" + }, + "examples": { + "maxPageSize": { + "description": "An integer between 1 and 10, inclusive", + "value": 10 + } + } + }, + { + "name": "pageToken", + "in": "query", + "description": "A page token, received from a previous call, to retrieve a subsequent page.\n\nWhen paginating, all other parameters provided to the subsequent call must\nmatch the call that provided the page token.", + "schema": { + "type": "string" + } + }, + { + "name": "filter", + "in": "query", + "description": "This field may be set in order to filter the resources returned.\n\nFiltering conforms to Common Expression Language (CEL). Only the boolean\nfield `archived` and `==` operator are supported. If `archived=false`,\narchived categories are not returned. The default value is `false`.\nExample: `\"filter=archived==true\"`", + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "OK", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ListGroupForumCategoriesResponse" + } + } + } + } + }, + "x-visibility": "BETA", + "x-roblox-engine-usability": { + "apiKeyWithHttpService": true + }, + "x-roblox-scopes": [ + { + "name": "group-forum:read" + } + ], + "x-roblox-docs": { + "category": "Users and groups", + "methodProperties": { + "scopes": ["group-forum:read"] + }, + "resource": { + "$ref": "#/components/schemas/GroupForumCategory", + "name": "GroupForumCategory" + } + }, + "x-roblox-stability": "BETA", + "x-roblox-rate-limits": { + "perApiKeyOwner": { + "period": "MINUTE", + "maxInPeriod": 150 + }, + "perOauth2Authorization": { + "period": "MINUTE", + "maxInPeriod": 30 + } + }, + "security": [ + { + "roblox-api-key": [] + }, + { + "roblox-oauth2": [] + } + ] + } + }, + "/cloud/v2/groups/{group_id}/forum-categories/{forum_category_id}/posts": { + "get": { + "tags": ["Cloud"], + "summary": "List Group Forum Posts", + "description": "Lists forum posts in the group's forum category.", + "operationId": "Cloud_ListGroupForumPosts", + "parameters": [ + { + "name": "group_id", + "in": "path", + "description": "The group ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "forum_category_id", + "in": "path", + "description": "The forum-category ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "maxPageSize", + "in": "query", + "description": "The maximum number of group forum posts to return. The service might return\nfewer than this value. If unspecified, at most 10 group forum posts are\nreturned. The maximum value is 100 and higher values are set to 100.", + "schema": { + "type": "integer", + "format": "int32" + }, + "examples": { + "maxPageSize": { + "description": "An integer between 1 and 100, inclusive", + "value": 10 + } + } + }, + { + "name": "pageToken", + "in": "query", + "description": "A page token, received from a previous call, to retrieve a subsequent page.\n\nWhen paginating, all other parameters provided to the subsequent call must\nmatch the call that provided the page token.", + "schema": { + "type": "string" + } + }, + { + "name": "filter", + "in": "query", + "description": "This field may be set in order to filter the resources returned.\n\nFiltering conforms to Common Expression Language (CEL). Only the query\n`filter=pinned==true` is supported, where only pinned posts are returned.\n`filter=pinned==false`is not supported.", + "schema": { + "type": "string" + } + }, + { + "name": "view", + "in": "query", + "description": "The view in which to retrieve the group forum post.\n\nSupports FULL and FULL_WITH_FIRST_COMMENT.\n\nDefaults to FULL.\n\nPossible values:\n\n | Value | Description |\n | --- | --- |\n | VIEW_UNSPECIFIED | The group forum post view is not specified; the default will be used. |\n | FULL | Includes all fields but does not dereference `first_comment`. Only the `path` field is populated. |\n | FULL_WITH_FIRST_COMMENT | Includes all fields and also fully populates `first_comment`. The default view. |", + "schema": { + "example": "VIEW_UNSPECIFIED", + "enum": ["VIEW_UNSPECIFIED", "FULL", "FULL_WITH_FIRST_COMMENT"], + "type": "string", + "format": "enum" + } + } + ], + "responses": { + "200": { + "description": "OK", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ListGroupForumPostsResponse" + } + } + } + } + }, + "x-visibility": "BETA", + "x-roblox-engine-usability": { + "apiKeyWithHttpService": true + }, + "x-roblox-scopes": [ + { + "name": "group-forum:read" + } + ], + "x-roblox-docs": { + "category": "Users and groups", + "methodProperties": { + "scopes": ["group-forum:read"] + }, + "resource": { + "$ref": "#/components/schemas/GroupForumPost", + "name": "GroupForumPost" + } + }, + "x-roblox-stability": "BETA", + "x-roblox-rate-limits": { + "perApiKeyOwner": { + "period": "MINUTE", + "maxInPeriod": 150 + }, + "perOauth2Authorization": { + "period": "MINUTE", + "maxInPeriod": 30 + } + }, + "security": [ + { + "roblox-api-key": [] + }, + { + "roblox-oauth2": [] + } + ] + } + }, + "/cloud/v2/groups/{group_id}/forum-categories/{forum_category_id}/posts/{post_id}/comments": { + "get": { + "tags": ["Cloud"], + "summary": "List Group Forum Comments", + "description": "Lists forum comments on a group's forum post.", + "operationId": "Cloud_ListGroupForumComments", + "parameters": [ + { + "name": "group_id", + "in": "path", + "description": "The group ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "forum_category_id", + "in": "path", + "description": "The forum-category ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "post_id", + "in": "path", + "description": "The post ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "maxPageSize", + "in": "query", + "description": "The maximum number of group forum comments to return. The service might\nreturn fewer than this value. If unspecified, at most 10 group forum\ncomments are returned. The maximum value is 100 and higher values are set\nto 100.", + "schema": { + "type": "integer", + "format": "int32" + }, + "examples": { + "maxPageSize": { + "description": "An integer between 1 and 100, inclusive", + "value": 10 + } + } + }, + { + "name": "pageToken", + "in": "query", + "description": "A page token, received from a previous call, to retrieve a subsequent page.\n\nWhen paginating, all other parameters provided to the subsequent call must\nmatch the call that provided the page token.", + "schema": { + "type": "string" + } + }, + { + "name": "filter", + "in": "query", + "description": "This field may be set in order to filter the resources returned.\n\nFiltering conforms to Common Expression Language (CEL). Only the string\nfield `repliesToCommentId` and the `==` operator are supported.\nIf `repliesToCommentId` is specified, only comments that respond to the\ncomment are returned. The default value is `''`. If\n`repliesToCommentId=''`, only comments responding directly to the post are\nreturned. Example:\n`\"filter=repliesToCommentId==01234567-89ab-cdef-0123-456789abcdef\"`", + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "OK", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ListGroupForumCommentsResponse" + } + } + } + } + }, + "x-visibility": "BETA", + "x-roblox-engine-usability": { + "apiKeyWithHttpService": true + }, + "x-roblox-scopes": [ + { + "name": "group-forum:read" + } + ], + "x-roblox-docs": { + "category": "Users and groups", + "methodProperties": { + "scopes": ["group-forum:read"] + }, + "resource": { + "$ref": "#/components/schemas/GroupForumComment", + "name": "GroupForumComment" + } + }, + "x-roblox-stability": "BETA", + "x-roblox-rate-limits": { + "perApiKeyOwner": { + "period": "MINUTE", + "maxInPeriod": 150 + }, + "perOauth2Authorization": { + "period": "MINUTE", + "maxInPeriod": 30 + } + }, + "security": [ + { + "roblox-api-key": [] + }, + { + "roblox-oauth2": [] + } + ] + } + }, "/cloud/v2/groups/{group_id}/join-requests": { "get": { "tags": ["Cloud"], @@ -6943,6 +7281,48 @@ }, "x-resource": true }, + "ContentMessage": { + "type": "object", + "properties": { + "createTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the message was created.", + "format": "date-time" + }, + "updateTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the message was last updated.", + "format": "date-time" + }, + "plainText": { + "type": "string", + "description": "The plain text content of the message." + }, + "author": { + "readOnly": true, + "example": "users/156", + "type": "string", + "description": "The resource path of the user who created the message." + }, + "messageReaction": { + "readOnly": true, + "type": "array", + "items": { + "$ref": "#/components/schemas/MessageReaction" + }, + "description": "The reactions to the message, if this feature is enabled for this message." + } + }, + "description": "Represents the content of a message.", + "x-oneOf": { + "Author": ["author"], + "content": ["plainText"] + } + }, "CreatorStoreProduct": { "type": "object", "properties": { @@ -7386,6 +7766,175 @@ "Owner": ["owner"] } }, + "GroupForumCategory": { + "type": "object", + "properties": { + "path": { + "example": "groups/123/forum-categories/01234567-89ab-cdef-0123-456789abcdef", + "type": "string", + "description": "The resource path of the group forum category.\n\nFormat: `groups/{group_id}/forum-categories/{group_forum_category_id}`" + }, + "createTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum category was created.", + "format": "date-time" + }, + "updateTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum category was last updated.", + "format": "date-time" + }, + "groupForumCategoryId": { + "readOnly": true, + "type": "string", + "description": "The id of the group forum category." + }, + "displayName": { + "type": "string", + "description": "The display name of the group forum category." + }, + "creator": { + "readOnly": true, + "example": "users/156", + "type": "string", + "description": "The resource path of the user who created the group forum category." + }, + "archiveTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum category was archived.", + "format": "date-time" + }, + "archiver": { + "readOnly": true, + "example": "users/156", + "type": "string", + "description": "The resource path of the user who archived the group forum category." + } + }, + "description": "Represents a forum category within a group for user discussions.", + "x-aep-resource": { + "patterns": [ + "groups/{group_id}/forum-categories/{group_forum_category_id}" + ], + "plural": "group-forum-categories", + "singular": "group-forum-category" + }, + "x-resource": true + }, + "GroupForumComment": { + "type": "object", + "properties": { + "path": { + "example": "groups/123/forum-categories/01234567-89ab-cdef-0123-456789abcdef/posts/01234567-89ab-cdef-0123-456789abcdef/comments/01234567-89ab-cdef-0123-456789abcdef", + "type": "string", + "description": "The resource path of the group forum comment.\n\nFormat:\n`groups/{group_id}/forum-categories/{group_forum_category_id}/posts/{group_forum_post_id}/comments/{group_forum_comment_id}`" + }, + "groupForumCommentId": { + "readOnly": true, + "type": "string", + "description": "The id of the group forum comment." + }, + "message": { + "$ref": "#/components/schemas/ContentMessage", + "description": "The message content of the group forum comment." + } + }, + "description": "Represents a forum comment on a group's forum post.", + "x-aep-resource": { + "patterns": [ + "groups/{group_id}/forum-categories/{group_forum_category_id}/posts/{group_forum_post_id}/comments/{group_forum_comment_id}" + ], + "plural": "group-forum-comments", + "singular": "group-forum-comment" + }, + "x-resource": true + }, + "GroupForumPost": { + "type": "object", + "properties": { + "path": { + "example": "groups/123/forum-categories/01234567-89ab-cdef-0123-456789abcdef/posts/01234567-89ab-cdef-0123-456789abcdef", + "type": "string", + "description": "The resource path of the group forum post.\n\nFormat:\n`groups/{group_id}/forum-categories/{group_forum_category_id}/posts/{group_forum_post_id}`" + }, + "createTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum post was created.", + "format": "date-time" + }, + "updateTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum post was last updated.", + "format": "date-time" + }, + "pinned": { + "example": true, + "type": "boolean", + "description": "Whether the group forum post is pinned." + }, + "locked": { + "example": true, + "type": "boolean", + "description": "Whether the group forum post is locked." + }, + "groupForumPostId": { + "readOnly": true, + "type": "string", + "description": "The id of the group forum post." + }, + "title": { + "type": "string", + "description": "The title of the group forum post." + }, + "firstComment": { + "$ref": "#/components/schemas/GroupForumComment", + "description": "The resource path of the group forum post's first comment, which holds the\npost content.\n\nBy default, the `first_comment` field is fully populated. This **will not**\ndereference resource references within the first comment, such as\n`GroupForumComment.author`; only the `path` subfield of these fields will\nbe populated. However, if only the `path` field of the first comment is\nneeded, you can specify the FULL view." + }, + "author": { + "readOnly": true, + "example": "users/156", + "type": "string", + "description": "The resource path of the user who created the group forum post." + }, + "archiveTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum post was archived.", + "format": "date-time" + }, + "archiver": { + "readOnly": true, + "example": "users/156", + "type": "string", + "description": "The resource path of the user who archived the group forum post." + }, + "commentCount": { + "type": "integer", + "description": "The number of comments on the group forum post. This includes the first\ncomment (post content).", + "format": "int64" + } + }, + "description": "Represents a forum post within a group's forum category.", + "x-aep-resource": { + "patterns": [ + "groups/{group_id}/forum-categories/{group_forum_category_id}/posts/{group_forum_post_id}" + ], + "plural": "group-forum-posts", + "singular": "group-forum-post" + }, + "x-resource": true + }, "GroupJoinRequest": { "type": "object", "properties": { @@ -8069,6 +8618,57 @@ }, "description": "A list of DataStores in the parent collection." }, + "ListGroupForumCategoriesResponse": { + "type": "object", + "properties": { + "groupForumCategories": { + "type": "array", + "items": { + "$ref": "#/components/schemas/GroupForumCategory" + }, + "description": "The GroupForumCategories from the specified Group." + }, + "nextPageToken": { + "type": "string", + "description": "A token that you can send as a `pageToken` parameter to retrieve the next\npage. If this field is omitted, there are no subsequent pages." + } + }, + "description": "A list of GroupForumCategories in the parent collection." + }, + "ListGroupForumCommentsResponse": { + "type": "object", + "properties": { + "groupForumComments": { + "type": "array", + "items": { + "$ref": "#/components/schemas/GroupForumComment" + }, + "description": "The GroupForumComments from the specified GroupForumPost." + }, + "nextPageToken": { + "type": "string", + "description": "A token that you can send as a `pageToken` parameter to retrieve the next\npage. If this field is omitted, there are no subsequent pages." + } + }, + "description": "A list of GroupForumComments in the parent collection." + }, + "ListGroupForumPostsResponse": { + "type": "object", + "properties": { + "groupForumPosts": { + "type": "array", + "items": { + "$ref": "#/components/schemas/GroupForumPost" + }, + "description": "The GroupForumPosts from the specified GroupForumCategory." + }, + "nextPageToken": { + "type": "string", + "description": "A token that you can send as a `pageToken` parameter to retrieve the next\npage. If this field is omitted, there are no subsequent pages." + } + }, + "description": "A list of GroupForumPosts in the parent collection." + }, "ListGroupJoinRequestsResponse": { "type": "object", "properties": { @@ -8600,6 +9200,23 @@ "sortKey": ["stringSortKey", "numericSortKey"] } }, + "MessageReaction": { + "type": "object", + "properties": { + "emoteId": { + "readOnly": true, + "type": "string", + "description": "The id of the emote." + }, + "count": { + "readOnly": true, + "type": "integer", + "description": "The count of reactions for this emote.", + "format": "int64" + } + }, + "description": "Represents the reaction to a message." + }, "Money": { "type": "object", "properties": { @@ -9788,6 +10405,7 @@ "scopes": { "creator-store-product:write": "creator-store-product:write", "creator-store-product:read": "creator-store-product:read", + "group-forum:read": "group-forum:read", "group:read": "group:read", "group:write": "group:write", "universe:write": "universe:write", diff --git a/.cursor/rules/roblox/en-us/reference/cloud/messaging-service/v1.json b/.cursor/rules/roblox/en-us/reference/cloud/messaging-service/v1.json index 1655f910..40c239bf 100644 --- a/.cursor/rules/roblox/en-us/reference/cloud/messaging-service/v1.json +++ b/.cursor/rules/roblox/en-us/reference/cloud/messaging-service/v1.json @@ -18,6 +18,7 @@ "operationId": "CrossServerMessaging_Publish", "x-roblox-cloud-api-operation-name": "Publish a Message", "x-roblox-stability": "BETA", + "summary": "Publish a cross-server message to a universe", "description": "Publish a message to a pre-defined topic of an experience, with the size of the message up to 1,024 characters (1 KB). Requires the **Publish** permission for API keys and the **universe-messaging-service:publish** scope for OAuth 2.0 apps. See [Cross-server messaging](/cloud-services/cross-server-messaging.md#subscribe-users-to-receive-messages) for defining and subscribing users to a topic.", "parameters": [ { diff --git a/.cursor/rules/roblox/en-us/reference/cloud/openapi.json b/.cursor/rules/roblox/en-us/reference/cloud/openapi.json index 1b4becbd..bdfc4dac 100644 --- a/.cursor/rules/roblox/en-us/reference/cloud/openapi.json +++ b/.cursor/rules/roblox/en-us/reference/cloud/openapi.json @@ -316,6 +316,350 @@ } } }, + "/cloud/v2/groups/{group_id}/forum-categories": { + "get": { + "tags": ["Groups"], + "summary": "List Group Forum Categories", + "description": "Lists forum categories in the group.", + "operationId": "Cloud_ListGroupForumCategories", + "parameters": [ + { + "name": "group_id", + "in": "path", + "description": "The group ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "maxPageSize", + "in": "query", + "description": "The maximum number of group forum categories to return. The service might\nreturn fewer than this value. If unspecified, at most 10 group forum\ncategories are returned. The maximum value is 10 and higher values are set\nto 10.", + "schema": { + "type": "integer", + "format": "int32" + }, + "examples": { + "maxPageSize": { + "description": "An integer between 1 and 10, inclusive", + "value": 10 + } + } + }, + { + "name": "pageToken", + "in": "query", + "description": "A page token, received from a previous call, to retrieve a subsequent page.\n\nWhen paginating, all other parameters provided to the subsequent call must\nmatch the call that provided the page token.", + "schema": { + "type": "string" + } + }, + { + "name": "filter", + "in": "query", + "description": "This field may be set in order to filter the resources returned.\n\nFiltering conforms to Common Expression Language (CEL). Only the boolean\nfield `archived` and `==` operator are supported. If `archived=false`,\narchived categories are not returned. The default value is `false`.\nExample: `\"filter=archived==true\"`", + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "OK", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ListGroupForumCategoriesResponse" + } + } + } + } + }, + "x-roblox-engine-usability": { + "apiKeyWithHttpService": false + }, + "x-roblox-scopes": [ + { + "name": "group-forum:read" + } + ], + "x-roblox-docs": { + "category": "Users and groups", + "methodProperties": { + "scopes": ["group-forum:read"] + }, + "resource": { + "$ref": "#/components/schemas/GroupForumCategory", + "name": "GroupForumCategory" + } + }, + "x-roblox-stability": "BETA", + "x-roblox-rate-limits": { + "perApiKeyOwner": { + "period": "MINUTE", + "maxInPeriod": 150 + }, + "perOauth2Authorization": { + "period": "MINUTE", + "maxInPeriod": 30 + } + }, + "security": [ + { + "roblox-api-key": [] + }, + { + "roblox-oauth2": [] + } + ], + "externalDocs": { + "url": "https://create.roblox.com/docs/cloud/reference/features/groups#Cloud_ListGroupForumCategories" + } + } + }, + "/cloud/v2/groups/{group_id}/forum-categories/{forum_category_id}/posts": { + "get": { + "tags": ["Groups"], + "summary": "List Group Forum Posts", + "description": "Lists forum posts in the group's forum category.", + "operationId": "Cloud_ListGroupForumPosts", + "parameters": [ + { + "name": "group_id", + "in": "path", + "description": "The group ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "forum_category_id", + "in": "path", + "description": "The forum-category ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "maxPageSize", + "in": "query", + "description": "The maximum number of group forum posts to return. The service might return\nfewer than this value. If unspecified, at most 10 group forum posts are\nreturned. The maximum value is 100 and higher values are set to 100.", + "schema": { + "type": "integer", + "format": "int32" + }, + "examples": { + "maxPageSize": { + "description": "An integer between 1 and 100, inclusive", + "value": 10 + } + } + }, + { + "name": "pageToken", + "in": "query", + "description": "A page token, received from a previous call, to retrieve a subsequent page.\n\nWhen paginating, all other parameters provided to the subsequent call must\nmatch the call that provided the page token.", + "schema": { + "type": "string" + } + }, + { + "name": "filter", + "in": "query", + "description": "This field may be set in order to filter the resources returned.\n\nFiltering conforms to Common Expression Language (CEL). Only the query\n`filter=pinned==true` is supported, where only pinned posts are returned.\n`filter=pinned==false`is not supported.", + "schema": { + "type": "string" + } + }, + { + "name": "view", + "in": "query", + "description": "The view in which to retrieve the group forum post.\n\nSupports FULL and FULL_WITH_FIRST_COMMENT.\n\nDefaults to FULL.\n\nPossible values:\n\n | Value | Description |\n | --- | --- |\n | VIEW_UNSPECIFIED | The group forum post view is not specified; the default will be used. |\n | FULL | Includes all fields but does not dereference `first_comment`. Only the `path` field is populated. |\n | FULL_WITH_FIRST_COMMENT | Includes all fields and also fully populates `first_comment`. The default view. |", + "schema": { + "example": "VIEW_UNSPECIFIED", + "enum": ["VIEW_UNSPECIFIED", "FULL", "FULL_WITH_FIRST_COMMENT"], + "type": "string", + "format": "enum" + } + } + ], + "responses": { + "200": { + "description": "OK", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ListGroupForumPostsResponse" + } + } + } + } + }, + "x-roblox-engine-usability": { + "apiKeyWithHttpService": false + }, + "x-roblox-scopes": [ + { + "name": "group-forum:read" + } + ], + "x-roblox-docs": { + "category": "Users and groups", + "methodProperties": { + "scopes": ["group-forum:read"] + }, + "resource": { + "$ref": "#/components/schemas/GroupForumPost", + "name": "GroupForumPost" + } + }, + "x-roblox-stability": "BETA", + "x-roblox-rate-limits": { + "perApiKeyOwner": { + "period": "MINUTE", + "maxInPeriod": 150 + }, + "perOauth2Authorization": { + "period": "MINUTE", + "maxInPeriod": 30 + } + }, + "security": [ + { + "roblox-api-key": [] + }, + { + "roblox-oauth2": [] + } + ], + "externalDocs": { + "url": "https://create.roblox.com/docs/cloud/reference/features/groups#Cloud_ListGroupForumPosts" + } + } + }, + "/cloud/v2/groups/{group_id}/forum-categories/{forum_category_id}/posts/{post_id}/comments": { + "get": { + "tags": ["Groups"], + "summary": "List Group Forum Comments", + "description": "Lists forum comments on a group's forum post.", + "operationId": "Cloud_ListGroupForumComments", + "parameters": [ + { + "name": "group_id", + "in": "path", + "description": "The group ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "forum_category_id", + "in": "path", + "description": "The forum-category ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "post_id", + "in": "path", + "description": "The post ID.", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "maxPageSize", + "in": "query", + "description": "The maximum number of group forum comments to return. The service might\nreturn fewer than this value. If unspecified, at most 10 group forum\ncomments are returned. The maximum value is 100 and higher values are set\nto 100.", + "schema": { + "type": "integer", + "format": "int32" + }, + "examples": { + "maxPageSize": { + "description": "An integer between 1 and 100, inclusive", + "value": 10 + } + } + }, + { + "name": "pageToken", + "in": "query", + "description": "A page token, received from a previous call, to retrieve a subsequent page.\n\nWhen paginating, all other parameters provided to the subsequent call must\nmatch the call that provided the page token.", + "schema": { + "type": "string" + } + }, + { + "name": "filter", + "in": "query", + "description": "This field may be set in order to filter the resources returned.\n\nFiltering conforms to Common Expression Language (CEL). Only the string\nfield `repliesToCommentId` and the `==` operator are supported.\nIf `repliesToCommentId` is specified, only comments that respond to the\ncomment are returned. The default value is `''`. If\n`repliesToCommentId=''`, only comments responding directly to the post are\nreturned. Example:\n`\"filter=repliesToCommentId==01234567-89ab-cdef-0123-456789abcdef\"`", + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "OK", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ListGroupForumCommentsResponse" + } + } + } + } + }, + "x-roblox-engine-usability": { + "apiKeyWithHttpService": false + }, + "x-roblox-scopes": [ + { + "name": "group-forum:read" + } + ], + "x-roblox-docs": { + "category": "Users and groups", + "methodProperties": { + "scopes": ["group-forum:read"] + }, + "resource": { + "$ref": "#/components/schemas/GroupForumComment", + "name": "GroupForumComment" + } + }, + "x-roblox-stability": "BETA", + "x-roblox-rate-limits": { + "perApiKeyOwner": { + "period": "MINUTE", + "maxInPeriod": 150 + }, + "perOauth2Authorization": { + "period": "MINUTE", + "maxInPeriod": 30 + } + }, + "security": [ + { + "roblox-api-key": [] + }, + { + "roblox-oauth2": [] + } + ], + "externalDocs": { + "url": "https://create.roblox.com/docs/cloud/reference/features/groups#Cloud_ListGroupForumComments" + } + } + }, "/cloud/v2/groups/{group_id}/join-requests": { "get": { "tags": ["Groups"], @@ -9969,6 +10313,7 @@ "operationId": "CrossServerMessaging_Publish", "x-roblox-cloud-api-operation-name": "Publish a Message", "x-roblox-stability": "BETA", + "summary": "Publish a cross-server message to a universe", "description": "Publish a message to a pre-defined topic of an experience, with the size of the message up to 1,024 characters (1 KB). Requires the **Publish** permission for API keys and the **universe-messaging-service:publish** scope for OAuth 2.0 apps. See [Cross-server messaging](/cloud-services/cross-server-messaging.md#subscribe-users-to-receive-messages) for defining and subscribing users to a topic.", "parameters": [ { @@ -15352,6 +15697,11 @@ "roblox-oauth2": [] } ], + "x-roblox-scopes": [ + { + "name": "legacy-asset:manage" + } + ], "externalDocs": { "url": "https://create.roblox.com/docs/cloud/reference/features/assets#get_asset_delivery_api_v1_assetId__assetId_" } @@ -15539,6 +15889,11 @@ "roblox-oauth2": [] } ], + "x-roblox-scopes": [ + { + "name": "legacy-asset:manage" + } + ], "externalDocs": { "url": "https://create.roblox.com/docs/cloud/reference/features/assets#get_asset_delivery_api_v1_assetId__assetId__version__versionNumber_" } @@ -18861,6 +19216,64 @@ } } }, + "/v1/passkey/finish-ar-preauth-registration": { + "post": { + "summary": "Finishes account recovery pre-auth passkey registration by validating the recovery session,\r\ndeactivating the user's password, and completing passkey registration.", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Roblox.Authentication.Api.Models.Request.FinishARPreAuthPasskeyRegistrationRequest" + } + }, + "text/json": { + "schema": { + "$ref": "#/components/schemas/Roblox.Authentication.Api.Models.Request.FinishARPreAuthPasskeyRegistrationRequest" + } + } + }, + "description": "The request body containing recovery session and passkey details.", + "required": true + }, + "responses": { + "200": { + "description": "OK", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/Roblox.Web.WebAPI.ApiEmptyResponseModel" + } + }, + "text/json": { + "schema": { + "$ref": "#/components/schemas/Roblox.Web.WebAPI.ApiEmptyResponseModel" + } + } + } + }, + "400": { + "description": "0: An unknown error occurred with the request." + }, + "401": { + "description": "0: An unknown error occurred with the request." + }, + "403": { + "description": "0: Token Validation Failed\r\n1: Reached limit of pass keys registered." + }, + "500": { + "description": "0: An unknown error occurred with the request." + }, + "503": { + "description": "2: Feature disabled." + } + }, + "servers": [ + { + "url": "https://auth.roblox.com" + } + ] + } + }, "/v1/passkey/finish-preauth-registration": { "post": { "tags": ["Accounts"], @@ -21770,25 +22183,6 @@ "type": "integer", "format": "int64" } - }, - { - "in": "header", - "name": "RequestUserIdIsInGameServer", - "description": "Whether the user with the request userId is present in the game server.", - "required": true, - "schema": { - "type": "boolean" - } - }, - { - "in": "header", - "name": "Roblox-Place-Id", - "description": "Place identifier of the requesting place.", - "required": true, - "schema": { - "type": "integer", - "format": "int64" - } } ], "responses": { @@ -21842,25 +22236,6 @@ "type": "integer" } } - }, - { - "in": "header", - "name": "RequestUserIdIsInGameServer", - "description": "Whether the user with the request userId is present in the game server.", - "required": true, - "schema": { - "type": "boolean" - } - }, - { - "in": "header", - "name": "Roblox-Place-Id", - "description": "Place identifier of the requesting place.", - "required": true, - "schema": { - "type": "integer", - "format": "int64" - } } ], "responses": { @@ -28601,133 +28976,6 @@ } } }, - "/v1/my/trusted-friends/{userId}/status": { - "get": { - "tags": ["Connections"], - "summary": "Gets trusted friend status of a specified user against the requesting user.", - "parameters": [ - { - "in": "path", - "name": "userId", - "description": "The user to check the trusted friend status for.", - "required": true, - "schema": { - "type": "integer", - "format": "int64" - } - } - ], - "responses": { - "200": { - "description": "OK", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/Roblox.Friends.Api.TrustedFriendStatusResponse" - } - }, - "text/json": { - "schema": { - "$ref": "#/components/schemas/Roblox.Friends.Api.TrustedFriendStatusResponse" - } - } - } - }, - "400": { - "description": "1: The target user is invalid or does not exist." - }, - "401": { - "description": "0: Authorization has been denied for this request." - } - }, - "servers": [ - { - "url": "https://friends.roblox.com" - } - ], - "x-roblox-engine-usability": { - "apiKeyWithHttpService": false - }, - "security": [ - { - "roblox-legacy-cookie": [] - } - ], - "externalDocs": { - "url": "https://create.roblox.com/docs/cloud/reference/features/friends#friends_get_v1_my_trusted_friends__userId__status" - } - } - }, - "/v1/user/{userId}/multiget-are-trusted-friends": { - "get": { - "tags": ["Connections"], - "summary": "Check if the requesting user is trusted friends with the specified users.", - "parameters": [ - { - "in": "path", - "name": "userId", - "description": "The requesting userId.", - "required": true, - "schema": { - "type": "integer", - "format": "int64" - } - }, - { - "in": "query", - "name": "userIds", - "description": "The user ids to check against with the requesting user.", - "required": true, - "style": "form", - "explode": false, - "schema": { - "type": "array", - "items": { - "format": "int64", - "type": "integer" - } - } - } - ], - "responses": { - "200": { - "description": "OK", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/Roblox.Friends.Api.MultigetAreTrustedFriendsResponse" - } - }, - "text/json": { - "schema": { - "$ref": "#/components/schemas/Roblox.Friends.Api.MultigetAreTrustedFriendsResponse" - } - } - } - }, - "400": { - "description": "1: The target user is invalid or does not exist." - } - }, - "servers": [ - { - "url": "https://friends.roblox.com" - } - ], - "x-roblox-engine-usability": { - "apiKeyWithHttpService": false - }, - "security": [ - {}, - { - "roblox-legacy-cookie": [] - } - ], - "externalDocs": { - "url": "https://create.roblox.com/docs/cloud/reference/features/friends#friends_get_v1_user__userId__multiget_are_trusted_friends" - } - } - }, "/v1/user/friend-requests/count": { "get": { "tags": ["Connections"], @@ -39228,6 +39476,9 @@ }, "400": { "description": "8: The universe IDs specified are invalid.\r\n9: Too many universe IDs were requested." + }, + "429": { + "description": "4: Too many requests have been made." } }, "servers": [ @@ -39716,145 +39967,6 @@ ] } }, - "/v1/games/{universeId}/votes": { - "get": { - "tags": ["Interactions", "Universes"], - "summary": "Get the game vote status", - "parameters": [ - { - "in": "path", - "name": "universeId", - "description": "The id of the universe we get vote status from.", - "required": true, - "schema": { - "type": "integer", - "format": "int64" - } - } - ], - "responses": { - "200": { - "description": "OK", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/Roblox.Games.Api.Models.Response.GameVoteResponse" - } - }, - "text/json": { - "schema": { - "$ref": "#/components/schemas/Roblox.Games.Api.Models.Response.GameVoteResponse" - } - } - } - }, - "400": { - "description": "2: The universe's root place is invalid.\r\n3: The asset is not voteable." - }, - "404": { - "description": "1: The requested universe does not exist." - }, - "429": { - "description": "10: Internal service busy. Please try again later." - }, - "500": { - "description": "0: An unknown error occurred." - } - }, - "deprecated": true, - "servers": [ - { - "url": "https://games.roblox.com" - } - ], - "x-roblox-engine-usability": { - "apiKeyWithHttpService": false - }, - "security": [ - {}, - { - "roblox-legacy-cookie": [] - } - ], - "description": "Use https://games.roblox.com/docs#!/Votes/get_v1_games_votes instead", - "externalDocs": { - "url": "https://create.roblox.com/docs/cloud/reference/features/interactions#games_get_v1_games__universeId__votes" - }, - "x-roblox-recommended-alternatives": [ - { - "url": "https://games.roblox.com/v1/games/votes", - "httpMethod": "GET", - "documentationUrl": "https://create.roblox.com/docs/cloud/reference/features/interactions#games_get_v1_games_votes" - } - ] - } - }, - "/v1/games/{universeId}/votes/user": { - "get": { - "tags": ["Interactions", "Universes"], - "summary": "Get the user's vote status for a game", - "parameters": [ - { - "in": "path", - "name": "universeId", - "description": "The id of the universe we get user vote status from.", - "required": true, - "schema": { - "type": "integer", - "format": "int64" - } - } - ], - "responses": { - "200": { - "description": "OK", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/Roblox.Games.Api.Models.Response.UserGameVoteResponse" - } - }, - "text/json": { - "schema": { - "$ref": "#/components/schemas/Roblox.Games.Api.Models.Response.UserGameVoteResponse" - } - } - } - }, - "400": { - "description": "2: The universe's root place is invalid.\r\n3: The asset is not voteable." - }, - "401": { - "description": "0: Authorization has been denied for this request." - }, - "404": { - "description": "1: The requested universe does not exist." - }, - "429": { - "description": "10: Internal service busy. Please try again later." - }, - "500": { - "description": "0: An unknown error occurred." - } - }, - "servers": [ - { - "url": "https://games.roblox.com" - } - ], - "x-roblox-engine-usability": { - "apiKeyWithHttpService": false - }, - "security": [ - { - "roblox-legacy-cookie": [] - } - ], - "externalDocs": { - "url": "https://create.roblox.com/docs/cloud/reference/features/interactions#games_get_v1_games__universeId__votes_user" - } - } - }, "/v1/games/games-product-info": { "get": { "tags": ["Universes"], @@ -40131,72 +40243,6 @@ } } }, - "/v1/games/votes": { - "get": { - "tags": ["Interactions", "Universes"], - "summary": "Gets a list of universe vote status", - "parameters": [ - { - "in": "query", - "name": "universeIds", - "description": "A list of universe Ids. Cannot exceed a maximum of 50 IDs.", - "required": true, - "style": "form", - "explode": false, - "schema": { - "type": "array", - "items": { - "format": "int64", - "type": "integer" - } - } - } - ], - "responses": { - "200": { - "description": "OK", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/Roblox.Web.WebAPI.Models.ApiArrayResponse_Roblox.Games.Api.Models.Response.GameVoteResponse_" - } - }, - "text/json": { - "schema": { - "$ref": "#/components/schemas/Roblox.Web.WebAPI.Models.ApiArrayResponse_Roblox.Games.Api.Models.Response.GameVoteResponse_" - } - } - } - }, - "400": { - "description": "3: The asset is not voteable.\r\n8: No universe IDs were specified.\r\n9: Too many universe IDs were requested." - }, - "429": { - "description": "10: Internal service busy. Please try again later." - }, - "500": { - "description": "0: An unknown error occurred." - } - }, - "servers": [ - { - "url": "https://games.roblox.com" - } - ], - "x-roblox-engine-usability": { - "apiKeyWithHttpService": false - }, - "security": [ - {}, - { - "roblox-legacy-cookie": [] - } - ], - "externalDocs": { - "url": "https://create.roblox.com/docs/cloud/reference/features/interactions#games_get_v1_games_votes" - } - } - }, "/v1/private-servers/enabled-in-universe/{universeId}": { "get": { "tags": ["Private servers", "Universes"], @@ -40313,7 +40359,7 @@ } } }, - "/v2/groups/{groupId}/gamesV2#GroupsApi": { + "/v2/groups/{groupId}/gamesV2": { "get": { "tags": ["Groups", "Universes"], "summary": "Gets experiences created by the specified group id.", @@ -40408,7 +40454,7 @@ } } }, - "/v2/groups/{groupId}/games#GroupsApi": { + "/v2/groups/{groupId}/games": { "get": { "tags": ["Groups", "Universes"], "summary": "Gets experiences created by the specified group id.", @@ -43889,12 +43935,12 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/Roblox.Web.WebAPI.Models.ApiArrayResponse_Roblox.Groups.Api.SocialLinkResponse_" + "$ref": "#/components/schemas/Roblox.Groups.Api.GetSocialLinkResponse" } }, "text/json": { "schema": { - "$ref": "#/components/schemas/Roblox.Web.WebAPI.Models.ApiArrayResponse_Roblox.Groups.Api.SocialLinkResponse_" + "$ref": "#/components/schemas/Roblox.Groups.Api.GetSocialLinkResponse" } } } @@ -53723,10 +53769,16 @@ "url": "https://thumbnails.roblox.com" } ], + "x-roblox-stability": "BETA", "x-roblox-engine-usability": { "apiKeyWithHttpService": false }, - "security": [{}], + "security": [ + {}, + { + "roblox-legacy-cookie": [] + } + ], "externalDocs": { "url": "https://create.roblox.com/docs/cloud/reference/features/assets#thumbnails_get_v1_assets_thumbnail_3d" } @@ -54752,10 +54804,16 @@ "url": "https://thumbnails.roblox.com" } ], + "x-roblox-stability": "BETA", "x-roblox-engine-usability": { "apiKeyWithHttpService": false }, - "security": [{}], + "security": [ + {}, + { + "roblox-legacy-cookie": [] + } + ], "externalDocs": { "url": "https://create.roblox.com/docs/cloud/reference/features/avatars#thumbnails_get_v1_users_avatar_3d" } @@ -55001,6 +55059,7 @@ "url": "https://thumbnails.roblox.com" } ], + "x-roblox-stability": "BETA", "x-roblox-engine-usability": { "apiKeyWithHttpService": false }, @@ -61008,7 +61067,7 @@ "x-roblox-stability": "BETA", "x-roblox-rate-limits": { "perApiKeyOwner": { - "period": "DAY", + "period": "MINUTE", "maxInPeriod": 100 } }, @@ -64654,7 +64713,7 @@ } } }, - "/v1/games/{universeId}/user-votes#VotingApi": { + "/v1/games/{universeId}/user-votes": { "patch": { "tags": ["Interactions", "Universes"], "operationId": "Voting_VoteGame", @@ -64757,7 +64816,207 @@ } } }, - "/v1/vip-servers/{id}/permissions#PrivateServersApi": { + "/v1/games/{universeId}/votes": { + "get": { + "tags": ["Interactions", "Universes"], + "operationId": "Voting_GetGameVoteStatus", + "parameters": [ + { + "name": "universeId", + "in": "path", + "required": true, + "schema": { + "type": "integer", + "format": "int64" + } + }, + { + "name": "Roblox-Api-Key", + "in": "header", + "description": "Api Key (UUID).", + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "Success", + "content": { + "text/plain": { + "schema": { + "$ref": "#/components/schemas/GameVoteResponse" + } + }, + "application/json": { + "schema": { + "$ref": "#/components/schemas/GameVoteResponse" + } + }, + "text/json": { + "schema": { + "$ref": "#/components/schemas/GameVoteResponse" + } + } + } + } + }, + "servers": [ + { + "url": "https://games.roblox.com" + } + ], + "x-roblox-engine-usability": { + "apiKeyWithHttpService": false + }, + "security": [ + {}, + { + "roblox-legacy-cookie": [] + } + ], + "externalDocs": { + "url": "https://create.roblox.com/docs/cloud/reference/features/interactions#Voting_GetGameVoteStatus" + }, + "x-roblox-recommended-alternatives": [ + { + "url": "https://games.roblox.com/v1/games/votes", + "httpMethod": "GET", + "documentationUrl": "https://create.roblox.com/docs/cloud/reference/features/interactions#Voting_MultiGetGameVoteStatus" + } + ] + } + }, + "/v1/games/{universeId}/votes/user": { + "get": { + "tags": ["Interactions", "Universes"], + "operationId": "Voting_GetGameVoteStatusForUser", + "parameters": [ + { + "name": "universeId", + "in": "path", + "required": true, + "schema": { + "type": "integer", + "format": "int64" + } + }, + { + "name": "Roblox-Api-Key", + "in": "header", + "description": "Api Key (UUID).", + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "Success", + "content": { + "text/plain": { + "schema": { + "$ref": "#/components/schemas/UserGameVoteResponse" + } + }, + "application/json": { + "schema": { + "$ref": "#/components/schemas/UserGameVoteResponse" + } + }, + "text/json": { + "schema": { + "$ref": "#/components/schemas/UserGameVoteResponse" + } + } + } + } + }, + "servers": [ + { + "url": "https://games.roblox.com" + } + ], + "x-roblox-engine-usability": { + "apiKeyWithHttpService": false + }, + "security": [ + { + "roblox-legacy-cookie": [] + } + ], + "externalDocs": { + "url": "https://create.roblox.com/docs/cloud/reference/features/interactions#Voting_GetGameVoteStatusForUser" + } + } + }, + "/v1/games/votes": { + "get": { + "tags": ["Interactions", "Universes"], + "operationId": "Voting_MultiGetGameVoteStatus", + "parameters": [ + { + "name": "universeIds", + "in": "query", + "schema": { + "type": "array", + "items": { + "type": "integer", + "format": "int64" + } + } + }, + { + "name": "Roblox-Api-Key", + "in": "header", + "description": "Api Key (UUID).", + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "Success", + "content": { + "text/plain": { + "schema": { + "$ref": "#/components/schemas/GameVoteResponseApiArrayResponse" + } + }, + "application/json": { + "schema": { + "$ref": "#/components/schemas/GameVoteResponseApiArrayResponse" + } + }, + "text/json": { + "schema": { + "$ref": "#/components/schemas/GameVoteResponseApiArrayResponse" + } + } + } + } + }, + "servers": [ + { + "url": "https://games.roblox.com" + } + ], + "x-roblox-engine-usability": { + "apiKeyWithHttpService": false + }, + "security": [ + {}, + { + "roblox-legacy-cookie": [] + } + ], + "externalDocs": { + "url": "https://create.roblox.com/docs/cloud/reference/features/interactions#Voting_MultiGetGameVoteStatus" + } + } + }, + "/v1/vip-servers/{id}/permissions": { "patch": { "tags": ["Groups", "Private servers", "Universes"], "operationId": "PrivateServers_UpdatePrivateServerPermissions", @@ -64833,7 +65092,7 @@ } } }, - "/v1/vip-servers/{id}/subscription#PrivateServersApi": { + "/v1/vip-servers/{id}/subscription": { "patch": { "tags": ["Private servers", "Universes"], "operationId": "PrivateServers_UpdatePrivateServerSubscription", @@ -64909,7 +65168,7 @@ } } }, - "/v1/vip-servers/{id}#PrivateServersApi": { + "/v1/vip-servers/{id}": { "get": { "tags": ["Metadata", "Private servers", "Universes"], "operationId": "PrivateServers_GetPrivateServer", @@ -65028,7 +65287,7 @@ } } }, - "/v1/games/vip-servers/{universeId}#PrivateServersApi": { + "/v1/games/vip-servers/{universeId}": { "post": { "tags": ["Private servers", "Universes"], "operationId": "PrivateServers_CreatePrivateServer", @@ -65104,7 +65363,7 @@ } } }, - "/v1/vip-server/can-invite/{userId}#PrivateServersApi": { + "/v1/vip-server/can-invite/{userId}": { "get": { "tags": ["Private servers", "Users"], "operationId": "PrivateServers_CanInviteUser", @@ -65214,7 +65473,7 @@ } } }, - "/v1/private-servers/my-private-servers#PrivateServersApi": { + "/v1/private-servers/my-private-servers": { "get": { "tags": ["Private servers", "Universes"], "operationId": "PrivateServersApi.PrivateServers_GetMyPrivateServers", @@ -65412,6 +65671,48 @@ }, "x-resource": true }, + "ContentMessage": { + "type": "object", + "properties": { + "createTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the message was created.", + "format": "date-time" + }, + "updateTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the message was last updated.", + "format": "date-time" + }, + "plainText": { + "type": "string", + "description": "The plain text content of the message." + }, + "author": { + "readOnly": true, + "example": "users/156", + "type": "string", + "description": "The resource path of the user who created the message." + }, + "messageReaction": { + "readOnly": true, + "type": "array", + "items": { + "$ref": "#/components/schemas/MessageReaction" + }, + "description": "The reactions to the message, if this feature is enabled for this message." + } + }, + "description": "Represents the content of a message.", + "x-oneOf": { + "Author": ["author"], + "content": ["plainText"] + } + }, "CreatorStoreProduct": { "type": "object", "properties": { @@ -65855,6 +66156,175 @@ "Owner": ["owner"] } }, + "GroupForumCategory": { + "type": "object", + "properties": { + "path": { + "example": "groups/123/forum-categories/01234567-89ab-cdef-0123-456789abcdef", + "type": "string", + "description": "The resource path of the group forum category.\n\nFormat: `groups/{group_id}/forum-categories/{group_forum_category_id}`" + }, + "createTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum category was created.", + "format": "date-time" + }, + "updateTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum category was last updated.", + "format": "date-time" + }, + "groupForumCategoryId": { + "readOnly": true, + "type": "string", + "description": "The id of the group forum category." + }, + "displayName": { + "type": "string", + "description": "The display name of the group forum category." + }, + "creator": { + "readOnly": true, + "example": "users/156", + "type": "string", + "description": "The resource path of the user who created the group forum category." + }, + "archiveTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum category was archived.", + "format": "date-time" + }, + "archiver": { + "readOnly": true, + "example": "users/156", + "type": "string", + "description": "The resource path of the user who archived the group forum category." + } + }, + "description": "Represents a forum category within a group for user discussions.", + "x-aep-resource": { + "patterns": [ + "groups/{group_id}/forum-categories/{group_forum_category_id}" + ], + "plural": "group-forum-categories", + "singular": "group-forum-category" + }, + "x-resource": true + }, + "GroupForumComment": { + "type": "object", + "properties": { + "path": { + "example": "groups/123/forum-categories/01234567-89ab-cdef-0123-456789abcdef/posts/01234567-89ab-cdef-0123-456789abcdef/comments/01234567-89ab-cdef-0123-456789abcdef", + "type": "string", + "description": "The resource path of the group forum comment.\n\nFormat:\n`groups/{group_id}/forum-categories/{group_forum_category_id}/posts/{group_forum_post_id}/comments/{group_forum_comment_id}`" + }, + "groupForumCommentId": { + "readOnly": true, + "type": "string", + "description": "The id of the group forum comment." + }, + "message": { + "$ref": "#/components/schemas/ContentMessage", + "description": "The message content of the group forum comment." + } + }, + "description": "Represents a forum comment on a group's forum post.", + "x-aep-resource": { + "patterns": [ + "groups/{group_id}/forum-categories/{group_forum_category_id}/posts/{group_forum_post_id}/comments/{group_forum_comment_id}" + ], + "plural": "group-forum-comments", + "singular": "group-forum-comment" + }, + "x-resource": true + }, + "GroupForumPost": { + "type": "object", + "properties": { + "path": { + "example": "groups/123/forum-categories/01234567-89ab-cdef-0123-456789abcdef/posts/01234567-89ab-cdef-0123-456789abcdef", + "type": "string", + "description": "The resource path of the group forum post.\n\nFormat:\n`groups/{group_id}/forum-categories/{group_forum_category_id}/posts/{group_forum_post_id}`" + }, + "createTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum post was created.", + "format": "date-time" + }, + "updateTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum post was last updated.", + "format": "date-time" + }, + "pinned": { + "example": true, + "type": "boolean", + "description": "Whether the group forum post is pinned." + }, + "locked": { + "example": true, + "type": "boolean", + "description": "Whether the group forum post is locked." + }, + "groupForumPostId": { + "readOnly": true, + "type": "string", + "description": "The id of the group forum post." + }, + "title": { + "type": "string", + "description": "The title of the group forum post." + }, + "firstComment": { + "$ref": "#/components/schemas/GroupForumComment", + "description": "The resource path of the group forum post's first comment, which holds the\npost content.\n\nBy default, the `first_comment` field is fully populated. This **will not**\ndereference resource references within the first comment, such as\n`GroupForumComment.author`; only the `path` subfield of these fields will\nbe populated. However, if only the `path` field of the first comment is\nneeded, you can specify the FULL view." + }, + "author": { + "readOnly": true, + "example": "users/156", + "type": "string", + "description": "The resource path of the user who created the group forum post." + }, + "archiveTime": { + "readOnly": true, + "example": "2023-07-05T12:34:56Z", + "type": "string", + "description": "The timestamp when the group forum post was archived.", + "format": "date-time" + }, + "archiver": { + "readOnly": true, + "example": "users/156", + "type": "string", + "description": "The resource path of the user who archived the group forum post." + }, + "commentCount": { + "type": "integer", + "description": "The number of comments on the group forum post. This includes the first\ncomment (post content).", + "format": "int64" + } + }, + "description": "Represents a forum post within a group's forum category.", + "x-aep-resource": { + "patterns": [ + "groups/{group_id}/forum-categories/{group_forum_category_id}/posts/{group_forum_post_id}" + ], + "plural": "group-forum-posts", + "singular": "group-forum-post" + }, + "x-resource": true + }, "GroupJoinRequest": { "type": "object", "properties": { @@ -66538,6 +67008,57 @@ }, "description": "A list of DataStores in the parent collection." }, + "ListGroupForumCategoriesResponse": { + "type": "object", + "properties": { + "groupForumCategories": { + "type": "array", + "items": { + "$ref": "#/components/schemas/GroupForumCategory" + }, + "description": "The GroupForumCategories from the specified Group." + }, + "nextPageToken": { + "type": "string", + "description": "A token that you can send as a `pageToken` parameter to retrieve the next\npage. If this field is omitted, there are no subsequent pages." + } + }, + "description": "A list of GroupForumCategories in the parent collection." + }, + "ListGroupForumCommentsResponse": { + "type": "object", + "properties": { + "groupForumComments": { + "type": "array", + "items": { + "$ref": "#/components/schemas/GroupForumComment" + }, + "description": "The GroupForumComments from the specified GroupForumPost." + }, + "nextPageToken": { + "type": "string", + "description": "A token that you can send as a `pageToken` parameter to retrieve the next\npage. If this field is omitted, there are no subsequent pages." + } + }, + "description": "A list of GroupForumComments in the parent collection." + }, + "ListGroupForumPostsResponse": { + "type": "object", + "properties": { + "groupForumPosts": { + "type": "array", + "items": { + "$ref": "#/components/schemas/GroupForumPost" + }, + "description": "The GroupForumPosts from the specified GroupForumCategory." + }, + "nextPageToken": { + "type": "string", + "description": "A token that you can send as a `pageToken` parameter to retrieve the next\npage. If this field is omitted, there are no subsequent pages." + } + }, + "description": "A list of GroupForumPosts in the parent collection." + }, "ListGroupJoinRequestsResponse": { "type": "object", "properties": { @@ -67069,6 +67590,23 @@ "sortKey": ["stringSortKey", "numericSortKey"] } }, + "MessageReaction": { + "type": "object", + "properties": { + "emoteId": { + "readOnly": true, + "type": "string", + "description": "The id of the emote." + }, + "count": { + "readOnly": true, + "type": "integer", + "description": "The count of reactions for this emote.", + "format": "int64" + } + }, + "description": "Represents the reaction to a message." + }, "Money": { "type": "object", "properties": { @@ -71105,6 +71643,25 @@ }, "additionalProperties": false }, + "Roblox.Authentication.Api.Models.Request.FinishARPreAuthPasskeyRegistrationRequest": { + "type": "object", + "properties": { + "recoverySession": { + "type": "string" + }, + "passkeySessionId": { + "type": "string" + }, + "passkeyRegistrationResponse": { + "type": "string" + }, + "userId": { + "format": "int64", + "type": "integer" + } + }, + "additionalProperties": false + }, "Roblox.Authentication.Api.Models.Request.FinishPasskeyPreauthRegistrationRequest": { "type": "object", "properties": { @@ -71539,6 +72096,10 @@ ], "type": "integer" }, + "staticFacialAnimation": { + "description": "Indicates user choice for facial animation.\r\nstaticFacialAnimation=false, implies the toggle as on and face will animate.\r\nApplicable for dynamic head assets.", + "type": "boolean" + }, "version": { "format": "int32", "description": "Client-authoritative meta model format version\r\n- default is always 1", @@ -74161,6 +74722,11 @@ "additionalProperties": { "type": "string" } + }, + "versionV2": { + "format": "int64", + "description": "VersionV2 of the OTA asset.", + "type": "integer" } }, "additionalProperties": false @@ -76124,21 +76690,6 @@ }, "additionalProperties": false }, - "Roblox.Friends.Api.MultigetAreTrustedFriendsResponse": { - "description": "Response model for MultigetAreTrustedFriendsRequest", - "type": "object", - "properties": { - "trustedFriendsId": { - "description": "List of trusted friend ids", - "type": "array", - "items": { - "format": "int64", - "type": "integer" - } - } - }, - "additionalProperties": false - }, "Roblox.Friends.Api.PendingFriendRequestCountModel": { "description": "Response model for Pending Friend Request Count", "type": "object", @@ -76151,19 +76702,6 @@ }, "additionalProperties": false }, - "Roblox.Friends.Api.TrustedFriendStatusResponse": { - "description": "The trusted friend status response model", - "type": "object", - "properties": { - "status": { - "format": "int32", - "description": "The trusted friend status. ['Invalid' = 0, 'NotFriends' = 1, 'Friends' = 2, 'TrustedFriends' = 3, 'RequestSent' = 4, 'RequestReceived' = 5, 'RequestIgnored' = 6]", - "enum": [0, 1, 2, 3, 4, 5, 6], - "type": "integer" - } - }, - "additionalProperties": false - }, "Roblox.Web.Captcha.Models.Request.CaptchaTokenRequest": { "type": "object", "properties": { @@ -78352,28 +78890,6 @@ }, "additionalProperties": false }, - "Roblox.Games.Api.Models.Response.GameVoteResponse": { - "description": "Response model for getting the game vote", - "type": "object", - "properties": { - "id": { - "format": "int64", - "description": "The game universe id", - "type": "integer" - }, - "upVotes": { - "format": "int64", - "description": "The game up votes", - "type": "integer" - }, - "downVotes": { - "format": "int64", - "description": "The game down votes", - "type": "integer" - } - }, - "additionalProperties": false - }, "Roblox.Games.Api.Models.Response.GetPrivateServerListResponse": { "type": "object", "properties": { @@ -78546,25 +79062,6 @@ }, "additionalProperties": false }, - "Roblox.Games.Api.Models.Response.UserGameVoteResponse": { - "description": "Response model for getting the user vote status for a game", - "type": "object", - "properties": { - "canVote": { - "description": "Whether the user can vote the game", - "type": "boolean" - }, - "userVote": { - "description": "The user's vote status for the game", - "type": "boolean" - }, - "reasonForNotVoteable": { - "description": "The reason why the user can't vote", - "type": "string" - } - }, - "additionalProperties": false - }, "Roblox.Games.Api.Models.Response.VerifiedBadgeUserResponse": { "description": "A response model specific to multi-get user.", "type": "object", @@ -78685,18 +79182,6 @@ }, "additionalProperties": false }, - "Roblox.Web.WebAPI.Models.ApiArrayResponse_Roblox.Games.Api.Models.Response.GameVoteResponse_": { - "type": "object", - "properties": { - "data": { - "type": "array", - "items": { - "$ref": "#/components/schemas/Roblox.Games.Api.Models.Response.GameVoteResponse" - } - } - }, - "additionalProperties": false - }, "Roblox.Web.WebAPI.Models.ApiPageResponse_Roblox.Web.Responses.Games.GameServerResponse_": { "type": "object", "properties": { @@ -79092,6 +79577,24 @@ }, "additionalProperties": false }, + "Roblox.Groups.Api.GetSocialLinkResponse": { + "description": "A social link response from a create request", + "type": "object", + "properties": { + "data": { + "type": "array", + "items": { + "$ref": "#/components/schemas/Roblox.Groups.Api.SocialLinkResponse" + } + }, + "socialLinksVerificationStatus": { + "format": "int32", + "enum": [0, 1, 2, 3], + "type": "integer" + } + }, + "additionalProperties": false + }, "Roblox.Groups.Api.GroupAllRolesResponse": { "description": "A group roles response model", "type": "object", @@ -80785,18 +81288,6 @@ }, "additionalProperties": false }, - "Roblox.Web.WebAPI.Models.ApiArrayResponse_Roblox.Groups.Api.SocialLinkResponse_": { - "type": "object", - "properties": { - "data": { - "type": "array", - "items": { - "$ref": "#/components/schemas/Roblox.Groups.Api.SocialLinkResponse" - } - } - }, - "additionalProperties": false - }, "Roblox.Web.WebAPI.Models.ApiArrayResponse_Roblox.Groups.Api.UserGroupMembershipResponse_": { "type": "object", "properties": { @@ -83669,6 +84160,10 @@ "accessContext": { "description": "Access context for asset permission checks (only relevant for Asset thumbnail types)", "type": "string" + }, + "headShape": { + "description": "Head shape for dynamic head thumbnails (only relevant for Asset thumbnail types with dynamic head)", + "type": "string" } }, "additionalProperties": false @@ -85462,6 +85957,12 @@ "format": "date-time", "nullable": true }, + "creatingUniverseId": { + "type": "integer", + "description": "Universe ID if the asset was created within a specific universe.", + "format": "int64", + "nullable": true + }, "categoryPath": { "type": "string", "description": "Category Path of the asset, if it exists.\r\ne.g. \"3d__characters\"", @@ -85616,6 +86117,17 @@ "type": "boolean", "description": "Whether the creator is verified.", "nullable": true + }, + "latestGroupUpdaterUserId": { + "type": "integer", + "description": "This value only exists for group-based creators and represents the User ID of the creator who most recently updated the asset.", + "format": "int64", + "nullable": true + }, + "latestGroupUpdaterUserName": { + "type": "string", + "description": "This value only exists for group-based creators and represents the name of the creator who most recently updated the asset.", + "nullable": true } }, "additionalProperties": false, @@ -85687,6 +86199,18 @@ "description": "Representation of a creator store asset.", "nullable": true }, + "CreatorStoreCapabilitiesModel": { + "type": "object", + "properties": { + "shouldSandbox": { + "type": "boolean", + "description": "Controls whether the model will be sandboxed upon insertion when it has scripts." + } + }, + "additionalProperties": false, + "description": "Represents the capabilities that a model has (e.g. sandboxing).\r\nOnly present for model asset types; AMRS includes it in the response when available.", + "nullable": true + }, "ToolboxService.CreatorStoreProduct": { "type": "object", "properties": { @@ -85993,6 +86517,15 @@ ], "description": "The number of studio Instance types in the model.\r\nSee [Instance](https://create.roblox.com/docs/reference/engine/classes/Instance).\r\nThese are broken down into specific types, such as MeshPart, Part, Script,\r\nTool, Animation, etc.", "nullable": true + }, + "capabilities": { + "allOf": [ + { + "$ref": "#/components/schemas/CreatorStoreCapabilitiesModel" + } + ], + "description": "Script capabilities for the asset (sandboxing)", + "nullable": true } }, "additionalProperties": false, @@ -87801,6 +88334,20 @@ "type": "integer", "description": "If BleedOffServers=true, how long will we allow old servers to stay up before shutting them down.\r\nValid values: 1 - 60 minutes", "format": "int32" + }, + "placeIdToVersions": { + "type": "object", + "additionalProperties": { + "uniqueItems": true, + "type": "array", + "items": { + "type": "integer", + "format": "int32" + }, + "nullable": true + }, + "description": "Optional. A mapping of PlaceId to the specific versions to restart/bleed off for that place.\r\nWhen set for a place, only servers running those exact versions will be affected.\r\nIf a place is not in this dictionary or has an empty set, behavior falls back to CloseOldVersionsOnly logic.", + "nullable": true } }, "additionalProperties": false, @@ -88319,6 +88866,24 @@ "isInUniverse": { "type": "boolean", "description": "Indicates whether the place is still part of the universe.\r\nA place may have active games but is no longer in the universe if it has been removed from the universe." + }, + "versionToPlayerCounts": { + "type": "object", + "additionalProperties": { + "type": "integer", + "format": "int32" + }, + "description": "Player counts broken down by place version.\r\nKey is the place version, value is the number of players on that version.", + "nullable": true + }, + "versionToInstanceCounts": { + "type": "object", + "additionalProperties": { + "type": "integer", + "format": "int32" + }, + "description": "Instance counts broken down by place version.\r\nKey is the place version, value is the number of instances on that version.", + "nullable": true } }, "additionalProperties": false, @@ -88726,6 +89291,37 @@ "additionalProperties": false, "description": "The response for updating a matchmaking server attribute definition." }, + "GameVoteResponse": { + "type": "object", + "properties": { + "id": { + "type": "integer", + "format": "int64" + }, + "upVotes": { + "type": "integer", + "format": "int64" + }, + "downVotes": { + "type": "integer", + "format": "int64" + } + }, + "additionalProperties": false + }, + "GameVoteResponseApiArrayResponse": { + "type": "object", + "properties": { + "data": { + "type": "array", + "items": { + "$ref": "#/components/schemas/GameVoteResponse" + }, + "nullable": true + } + }, + "additionalProperties": false + }, "UserGameVoteRequest": { "type": "object", "properties": { @@ -88736,6 +89332,23 @@ }, "additionalProperties": false }, + "UserGameVoteResponse": { + "type": "object", + "properties": { + "canVote": { + "type": "boolean" + }, + "userVote": { + "type": "boolean", + "nullable": true + }, + "reasonForNotVoteable": { + "type": "string", + "nullable": true + } + }, + "additionalProperties": false + }, "CanInviteUserResponse": { "type": "object", "properties": { @@ -90049,9 +90662,7 @@ "friends_get_v1_my_friends_requests", "friends_delete_v1_my_new_friend_requests", "friends_get_v1_my_new_friend_requests_count", - "friends_get_v1_my_trusted_friends__userId__status", "friends_post_v1_user__userId__multiget_are_friends", - "friends_get_v1_user__userId__multiget_are_trusted_friends", "friends_get_v1_user_friend_requests_count", "friends_post_v1_user_friend_requests_decline_all", "friends_post_v1_users__requesterUserId__accept_friend_request", @@ -90133,6 +90744,9 @@ "Cloud_ListGroupRoles", "Cloud_GetGroupRole", "Cloud_GetGroupShout", + "Cloud_ListGroupForumCategories", + "Cloud_ListGroupForumPosts", + "Cloud_ListGroupForumComments", "get_legacy_develop_v1_user_groups_canmanage", "develop_get_v1_user_groups_canmanage", "get_legacy_groups_v1_groups__groupId__audit_log", @@ -90278,9 +90892,9 @@ "games_post_v1_games__universeId__favorites", "games_get_v1_games__universeId__favorites_count", "Voting_VoteGame", - "games_get_v1_games__universeId__votes", - "games_get_v1_games__universeId__votes_user", - "games_get_v1_games_votes", + "Voting_GetGameVoteStatus", + "Voting_GetGameVoteStatusForUser", + "Voting_MultiGetGameVoteStatus", "games_get_v2_users__userId__favorite_games", "inventory_get_v1_users__userId__categories_favorites" ] @@ -90968,14 +91582,14 @@ "games_get_v1_games__universeId__favorites_count", "games_get_v1_games__universeId__media", "Voting_VoteGame", - "games_get_v1_games__universeId__votes", - "games_get_v1_games__universeId__votes_user", + "Voting_GetGameVoteStatus", + "Voting_GetGameVoteStatusForUser", "games_get_v1_games_games_product_info", "games_get_v1_games_multiget_place_details", "games_get_v1_games_multiget_playability_status", "games_get_v1_games_recommendations_game__universeId_", "PrivateServers_CreatePrivateServer", - "games_get_v1_games_votes", + "Voting_MultiGetGameVoteStatus", "games_get_v1_private_servers_enabled_in_universe__universeId_", "PrivateServersApi.PrivateServers_GetMyPrivateServers", "PrivateServers_GetPrivateServer", diff --git a/.cursor/rules/roblox/en-us/reference/engine/STUDIO_VERSION b/.cursor/rules/roblox/en-us/reference/engine/STUDIO_VERSION index 4ec4c834..6002c259 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/STUDIO_VERSION +++ b/.cursor/rules/roblox/en-us/reference/engine/STUDIO_VERSION @@ -1,2 +1,2 @@ -0.707.0.7070732 -version-207b4196e6334893 \ No newline at end of file +0.709.0.7090870 +version-bb326014c89c4d27 \ No newline at end of file diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/AdGui.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/AdGui.yaml index 873ab807..d318758e 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/AdGui.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/AdGui.yaml @@ -50,6 +50,7 @@ properties: can_save: true capabilities: - UI + - Monetization - name: AdGui.FallbackImage summary: '' description: '' @@ -86,6 +87,7 @@ properties: can_save: false capabilities: - UI + - Monetization methods: [] events: [] callbacks: @@ -115,4 +117,5 @@ callbacks: thread_safety: Unsafe capabilities: - UI + - Monetization writeCapabilities: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/AdService.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/AdService.yaml index 3dbfbdc0..7f934397 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/AdService.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/AdService.yaml @@ -39,7 +39,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Monetization - name: AdService:GetAdAvailabilityNowAsync summary: | Checks if a video ad is available to be played to the current user inside @@ -63,7 +64,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Monetization - name: AdService:RegisterAdOpportunityAsync summary: | Tracks how many times a user had the chance to watch a video ad and the @@ -91,7 +93,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Monetization - name: AdService:ShowRewardedVideoAdAsync summary: | Plays the video ad to the current user inside the experience. @@ -126,7 +129,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI + - Monetization - name: AdService:ShowVideoAd summary: | Show mobile video advertisements. @@ -144,7 +149,8 @@ methods: no longer operational. security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Monetization - name: AdService:UnregisterAdOpportunity summary: '' description: '' @@ -161,7 +167,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Monetization events: - name: AdService.VideoAdClosed summary: | @@ -180,5 +187,6 @@ events: `Class.AdService.VideoAdClosed|VideoAdClosed` has been decommissioned and is no longer operational. security: None - capabilities: [] + capabilities: + - Monetization callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/AudioDeviceInput.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/AudioDeviceInput.yaml index dc0efeb9..0b43c2dd 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/AudioDeviceInput.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/AudioDeviceInput.yaml @@ -51,7 +51,9 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Audio + - Input - name: AudioDeviceInput.Active summary: | Controls whether the physical device is actively recording. @@ -73,7 +75,9 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Audio + - Input - name: AudioDeviceInput.IsReady summary: | Denotes whether this `Class.AudioDeviceInput` is ready to produce sound. @@ -94,7 +98,9 @@ properties: serialization: can_load: false can_save: true - capabilities: [] + capabilities: + - Audio + - Input - name: AudioDeviceInput.Muted summary: | Controls whether this `Class.AudioDeviceInput` is muted. @@ -119,7 +125,9 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Audio + - Input - name: AudioDeviceInput.Player summary: | Determines whose device is producing sound. @@ -140,7 +148,10 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Audio + - Input + - Players - name: AudioDeviceInput.Volume summary: | Volume level which is multiplied onto the output audio stream. @@ -159,7 +170,9 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Audio + - Input methods: - name: AudioDeviceInput:GetConnectedWires summary: | @@ -183,7 +196,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Audio + - Input - name: AudioDeviceInput:GetInputPins summary: | Gets the list of pins that `Class.Wire` can use in `Class.Wire.TargetName` @@ -203,7 +218,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Audio + - Input - name: AudioDeviceInput:GetOutputPins summary: | Gets the list of pins that `Class.Wire` can use in `Class.Wire.SourceName` @@ -223,7 +240,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Audio + - Input - name: AudioDeviceInput:GetUserIdAccessList summary: | Returns a list of user IDs that are either permitted to hear or blocked @@ -241,7 +260,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Audio + - Input - name: AudioDeviceInput:SetUserIdAccessList summary: | Sets a list of user IDs that are either permitted to hear or blocked from @@ -266,7 +287,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Audio + - Input events: - name: AudioDeviceInput.WiringChanged summary: | @@ -302,5 +325,7 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - Audio + - Input callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/AudioPlayer.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/AudioPlayer.yaml index 89a300ed..3e671894 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/AudioPlayer.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/AudioPlayer.yaml @@ -71,7 +71,8 @@ properties: description: '' code_samples: [] type: Content - tags: [] + tags: + - Hidden deprecation_message: '' security: read: None diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/AudioSpeechToText.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/AudioSpeechToText.yaml index ff80eb3d..05c354da 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/AudioSpeechToText.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/AudioSpeechToText.yaml @@ -15,6 +15,8 @@ description: | Roblox uses the following formula to throttle requests for this API based on the number of players in your experience: `max requests per minute per experience = 1 + (5 * number_of_concurrent_users)`. + You can purchase additional usage using + [Extended Services](../../../cloud-services/extended-services.md). For more information, see [Speech-to-text](../../../audio/objects.md#speech-to-text). diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/AvatarCreationService.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/AvatarCreationService.yaml index 6b57e97a..b6e89249 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/AvatarCreationService.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/AvatarCreationService.yaml @@ -71,7 +71,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - DynamicGeneration - name: AvatarCreationService:GenerateAvatar2DPreviewAsync summary: | Creates a 2D avatar preview and returns a previewId. @@ -119,7 +120,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - DynamicGeneration - name: AvatarCreationService:GenerateAvatarAsync summary: | Generates an avatar and returns a generationId. @@ -165,7 +167,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - DynamicGeneration - name: AvatarCreationService:GetBatchTokenDetailsAsync summary: | Gets the avatar creation token details for a list of avatar creation @@ -208,7 +211,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Monetization - name: AvatarCreationService:GetValidationRules summary: | Gets data regarding rules that assets must abide by to pass UGC @@ -400,7 +404,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - DynamicGeneration - name: AvatarCreationService:LoadGeneratedAvatarAsync summary: | Loads a generated avatar using an avatar generation ID. @@ -441,7 +446,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - DynamicGeneration - name: AvatarCreationService:PrepareAvatarForPreviewAsync summary: | Prepares in-experience avatar for preview. @@ -475,7 +481,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Avatar - name: AvatarCreationService:PromptCreateAvatarAssetAsync summary: | Prompts a `Class.Player` to purchase and create an avatar asset from an @@ -668,7 +675,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetCreateUpdate + - Monetization - name: AvatarCreationService:PromptSelectAvatarGenerationImageAsync summary: | Prompt the `Class.Player` to take a selfie and return the FileId. @@ -709,7 +718,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - SensitiveInput - name: AvatarCreationService:RequestAvatarGenerationSessionAsync summary: | Request an AvatarGeneration session for a `Class.Player`. @@ -779,7 +789,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - DynamicGeneration - name: AvatarCreationService:ValidateUGCAccessoryAsync summary: | Studio only. Runs UGC validation for an `Enum.AccessoryType`. @@ -942,7 +953,8 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - DynamicGeneration - name: AvatarCreationService.AvatarModerationCompleted summary: | Fires when an in-experience-created avatar's moderation status has been @@ -976,5 +988,6 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - DynamicGeneration callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/AvatarEditorService.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/AvatarEditorService.yaml index 569b2116..5ccae4f2 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/AvatarEditorService.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/AvatarEditorService.yaml @@ -393,7 +393,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Avatar - name: AvatarEditorService:CheckApplyDefaultClothing summary: | Used to apply default clothing to the `Class.HumanoidDescription` if @@ -417,7 +418,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Avatar - name: AvatarEditorService:ConformToAvatarRulesAsync summary: '' description: '' @@ -435,7 +437,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Avatar - name: AvatarEditorService:ConformToAvatarRules summary: '' description: '' @@ -454,7 +457,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Avatar - name: AvatarEditorService:GetAccessoryType summary: '' description: '' @@ -587,7 +591,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead - name: AvatarEditorService:GetBatchItemDetails summary: | Gets the item details for a list of items at once. @@ -616,7 +621,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead - name: AvatarEditorService:GetFavoriteAsync summary: | Returns if the `Class.Players.LocalPlayer` has favorited the given bundle @@ -645,7 +651,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - AssetRead - name: AvatarEditorService:GetFavorite summary: | Returns if the `Class.Players.LocalPlayer` has favorited the given bundle @@ -673,7 +681,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - AssetRead - name: AvatarEditorService:GetInventoryAsync summary: | Returns an `Class.InventoryPages` object with information about owned @@ -711,7 +721,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - AssetRead - name: AvatarEditorService:GetInventory summary: | Returns an `Class.InventoryPages` object with information about owned @@ -734,7 +746,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - AssetRead - name: AvatarEditorService:GetItemDetailsAsync summary: | Returns the item details for the given item. @@ -800,7 +814,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead - name: AvatarEditorService:GetItemDetails summary: | Returns the item details for the given item. @@ -829,7 +844,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead - name: AvatarEditorService:GetOutfitDetailsAsync summary: | Returns the outfit details for the given outfit. @@ -892,7 +908,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead - name: AvatarEditorService:GetOutfitDetails summary: | Returns the outfit details for the given outfit. @@ -915,7 +932,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead - name: AvatarEditorService:GetOutfitsAsync summary: | Returns outfit data for the `Class.Players.LocalPlayer`. @@ -983,7 +1001,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - AssetRead - name: AvatarEditorService:GetOutfits summary: | Returns outfit data for the `Class.Players.LocalPlayer`. @@ -1007,7 +1027,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - AssetRead - name: AvatarEditorService:GetRecommendedAssetsAsync summary: | Returns a list of recommended assets based on a given `Enum.AssetType` and @@ -1073,7 +1095,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead - name: AvatarEditorService:GetRecommendedAssets summary: | Returns a list of recommended assets based on a given `Enum.AssetType` and @@ -1105,7 +1128,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead - name: AvatarEditorService:GetRecommendedBundlesAsync summary: | Returns a list of recommended bundles for a given bundle id. @@ -1164,7 +1188,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead - name: AvatarEditorService:GetRecommendedBundles summary: | Returns a list of recommended bundles for a given bundle id. @@ -1186,7 +1211,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead - name: AvatarEditorService:PromptAllowInventoryReadAccess summary: | Prompts the `Class.Players.LocalPlayer` to allow the developer to read @@ -1209,7 +1235,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - AssetRead - name: AvatarEditorService:PromptCreateOutfit summary: | Prompts the `Class.Players.LocalPlayer` to save the given @@ -1239,7 +1267,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing - name: AvatarEditorService:PromptDeleteOutfit summary: | Prompts the `Class.Players.LocalPlayer` to delete the given outfit. @@ -1261,7 +1291,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing - name: AvatarEditorService:PromptRenameOutfit summary: | Prompts the `Class.Players.LocalPlayer` to rename the given outfit. @@ -1283,7 +1315,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing - name: AvatarEditorService:PromptSaveAvatar summary: | Prompts the `Class.Players.LocalPlayer` to update their avatar based on @@ -1314,7 +1348,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing - name: AvatarEditorService:PromptSetFavorite summary: | Prompts the `Class.Players.LocalPlayer` to favorite or unfavorite the @@ -1345,7 +1381,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing - name: AvatarEditorService:PromptUpdateOutfit summary: | Prompts the `Class.Players.LocalPlayer` to update the given outfit. @@ -1376,7 +1414,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing - name: AvatarEditorService:SearchCatalogAsync summary: | Returns a `Class.CatalogPages` object containing the result of the given @@ -1403,7 +1443,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead - name: AvatarEditorService:SearchCatalog summary: | Returns a `Class.CatalogPages` object containing the result of the given @@ -1425,7 +1466,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - AssetRead events: - name: AvatarEditorService.PromptAllowInventoryReadAccessCompleted summary: | @@ -1448,7 +1490,9 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - Players + - AssetRead - name: AvatarEditorService.PromptCreateOutfitCompleted summary: | Fires when the PromptSaveOutfit operation is completed. @@ -1470,7 +1514,9 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing - name: AvatarEditorService.PromptDeleteOutfitCompleted summary: | Fires when the PromptDeleteOutfit operation is completed. @@ -1488,7 +1534,9 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing - name: AvatarEditorService.PromptRenameOutfitCompleted summary: | Fires when the PromptRenameOutfit operation is completed. @@ -1506,7 +1554,9 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing - name: AvatarEditorService.PromptSaveAvatarCompleted summary: | Fires when the `Class.AvatarEditorService:PromptSaveAvatar()` operation is @@ -1530,7 +1580,9 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing - name: AvatarEditorService.PromptSetFavoriteCompleted summary: | Fires when the `Class.AvatarEditorService:PromptSetFavorite()` operation @@ -1550,7 +1602,9 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing - name: AvatarEditorService.PromptUpdateOutfitCompleted summary: | Fires when the `Class.AvatarEditorService:PromptUpdateOutfit()` operation @@ -1570,5 +1624,7 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - Players + - PlatformAvatarEditing callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/ContentProvider.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/ContentProvider.yaml index 8dc4dc1c..f5ef466c 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/ContentProvider.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/ContentProvider.yaml @@ -230,6 +230,13 @@ methods: individual textures. Calling it on a `Class.SurfaceAppearance` instance will not do anything, but the associated textures will still be streamed in during runtime. + + If PreloadAsync is called on `Class.Instance|Instances` that are not + currently visible, such as a `Class.Decal` or an `Class.ImageLabel`, the + Engine will download and store textures used by those Instances in its + disk cache. Because the `Class.Instance|Instances` are not visible in + these cases, the Engine may reduce memory consumption by unloading the + textures after preloading them. code_samples: - ContentProvider-PreloadAsync1 parameters: diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/ExperienceNotificationService.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/ExperienceNotificationService.yaml index 7cd43b73..af368661 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/ExperienceNotificationService.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/ExperienceNotificationService.yaml @@ -61,7 +61,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - Social - name: ExperienceNotificationService:PromptOptIn summary: | Shows an in-experience prompt for the local player to enable @@ -95,7 +97,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players + - Social events: - name: ExperienceNotificationService.OptInPromptClosed summary: | @@ -117,5 +121,7 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - Players + - Social callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/Explosion.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/Explosion.yaml index 14843596..342f90eb 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/Explosion.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/Explosion.yaml @@ -97,7 +97,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - AvatarBehavior - name: Explosion.BlastRadius summary: | This property determines the radius of the `Class.Explosion`, in studs. @@ -133,7 +134,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - AvatarBehavior - name: Explosion.DestroyJointRadiusPercent summary: | Used to set the proportion of the `Class.Explosion.BlastRadius`, between 0 @@ -168,7 +170,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - AvatarBehavior - name: Explosion.ExplosionType summary: | This property determines how the `Class.Explosion` will interact with @@ -201,7 +204,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - AvatarBehavior - name: Explosion.LocalTransparencyModifier summary: '' description: '' @@ -219,7 +223,8 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - AvatarBehavior - name: Explosion.Position summary: | This property is the position of the center of the `Class.Explosion`. It @@ -256,7 +261,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - AvatarBehavior - name: Explosion.TimeScale summary: | Value between 0 and 1 that controls the speed of the particle effect. @@ -276,7 +282,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - AvatarBehavior - name: Explosion.Visible summary: | This property determines whether or not the visual effect of an @@ -305,7 +312,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - AvatarBehavior methods: [] events: - name: Explosion.Hit @@ -349,5 +357,6 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - AvatarBehavior callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/ForceField.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/ForceField.yaml index 68c589b0..c0d8a93e 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/ForceField.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/ForceField.yaml @@ -71,7 +71,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - AvatarBehavior methods: [] events: [] callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/GeometryService.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/GeometryService.yaml index e9ca83b1..42753da3 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/GeometryService.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/GeometryService.yaml @@ -168,6 +168,34 @@ methods: thread_safety: Unsafe capabilities: - CSG + - name: GeometryService:FragmentAsync + summary: | + This API has not been released. + description: '' + code_samples: [] + parameters: + - name: part + type: BasePart + default: + summary: '' + - name: sites + type: Array + default: + summary: '' + - name: options + type: Dictionary + default: nil + summary: '' + returns: + - type: Array + summary: '' + tags: + - Yields + deprecation_message: '' + security: None + thread_safety: Unsafe + capabilities: + - CSG - name: GeometryService:IntersectAsync summary: | Creates one or more `Class.PartOperation|PartOperations` from the @@ -432,6 +460,34 @@ methods: thread_safety: Unsafe capabilities: - CSG + - name: GeometryService:SweepPartAsync + summary: | + This API has not been released. + description: '' + code_samples: [] + parameters: + - name: part + type: BasePart + default: + summary: '' + - name: cframes + type: Array + default: + summary: '' + - name: options + type: Dictionary + default: nil + summary: '' + returns: + - type: MeshPart + summary: '' + tags: + - Yields + deprecation_message: '' + security: None + thread_safety: Unsafe + capabilities: + - CSG - name: GeometryService:UnionAsync summary: | Creates one or more `Class.PartOperation|PartOperations` from one part diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/GuiService.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/GuiService.yaml index 037b505a..0ca4e082 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/GuiService.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/GuiService.yaml @@ -44,7 +44,9 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - UI + - Input - name: GuiService.CoreGuiNavigationEnabled summary: | Toggles whether or not objects in the `Class.CoreGui` can be navigated @@ -66,7 +68,9 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - UI + - Input - name: GuiService.GuiNavigationEnabled summary: | Used to enable and disable the default controller GUI navigation. @@ -84,7 +88,9 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - UI + - Input - name: GuiService.IsModalDialog summary: | Indicates whether a modal dialog is visible. @@ -107,7 +113,8 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - UI - name: GuiService.IsWindows summary: | Indicates whether the user is playing on a computer running Windows. @@ -130,7 +137,8 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - UI - name: GuiService.MenuIsOpen summary: | Returns `true` if any menu of `Class.CoreGui` is open. @@ -150,7 +158,8 @@ properties: serialization: can_load: false can_save: true - capabilities: [] + capabilities: + - UI - name: GuiService.PreferredTextSize summary: | Gets the player's preferred text size as an `Enum.PreferredTextSize` @@ -211,7 +220,8 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - UI - name: GuiService.PreferredTransparency summary: | Gets the player's preferred transparency as a number between `0` and `1`. @@ -247,7 +257,8 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - UI - name: GuiService.ReducedMotionEnabled summary: | Returns `true` if the player has enabled reduced motion. @@ -273,7 +284,8 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - UI - name: GuiService.SelectedObject summary: | Sets the `Class.GuiObject` currently being focused on by the GUI @@ -299,7 +311,9 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - UI + - Input - name: GuiService.TopbarInset summary: | Used to determine the absolute size and position of unobstructed area @@ -330,7 +344,8 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - UI - name: GuiService.TouchControlsEnabled summary: | Used to enable and disable touch controls and touch control display UI. @@ -350,7 +365,9 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - UI + - Input - name: GuiService.ViewportDisplaySize summary: | Read-only property which represents the physical rendering size of the @@ -375,7 +392,8 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - UI methods: - name: GuiService:AddSelectionParent summary: | @@ -406,7 +424,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:AddSelectionTuple summary: | **AddSelectionTuple** works similarly to @@ -449,7 +468,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:CloseInspectMenu summary: | Closes the avatar inspection menu, if open. @@ -476,7 +496,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:DismissNotification summary: '' description: '' @@ -493,7 +514,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:GetEmotesMenuOpen summary: | Checks if the player emotes menu is open. @@ -511,7 +533,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:GetGameplayPausedNotificationEnabled summary: | Returns whether or not the `Class.Player.GameplayPaused` notification has @@ -534,7 +557,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:GetGuiInset summary: | Returns two `Datatype.Vector2` values representing the inset of user GUIs @@ -559,7 +583,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:GetInspectMenuEnabled summary: | Returns whether the avatar inspection menu is enabled. @@ -579,7 +604,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:InspectPlayerFromHumanoidDescription summary: | Allows the avatar inspection menu to appear showing the assets listed in a @@ -615,7 +641,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI + - Avatar - name: GuiService:InspectPlayerFromUserId summary: | Allows the avatar inspection menu to appear showing the user that has the @@ -645,7 +673,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI + - Avatar - name: GuiService:IsTenFootInterface summary: | Returns `true` if the client is using the ten foot interface, a special @@ -666,7 +696,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:RemoveSelectionGroup summary: | Removes a group that was created with @@ -690,7 +721,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:Select summary: | Sets `Class.GuiService.SelectedObject` to a child of a provided instance @@ -716,7 +748,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:SendNotification summary: '' description: '' @@ -733,7 +766,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:SetEmotesMenuOpen summary: | Opens or closes the player emotes menu. @@ -752,7 +786,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:SetGameplayPausedNotificationEnabled summary: | Lets you disable the built-in notification when a player's gameplay is @@ -781,7 +816,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI - name: GuiService:SetInspectMenuEnabled summary: | Allows you to enable or disable the avatar inspection menu. @@ -803,7 +839,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI events: - name: GuiService.MenuClosed summary: | @@ -815,7 +852,8 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - UI - name: GuiService.MenuOpened summary: | Fires when the user **opens** the Roblox `Class.CoreGui` escape menu. @@ -826,5 +864,6 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - UI callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/HttpService.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/HttpService.yaml index 58241c29..498c63d2 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/HttpService.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/HttpService.yaml @@ -433,7 +433,7 @@ methods: Timeout - Integer + Integer no An optional timeout value in seconds to make requests time out more quickly. Values must be greater than zero and no greater than the default request timeout. This can be useful when debugging hanging requests. diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/Humanoid.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/Humanoid.yaml index a2698510..09191f8d 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/Humanoid.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/Humanoid.yaml @@ -85,6 +85,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.AutomaticScalingEnabled summary: | When Enabled, AutomaticScalingEnabled causes the size of the character to @@ -165,6 +166,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.BreakJointsOnDeath summary: | Determines whether the humanoid's joints break when in the @@ -186,6 +188,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.CameraOffset summary: | An offset applied to the Camera's subject position when its CameraSubject @@ -214,6 +217,7 @@ properties: can_save: false capabilities: - Avatar + - AvatarBehavior - name: Humanoid.CollisionType summary: | Selects the `Enum.HumanoidCollisionType` for R15 and Rthro non-player @@ -360,6 +364,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.FloorMaterial summary: | Describes the `Enum.Material` that the `Class.Humanoid` is currently @@ -412,6 +417,7 @@ properties: can_save: false capabilities: - Avatar + - AvatarBehavior - name: Humanoid.Health summary: | Describes the current health of the Humanoid on the range [0, @@ -469,6 +475,7 @@ properties: can_save: false capabilities: - Avatar + - AvatarBehavior - name: Humanoid.HealthDisplayDistance summary: | Used in conjunction with the @@ -600,6 +607,7 @@ properties: can_save: false capabilities: - Avatar + - AvatarBehavior - name: Humanoid.JumpHeight summary: | Provides control over the height that the `Class.Humanoid` jumps to. @@ -631,6 +639,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.JumpPower summary: | Determines how much upwards force is applied to the `Class.Humanoid` when @@ -666,6 +675,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.LeftLeg summary: | A reference to the humanoid's _Left Leg_ part. @@ -717,6 +727,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.maxHealth summary: '' description: '' @@ -767,6 +778,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.MoveDirection summary: | Describes the direction that the `Class.Humanoid` is walking in. @@ -794,6 +806,7 @@ properties: can_save: false capabilities: - Avatar + - AvatarBehavior - name: Humanoid.NameDisplayDistance summary: | Used in conjunction with the `Class.Humanoid.DisplayDistanceType` property @@ -878,6 +891,7 @@ properties: can_save: false capabilities: - Avatar + - AvatarBehavior - name: Humanoid.RequiresNeck summary: | Allows developers to disable the behavior where a player @@ -901,6 +915,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.RightLeg summary: | A reference to the humanoid's _Right Leg_ part. @@ -984,6 +999,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.SeatPart summary: | A reference to the seat that a `Class.Humanoid` is currently sitting in, @@ -1014,6 +1030,7 @@ properties: can_save: false capabilities: - Avatar + - AvatarBehavior - name: Humanoid.Sit summary: | Describes whether the `Class.Humanoid` is currently sitting. @@ -1045,6 +1062,7 @@ properties: can_save: false capabilities: - Avatar + - AvatarBehavior - name: Humanoid.TargetPoint summary: | Describes the 3D position where the `Class.Player` controlling the @@ -1076,6 +1094,7 @@ properties: can_save: false capabilities: - Avatar + - AvatarBehavior - name: Humanoid.Torso summary: | A reference to a humanoid's root driving part. @@ -1128,6 +1147,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.WalkSpeed summary: | Describes the humanoid's maximum movement speed in studs per second. @@ -1168,6 +1188,7 @@ properties: can_save: true capabilities: - Avatar + - AvatarBehavior - name: Humanoid.WalkToPart summary: | A reference to a part whose position is trying to be reached by a @@ -1204,6 +1225,7 @@ properties: can_save: false capabilities: - Avatar + - AvatarBehavior - name: Humanoid.WalkToPoint summary: | The position that a humanoid is trying to reach, after a call to @@ -1235,6 +1257,7 @@ properties: can_save: false capabilities: - Avatar + - AvatarBehavior methods: - name: Humanoid:AddAccessory summary: | @@ -1522,6 +1545,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:ChangeState summary: | Sets the `Class.Humanoid` to enter the given `Enum.HumanoidStateType`. @@ -1572,6 +1596,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:EquipTool summary: | Makes the `Class.Humanoid` equip the given `Class.Tool`. @@ -1602,6 +1627,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:GetAccessories summary: | Returns an array of `Class.Accessory` objects that the humanoid's parent @@ -1701,6 +1727,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:GetLimb summary: | Returns the `Enum.Limb` enum that is associated with the given @@ -1740,6 +1767,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:GetMoveVelocity summary: '' description: '' @@ -1754,6 +1782,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:GetPlayingAnimationTracks summary: | Returns an array of all `Class.AnimationTrack|AnimationTracks` that are @@ -1805,6 +1834,7 @@ methods: thread_safety: Safe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:GetStateEnabled summary: | Returns whether a `Enum.HumanoidStateType` is enabled for the @@ -1844,6 +1874,7 @@ methods: thread_safety: Safe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:GetStatuses summary: | Returns a table of the Humanoid's statuses, and custom statuses. @@ -2016,6 +2047,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:MoveTo summary: | Causes the `Class.Humanoid` to attempt to walk to the given location by @@ -2076,6 +2108,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:PlayEmoteAsync summary: | Plays emotes and returns if was successfully ran. @@ -2101,6 +2134,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:PlayEmote summary: | Plays emotes and returns if was successfully ran. @@ -2126,6 +2160,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:RemoveAccessories summary: | Removes all `Class.Accessory` objects worn by the humanoid's parent. @@ -2276,6 +2311,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:TakeDamage summary: | Lowers the `Class.Humanoid.Health` of the `Class.Humanoid` by the given @@ -2323,6 +2359,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior - name: Humanoid:takeDamage summary: '' description: '' @@ -2372,6 +2409,7 @@ methods: thread_safety: Unsafe capabilities: - Avatar + - AvatarBehavior events: - name: Humanoid.AnimationPlayed summary: | @@ -2458,6 +2496,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.CustomStatusAdded summary: | Fired when a status is added to the Humanoid. @@ -2520,6 +2559,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.FallingDown summary: | Fires when the `Class.Humanoid` enters or leaves the `FallingDown` @@ -2545,6 +2585,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.FreeFalling summary: | Fires when the `Class.Humanoid` enters or leaves the `Freefall` @@ -2574,6 +2615,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.GettingUp summary: | Fires when the `Class.Humanoid` enters or leaves the `GettingUp` @@ -2605,6 +2647,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.HealthChanged summary: | Fires when the `Class.Humanoid.Health` changes (or when the @@ -2631,6 +2674,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.Jumping summary: | Fires when the `Class.Humanoid` enters and leaves the `Jumping` @@ -2660,6 +2704,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.MoveToFinished summary: | Fires when the `Class.Humanoid` finishes walking to a goal declared by @@ -2693,6 +2738,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.PlatformStanding summary: | Fires when the `Class.Humanoid` enters or leaves the `PlatformStanding` @@ -2724,6 +2770,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.Ragdoll summary: | Fires when the `Class.Humanoid` enters or leaves the `Ragdoll` @@ -2755,6 +2802,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.Running summary: | Fires when the speed at which a `Class.Humanoid` is running changes. @@ -2787,6 +2835,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.Seated summary: | Fired when a `Class.Humanoid` either sits in a `Class.Seat` or @@ -2828,6 +2877,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.StateChanged summary: | Fires when the state of the `Class.Humanoid` is changed. @@ -2865,6 +2915,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.StateEnabledChanged summary: | Fires when `Class.Humanoid:SetStateEnabled()` is called on the @@ -2901,6 +2952,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.StatusAdded summary: | Fired when a status is added to the Humanoid. @@ -2977,6 +3029,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.Swimming summary: | Fires when the speed at which a `Class.Humanoid` is swimming in @@ -3011,6 +3064,7 @@ events: security: None capabilities: - Avatar + - AvatarBehavior - name: Humanoid.Touched summary: | Fires when one of the humanoid's limbs come in contact with another @@ -3063,4 +3117,5 @@ events: security: None capabilities: - Avatar + - AvatarBehavior callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/InputBinding.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/InputBinding.yaml index c866ebf1..0309835b 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/InputBinding.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/InputBinding.yaml @@ -5,8 +5,9 @@ name: InputBinding type: class memory_category: Instances -summary: Defines which hardware binding should trigger the parent `Class.InputAction`. -description: |- +summary: | + Defines which hardware binding should trigger the parent `Class.InputAction`. +description: | An `InputBinding` defines which hardware binding should trigger the parent `Class.InputAction`, for example a key press, gamepad button, or tap on a touch‑enabled device. There can be multiple `InputBinding` instances parented @@ -19,10 +20,10 @@ tags: [] deprecation_message: '' properties: - name: InputBinding.Backward - summary: |- + summary: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "backward" inputs to the parent `Class.InputAction`. - description: |- + description: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "backward" inputs to `Class.InputAction:GetState()|GetState()` and the `Class.InputAction.StateChanged|StateChanged` event of the parent @@ -45,10 +46,10 @@ properties: capabilities: - Input - name: InputBinding.Down - summary: |- + summary: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "down" inputs to the parent `Class.InputAction`. - description: |- + description: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "down" inputs to `Class.InputAction:GetState()|GetState()` and the `Class.InputAction.StateChanged|StateChanged` event of the parent @@ -85,10 +86,10 @@ properties: capabilities: - Input - name: InputBinding.Forward - summary: |- + summary: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "forward" inputs to the parent `Class.InputAction`. - description: |- + description: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "forward" inputs to `Class.InputAction:GetState()|GetState()` and the `Class.InputAction.StateChanged|StateChanged` event of the parent @@ -111,10 +112,10 @@ properties: capabilities: - Input - name: InputBinding.KeyCode - summary: |- + summary: | Specifies the `Enum.KeyCode` which triggers the parent `Class.InputAction`. - description: |- + description: | Specifies the `Enum.KeyCode` which triggers the parent `Class.InputAction`. The code type should match the input action's `Class.InputAction.Type|Type`, for example `Enum.KeyCode.E` for an action @@ -138,10 +139,10 @@ properties: capabilities: - Input - name: InputBinding.Left - summary: |- + summary: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "left" inputs to the parent `Class.InputAction`. - description: |- + description: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "left" inputs to `Class.InputAction:GetState()|GetState()` and the `Class.InputAction.StateChanged|StateChanged` event of the parent @@ -174,10 +175,10 @@ properties: capabilities: - Input - name: InputBinding.PressedThreshold - summary: |- + summary: | Numerical value above which to fire an `Class.InputAction` with a `Class.InputAction.Type|Type` of `Enum.InputActionType|Bool`. - description: |- + description: | Numerical value above which to fire an `Class.InputAction` with a `Class.InputAction.Type|Type` of `Enum.InputActionType|Bool`, for example when a gamepad trigger such as `Enum.KeyCode.ButtonL2` exceeds `0.5` @@ -201,15 +202,21 @@ properties: capabilities: - Input - name: InputBinding.PrimaryModifier - summary: Specifies a primary `Enum.KeyCode` that must be pressed for the binding to activate. - description: |- - Specifies a `Enum.KeyCode` that serves as a required modifier for the binding. The `Class.InputBinding` - will only trigger the parent `Class.InputAction` if this input is pressed prior to - `Class.InputBinding.KeyCode|KeyCode`, `Class.InputBinding.UIButton|UIButton`, or composite directions. + summary: | + Specifies a primary `Enum.KeyCode` that must be pressed for the binding to + activate. + description: | + Specifies a `Enum.KeyCode` that serves as a required modifier for the + binding. The `Class.InputBinding` will only trigger the parent + `Class.InputAction` if this input is pressed prior to + `Class.InputBinding.KeyCode|KeyCode`, + `Class.InputBinding.UIButton|UIButton`, or composite directions. - There is no ordering requirement between `Class.InputBinding.PrimaryModifier|PrimaryModifier` and - `Class.InputBinding.SecondaryModifier|SecondaryModifier`. If both properties are set, both modifiers - must be pressed to activate the binding, but the order in which they are pressed does not matter. + There is no ordering requirement between + `Class.InputBinding.PrimaryModifier|PrimaryModifier` and + `Class.InputBinding.SecondaryModifier|SecondaryModifier`. If both + properties are set, both modifiers must be pressed to activate the + binding, but the order in which they are pressed does not matter. If set to `Enum.KeyCode.Unknown`, no primary modifier is required. code_samples: [] @@ -227,10 +234,10 @@ properties: capabilities: - Input - name: InputBinding.ReleasedThreshold - summary: |- + summary: | Numerical value below which to fire an `Class.InputAction` with a `Class.InputAction.Type|Type` of `Enum.InputActionType|Bool`. - description: |- + description: | Numerical value below which to fire an `Class.InputAction` with a `Class.InputAction.Type|Type` of `Enum.InputActionType|Bool`, for example when a gamepad trigger such as `Enum.KeyCode.ButtonL2` falls below `0.5` @@ -254,18 +261,22 @@ properties: capabilities: - Input - name: InputBinding.ResponseCurve - summary: Numerical value to configure scaling for more precise thumbstick aiming. - description: |- - A numerical value to control the sensitivity of gamepad thumbstick input by applying - a quadratic response curve. Commonly used for camera control to make small thumbstick inputs more - precise, while ramping up quickly to allow for large movements with higher magnitude input. + summary: | + Numerical value to configure scaling for more precise thumbstick aiming. + description: | + A numerical value to control the sensitivity of gamepad thumbstick input + by applying a quadratic response curve. Commonly used for camera control + to make small thumbstick inputs more precise, while ramping up quickly to + allow for large movements with higher magnitude input. - Ranges between `1` to `10`, with higher values providing a more intense curve. - Default is `1`, at which input is returned exactly as received. + Ranges between `1` to `10`, with higher values providing a more intense + curve. Default is `1`, at which input is returned exactly as received. Only applicable when the parent action's `Class.InputAction.Type|Type` is - `Enum.InputActionType|Direction2D` and the `Class.InputBinding.KeyCode|KeyCode` - is `Enum.KeyCode.Thumbstick1|Thumbstick1` or `Enum.KeyCode.Thumbstick2|Thumbstick2`. + `Enum.InputActionType|Direction2D` and the + `Class.InputBinding.KeyCode|KeyCode` is + `Enum.KeyCode.Thumbstick1|Thumbstick1` or + `Enum.KeyCode.Thumbstick2|Thumbstick2`. code_samples: [] type: float tags: [] @@ -281,10 +292,10 @@ properties: capabilities: - Input - name: InputBinding.Right - summary: |- + summary: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "right" inputs to the parent `Class.InputAction`. - description: |- + description: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "right" inputs to `Class.InputAction:GetState()|GetState()` and the `Class.InputAction.StateChanged|StateChanged` event of the parent @@ -317,10 +328,10 @@ properties: capabilities: - Input - name: InputBinding.Scale - summary: |- + summary: | Amount by which to linearly scale the values of a directional `Class.InputAction`. - description: |- + description: | Amount by which to linearly scale the values of an `Class.InputAction` with `Class.InputAction.Type|Type` of `Enum.InputActionType|Direction1D`, `Enum.InputActionType|Direction2D`, or `Enum.InputActionType|Direction3D`. @@ -339,15 +350,21 @@ properties: capabilities: - Input - name: InputBinding.SecondaryModifier - summary: Specifies a secondary `Enum.KeyCode` that must be pressed for the binding to activate. - description: |- - Specifies a `Enum.KeyCode` that serves as a required modifier for the binding. The `Class.InputBinding` - will only trigger the parent `Class.InputAction` if this input is pressed prior to - `Class.InputBinding.KeyCode|KeyCode`, `Class.InputBinding.UIButton|UIButton`, or composite directions. + summary: | + Specifies a secondary `Enum.KeyCode` that must be pressed for the binding + to activate. + description: | + Specifies a `Enum.KeyCode` that serves as a required modifier for the + binding. The `Class.InputBinding` will only trigger the parent + `Class.InputAction` if this input is pressed prior to + `Class.InputBinding.KeyCode|KeyCode`, + `Class.InputBinding.UIButton|UIButton`, or composite directions. - There is no ordering requirement between `Class.InputBinding.PrimaryModifier|PrimaryModifier` and - `Class.InputBinding.SecondaryModifier|SecondaryModifier`. If both properties are set, both modifiers - must be pressed to activate the binding, but the order in which they are pressed does not matter. + There is no ordering requirement between + `Class.InputBinding.PrimaryModifier|PrimaryModifier` and + `Class.InputBinding.SecondaryModifier|SecondaryModifier`. If both + properties are set, both modifiers must be pressed to activate the + binding, but the order in which they are pressed does not matter. If set to `Enum.KeyCode.Unknown`, no secondary modifier is required. code_samples: [] @@ -365,8 +382,10 @@ properties: capabilities: - Input - name: InputBinding.UIButton - summary: Connects a `Class.GuiButton` to a boolean action. - description: '`Class.GuiButton` to connect to a boolean action.' + summary: | + Connects a `Class.GuiButton` to a boolean action. + description: | + `Class.GuiButton` to connect to a boolean action. code_samples: [] type: GuiButton tags: [] @@ -382,10 +401,10 @@ properties: capabilities: - Input - name: InputBinding.Up - summary: |- + summary: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "up" inputs to the parent `Class.InputAction`. - description: |- + description: | Specifies an alternate `Enum.KeyCode` for dispatching directionally "up" inputs to `Class.InputAction:GetState()|GetState()` and the `Class.InputAction.StateChanged|StateChanged` event of the parent @@ -422,10 +441,10 @@ properties: capabilities: - Input - name: InputBinding.Vector2Scale - summary: |- + summary: | Amount by which to linearly scale the values of a two-directional `Class.InputAction`. - description: |- + description: | Amount by which to linearly scale the values of an `Class.InputAction` with `Class.InputAction.Type|Type` of `Enum.InputActionType|Direction2D`. code_samples: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/MaterialService.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/MaterialService.yaml index 1728aada..44be921b 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/MaterialService.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/MaterialService.yaml @@ -64,7 +64,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.BasaltName summary: | Specify `Class.MaterialVariant` name to override built-in Basalt. @@ -85,7 +86,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.BrickName summary: | Specify `Class.MaterialVariant` name to override built-in Brick. @@ -106,7 +108,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.CardboardName summary: '' description: '' @@ -123,7 +126,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.CarpetName summary: '' description: '' @@ -140,7 +144,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.CeramicTilesName summary: '' description: '' @@ -157,7 +162,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.ClayRoofTilesName summary: '' description: '' @@ -174,7 +180,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.CobblestoneName summary: | Specify `Class.MaterialVariant` name to override built-in Cobblestone. @@ -195,7 +202,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.ConcreteName summary: | Specify `Class.MaterialVariant` name to override built-in Concrete. @@ -216,7 +224,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.CorrodedMetalName summary: | Specify `Class.MaterialVariant` name to override built-in CorrodedMetal. @@ -237,7 +246,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.CrackedLavaName summary: | Specify `Class.MaterialVariant` name to override built-in CrackedLava. @@ -258,7 +268,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.DiamondPlateName summary: | Specify `Class.MaterialVariant` name to override built-in DiamondPlate. @@ -279,7 +290,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.FabricName summary: | Specify `Class.MaterialVariant` name to override built-in Fabric. @@ -300,7 +312,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.FoilName summary: | Specify `Class.MaterialVariant` name to override built-in Foil. @@ -321,7 +334,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.GlacierName summary: | Specify `Class.MaterialVariant` name to override built-in Glacier. @@ -342,7 +356,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.GraniteName summary: | Specify `Class.MaterialVariant` name to override built-in Granite. @@ -363,7 +378,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.GrassName summary: | Specify `Class.MaterialVariant` name to override built-in Grass. @@ -384,7 +400,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.GroundName summary: | Specify `Class.MaterialVariant` name to override built-in Ground. @@ -405,7 +422,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.IceName summary: | Specify `Class.MaterialVariant` name to override built-in Ice. @@ -426,7 +444,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.LeafyGrassName summary: | Specify `Class.MaterialVariant` name to override built-in LeafyGrass. @@ -447,7 +466,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.LeatherName summary: '' description: '' @@ -464,7 +484,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.LimestoneName summary: | Specify `Class.MaterialVariant` name to override built-in Limestone. @@ -485,7 +506,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.MarbleName summary: | Specify `Class.MaterialVariant` name to override built-in Marble. @@ -506,7 +528,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.MetalName summary: | Specify `Class.MaterialVariant` name to override built-in Metal. @@ -527,7 +550,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.MudName summary: | Specify `Class.MaterialVariant` name to override built-in Mud. @@ -548,7 +572,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.PavementName summary: | Specify `Class.MaterialVariant` name to override built-in Pavement. @@ -569,7 +594,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.PebbleName summary: | Specify `Class.MaterialVariant` name to override built-in Pebble. @@ -590,7 +616,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.PlasterName summary: '' description: '' @@ -607,7 +634,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.PlasticName summary: | Specify `Class.MaterialVariant` name to override built-in Plastic. @@ -628,7 +656,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.RockName summary: | Specify `Class.MaterialVariant` name to override built-in Rock. @@ -649,7 +678,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.RoofShinglesName summary: '' description: '' @@ -666,7 +696,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.RubberName summary: '' description: '' @@ -683,7 +714,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.SaltName summary: | Specify `Class.MaterialVariant` name to override built-in Salt. @@ -704,7 +736,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.SandName summary: | Specify `Class.MaterialVariant` name to override built-in Sand. @@ -725,7 +758,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.SandstoneName summary: | Specify `Class.MaterialVariant` name to override built-in Sandstone. @@ -746,7 +780,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.SlateName summary: | Specify `Class.MaterialVariant` name to override built-in Slate. @@ -767,7 +802,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.SmoothPlasticName summary: | Specify `Class.MaterialVariant` name to override built-in SmoothPlastic. @@ -788,7 +824,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.SnowName summary: | Specify `Class.MaterialVariant` name to override built-in Snow. @@ -809,7 +846,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.Use2022Materials summary: | Switch built-in material pack. @@ -829,7 +867,8 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - Material - name: MaterialService.WoodName summary: | Specify `Class.MaterialVariant` name to override built-in Wood. @@ -850,7 +889,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialService.WoodPlanksName summary: | Specify `Class.MaterialVariant` name to override built-in WoodPlanks. @@ -871,7 +911,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material methods: - name: MaterialService:GetBaseMaterialOverride summary: | @@ -893,7 +934,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Material - name: MaterialService:GetMaterialVariant summary: | Get the effective MaterialVariant reference given a name and Material. @@ -921,7 +963,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Material - name: MaterialService:SetBaseMaterialOverride summary: | Set a `Class.MaterialVariant` name that overrides a built-in material. @@ -946,6 +989,7 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Material events: [] callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/MaterialVariant.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/MaterialVariant.yaml index d99bd65b..7742e26c 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/MaterialVariant.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/MaterialVariant.yaml @@ -56,7 +56,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.BaseMaterial summary: | Category Material this variant belongs to. @@ -74,7 +75,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.ColorMap summary: | Determines the color of the surface. @@ -93,7 +95,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.ColorMapContent summary: | Determines the color of the surface. Only supports asset URIs as textures. @@ -112,7 +115,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.CustomPhysicalProperties summary: '' description: '' @@ -128,7 +132,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.EmissiveMaskContent summary: | Determines the emissivity across the surface. @@ -153,7 +158,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.EmissiveStrength summary: | Determines the strength of emissive contribution. @@ -179,7 +185,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.EmissiveTint summary: | Determines the tinting color for emissive contribution. @@ -197,7 +204,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.MaterialPattern summary: | Determines texture tiling method. @@ -215,7 +223,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.MetalnessMap summary: | Determines which parts of the surface are metal and are non-metal. @@ -248,7 +257,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.MetalnessMapContent summary: | Determines which parts of the surface are metal and are non-metal. Only @@ -282,7 +292,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.NormalMap summary: | Modifies the lighting of the surface by adding bumps, dents, cracks, and @@ -334,7 +345,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.NormalMapContent summary: | Modifies the lighting of the surface by adding bumps, dents, cracks, and @@ -386,7 +398,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.RoughnessMap summary: | Determines the apparent roughness across the surface. @@ -411,7 +424,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.RoughnessMapContent summary: | Determines the apparent roughness across the surface. Only supports asset @@ -437,7 +451,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material - name: MaterialVariant.StudsPerTile summary: | Determines the scale of textures. @@ -456,7 +471,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Material methods: [] events: [] callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/NetworkReplicator.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/NetworkReplicator.yaml index 0907d143..c46f1147 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/NetworkReplicator.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/NetworkReplicator.yaml @@ -37,6 +37,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Network + - Players events: [] callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/NetworkSettings.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/NetworkSettings.yaml index 839b8ee0..8415e112 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/NetworkSettings.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/NetworkSettings.yaml @@ -39,7 +39,8 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - Network - name: NetworkSettings.FreeMemoryMBytes summary: | Describes how much free memory is available, in MiBs. @@ -61,7 +62,8 @@ properties: serialization: can_load: false can_save: false - capabilities: [] + capabilities: + - Network - name: NetworkSettings.HttpProxyEnabled summary: '' description: '' @@ -77,7 +79,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Network - name: NetworkSettings.HttpProxyURL summary: '' description: '' @@ -93,7 +96,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Network - name: NetworkSettings.IncomingReplicationLag summary: | Simulates additional network latency in the network receiving path. @@ -112,7 +116,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Network - name: NetworkSettings.PrintJoinSizeBreakdown summary: | Prints diagnostic information about data sent on connect. @@ -134,7 +139,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Network - name: NetworkSettings.PrintPhysicsErrors summary: | Whether debug messages will be printed to the output pertaining to physics @@ -172,7 +178,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Network - name: NetworkSettings.PrintStreamInstanceQuota summary: | Whether debug information is printed to the output regarding the @@ -221,7 +228,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Network - name: NetworkSettings.RandomizeJoinInstanceOrder summary: | Special facility to help catch bugs related to how your experience loads. @@ -242,7 +250,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Network - name: NetworkSettings.RenderStreamedRegions summary: | Whether regions of space that are being streamed to the client will be @@ -263,7 +272,8 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Network - name: NetworkSettings.ShowActiveAnimationAsset summary: | Whether a label will be shown above each player character's head, showing @@ -284,7 +294,9 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Network + - Animation methods: [] events: [] callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/PlayerScripts.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/PlayerScripts.yaml index 1bb5dabc..b13aa37c 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/PlayerScripts.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/PlayerScripts.yaml @@ -45,7 +45,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players - name: PlayerScripts:ClearComputerMovementModes summary: | Unregisters all ComputerMovementMode enums from the experience's settings @@ -62,7 +63,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Input + - Players - name: PlayerScripts:ClearTouchCameraMovementModes summary: | Unregisters all TouchCameraMovementMode enums from the experience's @@ -79,7 +82,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players - name: PlayerScripts:ClearTouchMovementModes summary: | Unregisters all TouchMovementMode enums from the experience's settings @@ -96,7 +100,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Input + - Players - name: PlayerScripts:RegisterComputerCameraMovementMode summary: | Registers that a computer camera movement mode is available to be selected @@ -117,7 +123,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players - name: PlayerScripts:RegisterComputerMovementMode summary: | Registers that a computer movement mode is available to be selected from @@ -138,7 +145,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Input + - Players - name: PlayerScripts:RegisterTouchCameraMovementMode summary: | Registers that a touch camera movement mode is available to be selected @@ -159,7 +168,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Players - name: PlayerScripts:RegisterTouchMovementMode summary: | Registers that a touch movement mode is available to be selected from the @@ -180,6 +190,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Input + - Players events: [] callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/ProximityPrompt.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/ProximityPrompt.yaml index 2809cc8b..2ace2489 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/ProximityPrompt.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/ProximityPrompt.yaml @@ -91,6 +91,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.AutoLocalize summary: | Whether the prompt's `Class.ProximityPrompt.ActionText` and @@ -116,6 +117,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.ClickablePrompt summary: | Whether the prompt can be activated by clicking/tapping on the prompt UI. @@ -137,6 +139,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.Enabled summary: | Whether or not this prompt should be shown. @@ -157,6 +160,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.Exclusivity summary: | Used to customize which prompts can be shown at the same time. @@ -177,6 +181,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.GamepadKeyCode summary: | The gamepad button the player should press to trigger the prompt. @@ -198,6 +203,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.HoldDuration summary: | The duration, in seconds, that the player must hold the button/key down to @@ -219,6 +225,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.KeyboardKeyCode summary: | The key the player should press to trigger the prompt. @@ -239,6 +246,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.MaxActivationDistance summary: | The maximum distance a Player's `Class.Player.Character|character` can be @@ -261,6 +269,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.MaxIndicatorDistance summary: '' description: '' @@ -278,6 +287,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.ObjectText summary: | An optional property that determines the object name text shown to the @@ -299,6 +309,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.RequiresLineOfSight summary: | Whether the prompt is hidden if the path between the player's @@ -326,6 +337,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.RootLocalizationTable summary: | A reference to a `Class.LocalizationTable` to be used to apply automated @@ -357,6 +369,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.Style summary: | The style of the prompt's UI. @@ -387,6 +400,7 @@ properties: can_save: true capabilities: - UI + - Input - name: ProximityPrompt.UIOffset summary: | The pixel offset applied to the prompt's UI. @@ -406,6 +420,7 @@ properties: can_save: true capabilities: - UI + - Input methods: - name: ProximityPrompt:InputHoldBegin summary: | @@ -427,6 +442,7 @@ methods: thread_safety: Unsafe capabilities: - UI + - Input - name: ProximityPrompt:InputHoldEnd summary: | Fires a signal indicating that the user ended pressing the prompt GUI @@ -445,6 +461,7 @@ methods: thread_safety: Unsafe capabilities: - UI + - Input events: - name: ProximityPrompt.IndicatorHidden summary: '' @@ -456,6 +473,7 @@ events: security: None capabilities: - UI + - Input - name: ProximityPrompt.IndicatorShown summary: '' description: '' @@ -466,6 +484,7 @@ events: security: None capabilities: - UI + - Input - name: ProximityPrompt.PromptButtonHoldBegan summary: | Triggered when a player begins holding down the @@ -488,6 +507,7 @@ events: security: None capabilities: - UI + - Input - name: ProximityPrompt.PromptButtonHoldEnded summary: | Triggers when the player ends holding down the button on a prompt with a @@ -508,6 +528,7 @@ events: security: None capabilities: - UI + - Input - name: ProximityPrompt.PromptHidden summary: | Triggers when the `Class.ProximityPrompt|prompt` becomes hidden. @@ -521,6 +542,7 @@ events: security: None capabilities: - UI + - Input - name: ProximityPrompt.PromptShown summary: | Triggers when the `Class.ProximityPrompt|prompt` becomes visible. @@ -539,6 +561,7 @@ events: security: None capabilities: - UI + - Input - name: ProximityPrompt.Triggered summary: | Triggered when the prompt @@ -562,6 +585,7 @@ events: security: None capabilities: - UI + - Input - name: ProximityPrompt.TriggerEnded summary: | Triggers when `Class.ProximityPrompt.KeyboardKeyCode|key`/button is @@ -585,4 +609,5 @@ events: security: None capabilities: - UI + - Input callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/Sound.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/Sound.yaml index f5607c30..06e46954 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/Sound.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/Sound.yaml @@ -33,11 +33,14 @@ tags: [] deprecation_message: '' properties: - name: Sound.AudioContent - summary: '' - description: '' + summary: | + A reference to an audio asset. + description: | + This property is a reference to an audio asset. code_samples: [] type: Content - tags: [] + tags: + - Hidden deprecation_message: '' security: read: None diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/TeleportService.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/TeleportService.yaml index c011cf73..ae4e65d1 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/TeleportService.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/TeleportService.yaml @@ -6,13 +6,15 @@ name: TeleportService type: class memory_category: Instances summary: | - Enables transporting `Class.Player|Players` between places and servers. + Enables transporting `Class.Player|Players` between places and servers. For + more information on how to teleport players between servers, see + [Teleport between places](../../../projects/teleport.md). description: | **TeleportService** is responsible for transporting `Class.Player|Players` between different places and servers. For more information on how to teleport players between servers, see - [Teleporting Between Places](../../../projects/teleport.md). + [Teleport between places](../../../projects/teleport.md). code_samples: [] inherits: - Instance @@ -45,7 +47,8 @@ properties: serialization: can_load: true can_save: false - capabilities: [] + capabilities: + - Teleport methods: - name: TeleportService:GetArrivingTeleportGui summary: | @@ -86,7 +89,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI + - Teleport - name: TeleportService:GetLocalPlayerTeleportData summary: | Returns the _teleportData_ the `Class.Players.LocalPlayer` arrived into @@ -110,7 +115,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Teleport - name: TeleportService:GetPlayerPlaceInstanceAsync summary: | Returns the `Class.DataModel.PlaceId|PlaceId` and @@ -204,7 +210,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Teleport - name: TeleportService:GetTeleportSetting summary: | Retrieves a teleport setting saved using @@ -281,7 +288,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Teleport - name: TeleportService:PromptExperienceDetailsAsync summary: | Prompts a `Class.Player` with information about the specified experience. @@ -332,7 +340,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Teleport - name: TeleportService:ReserveServerAsync summary: | Returns an access code that can be used to teleport players to a reserved @@ -400,7 +409,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Teleport - name: TeleportService:ReserveServer summary: | Returns an access code that can be used to teleport players to a reserved @@ -428,7 +438,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Teleport - name: TeleportService:SetTeleportGui summary: | Sets the custom `Class.ScreenGui|teleport GUI` that will be shown to the @@ -484,7 +495,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI + - Teleport - name: TeleportService:SetTeleportSetting summary: | Stores a value under a given key that persists across all teleportations @@ -571,7 +584,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Teleport - name: TeleportService:Teleport summary: | Teleports a `Class.Player` to the place associated with the given @@ -616,7 +630,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI + - Teleport - name: TeleportService:TeleportAsync summary: | The all-encompassing method to teleport a player or group of players from @@ -683,7 +699,7 @@ methods: For more information on how to teleport players between servers and receive user data from a teleport, see - [Teleporting Between Places](../../../projects/teleport.md#sending-user-data-along-with-teleports). + [Teleport between places](../../../projects/teleport.md#sending-user-data-along-with-teleports). code_samples: [] parameters: - name: placeId @@ -714,7 +730,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI + - Teleport - name: TeleportService:TeleportPartyAsync summary: | Teleports a group of `Class.Player|Players` to the same server of the @@ -762,7 +780,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI + - Teleport - name: TeleportService:TeleportToPlaceInstance summary: | Teleports a `Class.Player` to the server instance associated with the @@ -816,7 +836,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI + - Teleport - name: TeleportService:TeleportToPrivateServer summary: | Teleport a group of `Class.Player|Players` to a reserved server created @@ -871,7 +893,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI + - Teleport - name: TeleportService:TeleportToSpawnByName summary: | A variant of `Class.TeleportService:Teleport()` that causes the @@ -921,7 +945,9 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - UI + - Teleport events: - name: TeleportService.LocalPlayerArrivedFromTeleport summary: | @@ -996,7 +1022,8 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - Teleport - name: TeleportService.TeleportInitFailed summary: | Fires when a teleport fails to start, leaving the player in their current @@ -1060,7 +1087,7 @@ events: For more information on how to teleport players between servers, see - [Teleporting Between Places](../../../projects/teleport.md). + [Teleport between places](../../../projects/teleport.md). code_samples: [] parameters: - name: player @@ -1092,5 +1119,6 @@ events: tags: [] deprecation_message: '' security: None - capabilities: [] + capabilities: + - Teleport callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/TestService.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/TestService.yaml index 291ffec9..02946364 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/TestService.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/TestService.yaml @@ -748,7 +748,7 @@ methods: summary: '' - name: source type: Instance - default: + default: nil summary: '' returns: - type: () diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/VideoSampler.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/VideoSampler.yaml index d7607f66..3cbf5f38 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/VideoSampler.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/VideoSampler.yaml @@ -17,6 +17,7 @@ inherits: descendants: [] tags: - NotCreatable + - NotReplicated deprecation_message: '' properties: - name: VideoSampler.TimeLength diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/VoiceChatService.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/VoiceChatService.yaml index 657efbda..264e5b21 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/VoiceChatService.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/VoiceChatService.yaml @@ -46,7 +46,9 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Audio + - Chat - name: VoiceChatService.EnableDefaultVoice summary: | Controls whether each voice-eligible player can be heard as though they @@ -74,7 +76,9 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Audio + - Chat - name: VoiceChatService.UseAudioApi summary: | Controls whether voice chat is represented and controlled by @@ -113,7 +117,9 @@ properties: serialization: can_load: true can_save: true - capabilities: [] + capabilities: + - Audio + - Chat methods: - name: VoiceChatService:GetChatGroupsAsync summary: | @@ -181,6 +187,7 @@ methods: security: None thread_safety: Unsafe capabilities: + - Audio - Chat - name: VoiceChatService:IsVoiceEnabledForUserIdAsync summary: | @@ -221,6 +228,8 @@ methods: deprecation_message: '' security: None thread_safety: Unsafe - capabilities: [] + capabilities: + - Audio + - Chat events: [] callbacks: [] diff --git a/.cursor/rules/roblox/en-us/reference/engine/classes/Wire.yaml b/.cursor/rules/roblox/en-us/reference/engine/classes/Wire.yaml index c39b19d5..7f1c41b5 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/classes/Wire.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/classes/Wire.yaml @@ -57,10 +57,11 @@ properties: `Class.Wire.SourceInstance|SourceInstance`, `Class.Wire.TargetInstance|TargetInstance`, `Class.Wire.SourceName|SourceName`, and `Class.Wire.TargetName|TargetName` - properties are all assigned, the connection is valid, and the connection - does not result in a cyclic processing graph. Cyclic processing graphs are - invalid since they can result in infinite computation; attempting to - create one will log an error. + properties are all assigned, the corresponding + `Class.Wire.SourceName|SourceName` and `Class.Wire.TargetName|TargetName` + pins exist on `Class.Wire.SourceInstance|SourceInstance` and + `Class.Wire.TargetInstance|TargetInstance` respectively, and the + connection does not result in a cyclic processing graph. code_samples: [] type: boolean tags: diff --git a/.cursor/rules/roblox/en-us/reference/engine/datatypes/PhysicalProperties.yaml b/.cursor/rules/roblox/en-us/reference/engine/datatypes/PhysicalProperties.yaml index a2252049..5e75c298 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/datatypes/PhysicalProperties.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/datatypes/PhysicalProperties.yaml @@ -105,6 +105,41 @@ constructors: code_samples: [] tags: [] deprecation_message: '' + - name: PhysicalProperties.new + summary: |- + Creates a `Datatype.PhysicalProperties` container with the specified density, + friction, elasticity, weight of friction, weight of elasticity, and acoustic absorption. + description: |- + Creates a `Datatype.PhysicalProperties` container with the specified density, + friction, elasticity, weight of friction, weight of elasticity, and acoustic absorption. + parameters: + - name: density + type: number + default: + summary: '' + - name: friction + type: number + default: + summary: '' + - name: elasticity + type: number + default: + summary: '' + - name: frictionWeight + type: number + default: + summary: '' + - name: elasticityWeight + type: number + default: + summary: '' + - name: acousticAbsorption + type: number + default: + summary: '' + code_samples: [] + tags: [] + deprecation_message: '' constants: [] properties: - name: PhysicalProperties.AcousticAbsorption diff --git a/.cursor/rules/roblox/en-us/reference/engine/enums/AlphaMode.yaml b/.cursor/rules/roblox/en-us/reference/engine/enums/AlphaMode.yaml index 8367be70..4664071e 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/enums/AlphaMode.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/enums/AlphaMode.yaml @@ -20,8 +20,6 @@ items: summary: | Overlays the `Class.SurfaceAppearance.ColorMap|ColorMap` on top of the underlying part color based on the map's alpha channel. - - value: 0 tags: [] deprecation_message: '' @@ -29,8 +27,6 @@ items: summary: | Uses the `Class.SurfaceAppearance.ColorMap|ColorMap` alpha channel to control the transparency of the surface. - - value: 1 tags: [] deprecation_message: '' @@ -41,3 +37,11 @@ items: value: 2 tags: [] deprecation_message: '' + - name: Opaque + summary: | + Ignores the `Class.SurfaceAppearance.ColorMap|ColorMap` alpha channel and + assumes an opacity value of 1 everywhere. + value: 3 + tags: + - NotBrowsable + deprecation_message: '' diff --git a/.cursor/rules/roblox/en-us/reference/engine/libraries/math.yaml b/.cursor/rules/roblox/en-us/reference/engine/libraries/math.yaml index 67a82b56..41912e6a 100644 --- a/.cursor/rules/roblox/en-us/reference/engine/libraries/math.yaml +++ b/.cursor/rules/roblox/en-us/reference/engine/libraries/math.yaml @@ -283,6 +283,56 @@ functions: tags: [] code_samples: [] deprecation_message: '' + - name: math.isfinite + summary: | + Returns true if `x` is a finite number. + description: | + Returns true if `x` is a finite number, meaning it is neither NaN nor + positive or negative infinity (±`Library.math.huge`). + parameters: + - name: x + type: number + default: + summary: '' + returns: + - type: boolean + summary: '' + tags: [] + code_samples: [] + deprecation_message: '' + - name: math.isinf + summary: | + Returns true if `x` is positive or negative infinity and false otherwise. + description: | + Returns true if `x` is positive or negative infinity + (±`Library.math.huge`) and false otherwise. + parameters: + - name: x + type: number + default: + summary: '' + returns: + - type: boolean + summary: '' + tags: [] + code_samples: [] + deprecation_message: '' + - name: math.isnan + summary: | + Returns true if `x` is not a number (NaN) and false otherwise. + description: | + Returns true if `x` is not a number (NaN) and false otherwise. + parameters: + - name: x + type: number + default: + summary: '' + returns: + - type: boolean + summary: '' + tags: [] + code_samples: [] + deprecation_message: '' - name: math.ldexp summary: | Returns `x`\*`2`^`e` (`e` should be an integer). diff --git a/.cursor/rules/roblox/en-us/scripting/attributes.md b/.cursor/rules/roblox/en-us/scripting/attributes.md index 9e43f030..b7b823d3 100644 --- a/.cursor/rules/roblox/en-us/scripting/attributes.md +++ b/.cursor/rules/roblox/en-us/scripting/attributes.md @@ -13,7 +13,11 @@ Making experiences interactive often means manipulating object properties and at Before you begin retrieving and manipulating objects, you must have an understanding of replication order. -The Roblox Engine doesn't guarantee the order in which objects are replicated from the server to the client, which makes the `Class.Instance:WaitForChild()` method essential for accessing objects in client scripts, particularly objects in the `Class.Workspace`. Still, some aspects of the process are predictable: +The Roblox Engine doesn't guarantee the order in which objects (and changes to objects) are replicated from the server to the client, which makes the `Class.Instance:WaitForChild()` method essential for accessing objects in client scripts, particularly objects in the `Class.Workspace`. + +For example, if a server script changes a property of some instance in the Workspace and then calls `Class.RemoteEvent:FireAllClients()`, the property change might replicate to the client before or after `Class.RemoteEvent.OnClientEvent` fires. Use the available methods and events to [detect changes](#detect-changes) rather than assuming a change has replicated. Changes of the same type, such as two attribute changes, generally **do** arrive in order. + +When clients first launch an experience, some aspects of replication and script execution are predictable: 1. The client loads the contents of `Class.ReplicatedFirst`, such as a loading screen, assets, and scripts. 1. `Class.LocalScript|LocalScripts` (and `Class.Script|Scripts` with a `Class.Script.RunContext|RunContext` of `Enum.RunContext.Client|Client`) in `ReplicatedFirst` run. These scripts can safely get objects from `ReplicatedFirst` without using `WaitForChild()`: diff --git a/.cursor/rules/roblox/en-us/scripting/security/access-control.md b/.cursor/rules/roblox/en-us/scripting/security/access-control.md index 2994dea6..d9c7a5d6 100644 --- a/.cursor/rules/roblox/en-us/scripting/security/access-control.md +++ b/.cursor/rules/roblox/en-us/scripting/security/access-control.md @@ -13,20 +13,30 @@ Understanding what information and assets are visible to clients is critical for Once inside a universe, clients can teleport to any place within that universe, potentially bypassing any intended access restrictions or progression gates. This can also lead to the unintended leak of unreleased content, for example from a development or staging experience. It is important to understand that: -- Disabling "Direct Access" only removes the Join button from subplaces on the website - it does not prevent client-initiated teleports +- Disabling "Direct Access" only removes the Join button from subplaces on the website — it does not prevent client-initiated teleports - Assume exploiters will discover the existence of all subplaces within your universe - A client can teleport to any subplace if they can access a single place, such as the root place, regardless of your intended design flow -Many developers attempt to prevent access by kicking unauthorized users from a subplace. This may work for blocking gameplay, but it does **not** prevent content from replicating to the client. For places intended to be private: +Many developers attempt to prevent access by kicking unauthorized users from a subplace. This may work for blocking gameplay, but it does **not** prevent content from replicating to the client, since replication begins as soon as the player joins and before server-side kick logic can execute. -- **Keep development and test places in separate, private universe**s - this is the only reliable way to ensure confidentiality - Never ship confidential event assets, scripts, or UI elements to the production environment before they're intended to be active +### Use secure teleports -For universes requiring a public place but restricted access, combine multiple layers: +The most effective way to prevent unauthorized access via teleportation is to set **Access Control for Places** to **Secure within universe only** in the Creator Dashboard. This restricts all non-start places to server-initiated teleports only, blocking client-initiated teleports at the platform level before the player ever joins the subplace. Because the player never joins, no content replicates to them. -- If feasible, use streaming to limit how much of the world will replicate to a new player. + +Enabling **Secure within universe only** is the recommended setting for any experience with restricted subplaces, progression-gated areas, or test places that players should not have access to. + + +For configuration steps and migration guidance for existing experiences that use client-initiated teleports, see [Teleport between places](../../projects/teleport.md#configure-secure-teleportation). + +### Defense-in-depth for restricted places + +For universes that cannot use secure teleports, or as additional layers of protection alongside them, combine the following measures: + +- **Keep development and test places in separate, private universes** — this is the only reliable way to ensure confidentiality for unreleased content. Never ship confidential event assets, scripts, or UI elements to the production environment before they are intended to be active. +- If feasible, use streaming to limit how much of the world replicates to a new player. - Add server-side group role or badge verification and verify player state or progression requirements. -- By default, disallow incoming players. This ensures no player will be allowed even if the verification process is inconclusive, such as in the case of an exception thrown from the engine. +- By default, disallow incoming players. This ensures no player will be allowed in even if the verification process is inconclusive, such as in the case of an exception thrown from the engine. - If practical, use the `Class.Players.BanAsync|Ban API` for players that fail validation. This prevents players from continuously attempting to join on the same account. ## Replication @@ -114,4 +124,4 @@ Confidentiality breaches can have significant consequences, such as: - **Content leaks**: Unreleased items, maps, or features may be discovered and shared publicly before official announcements - **Competitive disadvantage**: Mechanics, algorithms, or upcoming features may be reverse-engineered by competitors -- **Exploit Development**: Exposed server logic makes it faster and easier for exploiters to identify vulnerabilities and develop targeted attacks +- **Exploit development**: Exposed server logic makes it faster and easier for exploiters to identify vulnerabilities and develop targeted attacks diff --git a/.cursor/rules/roblox/en-us/scripting/sync.md b/.cursor/rules/roblox/en-us/scripting/sync.md index 79348887..422fd868 100644 --- a/.cursor/rules/roblox/en-us/scripting/sync.md +++ b/.cursor/rules/roblox/en-us/scripting/sync.md @@ -1,6 +1,6 @@ --- title: Script Sync -description: Use Studio's Script Sync feature to edit your Luau scripts in an external text editor. +description: Use Studio's Script Sync feature to edit your Luau scripts on your local disk using an external text editor or your favorite integrated development environment (IDE). --- import BetaAlert from '../includes/beta-features/beta-alert.md' @@ -9,7 +9,7 @@ import BetaAlert from '../includes/beta-features/beta-alert.md' Roblox Studio has a built-in [script editor](../studio/script-editor.md) for editing the code in your Roblox project. The editor includes key features like linting, type checking, autocomplete and collaborative editing. -Many developers, however, have a preferred text editor, such as [Visual Studio Code](https://code.visualstudio.com), [Cursor](https://cursor.com), [Sublime Text](https://www.sublimetext.com), or [Notepad++](https://notepad-plus-plus.org). Popular development environments often have features or plugins that don't exist in Studio. To use an external text editor with Studio, enable the **Script Sync** feature. +Many developers, however, have a preferred text editor or integrated development environment (IDE), such as [Visual Studio Code](https://code.visualstudio.com), [Cursor](https://cursor.com), [Sublime Text](https://www.sublimetext.com), or [Notepad++](https://notepad-plus-plus.org). Popular development environments often have features or plugins that don't exist in Studio. To use an external editor or IDE with Studio, enable the **Script Sync** feature. Script Sync works by synchronizing the scripts in your [data model](../projects/data-model.md) to text files on your local disk. Changes made to the Luau files on disk are applied to the scripts in Studio, and vice versa. This synchronization means that you can edit a script in an external editor like VS Code and immediately see the change in Studio. diff --git a/.cursor/rules/roblox/en-us/studio/ui-overview.md b/.cursor/rules/roblox/en-us/studio/ui-overview.md index 2c302286..9f1490a8 100644 --- a/.cursor/rules/roblox/en-us/studio/ui-overview.md +++ b/.cursor/rules/roblox/en-us/studio/ui-overview.md @@ -83,10 +83,6 @@ You can customize the appearance of the toolbar by right-clicking in an empty ar Beyond the default tool tabs, you can add **custom tabs** for your specific needs, either through Studio's interface or by editing local `.json` files. - -The tab configuration system is intended for customizing the Studio toolbar using existing controls/widgets and plugins. It's not meant to replace Studio [plugins](./plugins.md) or interactive [widgets](./build-studio-widgets.md). - - The easiest way to create custom tabs is through Studio's workflow. @@ -116,12 +112,14 @@ The easiest way to create custom tabs is through Studio's workflow. Custom tab configurations are saved **locally** on each machine in `.json` format, allowing you to build customized tabs and share them with your team or the community. -1. Open the custom tabs folder which is stored in a location unaffected by Studio updates. +1. To modify `.json` from a default tab toolset, begin by right‑clicking an existing tab and selecting **Duplicate tab**. + +2. Open the custom tabs folder which is stored in a location unaffected by Studio updates. - **Windows** — `%LOCALAPPDATA%\Roblox\\CustomRibbonTabs` - **Mac** — `~/Documents/Roblox//CustomRibbonTabs` -2. In the `CustomRibbonTabs` directory are `.json` files which define the toolbar's structure: +3. In the `CustomRibbonTabs` directory are `.json` files which define the toolbar's structure: - The `RibbonLayout.json` file controls the overall ordering and visibility of tabs. @@ -149,7 +147,7 @@ Custom tab configurations are saved **locally** on each machine in `.json` forma } ``` -3. In the `Controls` array for a tab file, insert various buttons, submenus, and more. Each control's `Id` value must be **unique** within the file. +4. In the `Controls` array for a tab file, insert various buttons, submenus, and more. Each control's `Id` value must be **unique** within the file. The following examples illustrate common elements in a custom tab: @@ -297,7 +295,7 @@ Custom tab configurations are saved **locally** on each machine in `.json` forma Beyond the above examples, you can right-click any default tab and select **Duplicate tab**. This will create a new custom tab and corresponding `.json` file in your local `CustomRibbonTabs` directory which you can inspect to explore how various default tools and menus are constructed. -4. Save the `.json` file and then reload Studio's plugins: +5. Save the `.json` file and then reload Studio's plugins: 1. Right-click anywhere in the toolbar or mezzanine and select **Manage tabs**. 2. Access the options menu from the popup's upper‑right corner and select **Reload custom tabs**. diff --git a/.cursor/rules/roblox/en-us/tutorials/use-case-tutorials/audio/speech-to-text.md b/.cursor/rules/roblox/en-us/tutorials/use-case-tutorials/audio/speech-to-text.md index ae1d954e..770b2e64 100644 --- a/.cursor/rules/roblox/en-us/tutorials/use-case-tutorials/audio/speech-to-text.md +++ b/.cursor/rules/roblox/en-us/tutorials/use-case-tutorials/audio/speech-to-text.md @@ -11,6 +11,10 @@ Using the [Gingerbread House - Start](https://www.roblox.com/games/1348125963709 For more general guidance on adding audio to your experience, see the [Add 2D audio](./add-2D-audio.md) and [Add 3D audio](./add-3D-audio.md) tutorials. + +For more reliable voice detection, use [push to chat](./add-voice-chat.md#add-push-to-chat) alongside STT. This ensures players' microphones are only active while they press and hold a specific button on their device. + + ## Audio objects To create STT audio, it's important to understand the audio objects that you will be working with throughout this tutorial. There are three types of audio objects for STT: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 00da9ee9..447eae0e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -28,6 +28,9 @@ jobs: - name: Install Rust uses: dtolnay/rust-toolchain@stable + - name: Install Rokit + uses: CompeyDev/setup-rokit@v0.1.2 + - name: Build Atlas run: cargo build --release --locked @@ -46,6 +49,13 @@ jobs: name: Atlas.rbxm path: Atlas.rbxm + - name: Upload Plugin to Roblox + env: + RBX_API_KEY: ${{ secrets.PLUGIN_UPLOAD_TOKEN }} + RBX_UNIVERSE_ID: ${{ vars.PLUGIN_CI_UNIVERSE_ID }} + RBX_PLACE_ID: ${{ vars.PLUGIN_CI_PLACE_ID }} + run: lune run upload-plugin Atlas.rbxm + build-vscode: needs: ["create-release"] name: Build VS Code Extension diff --git a/.gitmodules b/.gitmodules index 0de0d58e..a038db15 100644 --- a/.gitmodules +++ b/.gitmodules @@ -25,3 +25,6 @@ [submodule "vscode-rojo"] path = vscode-rojo url = https://github.com/UserGeneratedLLC/vscode-rojo.git +[submodule ".lune/opencloud-execute"] + path = .lune/opencloud-execute + url = https://github.com/Dekkonot/opencloud-luau-execute-lune.git diff --git a/.lune/.config.luau b/.lune/.config.luau new file mode 100644 index 00000000..43665fdb --- /dev/null +++ b/.lune/.config.luau @@ -0,0 +1,8 @@ +return { + luau = { + languagemode = "strict", + aliases = { + lune = "~/.lune/.typedefs/0.10.4/", + }, + }, +} diff --git a/.lune/opencloud-execute b/.lune/opencloud-execute new file mode 160000 index 00000000..8ae86dd3 --- /dev/null +++ b/.lune/opencloud-execute @@ -0,0 +1 @@ +Subproject commit 8ae86dd3ad35eb66b0a4edda7e8771a5986cd0c6 diff --git a/.lune/scripts/plugin-upload.luau b/.lune/scripts/plugin-upload.luau new file mode 100644 index 00000000..c5708796 --- /dev/null +++ b/.lune/scripts/plugin-upload.luau @@ -0,0 +1,51 @@ +local args: any = ... +assert(args, "no arguments passed to script") + +local input: buffer = args.BinaryInput + +local AssetService = game:GetService("AssetService") +local SerializationService = game:GetService("SerializationService") +local EncodingService = game:GetService("EncodingService") + +local input_hash: buffer = EncodingService:ComputeBufferHash(input, Enum.HashAlgorithm.Sha256) +local hex_hash: { string } = table.create(buffer.len(input_hash)) +for i = 0, buffer.len(input_hash) - 1 do + table.insert(hex_hash, string.format("%02x", buffer.readu8(input_hash, i))) +end + +print(`Deserializing plugin file (size: {buffer.len(input)} bytes, hash: {table.concat(hex_hash, "")})`) +local plugin = SerializationService:DeserializeInstancesAsync(input)[1] + +local UploadDetails = require(plugin.UploadDetails) :: any +local PLUGIN_ID = UploadDetails.assetId +local PLUGIN_NAME = UploadDetails.name +local PLUGIN_DESCRIPTION = UploadDetails.description +local PLUGIN_CREATOR_ID = UploadDetails.creatorId +local PLUGIN_CREATOR_TYPE = UploadDetails.creatorType + +assert(typeof(PLUGIN_ID) == "number", "UploadDetails did not contain a number field 'assetId'") +assert(typeof(PLUGIN_NAME) == "string", "UploadDetails did not contain a string field 'name'") +assert(typeof(PLUGIN_DESCRIPTION) == "string", "UploadDetails did not contain a string field 'description'") +assert(typeof(PLUGIN_CREATOR_ID) == "number", "UploadDetails did not contain a number field 'creatorId'") +assert(typeof(PLUGIN_CREATOR_TYPE) == "string", "UploadDetails did not contain a string field 'creatorType'") +assert( + Enum.AssetCreatorType:FromName(PLUGIN_CREATOR_TYPE) ~= nil, + "UploadDetails field 'creatorType' was not a valid member of Enum.AssetCreatorType" +) + +print(`Uploading to {PLUGIN_ID}`) +print(`Plugin Name: {PLUGIN_NAME}`) +print(`Plugin Description: {PLUGIN_DESCRIPTION}`) + +local result, version_or_err = AssetService:CreateAssetVersionAsync(plugin, Enum.AssetType.Plugin, PLUGIN_ID, { + ["Name"] = PLUGIN_NAME, + ["Description"] = PLUGIN_DESCRIPTION, + ["CreatorId"] = PLUGIN_CREATOR_ID, + ["CreatorType"] = Enum.AssetCreatorType:FromName(PLUGIN_CREATOR_TYPE), +}) + +if result ~= Enum.CreateAssetResult.Success then + error(`Plugin failed to upload because: {result.Name} - {version_or_err}`) +end + +print(`Plugin uploaded successfully. New version is {version_or_err}.`) diff --git a/.lune/upload-plugin.luau b/.lune/upload-plugin.luau new file mode 100644 index 00000000..1e8c71ef --- /dev/null +++ b/.lune/upload-plugin.luau @@ -0,0 +1,77 @@ +local fs = require("@lune/fs") +local process = require("@lune/process") +local stdio = require("@lune/stdio") + +local luau_execute = require("./opencloud-execute") + +local UNIVERSE_ID = process.env["RBX_UNIVERSE_ID"] +local PLACE_ID = process.env["RBX_PLACE_ID"] + +local version_string = string.gsub(fs.readFile("plugin/Version.txt"), "%s+$", "") +local versions = { string.match(version_string, "^v?(%d+)%.(%d+)%.(%d+)(.*)$") } +if versions[4] ~= "" then + print("This release is a pre-release. Skipping uploading plugin.") + process.exit(0) +end + +local plugin_path = process.args[1] +assert( + typeof(plugin_path) == "string", + "no plugin path provided, expected usage is `lune run upload-plugin [PATH TO RBXM]`." +) + +if process.env["CI"] ~= "true" then + local atlas = process.exec("atlas", { "build", "plugin.project.json", "--output", plugin_path }) + if not atlas.ok then + stdio.ewrite("plugin upload failed because: could not build plugin.rbxm\n\n") + stdio.ewrite(atlas.stderr) + stdio.ewrite("\n") + process.exit(1) + end +else + assert(fs.isFile(plugin_path), `Plugin file did not exist at {plugin_path}`) +end +local plugin_content = fs.readFile(plugin_path) + +local engine_script = fs.readFile(".lune/scripts/plugin-upload.luau") + +print("Creating task to upload plugin") +local task = luau_execute.create_task_latest(UNIVERSE_ID, PLACE_ID, engine_script, 300, false, plugin_content) + +print("Waiting for task to finish") +local success = luau_execute.await_finish(task) +if not success then + local error = luau_execute.get_error(task) + assert(error, "could not fetch error from task") + stdio.ewrite("plugin upload failed because: task did not finish successfully\n\n") + stdio.ewrite(error.code) + stdio.ewrite("\n") + stdio.ewrite(error.message) + stdio.ewrite("\n") + process.exit(1) +end + +print("Output from task:\n") +for _, log in luau_execute.get_structured_logs(task) do + if log.messageType == "ERROR" then + stdio.write(stdio.color("red")) + stdio.write(log.message) + stdio.write("\n") + stdio.write(stdio.color("reset")) + elseif log.messageType == "INFO" then + stdio.write(stdio.color("cyan")) + stdio.write(log.message) + stdio.write("\n") + stdio.write(stdio.color("reset")) + elseif log.messageType == "WARNING" then + stdio.write(stdio.color("yellow")) + stdio.write(log.message) + stdio.write("\n") + stdio.write(stdio.color("reset")) + else + stdio.write(stdio.color("reset")) + stdio.write(log.message) + stdio.write("\n") + stdio.write(stdio.color("reset")) + end +end diff --git a/.vscode/settings.json b/.vscode/settings.json index 62d42093..9d048071 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -25,7 +25,7 @@ "luau-lsp.completion.imports.requireStyle": "alwaysAbsolute", "luau-lsp.completion.fillCallArguments": false, "luau-lsp.completion.showDeprecatedItems": false, - "luau-lsp.sourcemap.generatorCommand": "atlas sourcemap default.project.json --output sourcemap.json --include-non-scripts --watch", + "luau-lsp.sourcemap.generatorCommand": "atlas sourcemap plugin.project.json --output sourcemap.json --include-non-scripts --watch", "luau-lsp.fflags.enableNewSolver": false, "luau-lsp.index.maxFiles": 100000, "luau-lsp.plugin.port": 3668, diff --git a/CHANGELOG.md b/CHANGELOG.md index 038e3f89..f7d9b258 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,167 @@ Making a new release? Simply add the new header with the version and date undern ## Unreleased +## [8.4.0] (February 19th, 2026) + +* Implement ambiguous path handling in the Reconciler with a 3-pass matching algorithm (exact match, property-weighted scoring, recursive change-count) that correctly reconciles instances when multiple children share the same name. +* Add default property caching and enhance property comparison logic for more accurate instance matching during reconciliation. +* Implement recursive change-count matching as a tiebreaker when property-weighted scoring produces ambiguous results. +* Refactor Reconciler equality checks and path handling for robustness with duplicate-named siblings. +* Refactor descendant search functions for `WeakDom` and `RojoTree` to support the new matching algorithm. +* Expose `snapshot`, `syncback`, and `variant_eq` modules publicly and add integration tests for the matching algorithm. +* Update ChangeBatcher tests to reflect encoding behavior for duplicate-named children. +* Enhance deduplication processes and refactor code for improved readability and consistency. +* Update rbx-dom subproject and reflection database. +* Implement live syncback via the Studio plugin with service properties encoding, allowing Roblox→filesystem sync during `atlas serve` sessions without the CLI command. +* Add `SyncbackRequest` structure and API protocol validation for plugin-initiated syncback. +* Add `SyncbackConfirm` UI component for reviewing and confirming live syncback operations in the plugin. +* Implement git-based sync direction defaults: use `git status` metadata to auto-select sync direction for changed files, with `stage_paths` support in `PatchSet` for automatic git staging. +* Enhance matching module with reference identity checks, matching session caching, valid children tracking, and `@native` codegen annotations. +* Sort syncback matched indices by new-child index for stable output ordering. +* Update deduplication suffix numbering to start at `~2` instead of `~1`. +* Refactor `RefPathIndex` initialization and attribute indexing; add regression tests for unique placeholder handling in reference paths. +* Update syncback rules and property filter defaults. +* Compact JSON5 array formatting for property values. +* Update color handling and enhance session management in the Reconciler. +* Refactor animation goal settings to use instant transitions in the plugin UI. +* Refactor version rejection logic in `ApiContext`. +* Add VFS prefetch cache to optimize file reads during project loading, reducing I/O overhead for large projects. +* Implement parallel I/O in the VFS backend for improved serve startup and file processing performance. +* Replace `float-cmp` dependency with custom float comparison and formatting using `lexical-write-float` for precise, round-trip-safe disk representation of float properties. +* Fix NaN equality in `trueEquals` for compound types (e.g., `CFrame`, `Vector3` with NaN components). +* Fix patch application logic to ensure correct resolution order for legacy `Rojo_Target_*` and path-based `Rojo_Ref_*` references. +* Fix multiple live syncback issues including reference handling and float precision. +* Add yield intervals for plugin diffing and hydrating processes to prevent UI freezes during large syncs. +* Add automatic dedup suffix stripping during syncback for cleaner output when suffixes are no longer needed. +* Add server-side and test-side port retry mechanisms to handle port race conditions. +* Refactor font comparison logic in `variant_eq` and `hash_variant` for correct Font property equality. +* Refactor syncback error handling and improve concurrency management. +* Refactor ChangeBatcher encoding logic for improved readability and duplicate handling. +* Consolidate filesystem name retrieval in `ref_target_path_from_tree`. +* Enhance file and directory removal logic in VFS. +* Enhance `PatchVisualizer` with change statistics in the syncback confirmation dialog. +* Add new Git configuration options in `InitCommand` for improved project initialization usability. +* Increase server initialization backoff duration and maximum retry attempts. +* Refactor logging levels to debug for improved performance. +* Update rbx-dom submodule and reflection database. +* Add plugin upload workflow using Lune and OpenCloud for automated Roblox marketplace publishing. +* Add `publish-plugin` scripts (PowerShell and Bash) and integrate plugin upload step into the GitHub Actions release workflow. + +
+Full commit log + +- `ba117f3a` Enhance Reconciler robustness and review dialogue compliance +- `9d57c8d6` Add support for Ambiguous Path Handling +- `cc77a2cd` Add comprehensive plan for Ambiguous Path Handling +- `d6162ac2` Complete implementation of the 3-pass matching algorithm for ambiguous path handling +- `e313f615` Refactor ChangeBatcher tests to remove skipped count checks +- `4cb45cd6` Add audit plan for Ambiguous Path Handling feature +- `1a1d9ce8` Implement new recursive change-count matching algorithm for instance reconciliation +- `3ad3c21c` Add ref path and dedup integration plan +- `bb9d2412` Refactor Reconciler tests and update path handling logic +- `aabbfb23` Update ChangeBatcher tests to reflect encoding behavior for duplicate-named children +- `e207a398` Refactor Reconciler equality checks and enhance path handling +- `f2cdb059` Add audit plans for ambiguous paths fixes and follow-up actions +- `ff48e059` Enhance audit plan for ambiguous paths follow-up actions +- `ed1f61c0` Complete audit plan updates and enhance deduplication processes +- `31350cd7` Consolidate glob patterns in MDC files for improved clarity +- `cc2c8287` update rules +- `85787f34` Refactor code for improved readability and consistency +- `0afe6be4` Update reflection database and enhance deduplication processes +- `e54f5ef4` Update rbx-dom subproject to latest commit for improved functionality +- `9c6966ac` Add default properties caching and update matching logic +- `d45b1c5b` Enhance property comparison and caching mechanisms +- `b6e34b9e` Expose snapshot, syncback, and variant_eq modules publicly; add integration tests for matching algorithm +- `21b3ee45` Refactor descendant search functions for WeakDom and RojoTree +- `6ef81cf8` Refactor matching logic for improved readability and consistency +- `5c4fd801` Enhance Reconciler with ambiguous path handling and improved matching logic +- `dea60eb4` Add formatting and static analysis scripts with usage instructions +- `df225001` Refactor code for improved clarity and maintainability +- `7ba04f7f` Update dependencies and implement git-based sync direction defaults +- `6c0098d8` Add git metadata handling and integration tests for sync direction defaults +- `dcd92537` Refactor version rejection logic in ApiContext +- `91c599ec` Enhance documentation and refactor syncback logic +- `6dec0686` Refactor code for improved readability and maintainability +- `bb4d40c4` Enhance git metadata handling in compute_git_metadata function +- `6de088be` Add batch script files and enhance git staging logic +- `a59d9d49` Add stage_paths field to PatchSet struct in tests +- `d836ebdd` Update deduplication logic and documentation for suffix handling +- `7c3cb8ab` Add deduplication plan to start suffix numbering at ~2 +- `a17852f9` Update syncback rules and property filter defaults +- `fe5852f9` Add compact JSON5 array formatting plan and update JSON serialization logic +- `201b9711` Refactor animation goal settings to use instant transitions +- `c5a5e973` Refactor RefPathIndex initialization and attribute indexing +- `a741f573` Add regression tests for unique placeholder handling in reference paths +- `d6169c6d` Refactor test assertions and improve code formatting +- `b8c3520d` Add Luau-style ref paths migration plan and update related tests +- `6f8e5a5c` Refactor code for improved readability and consistency +- `307ac81a` Implement live syncback feature and enhance API for service data transfer +- `b39c01e5` Add matching session caching and native codegen improvements +- `fa2b86c3` Refactor SyncbackConfirm component and enhance UI responsiveness +- `372a422b` Update SyncbackRequest structure and API protocol validation +- `93ea0bde` Refactor syncback process and improve code readability +- `bf4aef6e` Update sourcemap generator command in VSCode settings +- `b687c886` Enhance matching module with reference identity and caching improvements +- `ddd47b9a` Enhance matching module with valid children tracking and code readability improvements +- `30e032b2` Update color handling and enhance session management in the Reconciler +- `72ad84b5` Implement live syncback service properties encoding and enhance syncback functionality +- `c3cb9b4b` Enhance syncback functionality and improve matching process +- `8070129b` Refactor service encoding to conditionally include properties, attributes, tags, and references +- `e0fd9077` Refactor service encoding to streamline properties handling +- `a823db02` Refactor encoding logic in ChangeBatcher and improve matching depth handling +- `a75a1443` Enhance syncback process by integrating serialization of service children +- `7dc4f527` Implement live syncback fixes addressing multiple issues +- `39c8bd41` Enhance live syncback process with improved reference handling and float precision +- `3c026aea` Add float formatting functions and update serializer for improved precision +- `0450dabf` Refactor encoding logic in ChangeBatcher for improved readability and consistency +- `07e0100c` Add lexical-write-float and update float formatting in JSON serialization +- `ea902572` Update float cutoff values and adjust test cases for improved precision +- `b8a1dc26` Remove zero cutoff checks in float formatting functions +- `f5939258` Remove max significant digits and zero cutoff checks in float formatting functions +- `7f4b5c49` Refactor float formatting tests for improved precision and consistency +- `b47eef11` Add live syncback functionality and optimize service encoding +- `e6c06276` Add luau-optimize cursor command for performance optimization +- `24dec407` Enhance luau-optimize documentation with detailed type safety guidelines +- `9d8762bb` Refactor encoding logic in ChangeBatcher and enhance duplicate handling +- `9bb237f4` Add ChangeBatcher Encoding Audit Plan +- `d3a9155e` Update patch application logic to ensure correct resolution order for legacy and path-based references +- `b0f9b06c` Refactor filesystem name retrieval in `ref_target_path_from_tree` +- `23874d8a` Consolidate filesystem name retrieval in `ref_target_path_from_tree` +- `b1180d73` Enhance Luau documentation with additional context and examples +- `a283fd1d` Add Git Sync Defaults Audit Plan +- `7e0538b9` Remove `float-cmp` dependency and implement custom float comparison logic +- `69f0b9fd` Add Audit Plan for Ambiguous Path Handling +- `5b38648e` Update dependencies and improve syncback handling +- `49b4d858` Add Live Syncback Audit Fixes Plan +- `532425df` Update audit documentation and rules for live syncback enhancements +- `17d52e3e` Enhance float formatting and comparison functions for disk representation +- `7d0232c4` Add automatic dedup suffix stripping to enhance syncback functionality +- `bf56dc47` Refactor syncback error handling and improve concurrency management +- `d9fa62c8` Add server-side and test-side port retry mechanisms to handle race conditions +- `b85c5f5e` Refactor font comparison logic in variant_eq and hash_variant functions +- `354d4c8b` Add yield intervals for diffing and hydrating processes +- `b105d865` Fix NaN equality for trueEquals for compound types +- `1ffed121` Add prefetch cache for VFS to optimize file reads +- `9d3d02cd` Add new Git configuration options in InitCommand +- `0462b91a` Enhance PrefetchCache functionality and improve project file prefetching +- `5a943c33` Refactor prefetch_project_files function for improved readability +- `3ccaaa63` Enhance PrefetchCache and improve directory handling in VFS +- `5fab0d57` Refactor ServeCommand and enhance placeholder replacement in syncback +- `56e1ab4a` Remove unused methods from ServeSession and clean up syncback logic +- `1c25b928` Refactor logging levels to debug for improved performance and clarity +- `43834f35` Update submodule and implement parallel I/O audit fixes +- `a071d1f0` Update submodule and enhance PatchVisualizer with change statistics +- `8d17b1de` Enhance InitCommand Git configuration for improved usability +- `57112a9f` Update submodule and enhance syncback confirmation dialog +- `4f0afe87` Update submodule: rbx_binary Font snapshots remove cachedFaceId +- `d9459638` Update submodule: rbx-dom to latest commit b75dd401 +- `d360ebe6` Enhance file and directory removal logic in VFS +- `d252b138` Increase backoff duration and maximum retry attempts in server initialization +- `8cf140af` Add enhancements and fixes to VFS and syncback processes +- `4918fb80` Add plugin upload workflow and related enhancements + +
+ ## [8.3.0] (February 15th, 2026) * Add two-way sync support for Ref properties via `Rojo_Ref_*` attributes. The plugin now detects Ref property changes in Studio and encodes them as path-based attributes; the server resolves paths back to instance Refs. ([#2]) @@ -61,6 +222,7 @@ Making a new release? Simply add the new header with the version and date undern +[8.4.0]: https://github.com/UserGeneratedLLC/rojo/releases/tag/v8.4.0 [8.3.0]: https://github.com/UserGeneratedLLC/rojo/releases/tag/v8.3.0 [#2]: https://github.com/UserGeneratedLLC/rojo/pull/2 diff --git a/Cargo.lock b/Cargo.lock index 2ae6ddb4..5df9baef 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -160,9 +160,9 @@ checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50" [[package]] name = "async-compression" -version = "0.4.39" +version = "0.4.40" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "68650b7df54f0293fd061972a0fb05aaf4fc0879d3b3d21a638a182c5c543b9f" +checksum = "7d67d43201f4d20c78bcda740c142ca52482d81da80681533d33bf3f0596c8e2" dependencies = [ "compression-codecs", "compression-core", @@ -172,21 +172,20 @@ dependencies = [ [[package]] name = "atlas" -version = "8.3.0" +version = "8.4.0" dependencies = [ "anyhow", "backtrace", "bincode", "blake3", "bytes", - "clap 4.5.58", + "clap 4.5.59", "criterion", "crossbeam-channel", "csv", "data-encoding", "embed-resource", "env_logger 0.11.9", - "float-cmp", "fs-err", "futures", "globset", @@ -199,6 +198,7 @@ dependencies = [ "insta", "jod-thread", "json5", + "lexical-write-float", "log", "maplit", "memofs", @@ -227,6 +227,7 @@ dependencies = [ "serde_bytes", "serde_json", "serde_yaml", + "sha1 0.10.6", "strum", "tempfile", "termcolor", @@ -264,9 +265,9 @@ checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8" [[package]] name = "aws-lc-rs" -version = "1.15.4" +version = "1.16.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7b7b6141e96a8c160799cc2d5adecd5cbbe5054cb8c7c4af53da0f83bb7ad256" +checksum = "d9a7b350e3bb1767102698302bc37256cbd48422809984b98d292c40e2579aa9" dependencies = [ "aws-lc-sys", "zeroize", @@ -417,9 +418,9 @@ dependencies = [ [[package]] name = "bumpalo" -version = "3.19.1" +version = "3.20.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5dd9dc738b7a8311c7ade152424974d8115f2cdad61e8dab8dac9f2362298510" +checksum = "5c6f81257d10a0f602a294ae4182251151ff97dbb504ef9afcdda4a64b24d9b4" [[package]] name = "byteorder" @@ -536,9 +537,9 @@ dependencies = [ [[package]] name = "clap" -version = "4.5.58" +version = "4.5.59" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "63be97961acde393029492ce0be7a1af7e323e6bae9511ebfac33751be5e6806" +checksum = "c5caf74d17c3aec5495110c34cc3f78644bfa89af6c8993ed4de2790e49b6499" dependencies = [ "clap_builder", "clap_derive", @@ -546,9 +547,9 @@ dependencies = [ [[package]] name = "clap_builder" -version = "4.5.58" +version = "4.5.59" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7f13174bda5dfd69d7e947827e5af4b0f2f94a4a3ee92912fba07a66150f21e2" +checksum = "370daa45065b80218950227371916a1633217ae42b2715b2287b606dcd618e24" dependencies = [ "anstream", "anstyle", @@ -601,9 +602,9 @@ dependencies = [ [[package]] name = "compression-codecs" -version = "0.4.36" +version = "0.4.37" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "00828ba6fd27b45a448e57dbfe84f1029d4c9f26b368157e9a448a5f49a2ec2a" +checksum = "eb7b51a7d9c967fc26773061ba86150f19c50c0d65c887cb1fbe295fd16619b7" dependencies = [ "brotli", "compression-core", @@ -710,7 +711,7 @@ dependencies = [ "anes", "cast", "ciborium", - "clap 4.5.58", + "clap 4.5.59", "criterion-plot", "itertools", "num-traits", @@ -1031,15 +1032,6 @@ dependencies = [ "miniz_oxide", ] -[[package]] -name = "float-cmp" -version = "0.10.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b09cf3155332e944990140d967ff5eceb70df778b34f77d8075db46e4704e6d8" -dependencies = [ - "num-traits", -] - [[package]] name = "foldhash" version = "0.1.5" @@ -1767,6 +1759,31 @@ version = "0.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2" +[[package]] +name = "lexical-util" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2604dd126bb14f13fb5d1bd6a66155079cb9fa655b37f875b3a742c705dbed17" + +[[package]] +name = "lexical-write-float" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "50c438c87c013188d415fbabbb1dceb44249ab81664efbd31b14ae55dabb6361" +dependencies = [ + "lexical-util", + "lexical-write-integer", +] + +[[package]] +name = "lexical-write-integer" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "409851a618475d2d5796377cad353802345cba92c867d9fbcde9cf4eac4e14df" +dependencies = [ + "lexical-util", +] + [[package]] name = "libc" version = "0.2.182" @@ -3555,9 +3572,9 @@ dependencies = [ [[package]] name = "toml_parser" -version = "1.0.8+spec-1.1.0" +version = "1.0.9+spec-1.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0742ff5ff03ea7e67c8ae6c93cac239e0d9784833362da3f9a9c1da8dfefcbdc" +checksum = "702d4415e08923e7e1ef96cd5727c0dfed80b4d2fa25db9647fe5eb6f7c5a4c4" dependencies = [ "winnow", ] @@ -3737,9 +3754,9 @@ checksum = "dbc4bc3a9f746d862c45cb89d705aa10f187bb96c76001afab07a0d35ce60142" [[package]] name = "unicode-ident" -version = "1.0.23" +version = "1.0.24" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "537dd038a89878be9b64dd4bd1b260315c1bb94f4d784956b81e27a088d9a09e" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" [[package]] name = "unicode-width" diff --git a/Cargo.toml b/Cargo.toml index ecf5f832..cc5ce1d6 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "atlas" -version = "8.3.0" +version = "8.4.0" rust-version = "1.88" authors = [ "Lucien Greathouse ", @@ -75,6 +75,7 @@ log = "0.4.29" num_cpus = "1.17.0" opener = "0.8.4" rayon = "1.11.0" +walkdir = "2.5.0" rbx_cookie = "0.1.5" reqwest = { version = "0.13", default-features = false, features = [ "blocking", @@ -91,11 +92,12 @@ roblox_install = "1.0.0" serde = { version = "1.0.228", features = ["derive", "rc"] } serde_json = "1.0.149" json5 = "1.3.1" +lexical-write-float = "1.0.6" strum = { version = "0.27", features = ["derive"] } toml = "0.9" termcolor = "1.4.1" thiserror = "2" -tokio = { version = "1.49.0", features = ["rt", "rt-multi-thread", "macros"] } +tokio = { version = "1.49.0", features = ["rt", "rt-multi-thread", "macros", "sync"] } uuid = { version = "1.21.0", features = ["v4", "serde"] } clap = { version = "4", features = ["derive"] } profiling = "1.0.17" @@ -104,7 +106,7 @@ data-encoding = "2.10.0" pathdiff = "0.2.3" blake3 = "1.8.3" -float-cmp = "0.10.0" +sha1 = "0.10" indexmap = { version = "2.13.0", features = ["serde"] } rmp-serde = "1.3.1" serde_bytes = "0.11.19" diff --git a/build.rs b/build.rs index b03589d0..57e5a472 100644 --- a/build.rs +++ b/build.rs @@ -96,6 +96,7 @@ fn main() -> Result<(), anyhow::Error> { "src" => snapshot_from_fs_path(&plugin_dir.join("src"))?, "Packages" => snapshot_from_fs_path(&plugin_dir.join("Packages"))?, "Version.txt" => snapshot_from_fs_path(&plugin_dir.join("Version.txt"))?, + "UploadDetails.json" => snapshot_from_fs_path(&plugin_dir.join("UploadDetails.json"))?, }), }); diff --git a/crates/memofs/src/lib.rs b/crates/memofs/src/lib.rs index 7b3993a3..2eaaba1c 100644 --- a/crates/memofs/src/lib.rs +++ b/crates/memofs/src/lib.rs @@ -22,6 +22,7 @@ mod noop_backend; mod snapshot; mod std_backend; +use std::collections::HashMap; use std::path::{Path, PathBuf}; use std::sync::{Arc, Mutex, MutexGuard}; use std::{io, str}; @@ -31,6 +32,21 @@ pub use noop_backend::NoopBackend; pub use snapshot::VfsSnapshot; pub use std_backend::{CriticalErrorHandler, StdBackend, WatcherCriticalError}; +/// Pre-read file contents, canonical paths, and metadata for fast startup. +/// +/// Populated in parallel (via rayon + walkdir) before snapshot building, +/// then consumed by VFS reads during `snapshot_from_vfs()`. Cleared after +/// the initial tree build to avoid stale data during live operation. +pub struct PrefetchCache { + pub files: HashMap>, + pub canonical: HashMap, + /// `true` = file, `false` = directory. Paths not in the map fall through + /// to the backend (e.g. init-file probes for paths that don't exist). + pub is_file: HashMap, + /// Directory path -> sorted child paths. Consumed once per directory. + pub children: HashMap>, +} + mod sealed { use super::*; @@ -145,27 +161,39 @@ pub enum VfsEvent { struct VfsInner { backend: Box, watch_enabled: bool, + prefetch_cache: Option, } impl VfsInner { - fn read>(&mut self, path: P) -> io::Result>> { - let path = path.as_ref(); + /// Read raw bytes from the prefetch cache or the backend. + /// Removes the entry from the cache on hit to free memory. + fn read_raw(&mut self, path: &Path) -> io::Result> { + if let Some(cache) = &mut self.prefetch_cache { + if let Some(contents) = cache.files.remove(path) { + if self.watch_enabled { + self.backend.watch(path)?; + } + return Ok(contents); + } + } + let contents = self.backend.read(path)?; if self.watch_enabled { self.backend.watch(path)?; } - Ok(Arc::new(contents)) + Ok(contents) + } + + fn read>(&mut self, path: P) -> io::Result>> { + let path = path.as_ref(); + Ok(Arc::new(self.read_raw(path)?)) } fn read_to_string>(&mut self, path: P) -> io::Result> { let path = path.as_ref(); - let contents = self.backend.read(path)?; - - if self.watch_enabled { - self.backend.watch(path)?; - } + let contents = self.read_raw(path)?; let contents_str = str::from_utf8(&contents).map_err(|_| { io::Error::new( @@ -190,6 +218,19 @@ impl VfsInner { fn read_dir>(&mut self, path: P) -> io::Result { let path = path.as_ref(); + + if let Some(cache) = &mut self.prefetch_cache { + if let Some(child_paths) = cache.children.remove(path) { + if self.watch_enabled { + self.backend.watch(path)?; + } + let inner = child_paths.into_iter().map(|p| Ok(DirEntry { path: p })); + return Ok(ReadDir { + inner: Box::new(inner), + }); + } + } + let dir = self.backend.read_dir(path)?; if self.watch_enabled { @@ -211,23 +252,41 @@ impl VfsInner { fn remove_file>(&mut self, path: P) -> io::Result<()> { let path = path.as_ref(); - let _ = self.backend.unwatch(path); + if self.watch_enabled { + let _ = self.backend.unwatch(path); + } self.backend.remove_file(path) } fn remove_dir_all>(&mut self, path: P) -> io::Result<()> { let path = path.as_ref(); - let _ = self.backend.unwatch(path); + if self.watch_enabled { + let _ = self.backend.unwatch(path); + } self.backend.remove_dir_all(path) } fn metadata>(&mut self, path: P) -> io::Result { let path = path.as_ref(); + + if let Some(cache) = &self.prefetch_cache { + if let Some(&is_file) = cache.is_file.get(path) { + return Ok(Metadata { is_file }); + } + } + self.backend.metadata(path) } fn canonicalize>(&mut self, path: P) -> io::Result { let path = path.as_ref(); + + if let Some(cache) = &mut self.prefetch_cache { + if let Some(canonical) = cache.canonical.remove(path) { + return Ok(canonical); + } + } + self.backend.canonicalize(path) } @@ -300,6 +359,7 @@ impl Vfs { let lock = VfsInner { backend: Box::new(backend), watch_enabled: true, + prefetch_cache: None, }; Self { @@ -307,6 +367,24 @@ impl Vfs { } } + /// Load a prefetch cache for fast initial reads. + /// + /// File reads and canonicalize calls will check the cache before hitting + /// the backend. Call [`clear_prefetch_cache`] after the initial snapshot + /// build to free memory and ensure live operations get fresh data. + pub fn set_prefetch_cache(&self, cache: PrefetchCache) { + let mut inner = self.inner.lock().unwrap(); + inner.prefetch_cache = Some(cache); + } + + /// Drop the prefetch cache, freeing memory. + /// + /// After this call, all reads go through the backend as normal. + pub fn clear_prefetch_cache(&self) { + let mut inner = self.inner.lock().unwrap(); + inner.prefetch_cache = None; + } + /// Manually lock the Vfs, useful for large batches of operations. pub fn lock(&self) -> VfsLock<'_> { VfsLock { @@ -617,7 +695,8 @@ impl VfsLock<'_> { #[cfg(test)] mod test { - use crate::{InMemoryFs, StdBackend, Vfs, VfsSnapshot}; + use crate::{InMemoryFs, PrefetchCache, StdBackend, Vfs, VfsSnapshot}; + use std::collections::HashMap; use std::io; use std::path::PathBuf; @@ -693,4 +772,295 @@ mod test { let err = vfs.canonicalize(&file_path).unwrap_err(); assert_eq!(err.kind(), io::ErrorKind::NotFound); } + + fn make_prefetch(files: Vec<(&str, &[u8])>, canonical: Vec<(&str, &str)>) -> PrefetchCache { + PrefetchCache { + files: files + .into_iter() + .map(|(k, v)| (PathBuf::from(k), v.to_vec())) + .collect(), + canonical: canonical + .into_iter() + .map(|(k, v)| (PathBuf::from(k), PathBuf::from(v))) + .collect(), + is_file: HashMap::new(), + children: HashMap::new(), + } + } + + #[test] + fn prefetch_cache_read_hit() { + let imfs = InMemoryFs::new(); + let vfs = Vfs::new(imfs); + vfs.set_prefetch_cache(make_prefetch(vec![("test", b"cached")], vec![])); + + assert_eq!(vfs.read("test").unwrap().as_slice(), b"cached"); + } + + #[test] + fn prefetch_cache_read_depletion() { + let mut imfs = InMemoryFs::new(); + imfs.load_snapshot("test", VfsSnapshot::file("backend")) + .unwrap(); + let vfs = Vfs::new(imfs); + vfs.set_prefetch_cache(make_prefetch(vec![("test", b"cached")], vec![])); + + assert_eq!(vfs.read("test").unwrap().as_slice(), b"cached"); + assert_eq!(vfs.read("test").unwrap().as_slice(), b"backend"); + } + + #[test] + fn prefetch_cache_read_miss_falls_through() { + let mut imfs = InMemoryFs::new(); + imfs.load_snapshot("other", VfsSnapshot::file("backend")) + .unwrap(); + let vfs = Vfs::new(imfs); + vfs.set_prefetch_cache(make_prefetch(vec![("test", b"cached")], vec![])); + + assert_eq!(vfs.read("other").unwrap().as_slice(), b"backend"); + } + + #[test] + fn prefetch_cache_read_to_string_hit() { + let imfs = InMemoryFs::new(); + let vfs = Vfs::new(imfs); + vfs.set_prefetch_cache(make_prefetch(vec![("test", b"hello")], vec![])); + + assert_eq!(vfs.read_to_string("test").unwrap().as_str(), "hello"); + } + + #[test] + fn prefetch_cache_lf_normalized_hit() { + let imfs = InMemoryFs::new(); + let vfs = Vfs::new(imfs); + vfs.set_prefetch_cache(make_prefetch(vec![("test", b"line1\r\nline2\r\n")], vec![])); + + assert_eq!( + vfs.read_to_string_lf_normalized("test").unwrap().as_str(), + "line1\nline2\n" + ); + } + + #[test] + fn prefetch_cache_canonicalize_hit() { + let imfs = InMemoryFs::new(); + let vfs = Vfs::new(imfs); + vfs.set_prefetch_cache(make_prefetch(vec![], vec![("src/foo", "/abs/src/foo")])); + + assert_eq!( + vfs.canonicalize("src/foo").unwrap(), + PathBuf::from("/abs/src/foo") + ); + } + + #[test] + fn prefetch_cache_canonicalize_miss_falls_through() { + let mut imfs = InMemoryFs::new(); + imfs.load_snapshot("/real/file.txt", VfsSnapshot::file("x")) + .unwrap(); + let vfs = Vfs::new(imfs); + vfs.set_prefetch_cache(make_prefetch(vec![], vec![("other", "/abs/other")])); + + assert_eq!( + vfs.canonicalize("/real/file.txt").unwrap(), + PathBuf::from("/real/file.txt") + ); + } + + #[test] + fn prefetch_cache_clear_restores_backend() { + let mut imfs = InMemoryFs::new(); + imfs.load_snapshot("test", VfsSnapshot::file("backend")) + .unwrap(); + let vfs = Vfs::new(imfs); + vfs.set_prefetch_cache(make_prefetch(vec![("test", b"cached")], vec![])); + vfs.clear_prefetch_cache(); + + assert_eq!(vfs.read("test").unwrap().as_slice(), b"backend"); + } + + #[test] + fn prefetch_cache_no_cache_same_as_none() { + let mut imfs = InMemoryFs::new(); + imfs.load_snapshot("test", VfsSnapshot::file("data")) + .unwrap(); + let vfs = Vfs::new(imfs); + + assert_eq!(vfs.read("test").unwrap().as_slice(), b"data"); + } + + #[test] + fn prefetch_cache_watch_registered_on_cache_hit() { + let contents = "hello world".to_string(); + let dir = tempfile::tempdir().unwrap(); + let file_path = dir.path().join("file.txt"); + fs_err::write(&file_path, &contents).unwrap(); + + let vfs = Vfs::new(StdBackend::new_for_testing()); + let mut cache_files = HashMap::new(); + cache_files.insert(file_path.clone(), contents.as_bytes().to_vec()); + vfs.set_prefetch_cache(PrefetchCache { + files: cache_files, + canonical: HashMap::new(), + is_file: HashMap::new(), + children: HashMap::new(), + }); + + let result = vfs.read(&file_path).unwrap(); + assert_eq!(result.as_slice(), contents.as_bytes()); + } + + #[test] + fn prefetch_cache_read_to_string_invalid_utf8() { + let imfs = InMemoryFs::new(); + let vfs = Vfs::new(imfs); + vfs.set_prefetch_cache(make_prefetch( + vec![("test", &[0xFF, 0xFE, 0x00, 0x80])], + vec![], + )); + + let err = vfs.read_to_string("test").unwrap_err(); + assert_eq!(err.kind(), io::ErrorKind::InvalidData); + } + + #[test] + fn prefetch_cache_canonicalize_depletion() { + let mut imfs = InMemoryFs::new(); + imfs.load_snapshot("/real/path", VfsSnapshot::file("x")) + .unwrap(); + let vfs = Vfs::new(imfs); + vfs.set_prefetch_cache(make_prefetch( + vec![], + vec![("/real/path", "/canonical/path")], + )); + + assert_eq!( + vfs.canonicalize("/real/path").unwrap(), + PathBuf::from("/canonical/path"), + "First call should hit cache" + ); + assert_eq!( + vfs.canonicalize("/real/path").unwrap(), + PathBuf::from("/real/path"), + "Second call should fall through to InMemoryFs backend" + ); + } + + #[test] + fn prefetch_cache_set_overwrite_previous() { + let imfs = InMemoryFs::new(); + let vfs = Vfs::new(imfs); + + vfs.set_prefetch_cache(make_prefetch(vec![("a", b"first")], vec![])); + vfs.set_prefetch_cache(make_prefetch(vec![("a", b"second")], vec![])); + + assert_eq!(vfs.read("a").unwrap().as_slice(), b"second"); + } + + #[test] + fn prefetch_cache_concurrent_reads_from_threads() { + use std::sync::Arc; + + let dir = tempfile::tempdir().unwrap(); + let mut cache_files = HashMap::new(); + for i in 0..100 { + let path = dir.path().join(format!("file_{i}.txt")); + let content = format!("content_{i}"); + fs_err::write(&path, &content).unwrap(); + cache_files.insert(path, content.into_bytes()); + } + + let vfs = Arc::new(Vfs::new(StdBackend::new_for_testing())); + vfs.set_prefetch_cache(PrefetchCache { + files: cache_files, + canonical: HashMap::new(), + is_file: HashMap::new(), + children: HashMap::new(), + }); + + let handles: Vec<_> = (0..100) + .map(|i| { + let vfs = Arc::clone(&vfs); + let path = dir.path().join(format!("file_{i}.txt")); + let expected = format!("content_{i}"); + std::thread::spawn(move || { + let data = vfs.read(&path).unwrap(); + assert_eq!( + String::from_utf8(data.to_vec()).unwrap(), + expected, + "File {i} content mismatch" + ); + }) + }) + .collect(); + + for h in handles { + h.join().expect("Thread panicked"); + } + } + + #[test] + fn prefetch_cache_many_files_std_backend() { + let dir = tempfile::tempdir().unwrap(); + let mut cache_files = HashMap::new(); + let mut canonical = HashMap::new(); + + for i in 0..50 { + let path = dir.path().join(format!("f{i}.txt")); + let content = format!("data_{i}"); + fs_err::write(&path, &content).unwrap(); + cache_files.insert(path.clone(), content.into_bytes()); + if let Ok(c) = path.canonicalize() { + canonical.insert(path, c); + } + } + + let vfs = Vfs::new(StdBackend::new_for_testing()); + vfs.set_prefetch_cache(PrefetchCache { + files: cache_files, + canonical, + is_file: HashMap::new(), + children: HashMap::new(), + }); + + for i in 0..50 { + let path = dir.path().join(format!("f{i}.txt")); + let data = vfs.read_to_string(&path).unwrap(); + assert_eq!(data.as_str(), &format!("data_{i}")); + } + + for i in 0..50 { + let path = dir.path().join(format!("f{i}.txt")); + let data = vfs.read_to_string(&path).unwrap(); + assert_eq!( + data.as_str(), + &format!("data_{i}"), + "Second read (backend) of f{i}.txt diverged" + ); + } + } + + #[test] + fn prefetch_cache_read_after_write_ignores_cache() { + let mut imfs = InMemoryFs::new(); + imfs.load_snapshot("test", VfsSnapshot::file("original")) + .unwrap(); + let vfs = Vfs::new(imfs); + vfs.set_prefetch_cache(make_prefetch(vec![("test", b"cached")], vec![])); + + vfs.write("test", b"written").unwrap(); + let data = vfs.read("test").unwrap(); + assert_eq!( + data.as_slice(), + b"cached", + "Cache entry should still be consumed first" + ); + + let data2 = vfs.read("test").unwrap(); + assert_eq!( + data2.as_slice(), + b"written", + "After cache depleted, should see the written data" + ); + } } diff --git a/crates/memofs/src/std_backend.rs b/crates/memofs/src/std_backend.rs index ac97383d..67bc1651 100644 --- a/crates/memofs/src/std_backend.rs +++ b/crates/memofs/src/std_backend.rs @@ -82,6 +82,11 @@ impl StdBackend { ); false // keep watcher running } + WatcherCriticalError::ChannelSendFailed(_) => { + // The event receiver was dropped (VFS shutdown). This is + // normal during process exit -- stop the thread silently. + true + } _ => { log::error!("{}. File watching is no longer reliable.", err); std::process::exit(1); @@ -339,7 +344,7 @@ impl VfsBackend for StdBackend { // This prevents a failed watch from permanently marking the path as "watched" match self.debouncer.watch(path, RecursiveMode::Recursive) { Ok(()) => { - log::info!("Watching path: {}", path.display()); + log::debug!("Watching path: {}", path.display()); self.watches.insert(path.to_path_buf()); Ok(()) } @@ -352,22 +357,31 @@ impl VfsBackend for StdBackend { } fn unwatch(&mut self, path: &Path) -> io::Result<()> { - // Only remove from watches if unwatch succeeds - // This keeps state consistent if unwatch fails (e.g., path wasn't directly watched) + let was_watched = self.watches.contains(path); + match self.debouncer.unwatch(path) { Ok(()) => { - log::info!("Unwatched path: {}", path.display()); + if was_watched { + log::info!("Unwatched path: {}", path.display()); + } else { + log::trace!( + "Unwatched path (was not actively watched): {}", + path.display() + ); + } self.watches.remove(path); Ok(()) } Err(err) => { - // If the path wasn't being watched (common when parent dir is watched), - // still remove from our tracking set but don't propagate the error - if matches!( + let is_not_watched = matches!( err.kind, notify::ErrorKind::WatchNotFound | notify::ErrorKind::PathNotFound - ) { - log::info!( + ) || matches!( + &err.kind, + notify::ErrorKind::Generic(msg) if msg.contains("No watch was found") + ); + if is_not_watched { + log::trace!( "Path was not directly watched (likely covered by parent): {}", path.display() ); diff --git a/plugin.project.json b/plugin.project.json index 83265250..1742547b 100644 --- a/plugin.project.json +++ b/plugin.project.json @@ -22,6 +22,9 @@ }, "Version": { "$path": "plugin/Version.txt" + }, + "UploadDetails": { + "$path": "plugin/UploadDetails.json" } } } diff --git a/plugin/UploadDetails.json b/plugin/UploadDetails.json new file mode 100644 index 00000000..ef9fa736 --- /dev/null +++ b/plugin/UploadDetails.json @@ -0,0 +1,7 @@ +{ + "assetId": 111151139098227, + "name": "Atlas", + "description": "The plugin portion of Atlas, a tool to enable professional tooling for Roblox developers.", + "creatorId": 422368266, + "creatorType": "Group" +} diff --git a/plugin/Version.txt b/plugin/Version.txt index 2bf50aaf..a2f28f43 100644 --- a/plugin/Version.txt +++ b/plugin/Version.txt @@ -1 +1 @@ -8.3.0 +8.4.0 diff --git a/plugin/rbx_dom_lua/allValues.json b/plugin/rbx_dom_lua/allValues.json index 6c74956d..0988ee67 100644 --- a/plugin/rbx_dom_lua/allValues.json +++ b/plugin/rbx_dom_lua/allValues.json @@ -261,113 +261,7 @@ }, "MaterialColors": { "value": { - "MaterialColors": { - "Grass": [ - 106, - 127, - 63 - ], - "Slate": [ - 63, - 127, - 107 - ], - "Concrete": [ - 127, - 102, - 63 - ], - "Brick": [ - 138, - 86, - 62 - ], - "Sand": [ - 143, - 126, - 95 - ], - "WoodPlanks": [ - 139, - 109, - 79 - ], - "Rock": [ - 102, - 108, - 111 - ], - "Glacier": [ - 101, - 176, - 234 - ], - "Snow": [ - 195, - 199, - 218 - ], - "Sandstone": [ - 137, - 90, - 71 - ], - "Mud": [ - 58, - 46, - 36 - ], - "Basalt": [ - 30, - 30, - 37 - ], - "Ground": [ - 102, - 92, - 59 - ], - "CrackedLava": [ - 232, - 156, - 74 - ], - "Asphalt": [ - 115, - 123, - 107 - ], - "Cobblestone": [ - 132, - 123, - 90 - ], - "Ice": [ - 129, - 194, - 224 - ], - "LeafyGrass": [ - 115, - 132, - 74 - ], - "Salt": [ - 198, - 189, - 181 - ], - "Limestone": [ - 206, - 173, - 148 - ], - "Pavement": [ - 148, - 148, - 140 - ] - } + "MaterialColors": {} }, "ty": "MaterialColors" }, diff --git a/plugin/rbx_dom_lua/database.json b/plugin/rbx_dom_lua/database.json index 1856f932..add83531 100644 --- a/plugin/rbx_dom_lua/database.json +++ b/plugin/rbx_dom_lua/database.json @@ -1,9 +1,9 @@ { "Version": [ 0, - 700, + 708, 0, - 7000935 + 7080878 ], "Classes": { "Accessory": { @@ -1689,9 +1689,7 @@ }, "AnimationConstraint": { "Name": "AnimationConstraint", - "Tags": [ - "NotBrowsable" - ], + "Tags": [], "Superclass": "Constraint", "Properties": { "C0": { @@ -2070,6 +2068,19 @@ } } }, + "NodeId": { + "Name": "NodeId", + "Scriptability": "None", + "DataType": { + "Value": "String" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "NodeType": { "Name": "NodeType", "Scriptability": "ReadWrite", @@ -2100,6 +2111,9 @@ "InputPinData": { "BinaryString": "AQAAAAAAAAA=" }, + "NodeId": { + "String": "" + }, "NodeType": { "Enum": 0 }, @@ -3375,6 +3389,27 @@ } } }, + "AssetQualityService": { + "Name": "AssetQualityService", + "Tags": [ + "NotCreatable", + "NotReplicated", + "Service" + ], + "Superclass": "Instance", + "Properties": {}, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "AssetService": { "Name": "AssetService", "Tags": [ @@ -4432,6 +4467,7 @@ "Value": "Bool" }, "Tags": [ + "Hidden", "NotReplicated" ], "Kind": { @@ -4809,7 +4845,9 @@ "DataType": { "Value": "Ref" }, - "Tags": [], + "Tags": [ + "Hidden" + ], "Kind": { "Canonical": { "Serialization": "Serializes" @@ -5575,7 +5613,9 @@ "DataType": { "Value": "Ref" }, - "Tags": [], + "Tags": [ + "Hidden" + ], "Kind": { "Canonical": { "Serialization": "Serializes" @@ -5826,7 +5866,9 @@ "DataType": { "Value": "Bool" }, - "Tags": [], + "Tags": [ + "Hidden" + ], "Kind": { "Canonical": { "Serialization": "Serializes" @@ -9732,6 +9774,40 @@ } } }, + "BaseCoreGuiConfiguration": { + "Name": "BaseCoreGuiConfiguration", + "Tags": [ + "NotCreatable", + "NotReplicated" + ], + "Superclass": "Instance", + "Properties": { + "Enabled": { + "Name": "Enabled", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + } + }, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "BaseImportData": { "Name": "BaseImportData", "Tags": [ @@ -15427,6 +15503,40 @@ } } }, + "CapturesViewConfiguration": { + "Name": "CapturesViewConfiguration", + "Tags": [ + "NotCreatable", + "NotReplicated" + ], + "Superclass": "BaseCoreGuiConfiguration", + "Properties": { + "Open": { + "Name": "Open", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + } + }, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "CatalogPages": { "Name": "CatalogPages", "Tags": [ @@ -17739,6 +17849,55 @@ } } }, + "CompositeValueCurve": { + "Name": "CompositeValueCurve", + "Tags": [], + "Superclass": "Instance", + "Properties": { + "CurveType": { + "Name": "CurveType", + "Scriptability": "ReadWrite", + "DataType": { + "Enum": "CompositeValueCurveType" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + } + }, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "Attributes": { + "Attributes": {} + }, + "Capabilities": { + "SecurityCapabilities": 0 + }, + "CurveType": { + "Enum": 2 + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "Sandboxed": { + "Bool": false + }, + "SourceAssetId": { + "Int64": -1 + }, + "Tags": { + "Tags": [] + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "CompressorSoundEffect": { "Name": "CompressorSoundEffect", "Tags": [], @@ -18912,6 +19071,67 @@ } } }, + "CoreGuiConfiguration": { + "Name": "CoreGuiConfiguration", + "Tags": [ + "NotCreatable", + "NotReplicated", + "Service" + ], + "Superclass": "Instance", + "Properties": { + "CapturesViewConfiguration": { + "Name": "CapturesViewConfiguration", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Ref" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "PlayerListConfiguration": { + "Name": "PlayerListConfiguration", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Ref" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "SelfViewConfiguration": { + "Name": "SelfViewConfiguration", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Ref" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + } + }, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "CorePackages": { "Name": "CorePackages", "Tags": [ @@ -20332,6 +20552,22 @@ } } }, + "RunService": { + "Name": "RunService", + "Scriptability": "Read", + "DataType": { + "Value": "Ref" + }, + "Tags": [ + "NotReplicated", + "ReadOnly" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, "VIPServerId": { "Name": "VIPServerId", "Scriptability": "Read", @@ -23456,7 +23692,7 @@ "Orientation": { "Vector3": [ -0.0, - 179.99998, + 180.0, 90.0 ] }, @@ -25081,6 +25317,22 @@ } } }, + "InternalOverrideFACSData": { + "Name": "InternalOverrideFACSData", + "Scriptability": "None", + "DataType": { + "Value": "BinaryString" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "JawDrop": { "Name": "JawDrop", "Scriptability": "ReadWrite", @@ -25710,6 +25962,9 @@ "HistoryId": { "UniqueId": "00000000000000000000000000000000" }, + "InternalOverrideFACSData": { + "BinaryString": "" + }, "Sandboxed": { "Bool": false }, @@ -26043,6 +26298,27 @@ } } }, + "FileManagerService": { + "Name": "FileManagerService", + "Tags": [ + "NotCreatable", + "NotReplicated", + "Service" + ], + "Superclass": "Instance", + "Properties": {}, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "FileMesh": { "Name": "FileMesh", "Tags": [], @@ -29787,6 +30063,21 @@ } } }, + "DisplayScalingMode": { + "Name": "DisplayScalingMode", + "Scriptability": "None", + "DataType": { + "Enum": "DisplayScalingMode" + }, + "Tags": [ + "Hidden" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "GuiNavigationEnabled": { "Name": "GuiNavigationEnabled", "Scriptability": "ReadWrite", @@ -34018,6 +34309,19 @@ } } }, + "UseAvatarSettings": { + "Name": "UseAvatarSettings", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "WaistAccessory": { "Name": "WaistAccessory", "Scriptability": "ReadWrite", @@ -34139,6 +34443,9 @@ "UniqueId": { "UniqueId": "00000000000000000000000000000000" }, + "UseAvatarSettings": { + "Bool": false + }, "WalkAnimation": { "Int64": 0 }, @@ -38060,6 +38367,23 @@ "Serialization": "Serializes" } } + }, + "ViewportPositionState": { + "Name": "ViewportPositionState", + "Scriptability": "None", + "DataType": { + "Value": "Vector2" + }, + "Tags": [ + "NotReplicated", + "NotScriptable", + "ReadOnly" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } } }, "DefaultProperties": { @@ -38165,6 +38489,19 @@ } } }, + "PointerIndex": { + "Name": "PointerIndex", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Int32" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "PressedThreshold": { "Name": "PressedThreshold", "Scriptability": "ReadWrite", @@ -38178,6 +38515,19 @@ } } }, + "PrimaryModifier": { + "Name": "PrimaryModifier", + "Scriptability": "ReadWrite", + "DataType": { + "Enum": "KeyCode" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "ReleasedThreshold": { "Name": "ReleasedThreshold", "Scriptability": "ReadWrite", @@ -38230,6 +38580,19 @@ } } }, + "SecondaryModifier": { + "Name": "SecondaryModifier", + "Scriptability": "ReadWrite", + "DataType": { + "Enum": "KeyCode" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "UIButton": { "Name": "UIButton", "Scriptability": "ReadWrite", @@ -38268,6 +38631,19 @@ "Serialization": "Serializes" } } + }, + "Vector3Scale": { + "Name": "Vector3Scale", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Vector3" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } } }, "DefaultProperties": { @@ -38298,9 +38674,15 @@ "Left": { "Enum": 0 }, + "PointerIndex": { + "Int32": 0 + }, "PressedThreshold": { "Float32": 0.5 }, + "PrimaryModifier": { + "Enum": 0 + }, "ReleasedThreshold": { "Float32": 0.2 }, @@ -38316,6 +38698,9 @@ "Scale": { "Float32": 1.0 }, + "SecondaryModifier": { + "Enum": 0 + }, "SourceAssetId": { "Int64": -1 }, @@ -38333,6 +38718,13 @@ 1.0, 1.0 ] + }, + "Vector3Scale": { + "Vector3": [ + 1.0, + 1.0, + 1.0 + ] } } }, @@ -38744,6 +39136,23 @@ } } }, + "PredictionMode": { + "Name": "PredictionMode", + "Scriptability": "None", + "DataType": { + "Enum": "PredictionMode" + }, + "Tags": [ + "NotReplicated", + "NotScriptable", + "ReadOnly" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, "PropertyStatusStudio": { "Name": "PropertyStatusStudio", "Scriptability": "None", @@ -38939,6 +39348,27 @@ } } }, + "InstanceFileSyncService": { + "Name": "InstanceFileSyncService", + "Tags": [ + "NotCreatable", + "NotReplicated", + "Service" + ], + "Superclass": "Instance", + "Properties": {}, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "IntConstrainedValue": { "Name": "IntConstrainedValue", "Tags": [ @@ -39388,7 +39818,7 @@ "Bool": false }, "PhysicalConfigData": { - "SharedString": "" + "SharedString": "Q1NHUEhTAAAAAEJMT0NL" }, "PhysicsData": { "BinaryString": "" @@ -41657,6 +42087,38 @@ "Serialization": "DoesNotSerialize" } } + }, + "SlimAnimationSource": { + "Name": "SlimAnimationSource", + "Scriptability": "None", + "DataType": { + "Value": "Ref" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimReplicationTimestampSec": { + "Name": "SlimReplicationTimestampSec", + "Scriptability": "None", + "DataType": { + "Value": "Float64" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } } }, "DefaultProperties": { @@ -42034,6 +42496,102 @@ "Properties": {}, "DefaultProperties": {} }, + "MakeupDescription": { + "Name": "MakeupDescription", + "Tags": [ + "NotBrowsable" + ], + "Superclass": "Instance", + "Properties": { + "AssetId": { + "Name": "AssetId", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Int64" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "Instance": { + "Name": "Instance", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Ref" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "MakeupType": { + "Name": "MakeupType", + "Scriptability": "ReadWrite", + "DataType": { + "Enum": "MakeupType" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "Order": { + "Name": "Order", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Int32" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + } + }, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "AssetId": { + "Int64": 0 + }, + "Attributes": { + "Attributes": {} + }, + "Capabilities": { + "SecurityCapabilities": 0 + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "MakeupType": { + "Enum": 0 + }, + "Order": { + "Int32": 0 + }, + "Sandboxed": { + "Bool": false + }, + "SourceAssetId": { + "Int64": -1 + }, + "Tags": { + "Tags": [] + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "ManualGlue": { "Name": "ManualGlue", "Tags": [ @@ -43261,6 +43819,21 @@ "Tags": [], "Superclass": "Instance", "Properties": { + "AlphaMode": { + "Name": "AlphaMode", + "Scriptability": "ReadWrite", + "DataType": { + "Enum": "AlphaMode" + }, + "Tags": [ + "NotBrowsable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "AvgMetalness": { "Name": "AvgMetalness", "Scriptability": "None", @@ -43528,6 +44101,9 @@ } }, "DefaultProperties": { + "AlphaMode": { + "Enum": 3 + }, "Archivable": { "Bool": true }, @@ -44509,7 +45085,7 @@ "Content": "None" }, "PhysicalConfigData": { - "SharedString": "" + "SharedString": "Q1NHUEhTAAAAAEJMT0NL" }, "PhysicsData": { "BinaryString": "" @@ -45244,6 +45820,22 @@ } } }, + "SlimAnimationTarget": { + "Name": "SlimAnimationTarget", + "Scriptability": "None", + "DataType": { + "Value": "Ref" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, "SlimHash": { "Name": "SlimHash", "Scriptability": "None", @@ -46356,7 +46948,7 @@ "Bool": false }, "PhysicalConfigData": { - "SharedString": "" + "SharedString": "Q1NHUEhTAAAAAEJMT0NL" }, "PhysicsData": { "BinaryString": "" @@ -48752,7 +49344,7 @@ "Bool": false }, "PhysicalConfigData": { - "SharedString": "" + "SharedString": "Q1NHUEhTAAAAAEJMT0NL" }, "PhysicsData": { "BinaryString": "" @@ -51314,6 +51906,26 @@ } } }, + "PlatformLibraries": { + "Name": "PlatformLibraries", + "Tags": [ + "NotCreatable", + "Service" + ], + "Superclass": "Instance", + "Properties": {}, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "Player": { "Name": "Player", "Tags": [], @@ -51382,6 +51994,22 @@ } } }, + "CameraFrustumRequested": { + "Name": "CameraFrustumRequested", + "Scriptability": "None", + "DataType": { + "Value": "Bool" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, "CameraMaxZoomDistance": { "Name": "CameraMaxZoomDistance", "Scriptability": "ReadWrite", @@ -51933,6 +52561,22 @@ } } }, + "NeedRegionalFallback": { + "Name": "NeedRegionalFallback", + "Scriptability": "None", + "DataType": { + "Value": "Bool" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, "Neutral": { "Name": "Neutral", "Scriptability": "ReadWrite", @@ -52820,6 +53464,40 @@ } } }, + "PlayerListConfiguration": { + "Name": "PlayerListConfiguration", + "Tags": [ + "NotCreatable", + "NotReplicated" + ], + "Superclass": "BaseCoreGuiConfiguration", + "Properties": { + "Open": { + "Name": "Open", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + } + }, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "PlayerMouse": { "Name": "PlayerMouse", "Tags": [ @@ -57451,6 +58129,230 @@ } } }, + "RolloutValidation": { + "Name": "RolloutValidation", + "Tags": [ + "NotCreatable" + ], + "Superclass": "Instance", + "Properties": { + "FirstBinaryExpectedValue": { + "Name": "FirstBinaryExpectedValue", + "Scriptability": "None", + "DataType": { + "Value": "BinaryString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "FirstBinaryString": { + "Name": "FirstBinaryString", + "Scriptability": "None", + "DataType": { + "Value": "BinaryString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "FirstSharedExpectedValue": { + "Name": "FirstSharedExpectedValue", + "Scriptability": "None", + "DataType": { + "Value": "SharedString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "FirstSharedString": { + "Name": "FirstSharedString", + "Scriptability": "None", + "DataType": { + "Value": "SharedString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "SecondBinaryExpectedValue": { + "Name": "SecondBinaryExpectedValue", + "Scriptability": "None", + "DataType": { + "Value": "BinaryString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "SecondBinaryString": { + "Name": "SecondBinaryString", + "Scriptability": "None", + "DataType": { + "Value": "BinaryString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "SecondSharedExpectedValue": { + "Name": "SecondSharedExpectedValue", + "Scriptability": "None", + "DataType": { + "Value": "SharedString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "SecondSharedString": { + "Name": "SecondSharedString", + "Scriptability": "None", + "DataType": { + "Value": "SharedString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "ThirdBinaryExpectedValue": { + "Name": "ThirdBinaryExpectedValue", + "Scriptability": "None", + "DataType": { + "Value": "BinaryString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "ThirdBinaryString": { + "Name": "ThirdBinaryString", + "Scriptability": "None", + "DataType": { + "Value": "BinaryString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "ThirdSharedExpectedValue": { + "Name": "ThirdSharedExpectedValue", + "Scriptability": "None", + "DataType": { + "Value": "SharedString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "ThirdSharedString": { + "Name": "ThirdSharedString", + "Scriptability": "None", + "DataType": { + "Value": "SharedString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + } + }, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "RomarkRbxAnalyticsService": { "Name": "RomarkRbxAnalyticsService", "Tags": [ @@ -60791,6 +61693,40 @@ } } }, + "SelfViewConfiguration": { + "Name": "SelfViewConfiguration", + "Tags": [ + "NotCreatable", + "NotReplicated" + ], + "Superclass": "BaseCoreGuiConfiguration", + "Properties": { + "Open": { + "Name": "Open", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + } + }, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "SensorBase": { "Name": "SensorBase", "Tags": [ @@ -61732,6 +62668,19 @@ } } }, + "MoonTextureContent": { + "Name": "MoonTextureContent", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Content" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "MoonTextureId": { "Name": "MoonTextureId", "Scriptability": "ReadWrite", @@ -61745,6 +62694,19 @@ } } }, + "SkyboxBackContent": { + "Name": "SkyboxBackContent", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Content" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "SkyboxBk": { "Name": "SkyboxBk", "Scriptability": "ReadWrite", @@ -61771,6 +62733,32 @@ } } }, + "SkyboxDownContent": { + "Name": "SkyboxDownContent", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Content" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "SkyboxFrontContent": { + "Name": "SkyboxFrontContent", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Content" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "SkyboxFt": { "Name": "SkyboxFt", "Scriptability": "ReadWrite", @@ -61784,6 +62772,19 @@ } } }, + "SkyboxLeftContent": { + "Name": "SkyboxLeftContent", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Content" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "SkyboxLf": { "Name": "SkyboxLf", "Scriptability": "ReadWrite", @@ -61810,6 +62811,19 @@ } } }, + "SkyboxRightContent": { + "Name": "SkyboxRightContent", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Content" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "SkyboxRt": { "Name": "SkyboxRt", "Scriptability": "ReadWrite", @@ -61836,6 +62850,19 @@ } } }, + "SkyboxUpContent": { + "Name": "SkyboxUpContent", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Content" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "StarCount": { "Name": "StarCount", "Scriptability": "ReadWrite", @@ -61862,6 +62889,19 @@ } } }, + "SunTextureContent": { + "Name": "SunTextureContent", + "Scriptability": "ReadWrite", + "DataType": { + "Value": "Content" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "SunTextureId": { "Name": "SunTextureId", "Scriptability": "ReadWrite", @@ -62166,6 +63206,638 @@ } } }, + "SlimAnimationDataEntity": { + "Name": "SlimAnimationDataEntity", + "Tags": [ + "NotCreatable" + ], + "Superclass": "Instance", + "Properties": { + "EntitySource": { + "Name": "EntitySource", + "Scriptability": "None", + "DataType": { + "Value": "Ref" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "Handle": { + "Name": "Handle", + "Scriptability": "None", + "DataType": { + "Value": "Int32" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "IsSlimEnabled": { + "Name": "IsSlimEnabled", + "Scriptability": "None", + "DataType": { + "Value": "Bool" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "NumBones": { + "Name": "NumBones", + "Scriptability": "None", + "DataType": { + "Value": "Int32" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone0": { + "Name": "SlimBone0", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone1": { + "Name": "SlimBone1", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone10": { + "Name": "SlimBone10", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone11": { + "Name": "SlimBone11", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone12": { + "Name": "SlimBone12", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone13": { + "Name": "SlimBone13", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone14": { + "Name": "SlimBone14", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone15": { + "Name": "SlimBone15", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone16": { + "Name": "SlimBone16", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone17": { + "Name": "SlimBone17", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone18": { + "Name": "SlimBone18", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone19": { + "Name": "SlimBone19", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone2": { + "Name": "SlimBone2", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone20": { + "Name": "SlimBone20", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone21": { + "Name": "SlimBone21", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone22": { + "Name": "SlimBone22", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone23": { + "Name": "SlimBone23", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone24": { + "Name": "SlimBone24", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone25": { + "Name": "SlimBone25", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone26": { + "Name": "SlimBone26", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone27": { + "Name": "SlimBone27", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone28": { + "Name": "SlimBone28", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone29": { + "Name": "SlimBone29", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone3": { + "Name": "SlimBone3", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone30": { + "Name": "SlimBone30", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone31": { + "Name": "SlimBone31", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone4": { + "Name": "SlimBone4", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone5": { + "Name": "SlimBone5", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone6": { + "Name": "SlimBone6", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone7": { + "Name": "SlimBone7", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone8": { + "Name": "SlimBone8", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimBone9": { + "Name": "SlimBone9", + "Scriptability": "None", + "DataType": { + "Value": "CFrame" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "SlimReplicationTimestampSec": { + "Name": "SlimReplicationTimestampSec", + "Scriptability": "None", + "DataType": { + "Value": "Float64" + }, + "Tags": [ + "Hidden", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + } + }, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, + "SlimAnimationReplicationService": { + "Name": "SlimAnimationReplicationService", + "Tags": [ + "NotCreatable", + "Service" + ], + "Superclass": "Instance", + "Properties": {}, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "SlimContentProvider": { "Name": "SlimContentProvider", "Tags": [ @@ -62186,6 +63858,27 @@ } } }, + "SlimReplicationService": { + "Name": "SlimReplicationService", + "Tags": [ + "NotCreatable", + "NotReplicated", + "Service" + ], + "Superclass": "Instance", + "Properties": {}, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "SlimService": { "Name": "SlimService", "Tags": [ @@ -62614,6 +64307,7 @@ "Value": "Int32" }, "Tags": [ + "Hidden", "NotBrowsable", "NotReplicated", "ReadOnly" @@ -62661,7 +64355,9 @@ "DataType": { "Value": "Bool" }, - "Tags": [], + "Tags": [ + "Hidden" + ], "Kind": { "Canonical": { "Serialization": "Serializes" @@ -65052,7 +66748,6 @@ "Enum": "RolloutState" }, "Tags": [ - "NotBrowsable", "NotReplicated", "NotScriptable" ], @@ -65201,9 +66896,7 @@ "DataType": { "Value": "Bool" }, - "Tags": [ - "NotBrowsable" - ], + "Tags": [], "Kind": { "Canonical": { "Serialization": "Serializes" @@ -67034,6 +68727,19 @@ } } }, + "CommandBarEnterExec": { + "Name": "CommandBarEnterExec", + "Scriptability": "None", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "CommandBarLocalState": { "Name": "CommandBarLocalState", "Scriptability": "ReadWrite", @@ -67175,6 +68881,19 @@ } } }, + "DraggerLengthFactor": { + "Name": "DraggerLengthFactor", + "Scriptability": "None", + "DataType": { + "Value": "Float32" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "DraggerMajorGridIncrement": { "Name": "DraggerMajorGridIncrement", "Scriptability": "None", @@ -67214,6 +68933,32 @@ } } }, + "DraggerScaleFactor": { + "Name": "DraggerScaleFactor", + "Scriptability": "None", + "DataType": { + "Value": "Float32" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "DraggerShowAxisTicks": { + "Name": "DraggerShowAxisTicks", + "Scriptability": "None", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "DraggerShowHoverRuler": { "Name": "DraggerShowHoverRuler", "Scriptability": "None", @@ -67240,6 +68985,32 @@ } } }, + "DraggerShowNegativeAxes": { + "Name": "DraggerShowNegativeAxes", + "Scriptability": "None", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "DraggerShowPlanes": { + "Name": "DraggerShowPlanes", + "Scriptability": "None", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "DraggerShowTargetSnap": { "Name": "DraggerShowTargetSnap", "Scriptability": "None", @@ -67253,6 +69024,32 @@ } } }, + "DraggerShowTrackball": { + "Name": "DraggerShowTrackball", + "Scriptability": "None", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "DraggerShowWhileDragging": { + "Name": "DraggerShowWhileDragging", + "Scriptability": "None", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "DraggerSoftSnapMarginFactor": { "Name": "DraggerSoftSnapMarginFactor", "Scriptability": "None", @@ -68742,6 +70539,19 @@ } } }, + "UseDefaultExternalEditor": { + "Name": "UseDefaultExternalEditor", + "Scriptability": "None", + "DataType": { + "Value": "Bool" + }, + "Tags": [], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "Warning Color": { "Name": "Warning Color", "Scriptability": "ReadWrite", @@ -69813,14 +71623,17 @@ "Properties": { "AspectRatioRange": { "Name": "AspectRatioRange", - "Scriptability": "ReadWrite", + "Scriptability": "None", "DataType": { "Value": "NumberRange" }, - "Tags": [], + "Tags": [ + "NotReplicated", + "NotScriptable" + ], "Kind": { "Canonical": { - "Serialization": "Serializes" + "Serialization": "DoesNotSerialize" } } }, @@ -69858,27 +71671,81 @@ }, "MaxSize": { "Name": "MaxSize", - "Scriptability": "ReadWrite", + "Scriptability": "None", "DataType": { "Value": "Vector2" }, - "Tags": [], + "Tags": [ + "NotReplicated", + "NotScriptable" + ], "Kind": { "Canonical": { - "Serialization": "Serializes" + "Serialization": "DoesNotSerialize" } } }, "MinSize": { "Name": "MinSize", - "Scriptability": "ReadWrite", + "Scriptability": "None", "DataType": { "Value": "Vector2" }, - "Tags": [], + "Tags": [ + "NotReplicated", + "NotScriptable" + ], "Kind": { "Canonical": { - "Serialization": "Serializes" + "Serialization": "DoesNotSerialize" + } + } + }, + "PreferredInput": { + "Name": "PreferredInput", + "Scriptability": "None", + "DataType": { + "Enum": "PreferredInput" + }, + "Tags": [ + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "ReducedMotionEnabled": { + "Name": "ReducedMotionEnabled", + "Scriptability": "None", + "DataType": { + "Value": "Bool" + }, + "Tags": [ + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" + } + } + }, + "ViewportDisplaySize": { + "Name": "ViewportDisplaySize", + "Scriptability": "None", + "DataType": { + "Enum": "DisplaySize" + }, + "Tags": [ + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "DoesNotSerialize" } } } @@ -72105,113 +73972,7 @@ "Enum": 256 }, "MaterialColors": { - "MaterialColors": { - "Grass": [ - 106, - 127, - 63 - ], - "Slate": [ - 63, - 127, - 107 - ], - "Concrete": [ - 127, - 102, - 63 - ], - "Brick": [ - 138, - 86, - 62 - ], - "Sand": [ - 143, - 126, - 95 - ], - "WoodPlanks": [ - 139, - 109, - 79 - ], - "Rock": [ - 102, - 108, - 111 - ], - "Glacier": [ - 101, - 176, - 234 - ], - "Snow": [ - 195, - 199, - 218 - ], - "Sandstone": [ - 137, - 90, - 71 - ], - "Mud": [ - 58, - 46, - 36 - ], - "Basalt": [ - 30, - 30, - 37 - ], - "Ground": [ - 102, - 92, - 59 - ], - "CrackedLava": [ - 232, - 156, - 74 - ], - "Asphalt": [ - 115, - 123, - 107 - ], - "Cobblestone": [ - 132, - 123, - 90 - ], - "Ice": [ - 129, - 194, - 224 - ], - "LeafyGrass": [ - 115, - 132, - 74 - ], - "Salt": [ - 198, - 189, - 181 - ], - "Limestone": [ - 206, - 173, - 148 - ], - "Pavement": [ - 148, - 148, - 140 - ] - } + "MaterialColors": {} }, "MaterialVariant": { "String": "" @@ -77106,6 +78867,26 @@ } } }, + "TraceRouteService": { + "Name": "TraceRouteService", + "Tags": [ + "NotCreatable", + "Service" + ], + "Superclass": "Instance", + "Properties": {}, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "TracerService": { "Name": "TracerService", "Tags": [ @@ -80669,7 +82450,7 @@ "Bool": false }, "PhysicalConfigData": { - "SharedString": "" + "SharedString": "Q1NHUEhTAAAAAEJMT0NL" }, "PhysicsData": { "BinaryString": "" @@ -84040,9 +85821,7 @@ }, "VideoDisplay": { "Name": "VideoDisplay", - "Tags": [ - "NotBrowsable" - ], + "Tags": [], "Superclass": "GuiObject", "Properties": { "ResampleMode": { @@ -84635,9 +86414,7 @@ }, "VideoPlayer": { "Name": "VideoPlayer", - "Tags": [ - "NotBrowsable" - ], + "Tags": [], "Superclass": "Instance", "Properties": { "AutoLoadInStudio": { @@ -86729,7 +88506,7 @@ }, "AuthorityMode": { "Name": "AuthorityMode", - "Scriptability": "ReadWrite", + "Scriptability": "None", "DataType": { "Enum": "AuthorityMode" }, @@ -87069,6 +88846,21 @@ } } }, + "NextGenerationReplication": { + "Name": "NextGenerationReplication", + "Scriptability": "None", + "DataType": { + "Enum": "RolloutState" + }, + "Tags": [ + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "PathfindingUseImprovedSearch": { "Name": "PathfindingUseImprovedSearch", "Scriptability": "None", @@ -87129,6 +88921,21 @@ } } }, + "PlayerScriptsUseInputActionSystem": { + "Name": "PlayerScriptsUseInputActionSystem", + "Scriptability": "None", + "DataType": { + "Enum": "RolloutState" + }, + "Tags": [ + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "PrimalPhysicsSolver": { "Name": "PrimalPhysicsSolver", "Scriptability": "None", @@ -87202,6 +89009,74 @@ } } }, + "RolloutValidationBinaryExpectedValue": { + "Name": "RolloutValidationBinaryExpectedValue", + "Scriptability": "None", + "DataType": { + "Value": "BinaryString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "RolloutValidationBinaryString": { + "Name": "RolloutValidationBinaryString", + "Scriptability": "None", + "DataType": { + "Value": "BinaryString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "RolloutValidationSharedExpectedValue": { + "Name": "RolloutValidationSharedExpectedValue", + "Scriptability": "None", + "DataType": { + "Value": "SharedString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, + "RolloutValidationSharedString": { + "Name": "RolloutValidationSharedString", + "Scriptability": "None", + "DataType": { + "Value": "SharedString" + }, + "Tags": [ + "Hidden", + "NotReplicated", + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "SandboxedInstanceMode": { "Name": "SandboxedInstanceMode", "Scriptability": "None", @@ -87403,6 +89278,21 @@ } } }, + "UseFixedSimulation": { + "Name": "UseFixedSimulation", + "Scriptability": "None", + "DataType": { + "Enum": "RolloutState" + }, + "Tags": [ + "NotScriptable" + ], + "Kind": { + "Canonical": { + "Serialization": "Serializes" + } + } + }, "UseNewLuauTypeSolver": { "Name": "UseNewLuauTypeSolver", "Scriptability": "None", @@ -87448,7 +89338,7 @@ "Enum": 0 }, "CollisionGroupData": { - "BinaryString": "AQEABP////8HRGVmYXVsdA==" + "BinaryString": "AQQABP////8HRGVmYXVsdAEE+////xBGb3JnZU1vdXNlSWdub3JlAgT9////EFN0dWRpb1NlbGVjdGFibGUDBPf///8LRm9yZ2VEZWJyaXM=" }, "DistributedGameTime": { "Float64": 0.0 @@ -87538,6 +89428,9 @@ "NeedsPivotMigration": { "Bool": false }, + "NextGenerationReplication": { + "Enum": 0 + }, "PathfindingUseImprovedSearch": { "Enum": 0 }, @@ -87550,6 +89443,9 @@ "PlayerCharacterDestroyBehavior": { "Enum": 0 }, + "PlayerScriptsUseInputActionSystem": { + "Enum": 0 + }, "PrimalPhysicsSolver": { "Enum": 0 }, @@ -87565,6 +89461,18 @@ "Retargeting": { "Enum": 0 }, + "RolloutValidationBinaryExpectedValue": { + "BinaryString": "" + }, + "RolloutValidationBinaryString": { + "BinaryString": "" + }, + "RolloutValidationSharedExpectedValue": { + "SharedString": "" + }, + "RolloutValidationSharedString": { + "SharedString": "" + }, "Sandboxed": { "Bool": false }, @@ -87613,6 +89521,9 @@ "UniqueId": { "UniqueId": "00000000000000000000000000000000" }, + "UseFixedSimulation": { + "Enum": 0 + }, "UseNewLuauTypeSolver": { "Enum": 2 }, @@ -87794,6 +89705,27 @@ } } }, + "WrapDeformMeshProvider": { + "Name": "WrapDeformMeshProvider", + "Tags": [ + "NotCreatable", + "NotReplicated", + "Service" + ], + "Superclass": "Instance", + "Properties": {}, + "DefaultProperties": { + "Archivable": { + "Bool": true + }, + "HistoryId": { + "UniqueId": "00000000000000000000000000000000" + }, + "UniqueId": { + "UniqueId": "00000000000000000000000000000000" + } + } + }, "WrapDeformer": { "Name": "WrapDeformer", "Tags": [], @@ -88432,9 +90364,7 @@ }, "WrapTextureTransfer": { "Name": "WrapTextureTransfer", - "Tags": [ - "NotBrowsable" - ], + "Tags": [], "Superclass": "Instance", "Properties": { "ReferenceCageMeshContent": { @@ -88699,6 +90629,7 @@ "AlphaMode": { "name": "AlphaMode", "items": { + "Opaque": 3, "Overlay": 0, "TintMask": 2, "Transparency": 1 @@ -88785,6 +90716,21 @@ "Timeout": 10 } }, + "AnimationNodeBlend2DInputMode": { + "name": "AnimationNodeBlend2DInputMode", + "items": { + "Cartesian": 0, + "Polar": 1 + } + }, + "AnimationNodeInterruptible": { + "name": "AnimationNodeInterruptible", + "items": { + "Always": 0, + "ClipFinished": 1, + "Expression": 2 + } + }, "AnimationNodePlayMode": { "name": "AnimationNodePlayMode", "items": { @@ -88821,6 +90767,13 @@ "SubtractNode": 13 } }, + "AnimationNodeWaitFor": { + "name": "AnimationNodeWaitFor", + "items": { + "ClipFinished": 0, + "Expression": 1 + } + }, "AnimationPriority": { "name": "AnimationPriority", "items": { @@ -89568,13 +91521,6 @@ "Relevance": 1 } }, - "CatalogTimedOptionFilter": { - "name": "CatalogTimedOptionFilter", - "items": { - "All": 1, - "TimedOptionOnly": 2 - } - }, "CellBlock": { "name": "CellBlock", "items": { @@ -89807,6 +91753,19 @@ "TriggerForIncompleteCompletions": 3 } }, + "CompositeValueCurveType": { + "name": "CompositeValueCurveType", + "items": { + "ColorHSV": 1, + "ColorRGB": 0, + "NumberRange": 2, + "Rect": 3, + "UDim": 4, + "UDim2": 5, + "Vector2": 6, + "Vector3": 7 + } + }, "CompressionAlgorithm": { "name": "CompressionAlgorithm", "items": { @@ -89853,8 +91812,13 @@ "DisconnectClientFailure": 284, "DisconnectClientRequest": 285, "DisconnectCloudEditKick": 270, + "DisconnectCollaboratorNotAgeVerified": 310, + "DisconnectCollaboratorOwnerActionRequired": 312, "DisconnectCollaboratorPermissionRevoked": 294, + "DisconnectCollaboratorTooManyCollaborators": 313, + "DisconnectCollaboratorTrustedConnectionsRequired": 311, "DisconnectCollaboratorUnderage": 295, + "DisconnectCollaboratorUnknownError": 314, "DisconnectConnectionLost": 277, "DisconnectDevMaintenance": 274, "DisconnectDuplicatePlayer": 264, @@ -89973,6 +91937,7 @@ "name": "CoreGuiType", "items": { "All": 4, + "AvatarSwitcher": 8, "Backpack": 2, "Captures": 7, "Chat": 3, @@ -90030,6 +91995,14 @@ "Follow": 2 } }, + "DataModelExtractorFileType": { + "name": "DataModelExtractorFileType", + "items": { + "FirstSlice": 1, + "NonFirstSlice": 2, + "PlaceFile": 0 + } + }, "DataStoreRequestType": { "name": "DataStoreRequestType", "items": { @@ -90245,6 +92218,14 @@ "Neutral": 0 } }, + "DisplayScalingMode": { + "name": "DisplayScalingMode", + "items": { + "Default": 0, + "Legacy": 1, + "Responsive": 2 + } + }, "DisplaySize": { "name": "DisplaySize", "items": { @@ -90361,6 +92342,14 @@ "WhenScrollable": 0 } }, + "EngineFolder": { + "name": "EngineFolder", + "items": { + "Logs": 2, + "Screenshots": 0, + "Videos": 1 + } + }, "EnviromentalPhysicsThrottle": { "name": "EnviromentalPhysicsThrottle", "items": { @@ -90562,6 +92551,13 @@ "Rejected": 1 } }, + "FilterType": { + "name": "FilterType", + "items": { + "Exclude": 0, + "Include": 1 + } + }, "FinishRecordingOperation": { "name": "FinishRecordingOperation", "items": { @@ -90947,6 +92943,7 @@ "ConnectionClosed": 13, "CreatorEnvironmentsNotSupportedByService": 15, "DnsResolve": 2, + "InactivityTimeout": 16, "InvalidRedirect": 7, "InvalidUrl": 1, "NetFail": 8, @@ -90956,6 +92953,7 @@ "SslConnectFail": 10, "SslVerificationFail": 11, "TimedOut": 5, + "TooManyOutstandingRequests": 17, "TooManyRedirects": 6, "Unknown": 12 } @@ -91125,6 +93123,16 @@ "Sin": 13 } }, + "InstanceFileSyncStatus": { + "name": "InstanceFileSyncStatus", + "items": { + "AncestorErrored": 4, + "Errored": 1, + "NotSynced": 0, + "SyncedAsDescendant": 3, + "SyncedAsRoot": 2 + } + }, "IntermediateMeshGenerationResult": { "name": "IntermediateMeshGenerationResult", "items": { @@ -91278,11 +93286,13 @@ "Minus": 45, "Mode": 313, "MouseBackButton": 1029, + "MouseDelta": 1048, "MouseLeftButton": 1026, "MouseMiddleButton": 1028, "MouseNoButton": 1030, "MousePosition": 1033, "MouseRightButton": 1027, + "MouseWheel": 1035, "MouseX": 1031, "MouseY": 1032, "N": 110, @@ -91337,6 +93347,9 @@ "Thumbstick2Right": 1025, "Thumbstick2Up": 1022, "Tilde": 126, + "Touch": 1034, + "TrackpadPan": 1040, + "TrackpadPinch": 1045, "Two": 50, "U": 117, "Underscore": 95, @@ -91602,6 +93615,14 @@ "Strict": 3 } }, + "MakeupType": { + "name": "MakeupType", + "items": { + "Eye": 2, + "Face": 0, + "Lip": 1 + } + }, "MarketplaceBulkPurchasePromptStatus": { "name": "MarketplaceBulkPurchasePromptStatus", "items": { @@ -92160,7 +94181,7 @@ "Linux": 13, "MetaOS": 20, "NX": 9, - "None": 21, + "None": 22, "OSX": 1, "Ouya": 10, "PS3": 6, @@ -92168,6 +94189,7 @@ "PS5": 19, "SteamOS": 14, "UWP": 18, + "Web": 21, "WebOS": 15, "WiiU": 8, "Windows": 0, @@ -92589,6 +94611,7 @@ "items": { "Active": 131072, "Hover": 262144, + "NegatedPart": 1048576, "Selection": 524288, "Standard": 0 } @@ -92601,6 +94624,13 @@ "RotationsReset": 1 } }, + "RestPoseModel": { + "name": "RestPoseModel", + "items": { + "FromRigInACE": 0, + "FromRigInFile": 1 + } + }, "ReturnKeyType": { "name": "ReturnKeyType", "items": { @@ -92665,6 +94695,34 @@ "Ungroup": 8 } }, + "RigLabel": { + "name": "RigLabel", + "items": { + "Chest": 4, + "HeadBase": 6, + "Invalid": 0, + "LeftAnkle": 17, + "LeftClavicle": 7, + "LeftElbow": 9, + "LeftHip": 15, + "LeftKnee": 16, + "LeftShoulder": 8, + "LeftToes": 18, + "LeftWrist": 10, + "Neck": 5, + "Pelvis": 2, + "RightAnkle": 21, + "RightClavicle": 11, + "RightElbow": 13, + "RightHip": 19, + "RightKnee": 20, + "RightShoulder": 12, + "RightToes": 22, + "RightWrist": 14, + "Root": 1, + "Waist": 3 + } + }, "RigScale": { "name": "RigScale", "items": { @@ -92858,6 +94916,15 @@ "Sensor": 4 } }, + "ScreenshotCaptureResult": { + "name": "ScreenshotCaptureResult", + "items": { + "NoDeviceSupport": 2, + "NoSpaceOnDevice": 3, + "OtherError": 1, + "Success": 0 + } + }, "ScrollBarInset": { "name": "ScrollBarInset", "items": { @@ -92879,27 +94946,41 @@ "items": { "AccessOutsideWrite": 2, "Animation": 15, + "AssetCreateUpdate": 37, + "AssetManagement": 34, + "AssetRead": 33, "AssetRequire": 3, "Assistant": 31, "Audio": 8, "Avatar": 16, + "AvatarBehavior": 50, "Basic": 7, "CSG": 13, "CapabilityControl": 22, + "Capture": 38, "Chat": 14, + "Consequences": 48, "CreateInstances": 6, "DataStore": 9, + "DynamicGeneration": 35, "Environment": 18, + "Groups": 46, "Input": 17, "InternalTest": 29, "LegacySound": 20, + "LoadOwnedAsset": 41, "LoadString": 4, "LocalUser": 24, + "Logging": 44, + "Material": 49, + "Monetization": 40, "Network": 10, "Physics": 11, + "PlatformAvatarEditing": 36, "Players": 21, "Plugin": 23, "PluginOrOpenCloud": 30, + "PromptExternalPurchase": 45, "RemoteCommand": 32, "RemoteEvent": 19, "RobloxEngine": 27, @@ -92907,6 +94988,10 @@ "RunClientScript": 0, "RunServerScript": 1, "ScriptGlobals": 5, + "SensitiveInput": 39, + "ServerCommunication": 43, + "Social": 42, + "Teleport": 47, "UI": 12, "Unassigned": 28, "WritePlayer": 25 @@ -93609,6 +95694,8 @@ "items": { "Default": 0, "Email": 3, + "NewPassword": 9, + "NewPasswordShown": 10, "NoSuggestions": 1, "Number": 2, "OneTimePassword": 8, diff --git a/plugin/rbx_dom_lua/init.lua b/plugin/rbx_dom_lua/init.lua index e26e3c2a..321b7b01 100644 --- a/plugin/rbx_dom_lua/init.lua +++ b/plugin/rbx_dom_lua/init.lua @@ -1,11 +1,147 @@ local database = require(script.database) local Error = require(script.Error) local PropertyDescriptor = require(script.PropertyDescriptor) +local EncodedValue = require(script.EncodedValue) + +-- Cache for default properties per class (includes inherited defaults). +local defaultPropertiesCache: { [string]: { [string]: any } } = {} + +-- Returns a table mapping property names to their encoded default values +-- for a given class, including inherited defaults from parent classes. +-- Subclass overrides take precedence. Results are cached per class. +local function findDefaultProperties(className: string): { [string]: any } + local cached = defaultPropertiesCache[className] + if cached then + return cached + end + + local defaults: { [string]: any } = {} + local currentClassName: string? = className + + repeat + local currentClass = database.Classes[currentClassName] + if currentClass == nil then + break + end + + local classDefaults = currentClass.DefaultProperties + if classDefaults then + for propName, encodedValue in classDefaults do + if defaults[propName] == nil then + defaults[propName] = encodedValue + end + end + end + + currentClassName = currentClass.Superclass + until currentClassName == nil + + defaultPropertiesCache[className] = defaults + return defaults +end + +-- Property names excluded from comparison (handled separately). +local EXCLUDED_PROP_NAMES: { [string]: boolean } = { + Tags = true, + Attributes = true, + Name = true, +} + +-- Encoded value type keys that cannot be compared during matching. +local EXCLUDED_ENCODED_TYPES: { [string]: boolean } = { + Ref = true, + UniqueId = true, + SharedString = true, + Region3 = true, +} + +-- Scriptabilities that allow reading from Studio instances. +local READABLE_SCRIPTABILITY: { [string]: boolean } = { + ReadWrite = true, + Read = true, +} + +-- Cache for class comparison keys (propNames + decoded defaults). +-- Computed once per ClassName, lives for the plugin's lifetime. +local classComparisonKeysCache: { [string]: any } = {} + +-- Returns a pre-computed comparison key set for a class: +-- propNames: ordered array of comparable property names +-- propNameSet: O(1) lookup set of the same names +-- defaults: map of propName → decoded native default value +-- +-- Filtered to: canonical, scriptable (readable), serializable, and +-- decodable. Excludes Tags, Attributes, Ref, UniqueId, SharedString. +-- All defaults are pre-decoded to native Roblox values. +local function getClassComparisonKeys(className: string) + local cached = classComparisonKeysCache[className] + if cached then + return cached + end + + local propNames: { string } = {} + local propNameSet: { [string]: boolean } = {} + local seenProps: { [string]: boolean } = {} + + -- Walk the class hierarchy for property metadata + local currentClassName: string? = className + repeat + local currentClass = database.Classes[currentClassName] + if currentClass == nil then + break + end + + for propertyName, propertyData in currentClass.Properties do + if seenProps[propertyName] then + continue + end + seenProps[propertyName] = true + + if EXCLUDED_PROP_NAMES[propertyName] then + continue + end + if not propertyData.Kind or not propertyData.Kind.Canonical then + continue + end + if not READABLE_SCRIPTABILITY[propertyData.Scriptability] then + continue + end + if propertyData.Kind.Canonical.Serialization == "DoesNotSerialize" then + continue + end + + table.insert(propNames, propertyName) + propNameSet[propertyName] = true + end + + currentClassName = currentClass.Superclass + until currentClassName == nil + + -- Pre-decode defaults for every comparable property + local defaults: { [string]: any } = {} + local encodedDefaults = findDefaultProperties(className) + for _, propName in propNames do + local encoded = encodedDefaults[propName] + if encoded then + local ty = next(encoded) + if not EXCLUDED_ENCODED_TYPES[ty] then + local pcallOk, decodeOk, decoded = pcall(EncodedValue.decode, encoded) + if pcallOk and decodeOk and decoded ~= nil then + defaults[propName] = decoded + end + end + end + end + + local result = table.freeze({ + propNames = table.freeze(propNames), + propNameSet = table.freeze(propNameSet), + defaults = table.freeze(defaults), + }) + classComparisonKeysCache[className] = result + return result +end --- Returns a class descriptor with all property names and metadata (including inherited ones) --- Returns nil if the class doesn't exist in the database --- The properties table maps property names to metadata (scriptability, serialization status) --- To get a full PropertyDescriptor for encoding, use findCanonicalPropertyDescriptor local function findClassDescriptor(className) local classData = database.Classes[className] if classData == nil then @@ -111,6 +247,8 @@ return { writeProperty = writeProperty, findCanonicalPropertyDescriptor = findCanonicalPropertyDescriptor, findClassDescriptor = findClassDescriptor, + findDefaultProperties = findDefaultProperties, + getClassComparisonKeys = getClassComparisonKeys, Error = Error, - EncodedValue = require(script.EncodedValue), + EncodedValue = EncodedValue, } diff --git a/plugin/src/ApiContext.lua b/plugin/src/ApiContext.lua index 7ed53fbc..61496366 100644 --- a/plugin/src/ApiContext.lua +++ b/plugin/src/ApiContext.lua @@ -24,20 +24,28 @@ local function rejectFailedRequests(response) return response end -local function rejectWrongProtocolVersion(infoResponseBody) +local function rejectWrongVersion(infoResponseBody) + local pluginVersion = Version.display(Config.version) + local serverVersion = infoResponseBody.serverVersion + + if serverVersion ~= pluginVersion then + local message = ( + "Found a server, but it's a different version than this plugin." + .. "\nThe Atlas plugin requires an exact version match with the server." + .. "\n\nYour plugin is version %s." + .. "\nYour server is version %s." + .. "\n\nPlease update your plugin or server so both are the same version." + ):format(pluginVersion, serverVersion) + + return Promise.reject(message) + end + if infoResponseBody.protocolVersion ~= Config.protocolVersion then local message = ( - "Found a server, but it's using a different protocol version, and is incompatible." - .. "\nMake sure you have matching versions of both the Atlas plugin and server!" - .. "\n\nYour client is version %s, with protocol version %s. It expects server version %s." - .. "\nYour server is version %s, with protocol version %s." - ):format( - Version.display(Config.version), - Config.protocolVersion, - Config.expectedServerVersionString, - infoResponseBody.serverVersion, - infoResponseBody.protocolVersion - ) + "Found a server with the same version, but a different protocol version." + .. "\nThis should not happen — please report this as a bug." + .. "\n\nPlugin protocol: %s, Server protocol: %s" + ):format(Config.protocolVersion, infoResponseBody.protocolVersion) return Promise.reject(message) end @@ -159,7 +167,7 @@ function ApiContext:connect() return Http.get(url) :andThen(rejectFailedRequests) :andThen(Http.Response.msgpack) - :andThen(rejectWrongProtocolVersion) + :andThen(rejectWrongVersion) :andThen(rejectWrongFork) :andThen(function(body) assert(validateApiInfo(body)) @@ -188,7 +196,7 @@ function ApiContext:read(ids) end) end -function ApiContext:write(patch) +function ApiContext:write(patch, stageIds) local url = ("%s/api/write"):format(self.__baseUrl) -- Log what we're sending to the server (syncback/pull) @@ -248,6 +256,7 @@ function ApiContext:write(patch) removed = patch.removed, updated = updated, added = added, + stageIds = stageIds, } body = Http.msgpackEncode(body) diff --git a/plugin/src/App/Components/Checkbox.lua b/plugin/src/App/Components/Checkbox.lua index 98cbf8c0..c2bfd134 100644 --- a/plugin/src/App/Components/Checkbox.lua +++ b/plugin/src/App/Components/Checkbox.lua @@ -23,10 +23,7 @@ end function Checkbox:didUpdate(lastProps) if lastProps.active ~= self.props.active then - self.motor:setGoal(Flipper.Spring.new(self.props.active and 1 or 0, { - frequency = 6, - dampingRatio = 1.1, - })) + self.motor:setGoal(Flipper.Instant.new(self.props.active and 1 or 0)) end end diff --git a/plugin/src/App/Components/Dropdown.lua b/plugin/src/App/Components/Dropdown.lua index 486a614c..a697135f 100644 --- a/plugin/src/App/Components/Dropdown.lua +++ b/plugin/src/App/Components/Dropdown.lua @@ -36,10 +36,7 @@ function Dropdown:didUpdate(prevProps) }) end - self.openMotor:setGoal(Flipper.Spring.new(self.state.open and 1 or 0, { - frequency = 6, - dampingRatio = 1.1, - })) + self.openMotor:setGoal(Flipper.Instant.new(self.state.open and 1 or 0)) end function Dropdown:render() diff --git a/plugin/src/App/Components/IconButton.lua b/plugin/src/App/Components/IconButton.lua index dc451b0d..1b74d835 100644 --- a/plugin/src/App/Components/IconButton.lua +++ b/plugin/src/App/Components/IconButton.lua @@ -8,11 +8,6 @@ local Flipper = require(Packages.Flipper) local Assets = require(Plugin.Assets) local bindingUtil = require(Plugin.App.bindingUtil) -local HOVER_SPRING_PROPS = { - frequency = 5, - dampingRatio = 1.1, -} - local e = Roact.createElement local IconButton = Roact.Component:extend("IconButton") @@ -38,11 +33,11 @@ function IconButton:render() [Roact.Event.Activated] = self.props.onClick, [Roact.Event.MouseEnter] = function() - self.motor:setGoal(Flipper.Spring.new(1, HOVER_SPRING_PROPS)) + self.motor:setGoal(Flipper.Instant.new(1)) end, [Roact.Event.MouseLeave] = function() - self.motor:setGoal(Flipper.Spring.new(0, HOVER_SPRING_PROPS)) + self.motor:setGoal(Flipper.Instant.new(0)) end, }, { Icon = e("ImageLabel", { diff --git a/plugin/src/App/Components/Notifications/Notification.lua b/plugin/src/App/Components/Notifications/Notification.lua index 1d9464c9..bdb288f5 100644 --- a/plugin/src/App/Components/Notifications/Notification.lua +++ b/plugin/src/App/Components/Notifications/Notification.lua @@ -40,17 +40,11 @@ function Notification:dismiss() end self.dismissed = true - self.motor:setGoal(Flipper.Spring.new(0, { - frequency = 5, - dampingRatio = 1, - })) + self.motor:setGoal(Flipper.Instant.new(0)) end function Notification:didMount() - self.motor:setGoal(Flipper.Spring.new(1, { - frequency = 3, - dampingRatio = 1, - })) + self.motor:setGoal(Flipper.Instant.new(1)) self.props.soundPlayer:play(Assets.Sounds.Notification) diff --git a/plugin/src/App/Components/PatchVisualizer/DomLabel.lua b/plugin/src/App/Components/PatchVisualizer/DomLabel.lua index f804cad7..268e1c0d 100644 --- a/plugin/src/App/Components/PatchVisualizer/DomLabel.lua +++ b/plugin/src/App/Components/PatchVisualizer/DomLabel.lua @@ -433,10 +433,7 @@ function DomLabel:didUpdate(prevProps) then -- Close the expansion when the domlabel is changed to a different thing self.expanded = false - self.motor:setGoal(Flipper.Spring.new(24, { - frequency = 5, - dampingRatio = 1, - })) + self.motor:setGoal(Flipper.Instant.new(24)) end end @@ -491,7 +488,7 @@ function DomLabel:render() end end - if depth ~= 1 then + if depth > 1 then lineGuides["Connector"] = e("Frame", { Size = UDim2.new(0, 8, 0, 2), Position = UDim2.new(0, 2 + (12 * props.depth), 0, 12), @@ -541,71 +538,50 @@ function DomLabel:render() SelectionService:Set({ props.instance }) end else - -- Single click expands the changes (after a delay to check for double click) - task.delay(0.3, function() - -- Guard against unmounted component or stale click - if not self.isMounted then + -- Removed script: show diff of old source vs empty + if + props.patchType == "Remove" + and props.instance + and props.changeInfo + and props.changeInfo.linesRemoved + and props.showStringDiff + then + local ok, source = pcall(function() + return (props.instance :: any).Source + end) + if ok and type(source) == "string" then + props.showStringDiff(source, "", props.instancePath) return end + end - if os.clock() - (self.lastDoubleClickTime or 0) <= 0.35 then - -- This was a double click, so don't expand - return - end - - -- Use self.props to get current props, not stale closure - local p = self.props - - -- Removed script: show diff of old source vs empty - if - p.patchType == "Remove" - and p.instance - and p.changeInfo - and p.changeInfo.linesRemoved - and p.showStringDiff - then - local ok, source = pcall(function() - return (p.instance :: any).Source - end) - if ok and type(source) == "string" then - p.showStringDiff(source, "", p.instancePath) + -- Added script: find Source in changeList and show diff of empty vs new source + if + props.patchType == "Add" + and props.changeList + and props.changeInfo + and props.changeInfo.linesAdded + and props.showStringDiff + then + for _, entry in props.changeList do + if entry[1] == "Source" and type(entry[3]) == "string" then + props.showStringDiff("", tostring(entry[3]), props.instancePath) return end end + end - -- Added script: find Source in changeList and show diff of empty vs new source - if - p.patchType == "Add" - and p.changeList - and p.changeInfo - and p.changeInfo.linesAdded - and p.showStringDiff - then - for _, entry in p.changeList do - if entry[1] == "Source" and type(entry[3]) == "string" then - p.showStringDiff("", tostring(entry[3]), p.instancePath) - return - end - end + if props.changeList then + local cl = props.changeList + if #cl == 2 and cl[2][1] == "Source" and props.showStringDiff then + props.showStringDiff(tostring(cl[2][2]), tostring(cl[2][3]), props.instancePath) + else + self.expanded = not self.expanded + local goalHeight = 24 + + (if self.expanded then math.clamp(#props.changeList * 24, 24, 24 * 6) else 0) + self.motor:setGoal(Flipper.Instant.new(goalHeight)) end - - if p.changeList then - -- If the only change is Source, open the diff viewer directly - -- changeList[1] is the header, changeList[2+] are entries - local cl = p.changeList - if #cl == 2 and cl[2][1] == "Source" and p.showStringDiff then - p.showStringDiff(tostring(cl[2][2]), tostring(cl[2][3]), p.instancePath) - else - self.expanded = not self.expanded - local goalHeight = 24 - + (if self.expanded then math.clamp(#p.changeList * 24, 24, 24 * 6) else 0) - self.motor:setGoal(Flipper.Spring.new(goalHeight, { - frequency = 5, - dampingRatio = 1, - })) - end - end - end) + end end end, }), @@ -653,8 +629,8 @@ function DomLabel:render() }), ChangeInfo = e("Frame", { BackgroundTransparency = 1, - Size = UDim2.new(1, -indent - (if props.patchType and props.onSelectionChange then 210 else 80), 0, 24), - Position = UDim2.new(1, if props.patchType and props.onSelectionChange then -130 else -2, 0, 0), + Size = UDim2.new(1, -indent - (if props.patchType and props.onSelectionChange then 218 else 80), 0, 24), + Position = UDim2.new(1, if props.patchType and props.onSelectionChange then -138 else -2, 0, 0), AnchorPoint = Vector2.new(1, 0), ClipsDescendants = true, }, { diff --git a/plugin/src/App/Components/TableDiffVisualizer/Dictionary.lua b/plugin/src/App/Components/TableDiffVisualizer/Dictionary.lua index f5ac319c..a73dc3a8 100644 --- a/plugin/src/App/Components/TableDiffVisualizer/Dictionary.lua +++ b/plugin/src/App/Components/TableDiffVisualizer/Dictionary.lua @@ -7,6 +7,7 @@ local Roact = require(Packages.Roact) local Timer = require(Plugin.Timer) local Assets = require(Plugin.Assets) local Theme = require(Plugin.App.Theme) +local trueEquals = require(Plugin.Reconciler.trueEquals) local ScrollingFrame = require(Plugin.App.Components.ScrollingFrame) local DisplayValue = require(Plugin.App.Components.PatchVisualizer.DisplayValue) @@ -35,8 +36,7 @@ function Dictionary:calculateDiff() key = key, patchType = "Remove", }) - elseif newValue ~= oldValue then - -- Note: should this do some sort of deep comparison for various types? + elseif not trueEquals(oldValue, newValue) then table.insert(diff, { key = key, patchType = "Edit", diff --git a/plugin/src/App/Components/TextButton.lua b/plugin/src/App/Components/TextButton.lua index 1fe871fd..1978fb0f 100644 --- a/plugin/src/App/Components/TextButton.lua +++ b/plugin/src/App/Components/TextButton.lua @@ -13,11 +13,6 @@ local getTextBoundsAsync = require(Plugin.App.getTextBoundsAsync) local SlicedImage = require(script.Parent.SlicedImage) local TouchRipple = require(script.Parent.TouchRipple) -local SPRING_PROPS = { - frequency = 5, - dampingRatio = 1, -} - local e = Roact.createElement local TextButton = Roact.Component:extend("TextButton") @@ -33,7 +28,7 @@ end function TextButton:didUpdate(lastProps) if lastProps.enabled ~= self.props.enabled then self.motor:setGoal({ - enabled = Flipper.Spring.new(self.props.enabled and 1 or 0), + enabled = Flipper.Instant.new(self.props.enabled and 1 or 0), }) end end @@ -61,13 +56,13 @@ function TextButton:render() [Roact.Event.MouseEnter] = function() self.motor:setGoal({ - hover = Flipper.Spring.new(1, SPRING_PROPS), + hover = Flipper.Instant.new(1), }) end, [Roact.Event.MouseLeave] = function() self.motor:setGoal({ - hover = Flipper.Spring.new(0, SPRING_PROPS), + hover = Flipper.Instant.new(0), }) end, }, { diff --git a/plugin/src/App/Components/TextInput.lua b/plugin/src/App/Components/TextInput.lua index 5cf83f3f..f15a7c39 100644 --- a/plugin/src/App/Components/TextInput.lua +++ b/plugin/src/App/Components/TextInput.lua @@ -11,11 +11,6 @@ local bindingUtil = require(Plugin.App.bindingUtil) local SlicedImage = require(script.Parent.SlicedImage) -local SPRING_PROPS = { - frequency = 5, - dampingRatio = 1, -} - local e = Roact.createElement local TextInput = Roact.Component:extend("TextInput") @@ -31,7 +26,7 @@ end function TextInput:didUpdate(lastProps) if lastProps.enabled ~= self.props.enabled then self.motor:setGoal({ - enabled = Flipper.Spring.new(self.props.enabled and 1 or 0), + enabled = Flipper.Instant.new(self.props.enabled and 1 or 0), }) end end @@ -98,13 +93,13 @@ function TextInput:render() [Roact.Event.MouseEnter] = function() self.motor:setGoal({ - hover = Flipper.Spring.new(1, SPRING_PROPS), + hover = Flipper.Instant.new(1), }) end, [Roact.Event.MouseLeave] = function() self.motor:setGoal({ - hover = Flipper.Spring.new(0, SPRING_PROPS), + hover = Flipper.Instant.new(0), }) end, diff --git a/plugin/src/App/Components/TouchRipple.lua b/plugin/src/App/Components/TouchRipple.lua index 1ea4c27d..a5d047a1 100644 --- a/plugin/src/App/Components/TouchRipple.lua +++ b/plugin/src/App/Components/TouchRipple.lua @@ -8,11 +8,6 @@ local Flipper = require(Packages.Flipper) local Assets = require(Plugin.Assets) local bindingUtil = require(Plugin.App.bindingUtil) -local EXPAND_SPRING = { - frequency = 7, - dampingRatio = 2, -} - local TouchRipple = Roact.Component:extend("TouchRipple") function TouchRipple:init() @@ -81,8 +76,8 @@ function TouchRipple:render() self.setPosition(relativePosition) self.motor:setGoal({ - scale = Flipper.Spring.new(1, EXPAND_SPRING), - opacity = Flipper.Spring.new(1, EXPAND_SPRING), + scale = Flipper.Instant.new(1), + opacity = Flipper.Instant.new(1), }) input:GetPropertyChangedSignal("UserInputState"):Connect(function() @@ -90,10 +85,7 @@ function TouchRipple:render() if userInputState == Enum.UserInputState.Cancel or userInputState == Enum.UserInputState.End then self.motor:setGoal({ - opacity = Flipper.Spring.new(0, { - frequency = 5, - dampingRatio = 1, - }), + opacity = Flipper.Instant.new(0), }) end end) diff --git a/plugin/src/App/Page.lua b/plugin/src/App/Page.lua index 42bc68b2..30e76220 100644 --- a/plugin/src/App/Page.lua +++ b/plugin/src/App/Page.lua @@ -62,10 +62,7 @@ end function Page:didUpdate(lastProps) if self.props.active ~= lastProps.active then - self.motor:setGoal(Flipper.Spring.new(self.props.active and 1 or 0, { - frequency = 6, - dampingRatio = 1, - })) + self.motor:setGoal(Flipper.Instant.new(self.props.active and 1 or 0)) end end diff --git a/plugin/src/App/StatusPages/Confirming.lua b/plugin/src/App/StatusPages/Confirming.lua index abf48793..fc828b8a 100644 --- a/plugin/src/App/StatusPages/Confirming.lua +++ b/plugin/src/App/StatusPages/Confirming.lua @@ -16,6 +16,94 @@ local TableDiffVisualizer = require(Plugin.App.Components.TableDiffVisualizer) local e = Roact.createElement +local function makeStatTag(text, color, theme, transparency, layoutOrder) + return e("Frame", { + Size = UDim2.new(0, 0, 0, 16), + AutomaticSize = Enum.AutomaticSize.X, + BackgroundColor3 = color, + BackgroundTransparency = transparency:map(function(t) + return 0.85 + 0.15 * t + end), + LayoutOrder = layoutOrder, + }, { + Corner = e("UICorner", { + CornerRadius = UDim.new(0, 3), + }), + Padding = e("UIPadding", { + PaddingLeft = UDim.new(0, 5), + PaddingRight = UDim.new(0, 5), + }), + Label = e("TextLabel", { + Text = text, + BackgroundTransparency = 1, + FontFace = theme.Font.Thin, + TextSize = theme.TextSize.Small, + TextColor3 = color, + TextXAlignment = Enum.TextXAlignment.Center, + TextTransparency = transparency, + Size = UDim2.new(0, 0, 1, 0), + AutomaticSize = Enum.AutomaticSize.X, + }), + }) +end + +local function makeStatBar(changeInfo, theme, transparency, layoutOrder) + local removed = changeInfo.linesRemoved or 0 + local added = changeInfo.linesAdded or 0 + local propChanges = changeInfo.propChanges or 0 + local total = removed + added + propChanges + if total == 0 then + return nil + end + + local children = { + Layout = e("UIListLayout", { + FillDirection = Enum.FillDirection.Horizontal, + SortOrder = Enum.SortOrder.LayoutOrder, + }), + Corner = e("UICorner", { + CornerRadius = UDim.new(0, 2), + }), + } + + if propChanges > 0 then + children.SegProp = e("Frame", { + Size = UDim2.fromScale(propChanges / total, 1), + BackgroundColor3 = theme.Diff.Property, + BackgroundTransparency = transparency, + BorderSizePixel = 0, + LayoutOrder = 1, + }) + end + + if added > 0 then + children.SegAdd = e("Frame", { + Size = UDim2.fromScale(added / total, 1), + BackgroundColor3 = theme.Diff.Add, + BackgroundTransparency = transparency, + BorderSizePixel = 0, + LayoutOrder = 2, + }) + end + + if removed > 0 then + children.SegRemove = e("Frame", { + Size = UDim2.fromScale(removed / total, 1), + BackgroundColor3 = theme.Diff.Remove, + BackgroundTransparency = transparency, + BorderSizePixel = 0, + LayoutOrder = 3, + }) + end + + return e("Frame", { + Size = UDim2.new(0, 40, 0, 8), + BackgroundTransparency = 1, + ClipsDescendants = true, + LayoutOrder = layoutOrder, + }, children) +end + local ConfirmingPage = Roact.Component:extend("ConfirmingPage") function ConfirmingPage:init() @@ -143,6 +231,10 @@ function ConfirmingPage:render() local hasChanges = self.props.patchTree and self.props.patchTree:getCount() > 0 return Theme.with(function(theme) + local rootChangeInfo = self.props.patchTree + and self.props.patchTree.ROOT + and self.props.patchTree.ROOT.changeInfo + local pageContent = Roact.createFragment({ Header = e("Frame", { Size = UDim2.new(1, 0, 0, 28), @@ -166,6 +258,53 @@ function ConfirmingPage:render() BackgroundTransparency = 1, }), + ChangeStats = if rootChangeInfo + then e("Frame", { + Size = UDim2.new(0.25, -10, 0, 20), + Position = UDim2.new(0.5, 0, 0.5, 0), + AnchorPoint = Vector2.new(0, 0.5), + BackgroundTransparency = 1, + }, { + Layout = e("UIListLayout", { + FillDirection = Enum.FillDirection.Horizontal, + HorizontalAlignment = Enum.HorizontalAlignment.Right, + VerticalAlignment = Enum.VerticalAlignment.Center, + SortOrder = Enum.SortOrder.LayoutOrder, + Padding = UDim.new(0, 4), + }), + PropsTag = if rootChangeInfo.propChanges + then makeStatTag( + rootChangeInfo.propChanges .. "P", + theme.Diff.Property, + theme, + self.props.transparency, + 1 + ) + else nil, + AddedTag = if rootChangeInfo.linesAdded ~= nil + then makeStatTag( + "+" .. (rootChangeInfo.linesAdded or 0), + theme.Diff.Add, + theme, + self.props.transparency, + 2 + ) + else nil, + RemovedTag = if rootChangeInfo.linesRemoved ~= nil + then makeStatTag( + "-" .. (rootChangeInfo.linesRemoved or 0), + theme.Diff.Remove, + theme, + self.props.transparency, + 3 + ) + else nil, + Bar = if rootChangeInfo + then makeStatBar(rootChangeInfo, theme, self.props.transparency, 4) + else nil, + }) + else nil, + FilterInput = e(TextInput, { text = self.state.filterText, placeholder = "Filter", diff --git a/plugin/src/App/StatusPages/NotConnected.lua b/plugin/src/App/StatusPages/NotConnected.lua index af98380a..719a99b7 100644 --- a/plugin/src/App/StatusPages/NotConnected.lua +++ b/plugin/src/App/StatusPages/NotConnected.lua @@ -124,11 +124,23 @@ function NotConnectedPage:render() }), }), + Syncback = e(TextButton, { + text = "Syncback", + style = "Danger", + transparency = self.props.transparency, + layoutOrder = 2, + onClick = self.props.onSyncback, + }, { + Tip = e(Tooltip.Trigger, { + text = "Overwrite project files with current Studio state", + }), + }), + Connect = e(TextButton, { text = "Sync", style = "Primary", transparency = self.props.transparency, - layoutOrder = 2, + layoutOrder = 3, onClick = self.props.onConnect, }, { Tip = e(Tooltip.Trigger, { diff --git a/plugin/src/App/init.lua b/plugin/src/App/init.lua index f5086e2a..4452cfc6 100644 --- a/plugin/src/App/init.lua +++ b/plugin/src/App/init.lua @@ -1,8 +1,26 @@ local ChangeHistoryService = game:GetService("ChangeHistoryService") local Players = game:GetService("Players") +local SerializationService = game:GetService("SerializationService") local ServerStorage = game:GetService("ServerStorage") local RunService = game:GetService("RunService") +local SYNCBACK_SERVICES = { + "Lighting", + "MaterialService", + "ReplicatedFirst", + "ReplicatedStorage", + "ServerScriptService", + "ServerStorage", + "SoundService", + "StarterGui", + "StarterPack", + "StarterPlayer", + "Teams", + "TextChatService", + "VoiceChatService", + "Workspace", +} + local Rojo = script:FindFirstAncestor("Rojo") local Plugin = Rojo.Plugin local Packages = Rojo.Packages @@ -27,8 +45,13 @@ local ignorePlaceIds = require(Plugin.ignorePlaceIds) local timeUtil = require(Plugin.timeUtil) local Theme = require(script.Theme) +local Http = require(Packages.Http) + +local encodeService = require(Plugin.ChangeBatcher.encodeService) + local Page = require(script.Page) local Notifications = require(script.Components.Notifications) +local TextButton = require(script.Components.TextButton) local Tooltip = require(script.Components.Tooltip) local StudioPluginAction = require(script.Components.Studio.StudioPluginAction) local StudioToolbar = require(script.Components.Studio.StudioToolbar) @@ -604,6 +627,72 @@ function App:useRunningConnectionInfo() self.setPort(port) end +function App:performSyncback() + self:setState({ showingSyncbackConfirm = false }) + + local host = self.host:getValue() + local port = self.port:getValue() + + if host == "" then + host = Config.defaultHost + end + if port == "" then + port = Config.defaultPort + end + + local services = {} + local allChildren = {} + local allCarriers = {} + for _, className in SYNCBACK_SERVICES do + local ok, service = pcall(game.FindService, game, className) + if ok and service then + local chunk, refTargets = encodeService(service) + for _, child in service:GetChildren() do + table.insert(allChildren, child) + end + for _, carrier in refTargets do + table.insert(allChildren, carrier) + table.insert(allCarriers, carrier) + end + table.insert(services, chunk) + end + end + + local data = buffer.create(0) + if #allChildren > 0 then + data = SerializationService:SerializeInstancesAsync(allChildren) + end + + for _, carrier in allCarriers do + carrier:Destroy() + end + + local url = ("http://%s:%s/api/syncback"):format(host, port) + local body = Http.msgpackEncode({ + protocolVersion = Config.protocolVersion, + serverVersion = Config.expectedServerVersionString, + placeId = game.PlaceId, + data = data, + services = services, + }) + + Http.post(url, body) + :andThen(function() + Log.info("Syncback data sent to server.") + self:addNotification({ + text = "Syncback data sent. Server is processing.", + timeout = 10, + }) + end) + :catch(function(err) + Log.warn("Syncback failed: " .. tostring(err)) + self:addNotification({ + text = "Syncback failed: " .. tostring(err), + timeout = 10, + }) + end) +end + function App:startSession() -- If a session is already in progress, tear it down first. -- In one-shot mode the sync lock is skipped, so this is the only guard @@ -653,10 +742,12 @@ function App:startSession() }) end) + local cachedServerInfo = nil + self.cleanupPrecommit = serveSession:hookPrecommit(function(patch, instanceMap) - -- Build new tree for patch + local gitMetadata = cachedServerInfo and cachedServerInfo.gitMetadata self:setState({ - patchTree = PatchTree.build(patch, instanceMap, { "Property", "Old", "New" }), + patchTree = PatchTree.build(patch, instanceMap, { "Property", "Old", "New" }, gitMetadata), }) end) self.cleanupPostcommit = serveSession:hookPostcommit(function(patch, instanceMap, unappliedPatch) @@ -790,6 +881,8 @@ function App:startSession() local initialSyncConfirmed = false serveSession:setConfirmCallback(function(instanceMap, patch, serverInfo) + cachedServerInfo = serverInfo + -- Filter out the DataModel name change from the patch -- The project name (DataModel.Name) is managed by Studio independently PatchSet.removeDataModelName(patch, instanceMap) @@ -864,7 +957,12 @@ function App:startSession() }) self:setState({ appStatus = AppStatus.Confirming, - patchTree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }), + patchTree = PatchTree.build( + patch, + instanceMap, + { "Property", "Current", "Incoming" }, + serverInfo.gitMetadata + ), confirmData = { serverInfo = serverInfo, }, @@ -914,8 +1012,9 @@ function App:startSession() -- Update the patchTree when new changes arrive during confirmation -- The changedIds parameter contains IDs of items that were modified -- so their selections can be reset (user must re-review) + local gitMetadata = cachedServerInfo and cachedServerInfo.gitMetadata self:setState({ - patchTree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }), + patchTree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata), changedIds = changedIds, -- Pass to UI so it can unselect changed items }) end) @@ -1027,6 +1126,12 @@ function App:render() self:startSession() end, + onSyncback = function() + self:setState({ + showingSyncbackConfirm = true, + }) + end, + onNavigateSettings = function() self.backPage = AppStatus.NotConnected self:setState({ @@ -1045,7 +1150,19 @@ function App:render() self.confirmationBindable:Fire("Abort") end, onConfirm = function(selections) - self.confirmationBindable:Fire({ type = "Confirm", selections = selections }) + local autoSelectedIds = {} + if self.state.patchTree then + self.state.patchTree:forEach(function(node) + if node.patchType and node.defaultSelection ~= nil then + autoSelectedIds[node.id] = true + end + end) + end + self.confirmationBindable:Fire({ + type = "Confirm", + selections = selections, + autoSelectedIds = autoSelectedIds, + }) end, }), @@ -1110,6 +1227,90 @@ function App:render() }), }), + SyncbackConfirm = e(Theme.StudioProvider, nil, { + e(StudioPluginGui, { + id = "Atlas_SyncbackConfirm", + title = "⚠️ Full Syncback (BETA) ⚠️", + active = self.state.showingSyncbackConfirm == true, + isEphemeral = true, + + initDockState = Enum.InitialDockState.Float, + overridePreviousState = true, + floatingSize = Vector2.new(400, 250), + minimumSize = Vector2.new(300, 220), + + zIndexBehavior = Enum.ZIndexBehavior.Sibling, + + onClose = function() + self:setState({ showingSyncbackConfirm = false }) + end, + }, { + Content = Theme.with(function(theme) + local noTransparency = Roact.createBinding(0) + + return e("Frame", { + Size = UDim2.fromScale(1, 1), + BackgroundColor3 = theme.BackgroundColor, + BorderSizePixel = 0, + }, { + Padding = e("UIPadding", { + PaddingLeft = UDim.new(0, 16), + PaddingRight = UDim.new(0, 16), + PaddingTop = UDim.new(0, 16), + PaddingBottom = UDim.new(0, 16), + }), + Layout = e("UIListLayout", { + FillDirection = Enum.FillDirection.Vertical, + SortOrder = Enum.SortOrder.LayoutOrder, + Padding = UDim.new(0, 16), + VerticalAlignment = Enum.VerticalAlignment.Center, + }), + Message = e("TextLabel", { + Text = "This will overwrite your project files with the current Studio state. This cannot be undone.\n\nNote: Live syncback produces slightly different formatting than CLI syncback (atlas syncback). This may cause minor git diffs even when the data is identical. Pick one method and stick with it to avoid unnecessary churn.", + TextWrapped = true, + FontFace = theme.Font.Main, + TextSize = theme.TextSize.Body, + TextColor3 = theme.TextColor, + Size = UDim2.new(1, 0, 0, 0), + AutomaticSize = Enum.AutomaticSize.Y, + BackgroundTransparency = 1, + LayoutOrder = 1, + }), + Buttons = e("Frame", { + Size = UDim2.new(1, 0, 0, 34), + BackgroundTransparency = 1, + LayoutOrder = 2, + }, { + Layout = e("UIListLayout", { + HorizontalAlignment = Enum.HorizontalAlignment.Right, + FillDirection = Enum.FillDirection.Horizontal, + SortOrder = Enum.SortOrder.LayoutOrder, + Padding = UDim.new(0, 10), + }), + Cancel = e(TextButton, { + text = "Cancel", + style = "Bordered", + transparency = noTransparency, + layoutOrder = 1, + onClick = function() + self:setState({ showingSyncbackConfirm = false }) + end, + }), + Confirm = e(TextButton, { + text = "Syncback", + style = "Danger", + transparency = noTransparency, + layoutOrder = 2, + onClick = function() + self:performSyncback() + end, + }), + }), + }) + end), + }), + }), + toggleAction = e(StudioPluginAction, { name = "AtlasConnection", title = "Atlas: Connect/Disconnect", diff --git a/plugin/src/ChangeBatcher/encodeHelpers.lua b/plugin/src/ChangeBatcher/encodeHelpers.lua new file mode 100644 index 00000000..83b198ce --- /dev/null +++ b/plugin/src/ChangeBatcher/encodeHelpers.lua @@ -0,0 +1,85 @@ +--[[ + Shared encoding helpers for ChangeBatcher encoders. Consolidates property + filter logic so encodeInstance.lua and encodeService.lua stay in sync. +]] + +local Packages = script.Parent.Parent.Parent.Packages +local RbxDom = require(Packages.RbxDom) + +local encodeProperty = require(script.Parent.encodeProperty) +local UNENCODABLE_DATA_TYPES = require(script.Parent.propertyFilter) + +local SKIP_PROPERTIES = { + Parent = true, + Name = true, + Archivable = true, +} + +local Helpers = {} + +function Helpers.encodeAttributes(instance, properties) + local success, attributes = pcall(function() + return instance:GetAttributes() + end) + if success and attributes and next(attributes) then + local descriptor = RbxDom.findCanonicalPropertyDescriptor(instance.ClassName, "Attributes") + if descriptor then + local ok, result = encodeProperty(instance, "Attributes", descriptor) + if ok and result ~= nil then + properties.Attributes = result + end + end + end +end + +function Helpers.encodeTags(instance, properties) + local success, tags = pcall(function() + return instance:GetTags() + end) + if success and tags and #tags > 0 then + local descriptor = RbxDom.findCanonicalPropertyDescriptor(instance.ClassName, "Tags") + if descriptor then + local ok, result = encodeProperty(instance, "Tags", descriptor) + if ok and result ~= nil then + properties.Tags = result + end + end + end +end + +-- Iterates all encodable properties for a class, applying common filters: +-- SKIP_PROPERTIES, per-class skip map, Attributes/Tags, scriptability, +-- serialization, and UNENCODABLE_DATA_TYPES. Calls callback(propertyName, +-- descriptor) for each property that passes all filters. +function Helpers.forEachEncodableProperty(className, classSkipProps, callback) + local classDescriptor = RbxDom.findClassDescriptor(className) + if not classDescriptor then + return + end + + for propertyName, propertyMeta in pairs(classDescriptor.properties) do + if SKIP_PROPERTIES[propertyName] then + continue + end + if classSkipProps and classSkipProps[propertyName] then + continue + end + if propertyName == "Attributes" or propertyName == "Tags" then + continue + end + + local isReadable = propertyMeta.scriptability == "ReadWrite" + or propertyMeta.scriptability == "Read" + or propertyMeta.scriptability == "Custom" + local doesSerialize = propertyMeta.serialization ~= "DoesNotSerialize" + + if isReadable and doesSerialize then + local descriptor = RbxDom.findCanonicalPropertyDescriptor(className, propertyName) + if descriptor and not UNENCODABLE_DATA_TYPES[descriptor.dataType] then + callback(propertyName, descriptor) + end + end + end +end + +return Helpers diff --git a/plugin/src/ChangeBatcher/encodeInstance.lua b/plugin/src/ChangeBatcher/encodeInstance.lua index b4856daf..f621511d 100644 --- a/plugin/src/ChangeBatcher/encodeInstance.lua +++ b/plugin/src/ChangeBatcher/encodeInstance.lua @@ -10,10 +10,10 @@ Children are recursively encoded and included. DUPLICATE HANDLING: - - Instances with duplicate-named siblings at ANY level of the path are skipped - - This matches the behavior in ref_properties.rs::is_path_unique - - A path is ambiguous if duplicates exist anywhere from the instance to the root - - A warning is logged when duplicates are encountered + - Instances with duplicate-named siblings are still encoded and sent to the + server. The server handles filesystem name collisions via the dedup suffix + system (~2, ~3, etc.) in syncback/file_names.rs. + - A debug message is logged when duplicates are detected (for diagnostics only). ]] local Packages = script.Parent.Parent.Parent.Packages @@ -21,7 +21,7 @@ local Log = require(Packages.Log) local RbxDom = require(Packages.RbxDom) local encodeProperty = require(script.Parent.encodeProperty) -local UNENCODABLE_DATA_TYPES = require(script.Parent.propertyFilter) +local Helpers = require(script.Parent.encodeHelpers) -- Script class names that need Source property local SCRIPT_CLASS_NAMES = { @@ -30,13 +30,6 @@ local SCRIPT_CLASS_NAMES = { ModuleScript = true, } --- Properties to always skip when encoding (internal Roblox properties that never serialize) -local SKIP_PROPERTIES = { - Parent = true, - Name = true, -- We handle this separately - Archivable = true, -} - -- Properties to skip per class (matches filter_out_property in Rust syncback) -- These are properties whose values are encoded in the file itself, not metadata local SKIP_PROPERTIES_BY_CLASS = { @@ -117,61 +110,15 @@ end -- Forward declaration for recursion local encodeInstance --- Track skipped duplicates for logging -local skippedDuplicateCount = 0 - --- Encode Attributes if present on any instance -local function encodeAttributes(instance, properties) - -- Try to get Attributes - this works for all instance types - local success, attributes = pcall(function() - return instance:GetAttributes() - end) - - if success and attributes and next(attributes) then - -- Attributes need to be encoded specially - -- The server expects them in the properties map under "Attributes" - local attributeDescriptor = RbxDom.findCanonicalPropertyDescriptor(instance.ClassName, "Attributes") - if attributeDescriptor then - local encodeSuccess, encodeResult = encodeProperty(instance, "Attributes", attributeDescriptor) - if encodeSuccess and encodeResult ~= nil then - properties.Attributes = encodeResult - end - end - end -end - --- Encode Tags if present on any instance -local function encodeTags(instance, properties) - -- Try to get Tags - this works for all instance types - local success, tags = pcall(function() - return instance:GetTags() - end) - - if success and tags and #tags > 0 then - -- Tags need to be encoded specially - local tagDescriptor = RbxDom.findCanonicalPropertyDescriptor(instance.ClassName, "Tags") - if tagDescriptor then - local encodeSuccess, encodeResult = encodeProperty(instance, "Tags", tagDescriptor) - if encodeSuccess and encodeResult ~= nil then - properties.Tags = encodeResult - end - end - end -end - encodeInstance = function(instance, parentId, _skipPathCheck) - -- Check if the entire path to this instance is unique (unless we're in a recursive call) - -- For top-level instances, we check the entire ancestor chain for duplicates - -- For children, the parent already verified the path is unique up to that point, - -- so we only need to check siblings at each level (done in the recursion below) + -- Log duplicate-named siblings as a debug message (server handles them via + -- dedup suffixes) but do NOT skip encoding. if not _skipPathCheck and not isPathUnique(instance) then - skippedDuplicateCount += 1 - Log.warn( - "Skipped instance '{}' ({}) - path contains duplicate-named siblings (cannot reliably sync)", + Log.debug( + "Instance '{}' ({}) has duplicate-named siblings in path (server will handle via dedup)", instance:GetFullName(), instance.ClassName ) - return nil end -- Log encoding at trace level (very detailed) @@ -181,9 +128,8 @@ encodeInstance = function(instance, parentId, _skipPathCheck) local properties = {} - -- Always try to encode Attributes and Tags for any instance type - encodeAttributes(instance, properties) - encodeTags(instance, properties) + Helpers.encodeAttributes(instance, properties) + Helpers.encodeTags(instance, properties) -- For scripts, encode the Source property first (required for script files) if SCRIPT_CLASS_NAMES[instance.ClassName] then @@ -202,78 +148,29 @@ encodeInstance = function(instance, parentId, _skipPathCheck) end end - -- Encode ALL properties for ANY instance type (matching Rust syncback behavior) - -- The server handles filtering defaults and serialization - local classDescriptor = RbxDom.findClassDescriptor(instance.ClassName) - if classDescriptor then - -- Get class-specific properties to skip - local classSkipProps = SKIP_PROPERTIES_BY_CLASS[instance.ClassName] or {} - - for propertyName, propertyMeta in pairs(classDescriptor.properties) do - -- Skip universally skipped properties - if SKIP_PROPERTIES[propertyName] then - continue - end - - -- Skip class-specific properties (Source, RunContext for scripts, etc.) - if classSkipProps[propertyName] then - continue - end - - -- Skip Attributes and Tags since we handle them separately above - if propertyName == "Attributes" or propertyName == "Tags" then - continue - end - - -- Only encode properties that: - -- 1. Are readable (ReadWrite or Read scriptability) - -- 2. Actually serialize (not "DoesNotSerialize") - local isReadable = propertyMeta.scriptability == "ReadWrite" or propertyMeta.scriptability == "Read" - local doesSerialize = propertyMeta.serialization ~= "DoesNotSerialize" - - if isReadable and doesSerialize then - -- Get the full PropertyDescriptor for encoding - local descriptor = RbxDom.findCanonicalPropertyDescriptor(instance.ClassName, propertyName) - if descriptor then - if UNENCODABLE_DATA_TYPES[descriptor.dataType] then - continue - end - - -- Ref properties cannot be encoded during instance addition - -- because the target instance has no server ID yet. They are - -- handled separately by encodePatchUpdate.lua during property - -- update encoding via the InstanceMap. - if descriptor.dataType == "Ref" then - continue - end - - local encodeSuccess, encodeResult = encodeProperty(instance, propertyName, descriptor) - if encodeSuccess and encodeResult ~= nil then - properties[propertyName] = encodeResult - end - end - end + -- Encode ALL properties for ANY instance type (matching Rust syncback behavior). + -- The server handles filtering defaults and serialization. + local classSkipProps = SKIP_PROPERTIES_BY_CLASS[instance.ClassName] + Helpers.forEachEncodableProperty(instance.ClassName, classSkipProps, function(propertyName, descriptor) + -- Ref properties cannot be encoded during instance addition because the + -- target instance has no server ID yet. They are handled separately by + -- encodePatchUpdate.lua via the InstanceMap. + if descriptor.dataType == "Ref" then + return end - end - -- Recursively encode children, skipping those with duplicate names + local encodeSuccess, encodeResult = encodeProperty(instance, propertyName, descriptor) + if encodeSuccess and encodeResult ~= nil then + properties[propertyName] = encodeResult + end + end) + + -- Recursively encode children (including those with duplicate names; + -- the server handles duplicates via dedup suffix system) local children = {} local childInstances = instance:GetChildren() - local childDuplicates = findDuplicateNames(childInstances) for _, child in ipairs(childInstances) do - -- Skip children with duplicate names - if childDuplicates[child.Name] then - skippedDuplicateCount += 1 - Log.warn( - "Skipped child instance '{}' ({}) - has duplicate-named siblings (cannot reliably sync)", - child:GetFullName(), - child.ClassName - ) - continue - end - - -- Pass true to skip duplicate check since we already checked above local encodedChild = encodeInstance(child, nil, true) if encodedChild then table.insert(children, encodedChild) @@ -296,20 +193,6 @@ encodeInstance = function(instance, parentId, _skipPathCheck) } end --- Wrapper function that resets the duplicate count and returns the final count -local function encodeInstanceWithDuplicateTracking(instance, parentId) - skippedDuplicateCount = 0 - local result = encodeInstance(instance, parentId, false) - - -- Log summary if any duplicates were skipped - if skippedDuplicateCount > 0 then - Log.warn( - "Skipped {} location(s) with duplicate-named siblings during pull (cannot reliably sync)", - skippedDuplicateCount - ) - end - - return result, skippedDuplicateCount +return function(instance, parentId) + return encodeInstance(instance, parentId, false) end - -return encodeInstanceWithDuplicateTracking diff --git a/plugin/src/ChangeBatcher/encodeInstance.spec.lua b/plugin/src/ChangeBatcher/encodeInstance.spec.lua index fbb35db0..596f16d6 100644 --- a/plugin/src/ChangeBatcher/encodeInstance.spec.lua +++ b/plugin/src/ChangeBatcher/encodeInstance.spec.lua @@ -232,7 +232,7 @@ return function() root:Destroy() end) - it("should skip children with duplicate names", function() + it("should encode children with duplicate names", function() local parent = Instance.new("Folder") parent.Name = "Parent" @@ -254,9 +254,8 @@ return function() local encoded = encodeInstance(parent, "PARENT_ID") expect(encoded).to.be.ok() - -- Should only have 1 child (UniqueName), duplicates skipped - expect(#encoded.children).to.equal(1) - expect(encoded.children[1].name).to.equal("UniqueName") + -- All children are encoded (server handles duplicates via dedup suffixes) + expect(#encoded.children).to.equal(3) parent:Destroy() end) diff --git a/plugin/src/ChangeBatcher/encodeInstance.stress.spec.lua b/plugin/src/ChangeBatcher/encodeInstance.stress.spec.lua index 241717d9..8604b85c 100644 --- a/plugin/src/ChangeBatcher/encodeInstance.stress.spec.lua +++ b/plugin/src/ChangeBatcher/encodeInstance.stress.spec.lua @@ -30,11 +30,10 @@ return function() local root = LargeTreeGenerator.createDeepTree({ depth = 10 }) root.Parent = container - local encoded, skippedCount = encodeInstance(root, "PARENT_ID") + local encoded = encodeInstance(root, "PARENT_ID") expect(encoded).to.be.ok() expect(encoded.name).to.equal("DeepRoot") - expect(skippedCount).to.equal(0) -- Verify depth (root + 10 children = 11 total levels) local current = encoded @@ -52,10 +51,9 @@ return function() local root = LargeTreeGenerator.createDeepTree({ depth = 30 }) root.Parent = container - local encoded, skippedCount = encodeInstance(root, "PARENT_ID") + local encoded = encodeInstance(root, "PARENT_ID") expect(encoded).to.be.ok() - expect(skippedCount).to.equal(0) root:Destroy() end) @@ -66,11 +64,10 @@ return function() local root = LargeTreeGenerator.createWideTree({ width = 50, levels = 1 }) root.Parent = container - local encoded, skippedCount = encodeInstance(root, "PARENT_ID") + local encoded = encodeInstance(root, "PARENT_ID") expect(encoded).to.be.ok() expect(#encoded.children).to.equal(50) - expect(skippedCount).to.equal(0) root:Destroy() end) @@ -79,11 +76,10 @@ return function() local root = LargeTreeGenerator.createWideTree({ width = 100, levels = 1 }) root.Parent = container - local encoded, skippedCount = encodeInstance(root, "PARENT_ID") + local encoded = encodeInstance(root, "PARENT_ID") expect(encoded).to.be.ok() expect(#encoded.children).to.equal(100) - expect(skippedCount).to.equal(0) root:Destroy() end) @@ -108,13 +104,11 @@ return function() unique.Name = "Unique" unique.Parent = root - local encoded, skippedCount = encodeInstance(root, "PARENT_ID") + local encoded = encodeInstance(root, "PARENT_ID") expect(encoded).to.be.ok() - -- Should only have the unique child - expect(#encoded.children).to.equal(1) - expect(encoded.children[1].name).to.equal("Unique") - expect(skippedCount).to.equal(2) -- Both duplicates skipped + -- Duplicates are now included alongside unique child + expect(#encoded.children).to.equal(3) end) it("should skip entire subtree for instances with duplicate-named siblings in path", function() @@ -139,12 +133,11 @@ return function() dup2Child.Name = "ChildOfDup2" dup2Child.Parent = dup2 - local encoded, skippedCount = encodeInstance(root, "PARENT_ID") + local encoded = encodeInstance(root, "PARENT_ID") expect(encoded).to.be.ok() - -- Both duplicates and their children should be skipped - expect(#encoded.children).to.equal(0) - expect(skippedCount).to.equal(2) -- The two duplicates + -- Duplicates are now included with their children + expect(#encoded.children).to.equal(2) end) end) diff --git a/plugin/src/ChangeBatcher/encodeService.lua b/plugin/src/ChangeBatcher/encodeService.lua new file mode 100644 index 00000000..30841826 --- /dev/null +++ b/plugin/src/ChangeBatcher/encodeService.lua @@ -0,0 +1,52 @@ +local encodeProperty = require(script.Parent.encodeProperty) +local Helpers = require(script.Parent.encodeHelpers) + +-- ClockTime and TimeOfDay on Lighting are linked properties representing +-- the same value (float vs string). Exclude ClockTime so only TimeOfDay +-- is sent, avoiding redundant/conflicting representations. +local EXCLUDE_PROPERTIES = { + Lighting = { ClockTime = true }, +} + +return function(service: Instance) + local properties = {} + local refs = {} + local children = service:GetChildren() + local refTargets = {} + + local classExcludes = EXCLUDE_PROPERTIES[service.ClassName] + Helpers.forEachEncodableProperty(service.ClassName, classExcludes, function(propertyName, descriptor) + if descriptor.dataType == "Ref" then + local readOk, target = descriptor:read(service) + if readOk and target then + local carrier = Instance.new("ObjectValue") + carrier.Name = propertyName + carrier.Value = target + table.insert(refTargets, carrier) + refs[propertyName] = #refTargets + end + return + end + + local encodeOk, encoded = encodeProperty(service, propertyName, descriptor) + if encodeOk and encoded ~= nil then + properties[propertyName] = encoded + end + end) + + Helpers.encodeAttributes(service, properties) + Helpers.encodeTags(service, properties) + + local chunk = { + className = service.ClassName, + childCount = #children, + refTargetCount = #refTargets, + } + if next(properties) then + chunk.properties = properties + end + if next(refs) then + chunk.refs = refs + end + return chunk, refTargets +end diff --git a/plugin/src/ChangeBatcher/encodeService.spec.lua b/plugin/src/ChangeBatcher/encodeService.spec.lua new file mode 100644 index 00000000..d0d79051 --- /dev/null +++ b/plugin/src/ChangeBatcher/encodeService.spec.lua @@ -0,0 +1,142 @@ +return function() + local encodeService = require(script.Parent.encodeService) + + describe("chunk structure", function() + it("should return className matching the service", function() + local lighting = game:GetService("Lighting") + local chunk = encodeService(lighting) + + expect(chunk.className).to.equal("Lighting") + end) + + it("should return childCount matching GetChildren", function() + local lighting = game:GetService("Lighting") + local chunk = encodeService(lighting) + + expect(chunk.childCount).to.equal(#lighting:GetChildren()) + end) + + it("should omit properties when none encode", function() + local folder = Instance.new("Folder") + folder.Name = "EmptyService" + + local chunk = encodeService(folder) + + expect(chunk.className).to.equal("Folder") + expect(chunk.childCount).to.equal(0) + expect(chunk.refTargetCount).to.equal(0) + end) + end) + + describe("property encoding", function() + it("should encode Lighting.TimeOfDay", function() + local lighting = game:GetService("Lighting") + local chunk = encodeService(lighting) + + expect(chunk.properties).to.be.ok() + expect(chunk.properties.TimeOfDay).to.be.ok() + end) + + it("should not include skipped properties (Parent, Name, Archivable)", function() + local lighting = game:GetService("Lighting") + local chunk = encodeService(lighting) + + if chunk.properties then + expect(chunk.properties.Parent).to.equal(nil) + expect(chunk.properties.Name).to.equal(nil) + expect(chunk.properties.Archivable).to.equal(nil) + end + end) + end) + + describe("EXCLUDE_PROPERTIES", function() + it("should not include Lighting.ClockTime", function() + local lighting = game:GetService("Lighting") + local chunk = encodeService(lighting) + + if chunk.properties then + expect(chunk.properties.ClockTime).to.equal(nil) + end + end) + end) + + describe("Attributes and Tags", function() + it("should encode Attributes when present", function() + local folder = Instance.new("Folder") + folder:SetAttribute("TestKey", 42) + + local chunk = encodeService(folder) + + expect(chunk.properties).to.be.ok() + expect(chunk.properties.Attributes).to.be.ok() + + folder:Destroy() + end) + + it("should encode Tags when present", function() + local folder = Instance.new("Folder") + folder:AddTag("SyncTag") + + local chunk = encodeService(folder) + + expect(chunk.properties).to.be.ok() + expect(chunk.properties.Tags).to.be.ok() + + folder:Destroy() + end) + + it("should omit Attributes when empty", function() + local folder = Instance.new("Folder") + local chunk = encodeService(folder) + + if chunk.properties then + expect(chunk.properties.Attributes).to.equal(nil) + end + + folder:Destroy() + end) + end) + + describe("Ref property carriers", function() + it("should create ObjectValue carriers for Ref properties", function() + local workspace = game:GetService("Workspace") + local _, refTargets = encodeService(workspace) + + for _, carrier in refTargets do + expect(carrier:IsA("ObjectValue")).to.equal(true) + expect(typeof(carrier.Name)).to.equal("string") + end + + for _, carrier in refTargets do + carrier:Destroy() + end + end) + + it("should report correct refTargetCount", function() + local workspace = game:GetService("Workspace") + local chunk, refTargets = encodeService(workspace) + + expect(chunk.refTargetCount).to.equal(#refTargets) + + for _, carrier in refTargets do + carrier:Destroy() + end + end) + + it("should map refs to 1-based carrier indices", function() + local workspace = game:GetService("Workspace") + local chunk, refTargets = encodeService(workspace) + + if chunk.refs then + for _, idx in chunk.refs do + expect(idx >= 1).to.equal(true) + expect(idx <= #refTargets).to.equal(true) + end + end + + for _, carrier in refTargets do + carrier:Destroy() + end + end) + end) +end diff --git a/plugin/src/Config.lua b/plugin/src/Config.lua index eac96626..acb7b2f6 100644 --- a/plugin/src/Config.lua +++ b/plugin/src/Config.lua @@ -26,7 +26,7 @@ return strict("Config", { isDevBuild = isDevBuild, codename = "Epiphany", version = realVersion, - expectedServerVersionString = ("%d.%d or newer"):format(realVersion[1], realVersion[2]), + expectedServerVersionString = ("%d.%d.%d"):format(realVersion[1], realVersion[2], realVersion[3]), protocolVersion = 6, defaultHost = "localhost", defaultPort = "34873", diff --git a/plugin/src/PatchTree.lua b/plugin/src/PatchTree.lua index 700c3812..9a6c5700 100644 --- a/plugin/src/PatchTree.lua +++ b/plugin/src/PatchTree.lua @@ -4,6 +4,7 @@ ]] local HttpService = game:GetService("HttpService") +local ScriptEditorService = game:GetService("ScriptEditorService") local Rojo = script:FindFirstAncestor("Rojo") local Plugin = Rojo.Plugin @@ -13,6 +14,7 @@ local Log = require(Packages.Log) local ChangeMetadata = require(Plugin.ChangeMetadata) local Config = require(Plugin.Config) +local SHA1 = require(Plugin.SHA1) local Timer = require(Plugin.Timer) local Types = require(Plugin.Types) local decodeValue = require(Plugin.Reconciler.decodeValue) @@ -78,8 +80,9 @@ function Tree.new() local tree = { idToNode = {}, ROOT = { + id = "ROOT", className = "DataModel", - name = "ROOT", + name = "game", children = {}, }, } @@ -208,11 +211,47 @@ function Tree:buildAncestryNodes(previousId: string?, ancestryIds: { string }, p end end +local function computeRootChangeInfo(tree) + local totalLinesAdded = 0 + local totalLinesRemoved = 0 + local totalPropChanges = 0 + local hasLineData = false + + tree:forEach(function(node) + if not node.changeInfo then + return + end + if node.changeInfo.linesAdded then + totalLinesAdded += node.changeInfo.linesAdded + hasLineData = true + end + if node.changeInfo.linesRemoved then + totalLinesRemoved += node.changeInfo.linesRemoved + hasLineData = true + end + if node.changeInfo.propChanges then + totalPropChanges += node.changeInfo.propChanges + end + end) + + if not hasLineData and totalPropChanges == 0 then + return nil + end + + return { + linesAdded = if hasLineData then totalLinesAdded else nil, + linesRemoved = if hasLineData then totalLinesRemoved else nil, + isWhitespaceOnly = false, + propChanges = if totalPropChanges > 0 then totalPropChanges else nil, + } +end + local PatchTree = {} -- Builds a new tree from a patch and instanceMap -- uses changeListHeaders in node.changeList -function PatchTree.build(patch, instanceMap, changeListHeaders) +-- gitMetadata is optional: { changedIds: {string}, scriptCommittedHashes: {[string]: {string}} } +function PatchTree.build(patch, instanceMap, changeListHeaders, gitMetadata) Timer.start("PatchTree.build") local clock = os.clock() @@ -220,6 +259,14 @@ function PatchTree.build(patch, instanceMap, changeListHeaders) local knownAncestors = {} + local changedIdSet = nil + if gitMetadata and gitMetadata.changedIds then + changedIdSet = {} + for _, id in gitMetadata.changedIds do + changedIdSet[id] = true + end + end + Timer.start("patch.updated") for _, change in patch.updated do clock = yieldIfNeeded(clock) @@ -304,8 +351,41 @@ function PatchTree.build(patch, instanceMap, changeListHeaders) table.insert(changeList, 1, changeListHeaders) end - -- Add this node to tree - -- Default to nil (unselected) - user must explicitly choose Push/Pull/Skip + -- Compute default selection from git metadata + local defaultSelection = nil + if changedIdSet then + if not changedIdSet[change.id] then + defaultSelection = "pull" + elseif instance:IsA("LuaSourceContainer") and gitMetadata.scriptCommittedHashes then + local hashes = gitMetadata.scriptCommittedHashes[change.id] + if hashes then + local source = instance.Source + local gitBlob = "blob " .. tostring(#source) .. "\0" .. source + local studioHash = SHA1(buffer.fromstring(gitBlob)) + for _, hash in hashes do + if studioHash == hash then + defaultSelection = "push" + break + end + end + + if defaultSelection == nil then + local ok, draft = pcall(ScriptEditorService.GetEditorSource, ScriptEditorService, instance) + if ok and draft ~= source then + local draftBlob = "blob " .. tostring(#draft) .. "\0" .. draft + local draftHash = SHA1(buffer.fromstring(draftBlob)) + for _, hash in hashes do + if draftHash == hash then + defaultSelection = "push" + break + end + end + end + end + end + end + end + tree:addNode(instanceMap.fromInstances[instance.Parent], { id = change.id, patchType = "Edit", @@ -314,7 +394,7 @@ function PatchTree.build(patch, instanceMap, changeListHeaders) instance = instance, changeInfo = changeInfo, changeList = changeList, - defaultSelection = nil, + defaultSelection = defaultSelection, }) end Timer.stop() @@ -453,6 +533,8 @@ function PatchTree.build(patch, instanceMap, changeListHeaders) end Timer.stop() + tree.ROOT.changeInfo = computeRootChangeInfo(tree) + Timer.stop() return tree end @@ -624,6 +706,8 @@ function PatchTree.updateMetadata(tree, patch, instanceMap, unappliedPatch) end) Timer.stop() + tree.ROOT.changeInfo = computeRootChangeInfo(tree) + Timer.stop() return tree end diff --git a/plugin/src/PatchTree.spec.lua b/plugin/src/PatchTree.spec.lua index 46d13ef0..88f066b0 100644 --- a/plugin/src/PatchTree.spec.lua +++ b/plugin/src/PatchTree.spec.lua @@ -256,6 +256,440 @@ return function() end) end) + describe("gitMetadata default selections", function() + local SHA1 = require(script.Parent.SHA1) + + local function computeBlobSha1(content) + local gitBlob = "blob " .. tostring(#content) .. "\0" .. content + return SHA1(buffer.fromstring(gitBlob)) + end + + it("should default to pull for unchanged files (not in changedIds)", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local workspace = game:GetService("Workspace") + local folder = Instance.new("Folder") + folder.Name = "UnchangedFolder" + folder.Parent = workspace + + local id = HttpService:GenerateGUID(false) + instanceMap:insert(id, folder) + + table.insert(patch.updated, { + id = id, + changedName = "Renamed", + changedProperties = {}, + }) + + local gitMetadata = { + changedIds = {}, -- empty = no files changed in git + scriptCommittedHashes = {}, + } + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[id]).to.equal("pull") + + folder:Destroy() + instanceMap:stop() + end) + + it("should default to nil when no gitMetadata provided", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local workspace = game:GetService("Workspace") + local folder = Instance.new("Folder") + folder.Name = "NoGitFolder" + folder.Parent = workspace + + local id = HttpService:GenerateGUID(false) + instanceMap:insert(id, folder) + + table.insert(patch.updated, { + id = id, + changedName = "Renamed", + changedProperties = {}, + }) + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, nil) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[id]).to.equal(nil) + + folder:Destroy() + instanceMap:stop() + end) + + it("should default to nil for non-script changed files", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local workspace = game:GetService("Workspace") + local folder = Instance.new("Folder") + folder.Name = "ChangedFolder" + folder.Parent = workspace + + local id = HttpService:GenerateGUID(false) + instanceMap:insert(id, folder) + + table.insert(patch.updated, { + id = id, + changedName = "Renamed", + changedProperties = {}, + }) + + local gitMetadata = { + changedIds = { id }, + scriptCommittedHashes = {}, + } + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[id]).to.equal(nil) + + folder:Destroy() + instanceMap:stop() + end) + + it("should default to push when script Source matches committed hash", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local workspace = game:GetService("Workspace") + local script = Instance.new("ModuleScript") + script.Name = "MatchingScript" + script.Source = "local x = 1\nreturn x" + script.Parent = workspace + + local id = HttpService:GenerateGUID(false) + instanceMap:insert(id, script) + + table.insert(patch.updated, { + id = id, + changedProperties = { Source = { String = "-- different from server" } }, + }) + + local committedHash = computeBlobSha1(script.Source) + local gitMetadata = { + changedIds = { id }, + scriptCommittedHashes = { + [id] = { committedHash }, + }, + } + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[id]).to.equal("push") + + script:Destroy() + instanceMap:stop() + end) + + it("should default to nil when script Source does not match any committed hash", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local workspace = game:GetService("Workspace") + local script = Instance.new("ModuleScript") + script.Name = "MismatchScript" + script.Source = "-- studio has different content" + script.Parent = workspace + + local id = HttpService:GenerateGUID(false) + instanceMap:insert(id, script) + + table.insert(patch.updated, { + id = id, + changedProperties = { Source = { String = "-- server version" } }, + }) + + local gitMetadata = { + changedIds = { id }, + scriptCommittedHashes = { + [id] = { computeBlobSha1("-- committed version, different from studio") }, + }, + } + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[id]).to.equal(nil) + + script:Destroy() + instanceMap:stop() + end) + + it("should match against staged hash when HEAD hash doesn't match", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local workspace = game:GetService("Workspace") + local script = Instance.new("ModuleScript") + script.Name = "StagedMatchScript" + script.Source = "-- staged version content" + script.Parent = workspace + + local id = HttpService:GenerateGUID(false) + instanceMap:insert(id, script) + + table.insert(patch.updated, { + id = id, + changedProperties = { Source = { String = "-- server" } }, + }) + + local headHash = computeBlobSha1("-- old committed version") + local stagedHash = computeBlobSha1(script.Source) + local gitMetadata = { + changedIds = { id }, + scriptCommittedHashes = { + [id] = { headHash, stagedHash }, + }, + } + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[id]).to.equal("push") + + script:Destroy() + instanceMap:stop() + end) + + it("should not auto-select Added items regardless of git status", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local parentId = HttpService:GenerateGUID(false) + local workspace = game:GetService("Workspace") + instanceMap:insert(parentId, workspace) + + local addId = HttpService:GenerateGUID(false) + patch.added[addId] = { + Id = addId, + Parent = parentId, + ClassName = "ModuleScript", + Name = "AddedScript", + Properties = {}, + } + + local gitMetadata = { + changedIds = { addId }, + scriptCommittedHashes = {}, + } + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[addId]).to.equal(nil) + + instanceMap:stop() + end) + + it("should not auto-select Removed items regardless of git status", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local workspace = game:GetService("Workspace") + local folder = Instance.new("Folder") + folder.Name = "ToRemove" + folder.Parent = workspace + + local id = HttpService:GenerateGUID(false) + instanceMap:insert(id, folder) + table.insert(patch.removed, id) + + local gitMetadata = { + changedIds = {}, + scriptCommittedHashes = {}, + } + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[id]).to.equal(nil) + + folder:Destroy() + instanceMap:stop() + end) + + it("should handle Script and LocalScript as LuaSourceContainer", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local workspace = game:GetService("Workspace") + local serverScript = Instance.new("Script") + serverScript.Name = "ServerTest" + serverScript.Source = "print('server')" + serverScript.Parent = workspace + + local localScript = Instance.new("LocalScript") + localScript.Name = "ClientTest" + localScript.Source = "print('client')" + localScript.Parent = workspace + + local serverId = HttpService:GenerateGUID(false) + local localId = HttpService:GenerateGUID(false) + instanceMap:insert(serverId, serverScript) + instanceMap:insert(localId, localScript) + + table.insert(patch.updated, { + id = serverId, + changedProperties = { Source = { String = "-- new" } }, + }) + table.insert(patch.updated, { + id = localId, + changedProperties = { Source = { String = "-- new" } }, + }) + + local gitMetadata = { + changedIds = { serverId, localId }, + scriptCommittedHashes = { + [serverId] = { computeBlobSha1(serverScript.Source) }, + [localId] = { computeBlobSha1(localScript.Source) }, + }, + } + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[serverId]).to.equal("push") + expect(selections[localId]).to.equal("push") + + serverScript:Destroy() + localScript:Destroy() + instanceMap:stop() + end) + + it("should handle mixed changed and unchanged files", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local workspace = game:GetService("Workspace") + local changedScript = Instance.new("ModuleScript") + changedScript.Name = "Changed" + changedScript.Source = "-- matches committed" + changedScript.Parent = workspace + + local unchangedFolder = Instance.new("Folder") + unchangedFolder.Name = "Unchanged" + unchangedFolder.Parent = workspace + + local changedId = HttpService:GenerateGUID(false) + local unchangedId = HttpService:GenerateGUID(false) + instanceMap:insert(changedId, changedScript) + instanceMap:insert(unchangedId, unchangedFolder) + + table.insert(patch.updated, { + id = changedId, + changedProperties = { Source = { String = "-- server" } }, + }) + table.insert(patch.updated, { + id = unchangedId, + changedName = "StudioRenamed", + changedProperties = {}, + }) + + local gitMetadata = { + changedIds = { changedId }, -- only the script is git-changed + scriptCommittedHashes = { + [changedId] = { computeBlobSha1(changedScript.Source) }, + }, + } + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[changedId]).to.equal("push") + expect(selections[unchangedId]).to.equal("pull") + + changedScript:Destroy() + unchangedFolder:Destroy() + instanceMap:stop() + end) + + it("should handle changed script with no committed hash (new file)", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local workspace = game:GetService("Workspace") + local script = Instance.new("ModuleScript") + script.Name = "NewScript" + script.Source = "return 'new'" + script.Parent = workspace + + local id = HttpService:GenerateGUID(false) + instanceMap:insert(id, script) + + table.insert(patch.updated, { + id = id, + changedProperties = { Source = { String = "-- different" } }, + }) + + local gitMetadata = { + changedIds = { id }, + scriptCommittedHashes = {}, -- no hash for this script (new file) + } + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[id]).to.equal(nil) + + script:Destroy() + instanceMap:stop() + end) + + it("should handle empty changedIds with gitMetadata present", function() + local patch = PatchSet.newEmpty() + local instanceMap = InstanceMap.new() + + local workspace = game:GetService("Workspace") + local folder = Instance.new("Folder") + folder.Name = "AllUnchanged" + folder.Parent = workspace + + local script = Instance.new("ModuleScript") + script.Name = "AlsoUnchanged" + script.Source = "return true" + script.Parent = workspace + + local folderId = HttpService:GenerateGUID(false) + local scriptId = HttpService:GenerateGUID(false) + instanceMap:insert(folderId, folder) + instanceMap:insert(scriptId, script) + + table.insert(patch.updated, { + id = folderId, + changedName = "StudioRenamed", + changedProperties = {}, + }) + table.insert(patch.updated, { + id = scriptId, + changedProperties = { Source = { String = "-- studio version" } }, + }) + + local gitMetadata = { + changedIds = {}, -- nothing changed in git + scriptCommittedHashes = {}, + } + + local tree = PatchTree.build(patch, instanceMap, { "Property", "Current", "Incoming" }, gitMetadata) + local selections = PatchTree.buildInitialSelections(tree) + + expect(selections[folderId]).to.equal("pull") + expect(selections[scriptId]).to.equal("pull") + + folder:Destroy() + script:Destroy() + instanceMap:stop() + end) + end) + describe("getSelectableNodeIds", function() it("should return empty array for empty tree", function() local patch = PatchSet.newEmpty() diff --git a/plugin/src/Reconciler/diff.lua b/plugin/src/Reconciler/diff.lua index b2b2adad..95e8a25a 100644 --- a/plugin/src/Reconciler/diff.lua +++ b/plugin/src/Reconciler/diff.lua @@ -12,137 +12,12 @@ local getProperty = require(script.Parent.getProperty) local Error = require(script.Parent.Error) local decodeValue = require(script.Parent.decodeValue) +local trueEquals = require(script.Parent.trueEquals) + local function isEmpty(table) return next(table) == nil end --- Finds duplicate names among children and returns a set of names that are duplicated -local function findDuplicateNames( - children: { Instance } | { string }, - virtualInstances: { [string]: any }? -): { [string]: boolean } - local nameCounts: { [string]: number } = {} - local duplicates: { [string]: boolean } = {} - - for _, child in children do - local name: string - if typeof(child) == "Instance" then - name = child.Name - elseif virtualInstances then - -- child is a virtual instance ID - local virtualChild = virtualInstances[child] - name = virtualChild and virtualChild.Name or "" - else - continue - end - - nameCounts[name] = (nameCounts[name] or 0) + 1 - if nameCounts[name] > 1 then - duplicates[name] = true - end - end - - return duplicates -end - -local function fuzzyEq(a: number, b: number, epsilon: number): boolean - local diff = math.abs(a - b) - -- Use both absolute and relative epsilon for better precision handling - -- Absolute: for small numbers near zero - -- Relative: for larger numbers where absolute epsilon is too tight - local maxVal = math.max(math.abs(a), math.abs(b), 1) - return diff < epsilon or diff < maxVal * epsilon -end - -local function trueEquals(a, b): boolean - -- Exit early for simple equality values - if a == b then - return true - end - - -- Treat nil and { Ref = "000...0" } as equal - if - (a == nil and type(b) == "table" and b.Ref == "00000000000000000000000000000000") - or (b == nil and type(a) == "table" and a.Ref == "00000000000000000000000000000000") - then - return true - end - - local typeA, typeB = typeof(a), typeof(b) - - -- For tables, try recursive deep equality - if typeA == "table" and typeB == "table" then - local checkedKeys = {} - for key, value in a do - checkedKeys[key] = true - if not trueEquals(value, b[key]) then - return false - end - end - for key, value in b do - if checkedKeys[key] then - continue - end - if not trueEquals(value, a[key]) then - return false - end - end - return true - - -- For NaN, check if both values are not equal to themselves - elseif a ~= a and b ~= b then - return true - - -- For numbers, compare with epsilon of 0.0001 to avoid floating point inequality - elseif typeA == "number" and typeB == "number" then - return fuzzyEq(a, b, 0.0001) - - -- For EnumItem->number, compare the EnumItem's value - elseif typeA == "number" and typeB == "EnumItem" then - return a == b.Value - elseif typeA == "EnumItem" and typeB == "number" then - return a.Value == b - - -- For Color3s, compare to RGB ints to avoid floating point inequality - elseif typeA == "Color3" and typeB == "Color3" then - local aR, aG, aB = math.floor(a.R * 255), math.floor(a.G * 255), math.floor(a.B * 255) - local bR, bG, bB = math.floor(b.R * 255), math.floor(b.G * 255), math.floor(b.B * 255) - return aR == bR and aG == bG and aB == bB - - -- For CFrames, compare to components with epsilon of 0.0001 to avoid floating point inequality - elseif typeA == "CFrame" and typeB == "CFrame" then - local aComponents, bComponents = { a:GetComponents() }, { b:GetComponents() } - for i, aComponent in aComponents do - if not fuzzyEq(aComponent, bComponents[i], 0.0001) then - return false - end - end - return true - - -- For Vector3s, compare to components with epsilon of 0.0001 to avoid floating point inequality - elseif typeA == "Vector3" and typeB == "Vector3" then - local aComponents, bComponents = { a.X, a.Y, a.Z }, { b.X, b.Y, b.Z } - for i, aComponent in aComponents do - if not fuzzyEq(aComponent, bComponents[i], 0.0001) then - return false - end - end - return true - - -- For Vector2s, compare to components with epsilon of 0.0001 to avoid floating point inequality - elseif typeA == "Vector2" and typeB == "Vector2" then - local aComponents, bComponents = { a.X, a.Y }, { b.X, b.Y } - for i, aComponent in aComponents do - if not fuzzyEq(aComponent, bComponents[i], 0.0001) then - return false - end - end - return true - end - - return false -end - local function shouldDeleteUnknownInstances(virtualInstance) if virtualInstance.Metadata ~= nil then return not virtualInstance.Metadata.ignoreUnknownInstances @@ -175,6 +50,8 @@ local function shouldDeleteChild(virtualInstance, childInstance) return false end +local DIFF_YIELD_INTERVAL = 1000 + local function diff(instanceMap, virtualInstances, rootId, serverInfo) local patch = { removed = {}, @@ -182,6 +59,8 @@ local function diff(instanceMap, virtualInstances, rootId, serverInfo) updated = {}, } + local diffCount = 0 + -- Build a lookup table for visible services (for ignoreHiddenServices check) local visibleServicesSet: { [string]: boolean } = {} if serverInfo and serverInfo.ignoreHiddenServices and serverInfo.visibleServices then @@ -190,86 +69,9 @@ local function diff(instanceMap, virtualInstances, rootId, serverInfo) end end - -- Count of locations with duplicate-named siblings - local skippedDuplicateCount = 0 - - -- Pre-scan to find ALL instances that are in an ambiguous path - -- (i.e., they or any of their ancestors have duplicate-named siblings) - local ambiguousIds: { [string]: boolean } = {} - - local function markSubtreeAmbiguous(id) - if ambiguousIds[id] then - return - end - ambiguousIds[id] = true - - local virtualInstance = virtualInstances[id] - if virtualInstance then - for _, childId in ipairs(virtualInstance.Children) do - markSubtreeAmbiguous(childId) - end - end - end - - -- Recursively scan for duplicates and mark ambiguous subtrees - local function scanForDuplicates(id) - local virtualInstance = virtualInstances[id] - if not virtualInstance then - return - end - - local instance = instanceMap.fromIds[id] - - -- Detect duplicate names among real DOM children (if instance exists) - local realDuplicates: { [string]: boolean } = {} - if instance then - local realChildren = instance:GetChildren() - realDuplicates = findDuplicateNames(realChildren) - - -- Count locations with duplicates - if next(realDuplicates) then - skippedDuplicateCount += 1 - end - end - - -- Detect duplicate names among virtual DOM children - local virtualDuplicates = findDuplicateNames(virtualInstance.Children, virtualInstances) - - -- Merge duplicates from both sides - local allDuplicates: { [string]: boolean } = {} - for name in realDuplicates do - allDuplicates[name] = true - end - for name in virtualDuplicates do - allDuplicates[name] = true - end - - -- Mark all children with duplicate names (and their entire subtrees) as ambiguous - for _, childId in ipairs(virtualInstance.Children) do - local virtualChild = virtualInstances[childId] - if virtualChild and allDuplicates[virtualChild.Name] then - markSubtreeAmbiguous(childId) - end - end - - -- Recurse into non-ambiguous children to find deeper duplicates - for _, childId in ipairs(virtualInstance.Children) do - if not ambiguousIds[childId] then - scanForDuplicates(childId) - end - end - end - - -- First pass: scan entire tree to find all ambiguous paths - scanForDuplicates(rootId) - -- Add a virtual instance and all of its descendants to the patch, marked as - -- being added. Skip ambiguous instances. + -- being added. local function markIdAdded(id) - if ambiguousIds[id] then - return - end - local virtualInstance = virtualInstances[id] patch.added[id] = virtualInstance @@ -280,11 +82,10 @@ local function diff(instanceMap, virtualInstances, rootId, serverInfo) -- Internal recursive kernel for diffing an instance with the given ID. local function diffInternal(id) - -- Skip entirely if this instance is in an ambiguous path - if ambiguousIds[id] then - return true + diffCount += 1 + if diffCount % DIFF_YIELD_INTERVAL == 0 then + task.wait() end - local virtualInstance = virtualInstances[id] local instance = instanceMap.fromIds[id] @@ -387,11 +188,6 @@ local function diff(instanceMap, virtualInstances, rootId, serverInfo) local childId = instanceMap.fromInstances[childInstance] - -- Skip children in ambiguous paths - if childId and ambiguousIds[childId] then - continue - end - if childId == nil then -- pcall to avoid security permission errors local success, skip = pcall(function() @@ -403,14 +199,6 @@ local function diff(instanceMap, virtualInstances, rootId, serverInfo) continue end - -- Check if this unmapped child has a duplicate name among siblings - -- If so, skip it (we can't reliably determine what to do with it) - local siblings = instance:GetChildren() - local siblingDuplicates = findDuplicateNames(siblings) - if siblingDuplicates[childInstance.Name] then - continue - end - -- Skip hidden services when ignoreHiddenServices is enabled. -- Services are direct children of game (DataModel), and if they're -- not in the visible services list, they weren't synced back to disk, @@ -441,11 +229,6 @@ local function diff(instanceMap, virtualInstances, rootId, serverInfo) -- Traverse the list of children in the virtual DOM. Any virtual -- instance that has no corresponding real instance should be created. for _, childId in ipairs(virtualInstance.Children) do - -- Skip children in ambiguous paths - if ambiguousIds[childId] then - continue - end - local childInstance = instanceMap.fromIds[childId] if childInstance == nil then @@ -460,11 +243,6 @@ local function diff(instanceMap, virtualInstances, rootId, serverInfo) local diffSuccess, err = diffInternal(rootId) - -- Log count of skipped duplicates - if skippedDuplicateCount > 0 then - Log.warn("Skipped {} location(s) with duplicate-named siblings (cannot reliably sync)", skippedDuplicateCount) - end - if not diffSuccess then return false, err end diff --git a/plugin/src/Reconciler/diff.spec.lua b/plugin/src/Reconciler/diff.spec.lua index 6d62b8b7..163606c7 100644 --- a/plugin/src/Reconciler/diff.spec.lua +++ b/plugin/src/Reconciler/diff.spec.lua @@ -276,7 +276,7 @@ return function() end) describe("duplicate-named siblings", function() - it("should skip real DOM children with duplicate names", function() + it("should remove real DOM children with duplicate names when not in virtual DOM", function() local knownInstances = InstanceMap.new() local virtualInstances = { ROOT = { @@ -305,11 +305,11 @@ return function() assert(ok, tostring(patch)) - -- Duplicate-named children should be skipped (not removed) - assert(isEmpty(patch.removed)) + -- Duplicate-named children should be removed (matching handles pairing) + expect(#patch.removed).to.equal(2) end) - it("should skip virtual children with duplicate names among siblings", function() + it("should add virtual children with duplicate names among siblings", function() local knownInstances = InstanceMap.new() local virtualInstances = { ROOT = { @@ -340,8 +340,8 @@ return function() assert(ok, tostring(patch)) - -- Duplicate-named virtual children should not be added - assert(isEmpty(patch.added)) + -- Duplicate-named virtual children should be added (matching handles pairing) + expect(size(patch.added)).to.equal(2) end) end) diff --git a/plugin/src/Reconciler/diff.stress.spec.lua b/plugin/src/Reconciler/diff.stress.spec.lua index 14b0468d..619e55b3 100644 --- a/plugin/src/Reconciler/diff.stress.spec.lua +++ b/plugin/src/Reconciler/diff.stress.spec.lua @@ -239,9 +239,8 @@ return function() local ok, patch = diff(instanceMap, virtualInstances, rootId) expect(ok).to.equal(true) - -- Duplicates should NOT be marked for removal - -- Only if they had unique names would they be candidates - expect(#patch.removed).to.equal(0) + -- Duplicates should be marked for removal (matching handles pairing) + expect(#patch.removed).to.equal(2) instanceMap:stop() end) @@ -290,9 +289,9 @@ return function() local ok, patch = diff(instanceMap, virtualInstances, rootId) expect(ok).to.equal(true) - -- Neither duplicate should be added since the path is ambiguous - expect(patch.added[dup1Id]).to.equal(nil) - expect(patch.added[dup2Id]).to.equal(nil) + -- Both duplicates should be added (matching handles pairing) + expect(patch.added[dup1Id]).to.be.ok() + expect(patch.added[dup2Id]).to.be.ok() instanceMap:stop() end) @@ -350,8 +349,8 @@ return function() local ok, patch = diff(instanceMap, virtualInstances, rootId) expect(ok).to.equal(true) - -- Grandchild should also NOT be added since its parent is ambiguous - expect(patch.added[grandchildId]).to.equal(nil) + -- Grandchild should be added along with its parent (matching handles pairing) + expect(patch.added[grandchildId]).to.be.ok() instanceMap:stop() end) @@ -432,13 +431,13 @@ return function() local part = Instance.new("Part") part.Name = "TestPart" - part.Color = Color3.new(1, 0.5, 0.25) + part.Color = Color3.fromRGB(255, 128, 64) part.Parent = container local rootId = generateId() instanceMap:insert(rootId, part) - -- Same color values (exact match) + -- Same color values (exact 8-bit match avoids quantization mismatch) local virtualInstances = { [rootId] = { Id = rootId, @@ -447,7 +446,7 @@ return function() Parent = nil, Properties = { Color = { - Color3 = { 1, 0.5, 0.25 }, + Color3 = { 255 / 255, 128 / 255, 64 / 255 }, }, }, Children = {}, diff --git a/plugin/src/Reconciler/hydrate.lua b/plugin/src/Reconciler/hydrate.lua index afaf49c6..e434483d 100644 --- a/plugin/src/Reconciler/hydrate.lua +++ b/plugin/src/Reconciler/hydrate.lua @@ -1,14 +1,33 @@ --[[ Defines the process of "hydration" -- matching up a virtual DOM with concrete instances and assigning them IDs. + + Uses the change-count matching algorithm to handle duplicate-named + instances: groups by (Name, ClassName), then greedy-assigns pairs + by fewest total reconciler changes (recursive into subtrees). ]] local Packages = script.Parent.Parent.Parent.Packages local Log = require(Packages.Log) local invariant = require(script.Parent.Parent.invariant) +local Matching = require(script.Parent.matching) + +local HYDRATE_YIELD_INTERVAL = 1000 + +local function hydrate(instanceMap, virtualInstances, rootId, rootInstance, session) + if not session then + session = Matching.newSession() + end + + if not session.hydrateCount then + session.hydrateCount = 0 + end + session.hydrateCount += 1 + if session.hydrateCount % HYDRATE_YIELD_INTERVAL == 0 then + task.wait() + end -local function hydrate(instanceMap, virtualInstances, rootId, rootInstance) local virtualInstance = virtualInstances[rootId] if virtualInstance == nil then @@ -18,50 +37,35 @@ local function hydrate(instanceMap, virtualInstances, rootId, rootInstance) instanceMap:insert(rootId, rootInstance) local existingChildren = rootInstance:GetChildren() + local virtualChildIds = virtualInstance.Children - -- For each existing child, we'll track whether it's been paired with an - -- instance that the Rojo server knows about. - local isExistingChildVisited = {} - for i = 1, #existingChildren do - isExistingChildVisited[i] = false - end - - for _, childId in ipairs(virtualInstance.Children) do - local virtualChild = virtualInstances[childId] - - if virtualChild == nil then + -- Filter out virtual children missing from virtualInstances + local validVirtualIds: { string } = {} + for _, childId in ipairs(virtualChildIds) do + if virtualInstances[childId] then + table.insert(validVirtualIds, childId) + else Log.warn( "Hydration: virtualInstances missing child ID {} (parent: {} '{}')", childId, rootId, virtualInstance.Name ) - continue end + end - local matched = false - for childIndex, childInstance in existingChildren do - if not isExistingChildVisited[childIndex] then - -- We guard accessing Name and ClassName in order to avoid - -- tripping over children of DataModel that Rojo won't have - -- permissions to access at all. - local accessSuccess, name, className = pcall(function() - return childInstance.Name, childInstance.ClassName - end) + local result = + Matching.matchChildren(session, validVirtualIds, existingChildren, virtualInstances, rootId, rootInstance) - -- This rule is very conservative and could be loosened in the - -- future, or more heuristics could be introduced. - if accessSuccess and name == virtualChild.Name and className == virtualChild.ClassName then - isExistingChildVisited[childIndex] = true - hydrate(instanceMap, virtualInstances, childId, childInstance) - matched = true - break - end - end - end + -- Recursively hydrate matched pairs + for _, pair in result.matched do + hydrate(instanceMap, virtualInstances, pair.virtualId, pair.studioInstance, session) + end - if not matched then - -- Log why the match failed to help diagnose hydration issues + -- Log unmatched virtual children (no Studio counterpart found) + for _, virtualId in result.unmatchedVirtual do + local virtualChild = virtualInstances[virtualId] + if virtualChild then Log.debug( "Hydration: No match for virtual child '{}' ({}) under '{}' ({})", virtualChild.Name, @@ -69,17 +73,6 @@ local function hydrate(instanceMap, virtualInstances, rootId, rootInstance) virtualInstance.Name, rootInstance:GetFullName() ) - -- List real children for comparison - local realChildNames = {} - for _, child in existingChildren do - local success, childName, childClass = pcall(function() - return child.Name, child.ClassName - end) - if success then - table.insert(realChildNames, string.format("%s (%s)", childName, childClass)) - end - end - Log.trace(" Real children: {}", table.concat(realChildNames, ", ")) end end end diff --git a/plugin/src/Reconciler/init.lua b/plugin/src/Reconciler/init.lua index fe51478c..f7269a4f 100644 --- a/plugin/src/Reconciler/init.lua +++ b/plugin/src/Reconciler/init.lua @@ -33,9 +33,9 @@ function Reconciler:applyPatch(patch) return unappliedPatch end -function Reconciler:hydrate(virtualInstances, rootId, rootInstance) +function Reconciler:hydrate(virtualInstances, rootId, rootInstance, session) Timer.start("Reconciler:hydrate") - local result = hydrate(self.__instanceMap, virtualInstances, rootId, rootInstance) + local result = hydrate(self.__instanceMap, virtualInstances, rootId, rootInstance, session) Timer.stop() return result diff --git a/plugin/src/Reconciler/matching.lua b/plugin/src/Reconciler/matching.lua new file mode 100644 index 00000000..12e3de05 --- /dev/null +++ b/plugin/src/Reconciler/matching.lua @@ -0,0 +1,656 @@ +--!strict +--!native +--!optimize 2 +--[[ + Instance matching algorithm for the plugin. + + Pairs server virtual instances to Studio instances during hydration + by minimizing total reconciler changes. matchChildren is fundamentally + computeSubtreeCost -- it finds the cost-minimizing pairing of children + and returns the total cost. The match assignments are a bonus output. + + Two-function mutual recursion: + matchChildren: given two child sets, find optimal pairing + totalCost + computePairCost: given one virtual + one studio instance, compute + total cost including subtree (calls matchChildren for children) + + A MatchingSession holds caches keyed by instance identity so that + recursive scoring results are reused when hydrate recurses into the + same children that were already scored. +]] + +local Packages = script.Parent.Parent.Parent.Packages +local RbxDom = require(Packages.RbxDom) + +local trueEquals = require(script.Parent.trueEquals) + +local UNMATCHED_PENALTY = 10000 +local MAX_SCORING_DEPTH = 3 + +-- ================================================================ +-- Types +-- ================================================================ + +type MatchPair = { + virtualId: string, + studioInstance: Instance, +} + +type MatchResult = { + matched: { MatchPair }, + unmatchedVirtual: { string }, + unmatchedStudio: { Instance }, + totalCost: number, +} + +type MatchingSession = { + matchCache: { [string]: { [Instance]: MatchResult } }, + costCache: { [string]: { [Instance]: number } }, + vCacheStore: { [string]: VCache }, +} + +type RefIdentity = { + name: string, + className: string, +} + +type VCache = { + props: { [string]: any }, + extraProps: { string }, + tags: { [string]: boolean }, + attrs: { [string]: any }, + refs: { [string]: RefIdentity }, + childCount: number, + validChildren: { string }, +} + +type SCache = { + instance: Instance, + props: { [string]: any }, + tags: { [string]: boolean }, + attrs: { [string]: any }, + refs: { [string]: RefIdentity }, + children: { Instance }, + childCount: number, +} + +type VirtualInstance = { + Name: string, + ClassName: string, + Children: { string }, + Properties: { [string]: any }?, +} + +type VirtualInstances = { [string]: VirtualInstance } + +type ClassComparisonKeys = { + propNames: { string }, + propNameSet: { [string]: boolean }, + defaults: { [string]: any }, +} + +type ScoredPair = { + vi: number, + si: number, + cost: number, + idx: number, +} + +-- ================================================================ +-- Session constructor +-- ================================================================ + +local function newSession(): MatchingSession + return { + matchCache = {}, + costCache = {}, + vCacheStore = {}, + } +end + +-- ================================================================ +-- Utilities +-- ================================================================ + +local function rawIndex(inst: any, key: string): any + return inst[key] +end + +local function safeGet(inst: Instance, key: string): (boolean, any) + return pcall(rawIndex, inst :: any, key) +end + +-- ================================================================ +-- Pre-computation helpers (called once per instance per group) +-- ================================================================ + +local function cacheVirtual( + vInst: VirtualInstance, + classKeys: ClassComparisonKeys, + virtualInstances: VirtualInstances +): VCache + local decoded: { [string]: any } = {} + local extraProps: { string } = {} + local refs: { [string]: RefIdentity } = {} + local vProps = vInst.Properties + + if vProps then + for propName, encodedValue in pairs(vProps) do + if propName == "Tags" or propName == "Attributes" then + continue + end + + local ty = next(encodedValue) + if ty == "Ref" then + local targetId = encodedValue.Ref + if targetId then + local target = virtualInstances[targetId] + if target then + refs[propName] = { name = target.Name, className = target.ClassName } + end + end + continue + end + + local pcallOk, decodeOk, value = pcall(RbxDom.EncodedValue.decode, encodedValue) + if pcallOk and decodeOk and value ~= nil then + decoded[propName] = value + if not classKeys.propNameSet[propName] then + table.insert(extraProps, propName) + end + end + end + end + + local decodedTags: { [string]: boolean } = {} + if vProps and vProps.Tags then + local ok, tags = RbxDom.EncodedValue.decode(vProps.Tags) + if ok and type(tags) == "table" then + for _, tag in ipairs(tags) do + decodedTags[tag] = true + end + end + end + + local decodedAttrs: { [string]: any } = {} + if vProps and vProps.Attributes then + local ok, attrs = RbxDom.EncodedValue.decode(vProps.Attributes) + if ok and type(attrs) == "table" then + decodedAttrs = attrs + end + end + + local validChildren: { string } = {} + if vInst.Children then + for _, childId in ipairs(vInst.Children) do + if virtualInstances[childId] then + table.insert(validChildren, childId) + end + end + end + + return { + props = decoded, + extraProps = extraProps, + tags = decodedTags, + attrs = decodedAttrs, + refs = refs, + childCount = if vInst.Children then #vInst.Children else 0, + validChildren = validChildren, + } +end + +local function cacheStudio( + studioInstance: Instance, + classKeys: ClassComparisonKeys, + extraPropNames: { string }, + refPropNames: { string } +): SCache + local props: { [string]: any } = {} + + for _, propName in ipairs(classKeys.propNames) do + local ok, value = safeGet(studioInstance, propName) + if ok then + props[propName] = value + end + end + + for _, propName in ipairs(extraPropNames) do + if props[propName] ~= nil then + continue + end + local ok, value = safeGet(studioInstance, propName) + if ok then + props[propName] = value + end + end + + local refs: { [string]: RefIdentity } = {} + for _, propName in ipairs(refPropNames) do + local ok, value = safeGet(studioInstance, propName) + if ok and typeof(value) == "Instance" then + refs[propName] = { name = (value :: Instance).Name, className = (value :: Instance).ClassName } + end + end + + local tags: { [string]: boolean } = {} + for _, tag in ipairs(studioInstance:GetTags()) do + tags[tag] = true + end + + local children = studioInstance:GetChildren() + + return { + instance = studioInstance, + props = props, + tags = tags, + attrs = studioInstance:GetAttributes(), + refs = refs, + children = children, + childCount = #children, + } +end + +-- ================================================================ +-- Hot-path scoring (ZERO decode, ZERO reflection lookup) +-- ================================================================ + +local function countOwnDiffs(vCache: VCache, sCache: SCache, classKeys: ClassComparisonKeys): number + local cost = 0 + local vProps = vCache.props + local sProps = sCache.props + local defaults = classKeys.defaults + + for _, propName in ipairs(classKeys.propNames) do + local vVal = vProps[propName] + if vVal == nil then + vVal = defaults[propName] + end + + local sVal = sProps[propName] + if vVal ~= sVal and not trueEquals(vVal, sVal) then + cost += 1 + end + end + + for _, propName in ipairs(vCache.extraProps) do + local vVal = vProps[propName] + local sVal = sProps[propName] + if vVal ~= sVal and not trueEquals(vVal, sVal) then + cost += 1 + end + end + + local vTags = vCache.tags + local sTags = sCache.tags + for tag, _ in pairs(vTags) do + if not sTags[tag] then + cost += 1 + end + end + for tag, _ in pairs(sTags) do + if not vTags[tag] then + cost += 1 + end + end + + local vAttrs = vCache.attrs + local sAttrs = sCache.attrs + for key, vVal in pairs(vAttrs) do + local sVal = sAttrs[key] + if vVal ~= sVal and not trueEquals(vVal, sVal) then + cost += 1 + end + end + for key, _ in pairs(sAttrs) do + if vAttrs[key] == nil then + cost += 1 + end + end + + local vRefs = vCache.refs + local sRefs = sCache.refs + for propName, vRef in pairs(vRefs) do + local sRef = sRefs[propName] + if sRef then + if vRef.name ~= sRef.name or vRef.className ~= sRef.className then + cost += UNMATCHED_PENALTY + end + else + cost += UNMATCHED_PENALTY + end + end + for propName, _ in pairs(sRefs) do + if not vRefs[propName] then + cost += UNMATCHED_PENALTY + end + end + + if vCache.childCount ~= sCache.childCount then + cost += 1 + end + + return cost +end + +-- ================================================================ +-- Core: 2-function mutual recursion +-- ================================================================ + +local matchChildren -- forward declare + +local function computePairCost( + session: MatchingSession, + virtualId: string, + studioInstance: Instance, + virtualInstances: VirtualInstances, + bestSoFar: number, + depth: number +): number + local vc = session.costCache[virtualId] + if vc then + local cached = vc[studioInstance] + if cached ~= nil then + return cached + end + end + + local vInst = virtualInstances[virtualId] + if not vInst then + return UNMATCHED_PENALTY + end + + local classKeys = RbxDom.getClassComparisonKeys(vInst.ClassName) + local vCache = session.vCacheStore[virtualId] + if not vCache then + vCache = cacheVirtual(vInst, classKeys, virtualInstances) + session.vCacheStore[virtualId] = vCache + end + local refPropNames: { string } = {} + for propName, _ in pairs(vCache.refs) do + table.insert(refPropNames, propName) + end + local sCache = cacheStudio(studioInstance, classKeys, vCache.extraProps, refPropNames) + + local cost = countOwnDiffs(vCache, sCache, classKeys) + if cost >= bestSoFar or depth >= MAX_SCORING_DEPTH then + return cost + end + + local validVChildren = vCache.validChildren + local studioKids = sCache.children + + if #validVChildren > 0 or #studioKids > 0 then + if #validVChildren == 0 then + cost += #studioKids * UNMATCHED_PENALTY + elseif #studioKids == 0 then + cost += #validVChildren * UNMATCHED_PENALTY + else + local childResult = matchChildren( + session, + validVChildren, + studioKids, + virtualInstances, + virtualId, + studioInstance, + depth + 1 + ) + cost += childResult.totalCost + end + end + + if cost < bestSoFar then + if not session.costCache[virtualId] then + session.costCache[virtualId] = {} + end + session.costCache[virtualId][studioInstance] = cost + end + + return cost +end + +matchChildren = function( + session: MatchingSession, + virtualChildren: { string }, + studioChildren: { Instance }, + virtualInstances: VirtualInstances, + parentVirtualId: string?, + parentStudioInstance: Instance?, + _depth: number? +): MatchResult + local depth = _depth or 0 + if parentVirtualId and parentStudioInstance then + local pc = session.matchCache[parentVirtualId] + if pc then + local cached = pc[parentStudioInstance] + if cached then + return cached + end + end + end + + local matched: { MatchPair } = {} + local matchedCosts: { number? } = {} + local matchedV: { [number]: boolean } = {} + local matchedS: { [number]: boolean } = {} + + -- ============================================================ + -- Group by (Name, ClassName) once + -- ============================================================ + local vByKey: { [string]: { number } } = {} + for i, id in ipairs(virtualChildren) do + local vInst = virtualInstances[id] + if vInst then + local key = vInst.Name .. "\0" .. vInst.ClassName + local group = vByKey[key] + if not group then + group = {} + vByKey[key] = group + end + table.insert(group, i) + end + end + + local sByKey: { [string]: { number } } = {} + for i, inst in ipairs(studioChildren) do + local key = inst.Name .. "\0" .. inst.ClassName + local group = sByKey[key] + if not group then + group = {} + sByKey[key] = group + end + table.insert(group, i) + end + + -- ============================================================ + -- 1:1 instant match + ambiguous scoring in single pass + -- ============================================================ + for key, vIndices in pairs(vByKey) do + local sIndices = sByKey[key] + if not sIndices then + continue + end + + if #vIndices == 1 and #sIndices == 1 then + local vi, si = vIndices[1], sIndices[1] + table.insert(matched, { + virtualId = virtualChildren[vi], + studioInstance = studioChildren[si], + }) + table.insert(matchedCosts, nil) + matchedV[vi] = true + matchedS[si] = true + continue + end + + local firstVInst = virtualInstances[virtualChildren[vIndices[1]]] + if not firstVInst then + continue + end + local classKeys = RbxDom.getClassComparisonKeys(firstVInst.ClassName) + + local vCaches: { [number]: VCache } = {} + local allExtraProps: { [string]: boolean } = {} + local allRefProps: { [string]: boolean } = {} + for _, vi in ipairs(vIndices) do + local vid = virtualChildren[vi] + local vInst = virtualInstances[vid] + if not vInst then + continue + end + local vCache = session.vCacheStore[vid] + if not vCache then + vCache = cacheVirtual(vInst, classKeys, virtualInstances) + session.vCacheStore[vid] = vCache + end + vCaches[vi] = vCache + for _, propName in ipairs(vCache.extraProps) do + allExtraProps[propName] = true + end + for propName, _ in pairs(vCache.refs) do + allRefProps[propName] = true + end + end + + local extraPropNamesArray: { string } = {} + for propName, _ in pairs(allExtraProps) do + table.insert(extraPropNamesArray, propName) + end + + local refPropNamesArray: { string } = {} + for propName, _ in pairs(allRefProps) do + table.insert(refPropNamesArray, propName) + end + + local sCaches: { [number]: SCache } = {} + for _, si in ipairs(sIndices) do + sCaches[si] = cacheStudio(studioChildren[si], classKeys, extraPropNamesArray, refPropNamesArray) + end + + local scoredPairs: { ScoredPair } = {} + local pairIdx = 0 + local bestSoFar = math.huge + + for _, vi in ipairs(vIndices) do + local vCache = vCaches[vi] + if not vCache then + continue + end + + for _, si in ipairs(sIndices) do + local sCache = sCaches[si] + if not sCache then + continue + end + + pairIdx += 1 + local cost = countOwnDiffs(vCache, sCache, classKeys) + + if cost < bestSoFar and depth < MAX_SCORING_DEPTH then + local validVChildren = vCache.validChildren + local studioKids = sCache.children + + if #validVChildren > 0 or #studioKids > 0 then + if #validVChildren == 0 then + cost += #studioKids * UNMATCHED_PENALTY + elseif #studioKids == 0 then + cost += #validVChildren * UNMATCHED_PENALTY + else + local childResult = matchChildren( + session, + validVChildren, + studioKids, + virtualInstances, + virtualChildren[vi], + sCache.instance, + depth + 1 + ) + cost += childResult.totalCost + end + end + end + + table.insert(scoredPairs, { vi = vi, si = si, cost = cost, idx = pairIdx }) + if cost < bestSoFar then + bestSoFar = cost + end + end + end + + table.sort(scoredPairs, function(a: ScoredPair, b: ScoredPair): boolean + if a.cost ~= b.cost then + return a.cost < b.cost + end + return a.idx < b.idx + end) + + for _, pair in ipairs(scoredPairs) do + if matchedV[pair.vi] or matchedS[pair.si] then + continue + end + table.insert(matched, { + virtualId = virtualChildren[pair.vi], + studioInstance = studioChildren[pair.si], + }) + table.insert(matchedCosts, pair.cost) + matchedV[pair.vi] = true + matchedS[pair.si] = true + end + end + + -- ============================================================ + -- Build unmatched lists + compute totalCost + -- ============================================================ + local unmatchedVirtual: { string } = {} + for i, id in ipairs(virtualChildren) do + if not matchedV[i] then + table.insert(unmatchedVirtual, id) + end + end + + local unmatchedStudio: { Instance } = {} + for i, inst in ipairs(studioChildren) do + if not matchedS[i] then + table.insert(unmatchedStudio, inst) + end + end + + local totalCost = 0 + for i, pair in ipairs(matched) do + local precomputed = matchedCosts[i] + if precomputed ~= nil then + totalCost += precomputed + else + totalCost += computePairCost( + session, + pair.virtualId, + pair.studioInstance, + virtualInstances, + math.huge, + depth + ) + end + end + totalCost += (#unmatchedVirtual + #unmatchedStudio) * UNMATCHED_PENALTY + + local result = { + matched = matched, + unmatchedVirtual = unmatchedVirtual, + unmatchedStudio = unmatchedStudio, + totalCost = totalCost, + } + + if parentVirtualId and parentStudioInstance then + if not session.matchCache[parentVirtualId] then + session.matchCache[parentVirtualId] = {} + end + session.matchCache[parentVirtualId][parentStudioInstance] = result + end + + return result +end + +return { + newSession = newSession, + matchChildren = matchChildren, +} diff --git a/plugin/src/Reconciler/matching.spec.lua b/plugin/src/Reconciler/matching.spec.lua new file mode 100644 index 00000000..55ad4efe --- /dev/null +++ b/plugin/src/Reconciler/matching.spec.lua @@ -0,0 +1,587 @@ +return function() + local Matching = require(script.Parent.matching) + + local function makeVirtualInstance(name, className, props, children) + return { + Name = name, + ClassName = className, + Properties = props or {}, + Children = children or {}, + } + end + + local function makeStudioFolder(name, parent) + local inst = Instance.new("Folder") + inst.Name = name + if parent then + inst.Parent = parent + end + return inst + end + + local function makeStudioPart(name, parent, transparency) + local inst = Instance.new("Part") + inst.Name = name + if transparency then + inst.Transparency = transparency + end + if parent then + inst.Parent = parent + end + return inst + end + + local function makeStudioModel(name, parent) + local inst = Instance.new("Model") + inst.Name = name + if parent then + inst.Parent = parent + end + return inst + end + + local container + beforeEach(function() + container = Instance.new("Folder") + container.Name = "MatchingTestContainer" + container.Parent = workspace + end) + + afterEach(function() + if container then + container:Destroy() + end + end) + + describe("session cache", function() + it("cache hit returns same reference", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Child", "Folder", nil, {}), + } + local studioChild = makeStudioFolder("Child", container) + + local parentInst = container + + local result1 = Matching.matchChildren( + session, + { "A" }, + { studioChild }, + virtualInstances, + "PARENT", + parentInst + ) + local result2 = Matching.matchChildren( + session, + { "A" }, + { studioChild }, + virtualInstances, + "PARENT", + parentInst + ) + expect(result1).to.equal(result2) + end) + + it("different parents get different cache entries", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("X", "Folder", nil, {}), + B = makeVirtualInstance("Y", "Folder", nil, {}), + } + local studioX = makeStudioFolder("X", container) + local studioY = makeStudioFolder("Y", container) + + local result1 = Matching.matchChildren( + session, + { "A" }, + { studioX }, + virtualInstances, + "PARENT_1", + container + ) + local result2 = Matching.matchChildren( + session, + { "B" }, + { studioY }, + virtualInstances, + "PARENT_2", + container + ) + expect(result1).never.to.equal(result2) + expect(result1.matched[1].virtualId).to.equal("A") + expect(result2.matched[1].virtualId).to.equal("B") + end) + + it("separate sessions are independent", function() + local session1 = Matching.newSession() + local session2 = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Child", "Folder", nil, {}), + } + local studioChild = makeStudioFolder("Child", container) + + local result1 = Matching.matchChildren(session1, { "A" }, { studioChild }, virtualInstances, "P", container) + local result2 = Matching.matchChildren(session2, { "A" }, { studioChild }, virtualInstances, "P", container) + expect(result1).never.to.equal(result2) + expect(result1.totalCost).to.equal(result2.totalCost) + end) + end) + + describe("totalCost", function() + it("zero for identical 1:1 match", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Alpha", "Folder", {}, {}), + } + local studioAlpha = makeStudioFolder("Alpha", container) + + local result = Matching.matchChildren( + session, + { "A" }, + { studioAlpha }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(1) + expect(result.totalCost).to.equal(0) + end) + + it("zero for multiple identical 1:1", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Alpha", "Folder", {}, {}), + B = makeVirtualInstance("Beta", "Folder", {}, {}), + } + local studioAlpha = makeStudioFolder("Alpha", container) + local studioBeta = makeStudioFolder("Beta", container) + + local result = Matching.matchChildren( + session, + { "A", "B" }, + { studioAlpha, studioBeta }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(2) + expect(result.totalCost).to.equal(0) + end) + + it("nonzero for property diff", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("P", "Part", { Transparency = { Float32 = 0.5 } }, {}), + } + local studioPart = makeStudioPart("P", container, 0) + + local result = Matching.matchChildren(session, { "A" }, { studioPart }, virtualInstances, "ROOT", container) + expect(#result.matched).to.equal(1) + expect(result.totalCost > 0).to.equal(true) + end) + + it("unmatched virtual adds penalty", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Alpha", "Folder", {}, {}), + B = makeVirtualInstance("Beta", "Folder", {}, {}), + } + local studioAlpha = makeStudioFolder("Alpha", container) + + local result = Matching.matchChildren( + session, + { "A", "B" }, + { studioAlpha }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(1) + expect(#result.unmatchedVirtual).to.equal(1) + expect(result.totalCost >= 10000).to.equal(true) + end) + + it("unmatched studio adds penalty", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Alpha", "Folder", {}, {}), + } + local studioAlpha = makeStudioFolder("Alpha", container) + local studioBeta = makeStudioFolder("Beta", container) + + local result = Matching.matchChildren( + session, + { "A" }, + { studioAlpha, studioBeta }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(1) + expect(#result.unmatchedStudio).to.equal(1) + expect(result.totalCost >= 10000).to.equal(true) + end) + + it("sums pair costs and unmatched", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Alpha", "Folder", {}, {}), + B = makeVirtualInstance("Beta", "Folder", {}, {}), + C = makeVirtualInstance("Gamma", "Folder", {}, {}), + } + local studioAlpha = makeStudioFolder("Alpha", container) + local studioBeta = makeStudioFolder("Beta", container) + + local result = Matching.matchChildren( + session, + { "A", "B", "C" }, + { studioAlpha, studioBeta }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(2) + expect(#result.unmatchedVirtual).to.equal(1) + expect(result.totalCost).to.equal(10000) + end) + end) + + describe("Ref scoring", function() + it("Ref match pairs correctly in 2x2 ambiguous", function() + local session = Matching.newSession() + local virtualInstances = { + MODEL_A = makeVirtualInstance("M", "Model", { + PrimaryPart = { Ref = "HANDLE" }, + }, {}), + MODEL_B = makeVirtualInstance("M", "Model", { + PrimaryPart = { Ref = "GRIP" }, + }, {}), + HANDLE = makeVirtualInstance("Handle", "Part", {}, {}), + GRIP = makeVirtualInstance("Grip", "Part", {}, {}), + } + + local studioModelX = makeStudioModel("M", container) + local studioHandleX = makeStudioPart("Handle", studioModelX, nil) + studioModelX.PrimaryPart = studioHandleX + + local studioModelY = makeStudioModel("M", container) + local studioGripY = makeStudioPart("Grip", studioModelY, nil) + studioModelY.PrimaryPart = studioGripY + + local result = Matching.matchChildren( + session, + { "MODEL_A", "MODEL_B" }, + { studioModelX, studioModelY }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(2) + + for _, pair in ipairs(result.matched) do + if pair.virtualId == "MODEL_A" then + expect(pair.studioInstance.PrimaryPart.Name).to.equal("Handle") + elseif pair.virtualId == "MODEL_B" then + expect(pair.studioInstance.PrimaryPart.Name).to.equal("Grip") + end + end + end) + + it("no Ref properties does not affect scoring", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Data", "Folder", {}, {}), + B = makeVirtualInstance("Data", "Folder", {}, {}), + } + local studioA = makeStudioFolder("Data", container) + local studioB = makeStudioFolder("Data", container) + + local result = Matching.matchChildren( + session, + { "A", "B" }, + { studioA, studioB }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(2) + expect(result.totalCost).to.equal(0) + end) + end) + + describe("ambiguous groups", function() + it("2x2 same name different children", function() + local session = Matching.newSession() + local virtualInstances = { + FOLDER_A = makeVirtualInstance("Data", "Folder", {}, { "CHILD_A" }), + FOLDER_B = makeVirtualInstance("Data", "Folder", {}, { "CHILD_B" }), + CHILD_A = makeVirtualInstance("Alpha", "Folder", {}, {}), + CHILD_B = makeVirtualInstance("Beta", "Folder", {}, {}), + } + + local studioFolderX = makeStudioFolder("Data", container) + makeStudioFolder("Alpha", studioFolderX) + + local studioFolderY = makeStudioFolder("Data", container) + makeStudioFolder("Beta", studioFolderY) + + local result = Matching.matchChildren( + session, + { "FOLDER_A", "FOLDER_B" }, + { studioFolderX, studioFolderY }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(2) + + for _, pair in ipairs(result.matched) do + local studioChildName = pair.studioInstance:GetChildren()[1].Name + if pair.virtualId == "FOLDER_A" then + expect(studioChildName).to.equal("Alpha") + elseif pair.virtualId == "FOLDER_B" then + expect(studioChildName).to.equal("Beta") + end + end + end) + + it("3x3 same name different children", function() + local session = Matching.newSession() + local virtualInstances = { + F1 = makeVirtualInstance("Data", "Folder", {}, { "C1" }), + F2 = makeVirtualInstance("Data", "Folder", {}, { "C2" }), + F3 = makeVirtualInstance("Data", "Folder", {}, { "C3" }), + C1 = makeVirtualInstance("One", "Folder", {}, {}), + C2 = makeVirtualInstance("Two", "Folder", {}, {}), + C3 = makeVirtualInstance("Three", "Folder", {}, {}), + } + + local s1 = makeStudioFolder("Data", container) + makeStudioFolder("One", s1) + local s2 = makeStudioFolder("Data", container) + makeStudioFolder("Two", s2) + local s3 = makeStudioFolder("Data", container) + makeStudioFolder("Three", s3) + + local result = Matching.matchChildren( + session, + { "F1", "F2", "F3" }, + { s1, s2, s3 }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(3) + + for _, pair in ipairs(result.matched) do + local studioChildName = pair.studioInstance:GetChildren()[1].Name + if pair.virtualId == "F1" then + expect(studioChildName).to.equal("One") + elseif pair.virtualId == "F2" then + expect(studioChildName).to.equal("Two") + elseif pair.virtualId == "F3" then + expect(studioChildName).to.equal("Three") + end + end + end) + + it("identical instances match all with zero cost", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Data", "Folder", {}, {}), + B = makeVirtualInstance("Data", "Folder", {}, {}), + } + local s1 = makeStudioFolder("Data", container) + local s2 = makeStudioFolder("Data", container) + + local result = Matching.matchChildren( + session, + { "A", "B" }, + { s1, s2 }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(2) + expect(result.totalCost).to.equal(0) + end) + + it("asymmetric 3v vs 2s", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Data", "Folder", {}, {}), + B = makeVirtualInstance("Data", "Folder", {}, {}), + C = makeVirtualInstance("Data", "Folder", {}, {}), + } + local s1 = makeStudioFolder("Data", container) + local s2 = makeStudioFolder("Data", container) + + local result = Matching.matchChildren( + session, + { "A", "B", "C" }, + { s1, s2 }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(2) + expect(#result.unmatchedVirtual).to.equal(1) + end) + end) + + describe("recursive matching", function() + it("nested ambiguous resolved by grandchildren", function() + local session = Matching.newSession() + local virtualInstances = { + OUTER_A = makeVirtualInstance("Data", "Folder", {}, { "INNER_A1", "INNER_A2" }), + OUTER_B = makeVirtualInstance("Data", "Folder", {}, { "INNER_B1", "INNER_B2" }), + INNER_A1 = makeVirtualInstance("P", "Part", { Transparency = { Float32 = 0.1 } }, {}), + INNER_A2 = makeVirtualInstance("P", "Part", { Transparency = { Float32 = 0.2 } }, {}), + INNER_B1 = makeVirtualInstance("P", "Part", { Transparency = { Float32 = 0.8 } }, {}), + INNER_B2 = makeVirtualInstance("P", "Part", { Transparency = { Float32 = 0.9 } }, {}), + } + + local studioOuterX = makeStudioFolder("Data", container) + makeStudioPart("P", studioOuterX, 0.1) + makeStudioPart("P", studioOuterX, 0.2) + + local studioOuterY = makeStudioFolder("Data", container) + makeStudioPart("P", studioOuterY, 0.8) + makeStudioPart("P", studioOuterY, 0.9) + + local result = Matching.matchChildren( + session, + { "OUTER_A", "OUTER_B" }, + { studioOuterY, studioOuterX }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(2) + + for _, pair in ipairs(result.matched) do + local kids = pair.studioInstance:GetChildren() + if pair.virtualId == "OUTER_A" then + expect(kids[1].Transparency < 0.5).to.equal(true) + elseif pair.virtualId == "OUTER_B" then + expect(kids[1].Transparency > 0.5).to.equal(true) + end + end + end) + end) + + describe("edge cases", function() + it("empty both sides", function() + local session = Matching.newSession() + local result = Matching.matchChildren(session, {}, {}, {}, "ROOT", container) + expect(#result.matched).to.equal(0) + expect(#result.unmatchedVirtual).to.equal(0) + expect(#result.unmatchedStudio).to.equal(0) + expect(result.totalCost).to.equal(0) + end) + + it("all virtual unmatched", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Alpha", "Folder", {}, {}), + B = makeVirtualInstance("Beta", "Folder", {}, {}), + } + local result = Matching.matchChildren(session, { "A", "B" }, {}, virtualInstances, "ROOT", container) + expect(#result.matched).to.equal(0) + expect(#result.unmatchedVirtual).to.equal(2) + expect(result.totalCost).to.equal(20000) + end) + + it("all studio unmatched", function() + local session = Matching.newSession() + local s1 = makeStudioFolder("Alpha", container) + local s2 = makeStudioFolder("Beta", container) + local result = Matching.matchChildren(session, {}, { s1, s2 }, {}, "ROOT", container) + expect(#result.matched).to.equal(0) + expect(#result.unmatchedStudio).to.equal(2) + expect(result.totalCost).to.equal(20000) + end) + + it("class name discrimination", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Foo", "Folder", {}, {}), + B = makeVirtualInstance("Foo", "Part", {}, {}), + } + local studioFolder = makeStudioFolder("Foo", container) + local studioPart = makeStudioPart("Foo", container, nil) + + local result = Matching.matchChildren( + session, + { "A", "B" }, + { studioPart, studioFolder }, + virtualInstances, + "ROOT", + container + ) + expect(#result.matched).to.equal(2) + + for _, pair in ipairs(result.matched) do + if pair.virtualId == "A" then + expect(pair.studioInstance.ClassName).to.equal("Folder") + elseif pair.virtualId == "B" then + expect(pair.studioInstance.ClassName).to.equal("Part") + end + end + end) + + it("single child each side", function() + local session = Matching.newSession() + local virtualInstances = { + A = makeVirtualInstance("Only", "Folder", {}, {}), + } + local studioOnly = makeStudioFolder("Only", container) + + local result = Matching.matchChildren(session, { "A" }, { studioOnly }, virtualInstances, "ROOT", container) + expect(#result.matched).to.equal(1) + expect(result.matched[1].virtualId).to.equal("A") + expect(result.matched[1].studioInstance).to.equal(studioOnly) + expect(result.totalCost).to.equal(0) + end) + end) + + describe("parity with Rust fixture", function() + it("4 Parts with different Transparency values in reversed order", function() + local session = Matching.newSession() + local transparencies = { 0, 0.3, 0.6, 0.9 } + + local virtualInstances = {} + local virtualChildren = {} + for i, t in ipairs(transparencies) do + local id = "V" .. tostring(i) + virtualInstances[id] = makeVirtualInstance("Line", "Part", { + Transparency = { Float32 = t }, + }, {}) + table.insert(virtualChildren, id) + end + + local studioChildren = {} + for i = #transparencies, 1, -1 do + local inst = Instance.new("Part") + inst.Name = "Line" + inst.Transparency = transparencies[i] + inst.Parent = container + table.insert(studioChildren, inst) + end + + local result = + Matching.matchChildren(session, virtualChildren, studioChildren, virtualInstances, "ROOT", container) + + expect(#result.matched).to.equal(4) + expect(#result.unmatchedVirtual).to.equal(0) + expect(#result.unmatchedStudio).to.equal(0) + + for _, pair in ipairs(result.matched) do + local vInst = virtualInstances[pair.virtualId] + local vT = vInst.Properties.Transparency.Float32 + local sT = pair.studioInstance.Transparency + expect(math.abs(vT - sT) < 0.001).to.equal(true) + end + end) + end) +end diff --git a/plugin/src/Reconciler/stats.json b/plugin/src/Reconciler/stats.json new file mode 100644 index 00000000..142ce60a --- /dev/null +++ b/plugin/src/Reconciler/stats.json @@ -0,0 +1,28 @@ +{ + "lines": 629, + "bytecode": 6261, + "bytecodeInstructionCount": 986, + "codegen": 0, + "readTime": 0.000094, + "miscTime": 0.000053, + "parseTime": 0.000251, + "compileTime": 0.000709, + "codegenTime": 0.000000, + "lowerStats": { + "totalFunctions": 0, + "skippedFunctions": 0, + "spillsToSlot": 0, + "spillsToRestore": 0, + "maxSpillSlotsUsed": 0, + "blocksPreOpt": 0, + "blocksPostOpt": 0, + "maxBlockInstructions": 0, + "regAllocErrors": 0, + "loweringErrors": 0, + "blockLinearizationStats": { + "constPropInstructionCount": 0, + "timeSeconds": 0.000000 + }, + "functions": [] + } + } \ No newline at end of file diff --git a/plugin/src/Reconciler/trueEquals.lua b/plugin/src/Reconciler/trueEquals.lua new file mode 100644 index 00000000..8588112e --- /dev/null +++ b/plugin/src/Reconciler/trueEquals.lua @@ -0,0 +1,170 @@ +--!strict +--!native +--!optimize 2 +--[[ + Shared value equality check used by both the diff and matching modules. + + If matching says "equal" but diff says "different" (or vice versa), + we get phantom changes from cross-paired instances. Both modules MUST + use this same function. +]] + +local NULL_REF: string = "00000000000000000000000000000000" +local EPSILON: number = 0.0001 + +local function fuzzyEq(a: number, b: number): boolean + if a == b then + return true + end + if a ~= a then + return b ~= b + end + local diff = math.abs(a - b) + local maxVal = math.max(math.abs(a), math.abs(b), 1) + return diff < EPSILON or diff < maxVal * EPSILON +end + +local function color3Eq(a: Color3, b: Color3): boolean + return fuzzyEq(a.R, b.R) and fuzzyEq(a.G, b.G) and fuzzyEq(a.B, b.B) +end + +local function vector2Eq(a: Vector2, b: Vector2): boolean + return fuzzyEq(a.X, b.X) and fuzzyEq(a.Y, b.Y) +end + +local function vector3Eq(a: Vector3, b: Vector3): boolean + return fuzzyEq(a.X, b.X) and fuzzyEq(a.Y, b.Y) and fuzzyEq(a.Z, b.Z) +end + +local function cframeEq(a: CFrame, b: CFrame): boolean + local ax, ay, az, aR00, aR01, aR02, aR10, aR11, aR12, aR20, aR21, aR22 = a:GetComponents() + local bx, by, bz, bR00, bR01, bR02, bR10, bR11, bR12, bR20, bR21, bR22 = b:GetComponents() + return fuzzyEq(ax, bx) + and fuzzyEq(ay, by) + and fuzzyEq(az, bz) + and fuzzyEq(aR00, bR00) + and fuzzyEq(aR01, bR01) + and fuzzyEq(aR02, bR02) + and fuzzyEq(aR10, bR10) + and fuzzyEq(aR11, bR11) + and fuzzyEq(aR12, bR12) + and fuzzyEq(aR20, bR20) + and fuzzyEq(aR21, bR21) + and fuzzyEq(aR22, bR22) +end + +local function trueEquals(a: any, b: any): boolean + if rawequal(a, b) then + return true + end + + if a == nil then + return type(b) == "table" and rawget(b, "Ref") == NULL_REF + end + if b == nil then + return type(a) == "table" and rawget(a, "Ref") == NULL_REF + end + + local t = typeof(a) + local tb = typeof(b) + if t ~= tb then + if t == "number" and tb == "EnumItem" then + return a == (b :: EnumItem).Value + end + if t == "EnumItem" and tb == "number" then + return (a :: EnumItem).Value == b + end + return false + end + + if t == "table" then + if rawlen(a) ~= rawlen(b) then + return false + end + for k, v in next, a do + local ov = rawget(b, k) + if ov == nil or not trueEquals(v, ov) then + return false + end + end + for k in next, b do + if rawget(a, k) == nil then + return false + end + end + return true + end + + if t == "number" then + return fuzzyEq(a, b) + end + if t == "Color3" then + return color3Eq(a, b) + end + if t == "Vector3" then + return vector3Eq(a, b) + end + if t == "Vector2" then + return vector2Eq(a, b) + end + if t == "CFrame" then + return cframeEq(a, b) + end + if t == "UDim" then + return fuzzyEq(a.Scale, b.Scale) and a.Offset == b.Offset + end + if t == "UDim2" then + return fuzzyEq(a.X.Scale, b.X.Scale) + and a.X.Offset == b.X.Offset + and fuzzyEq(a.Y.Scale, b.Y.Scale) + and a.Y.Offset == b.Y.Offset + end + if t == "NumberRange" then + return fuzzyEq(a.Min, b.Min) and fuzzyEq(a.Max, b.Max) + end + if t == "Rect" then + return fuzzyEq(a.Min.X, b.Min.X) + and fuzzyEq(a.Min.Y, b.Min.Y) + and fuzzyEq(a.Max.X, b.Max.X) + and fuzzyEq(a.Max.Y, b.Max.Y) + end + if t == "NumberSequence" then + local aKp = a.Keypoints + local bKp = b.Keypoints + if #aKp ~= #bKp then + return false + end + for i = 1, #aKp do + if + not ( + fuzzyEq(aKp[i].Time, bKp[i].Time) + and fuzzyEq(aKp[i].Value, bKp[i].Value) + and fuzzyEq(aKp[i].Envelope, bKp[i].Envelope) + ) + then + return false + end + end + return true + end + if t == "ColorSequence" then + local aKp = a.Keypoints + local bKp = b.Keypoints + if #aKp ~= #bKp then + return false + end + for i = 1, #aKp do + if not (fuzzyEq(aKp[i].Time, bKp[i].Time) and color3Eq(aKp[i].Value, bKp[i].Value)) then + return false + end + end + return true + end + if t == "Font" then + return a.Family == b.Family and a.Weight == b.Weight and a.Style == b.Style + end + + return a == b +end + +return trueEquals diff --git a/plugin/src/Reconciler/trueEquals.spec.lua b/plugin/src/Reconciler/trueEquals.spec.lua new file mode 100644 index 00000000..7b1fc526 --- /dev/null +++ b/plugin/src/Reconciler/trueEquals.spec.lua @@ -0,0 +1,912 @@ +return function() + local trueEquals = require(script.Parent.trueEquals) + + local NULL_REF = "00000000000000000000000000000000" + + describe("identity / rawequal", function() + it("same table reference", function() + local t = { a = 1 } + expect(trueEquals(t, t)).to.equal(true) + end) + + it("same string", function() + local s = "hello" + expect(trueEquals(s, s)).to.equal(true) + end) + + it("same number", function() + expect(trueEquals(42, 42)).to.equal(true) + end) + + it("same boolean", function() + expect(trueEquals(true, true)).to.equal(true) + end) + + it("same Vector3 reference", function() + local v = Vector3.new(1, 2, 3) + expect(trueEquals(v, v)).to.equal(true) + end) + + it("same Instance reference", function() + local inst = Instance.new("Folder") + expect(trueEquals(inst, inst)).to.equal(true) + inst:Destroy() + end) + end) + + describe("nil handling", function() + it("nil vs nil", function() + expect(trueEquals(nil, nil)).to.equal(true) + end) + + it("nil vs non-nil", function() + expect(trueEquals(nil, 1)).to.equal(false) + end) + + it("non-nil vs nil", function() + expect(trueEquals(1, nil)).to.equal(false) + end) + + it("nil vs null-ref table", function() + expect(trueEquals(nil, { Ref = NULL_REF })).to.equal(true) + end) + + it("null-ref table vs nil", function() + expect(trueEquals({ Ref = NULL_REF }, nil)).to.equal(true) + end) + + it("nil vs non-null ref", function() + expect(trueEquals(nil, { Ref = "abc123" })).to.equal(false) + end) + + it("nil vs null-ref with extra keys still matches", function() + expect(trueEquals(nil, { Ref = NULL_REF, extra = 1 })).to.equal(true) + end) + + it("nil vs empty table", function() + expect(trueEquals(nil, {})).to.equal(false) + end) + + it("nil vs table with wrong key", function() + expect(trueEquals(nil, { NotRef = NULL_REF })).to.equal(false) + end) + end) + + describe("numbers -- absolute epsilon", function() + it("0 vs 0", function() + expect(trueEquals(0, 0)).to.equal(true) + end) + + it("0 vs 0.00005 within epsilon", function() + expect(trueEquals(0, 0.00005)).to.equal(true) + end) + + it("0 vs 0.0001 at boundary (strict less-than)", function() + expect(trueEquals(0, 0.0001)).to.equal(false) + end) + + it("0 vs 0.00009999 just under", function() + expect(trueEquals(0, 0.00009999)).to.equal(true) + end) + + it("0 vs -0.00005 negative within", function() + expect(trueEquals(0, -0.00005)).to.equal(true) + end) + + it("0 vs 0.001 beyond", function() + expect(trueEquals(0, 0.001)).to.equal(false) + end) + end) + + describe("numbers -- relative epsilon", function() + it("1000 vs 1000.05 within relative", function() + expect(trueEquals(1000, 1000.05)).to.equal(true) + end) + + it("1000 vs 1000.5 beyond relative", function() + expect(trueEquals(1000, 1000.5)).to.equal(false) + end) + + it("1e6 vs 1e6+50 within relative", function() + expect(trueEquals(1000000, 1000050)).to.equal(true) + end) + + it("1e6 vs 1e6+200 beyond relative", function() + expect(trueEquals(1000000, 1000200)).to.equal(false) + end) + end) + + describe("numbers -- negative", function() + it("-5 vs -5.00005 within", function() + expect(trueEquals(-5, -5.00005)).to.equal(true) + end) + + it("-5 vs -5.001 beyond", function() + expect(trueEquals(-5, -5.001)).to.equal(false) + end) + + it("-1 vs 1 different", function() + expect(trueEquals(-1, 1)).to.equal(false) + end) + end) + + describe("numbers -- special values", function() + it("0 vs -0", function() + expect(trueEquals(0, -0)).to.equal(true) + end) + + it("huge vs huge", function() + expect(trueEquals(math.huge, math.huge)).to.equal(true) + end) + + it("-huge vs -huge", function() + expect(trueEquals(-math.huge, -math.huge)).to.equal(true) + end) + + it("huge vs -huge", function() + expect(trueEquals(math.huge, -math.huge)).to.equal(false) + end) + end) + + describe("NaN", function() + it("NaN vs NaN", function() + expect(trueEquals(0 / 0, 0 / 0)).to.equal(true) + end) + + it("NaN vs 0", function() + expect(trueEquals(0 / 0, 0)).to.equal(false) + end) + + it("NaN vs string", function() + expect(trueEquals(0 / 0, "nan")).to.equal(false) + end) + end) + + describe("Color3", function() + it("black vs black", function() + expect(trueEquals(Color3.new(0, 0, 0), Color3.new(0, 0, 0))).to.equal(true) + end) + + it("white vs white", function() + expect(trueEquals(Color3.new(1, 1, 1), Color3.new(1, 1, 1))).to.equal(true) + end) + + it("same mid-gray", function() + expect(trueEquals(Color3.new(0.5, 0.5, 0.5), Color3.new(0.5, 0.5, 0.5))).to.equal(true) + end) + + it("red vs green", function() + expect(trueEquals(Color3.new(1, 0, 0), Color3.new(0, 1, 0))).to.equal(false) + end) + + it("fromRGB same", function() + expect(trueEquals(Color3.fromRGB(128, 128, 128), Color3.fromRGB(128, 128, 128))).to.equal(true) + end) + + it("fromRGB one int off", function() + expect(trueEquals(Color3.fromRGB(128, 128, 128), Color3.fromRGB(129, 128, 128))).to.equal(false) + end) + + it("near-boundary within epsilon", function() + expect(trueEquals(Color3.new(0.5, 0, 0), Color3.new(0.50005, 0, 0))).to.equal(true) + end) + + it("near-boundary beyond epsilon", function() + expect(trueEquals(Color3.new(0.5, 0, 0), Color3.new(0.502, 0, 0))).to.equal(false) + end) + end) + + describe("Vector3", function() + it("zero vs zero", function() + expect(trueEquals(Vector3.new(0, 0, 0), Vector3.new(0, 0, 0))).to.equal(true) + end) + + it("same components", function() + expect(trueEquals(Vector3.new(1, 2, 3), Vector3.new(1, 2, 3))).to.equal(true) + end) + + it("within epsilon on Z", function() + expect(trueEquals(Vector3.new(1, 2, 3), Vector3.new(1, 2, 3.00005))).to.equal(true) + end) + + it("beyond epsilon on Z", function() + expect(trueEquals(Vector3.new(1, 2, 3), Vector3.new(1, 2, 3.001))).to.equal(false) + end) + + it("Z differs by 1", function() + expect(trueEquals(Vector3.new(1, 2, 3), Vector3.new(1, 2, 4))).to.equal(false) + end) + + it("negative components", function() + expect(trueEquals(Vector3.new(-100, -200, -300), Vector3.new(-100, -200, -300))).to.equal(true) + end) + + it("large values within relative epsilon", function() + expect(trueEquals(Vector3.new(1e6, 1e6, 1e6), Vector3.new(1e6 + 50, 1e6, 1e6))).to.equal(true) + end) + + it("NaN component equal", function() + expect(trueEquals(Vector3.new(0 / 0, 1, 2), Vector3.new(0 / 0, 1, 2))).to.equal(true) + end) + + it("NaN vs finite component", function() + expect(trueEquals(Vector3.new(0 / 0, 1, 2), Vector3.new(0, 1, 2))).to.equal(false) + end) + end) + + describe("Vector2", function() + it("zero vs zero", function() + expect(trueEquals(Vector2.new(0, 0), Vector2.new(0, 0))).to.equal(true) + end) + + it("within epsilon on Y", function() + expect(trueEquals(Vector2.new(1, 2), Vector2.new(1, 2.00005))).to.equal(true) + end) + + it("Y differs by 1", function() + expect(trueEquals(Vector2.new(1, 2), Vector2.new(1, 3))).to.equal(false) + end) + + it("infinity components", function() + expect(trueEquals(Vector2.new(math.huge, math.huge), Vector2.new(math.huge, math.huge))).to.equal(true) + end) + + it("mixed infinity and finite", function() + expect(trueEquals(Vector2.new(math.huge, 0), Vector2.new(math.huge, 1))).to.equal(false) + end) + + it("NaN components", function() + expect(trueEquals(Vector2.new(0 / 0, 0 / 0), Vector2.new(0 / 0, 0 / 0))).to.equal(true) + end) + + it("NaN vs finite component", function() + expect(trueEquals(Vector2.new(0 / 0, 1), Vector2.new(1, 1))).to.equal(false) + end) + end) + + describe("CFrame", function() + it("identity vs identity", function() + expect(trueEquals(CFrame.identity, CFrame.identity)).to.equal(true) + end) + + it("translation only same", function() + expect(trueEquals(CFrame.new(1, 2, 3), CFrame.new(1, 2, 3))).to.equal(true) + end) + + it("translation Z differs", function() + expect(trueEquals(CFrame.new(1, 2, 3), CFrame.new(1, 2, 4))).to.equal(false) + end) + + it("same rotation", function() + expect(trueEquals(CFrame.Angles(0, math.pi / 4, 0), CFrame.Angles(0, math.pi / 4, 0))).to.equal(true) + end) + + it("different rotation", function() + expect(trueEquals(CFrame.Angles(0, math.pi / 4, 0), CFrame.Angles(0, math.pi / 2, 0))).to.equal(false) + end) + + it("tiny translation perturbation within epsilon", function() + expect(trueEquals(CFrame.new(1, 2, 3), CFrame.new(1, 2, 3.00005))).to.equal(true) + end) + + it("constructed from components match", function() + local cf1 = CFrame.new(1, 2, 3) * CFrame.Angles(0.1, 0.2, 0.3) + local ax, ay, az, r00, r01, r02, r10, r11, r12, r20, r21, r22 = cf1:GetComponents() + local cf2 = CFrame.new(ax, ay, az, r00, r01, r02, r10, r11, r12, r20, r21, r22) + expect(trueEquals(cf1, cf2)).to.equal(true) + end) + + it("one rotation component perturbed beyond epsilon", function() + local cf1 = CFrame.new(1, 2, 3) * CFrame.Angles(0.1, 0.2, 0.3) + local ax, ay, az, r00, r01, r02, r10, r11, r12, r20, r21, r22 = cf1:GetComponents() + local cf2 = CFrame.new(ax, ay, az, r00, r01 + 0.001, r02, r10, r11, r12, r20, r21, r22) + expect(trueEquals(cf1, cf2)).to.equal(false) + end) + end) + + describe("tables -- deep equality", function() + it("empty vs empty", function() + expect(trueEquals({}, {})).to.equal(true) + end) + + it("same array", function() + expect(trueEquals({ 1, 2, 3 }, { 1, 2, 3 })).to.equal(true) + end) + + it("different rawlen", function() + expect(trueEquals({ 1, 2, 3 }, { 1, 2 })).to.equal(false) + end) + + it("same rawlen different element", function() + expect(trueEquals({ 1, 2, 3 }, { 1, 2, 4 })).to.equal(false) + end) + + it("same dict", function() + expect(trueEquals({ a = 1 }, { a = 1 })).to.equal(true) + end) + + it("different dict value", function() + expect(trueEquals({ a = 1 }, { a = 2 })).to.equal(false) + end) + + it("extra key in b", function() + expect(trueEquals({ a = 1 }, { a = 1, b = 2 })).to.equal(false) + end) + + it("extra key in a", function() + expect(trueEquals({ a = 1, b = 2 }, { a = 1 })).to.equal(false) + end) + + it("nested tables equal", function() + expect(trueEquals({ a = { b = 1 } }, { a = { b = 1 } })).to.equal(true) + end) + + it("nested tables different", function() + expect(trueEquals({ a = { b = 1 } }, { a = { b = 2 } })).to.equal(false) + end) + + it("mixed types in array", function() + expect(trueEquals({ 1, "two", true }, { 1, "two", true })).to.equal(true) + end) + + it("fuzzy number values in table", function() + expect(trueEquals({ x = 1.00005 }, { x = 1 })).to.equal(true) + end) + + it("encoded property table", function() + expect(trueEquals({ Vector3 = { 1, 2, 3 } }, { Vector3 = { 1, 2, 3 } })).to.equal(true) + end) + + it("null ref strings rawequal", function() + expect(trueEquals({ Ref = NULL_REF }, { Ref = NULL_REF })).to.equal(true) + end) + end) + + describe("EnumItem cross-type", function() + it("EnumItem vs matching number", function() + expect(trueEquals(Enum.SortOrder.LayoutOrder, Enum.SortOrder.LayoutOrder.Value)).to.equal(true) + end) + + it("number vs matching EnumItem", function() + expect(trueEquals(Enum.SortOrder.LayoutOrder.Value, Enum.SortOrder.LayoutOrder)).to.equal(true) + end) + + it("EnumItem vs wrong number", function() + expect(trueEquals(Enum.SortOrder.LayoutOrder, 999)).to.equal(false) + end) + + it("same EnumItem rawequal", function() + expect(trueEquals(Enum.SortOrder.LayoutOrder, Enum.SortOrder.LayoutOrder)).to.equal(true) + end) + + it("different EnumItem values", function() + expect(trueEquals(Enum.SortOrder.LayoutOrder, Enum.SortOrder.Name)).to.equal(false) + end) + end) + + describe("UDim", function() + it("identical", function() + expect(trueEquals(UDim.new(0.5, 10), UDim.new(0.5, 10))).to.equal(true) + end) + + it("scale within epsilon", function() + expect(trueEquals(UDim.new(0.5, 10), UDim.new(0.50005, 10))).to.equal(true) + end) + + it("scale beyond epsilon", function() + expect(trueEquals(UDim.new(0.5, 10), UDim.new(0.502, 10))).to.equal(false) + end) + + it("offset differs by 1", function() + expect(trueEquals(UDim.new(0.5, 10), UDim.new(0.5, 11))).to.equal(false) + end) + + it("zero vs zero", function() + expect(trueEquals(UDim.new(0, 0), UDim.new(0, 0))).to.equal(true) + end) + end) + + describe("UDim2", function() + it("identical", function() + expect(trueEquals(UDim2.new(1, 0, 0.5, 100), UDim2.new(1, 0, 0.5, 100))).to.equal(true) + end) + + it("X scale within epsilon", function() + expect(trueEquals(UDim2.new(1, 0, 0.5, 100), UDim2.new(1.00005, 0, 0.5, 100))).to.equal(true) + end) + + it("X scale beyond epsilon", function() + expect(trueEquals(UDim2.new(1, 0, 0.5, 100), UDim2.new(1.002, 0, 0.5, 100))).to.equal(false) + end) + + it("Y scale within epsilon", function() + expect(trueEquals(UDim2.new(1, 0, 0.5, 100), UDim2.new(1, 0, 0.50005, 100))).to.equal(true) + end) + + it("X offset differs", function() + expect(trueEquals(UDim2.new(1, 0, 0.5, 100), UDim2.new(1, 1, 0.5, 100))).to.equal(false) + end) + + it("Y offset differs", function() + expect(trueEquals(UDim2.new(1, 0, 0.5, 100), UDim2.new(1, 0, 0.5, 101))).to.equal(false) + end) + + it("fromOffset identical", function() + expect(trueEquals(UDim2.fromOffset(100, 200), UDim2.fromOffset(100, 200))).to.equal(true) + end) + + it("fromScale identical", function() + expect(trueEquals(UDim2.fromScale(0.5, 0.5), UDim2.fromScale(0.5, 0.5))).to.equal(true) + end) + end) + + describe("NumberRange", function() + it("identical", function() + expect(trueEquals(NumberRange.new(0, 10), NumberRange.new(0, 10))).to.equal(true) + end) + + it("min within epsilon", function() + expect(trueEquals(NumberRange.new(0, 10), NumberRange.new(0.00005, 10))).to.equal(true) + end) + + it("max beyond epsilon", function() + expect(trueEquals(NumberRange.new(0, 10), NumberRange.new(0, 10.002))).to.equal(false) + end) + + it("single-value range", function() + expect(trueEquals(NumberRange.new(5), NumberRange.new(5))).to.equal(true) + end) + + it("different ranges", function() + expect(trueEquals(NumberRange.new(0, 10), NumberRange.new(0, 20))).to.equal(false) + end) + end) + + describe("Rect", function() + it("identical", function() + expect(trueEquals(Rect.new(0, 0, 100, 200), Rect.new(0, 0, 100, 200))).to.equal(true) + end) + + it("min X within epsilon", function() + expect(trueEquals(Rect.new(0, 0, 100, 200), Rect.new(0.00005, 0, 100, 200))).to.equal(true) + end) + + it("max Y beyond epsilon", function() + expect(trueEquals(Rect.new(0, 0, 100, 200), Rect.new(0, 0, 100, 200.5))).to.equal(false) + end) + + it("from Vector2 identical", function() + expect( + trueEquals( + Rect.new(Vector2.new(10, 20), Vector2.new(30, 40)), + Rect.new(Vector2.new(10, 20), Vector2.new(30, 40)) + ) + ).to.equal(true) + end) + end) + + describe("NumberSequence", function() + it("identical two-keypoint", function() + expect( + trueEquals( + NumberSequence.new({ NumberSequenceKeypoint.new(0, 0), NumberSequenceKeypoint.new(1, 1) }), + NumberSequence.new({ NumberSequenceKeypoint.new(0, 0), NumberSequenceKeypoint.new(1, 1) }) + ) + ).to.equal(true) + end) + + it("value within epsilon", function() + expect( + trueEquals( + NumberSequence.new({ NumberSequenceKeypoint.new(0, 0), NumberSequenceKeypoint.new(1, 1) }), + NumberSequence.new({ NumberSequenceKeypoint.new(0, 0), NumberSequenceKeypoint.new(1, 1.00005) }) + ) + ).to.equal(true) + end) + + it("value beyond epsilon", function() + expect( + trueEquals( + NumberSequence.new({ NumberSequenceKeypoint.new(0, 0), NumberSequenceKeypoint.new(1, 1) }), + NumberSequence.new({ NumberSequenceKeypoint.new(0, 0), NumberSequenceKeypoint.new(1, 1.002) }) + ) + ).to.equal(false) + end) + + it("different keypoint count", function() + expect( + trueEquals( + NumberSequence.new({ NumberSequenceKeypoint.new(0, 0), NumberSequenceKeypoint.new(1, 1) }), + NumberSequence.new({ + NumberSequenceKeypoint.new(0, 0), + NumberSequenceKeypoint.new(0.5, 0.5), + NumberSequenceKeypoint.new(1, 1), + }) + ) + ).to.equal(false) + end) + + it("simple constructor identical", function() + expect(trueEquals(NumberSequence.new(5), NumberSequence.new(5))).to.equal(true) + end) + + it("envelope within epsilon", function() + expect( + trueEquals( + NumberSequence.new({ NumberSequenceKeypoint.new(0, 1, 0.5), NumberSequenceKeypoint.new(1, 1, 0.5) }), + NumberSequence.new({ + NumberSequenceKeypoint.new(0, 1, 0.50005), + NumberSequenceKeypoint.new(1, 1, 0.5), + }) + ) + ).to.equal(true) + end) + end) + + describe("ColorSequence", function() + it("identical two-keypoint", function() + expect(trueEquals( + ColorSequence.new({ + ColorSequenceKeypoint.new(0, Color3.new(1, 0, 0)), + ColorSequenceKeypoint.new(1, Color3.new(0, 0, 1)), + }), + ColorSequence.new({ + ColorSequenceKeypoint.new(0, Color3.new(1, 0, 0)), + ColorSequenceKeypoint.new(1, Color3.new(0, 0, 1)), + }) + )).to.equal(true) + end) + + it("color within epsilon", function() + expect(trueEquals( + ColorSequence.new({ + ColorSequenceKeypoint.new(0, Color3.new(0.5, 0, 0)), + ColorSequenceKeypoint.new(1, Color3.new(0, 0, 1)), + }), + ColorSequence.new({ + ColorSequenceKeypoint.new(0, Color3.new(0.50005, 0, 0)), + ColorSequenceKeypoint.new(1, Color3.new(0, 0, 1)), + }) + )).to.equal(true) + end) + + it("color beyond epsilon", function() + expect(trueEquals( + ColorSequence.new({ + ColorSequenceKeypoint.new(0, Color3.new(0.5, 0, 0)), + ColorSequenceKeypoint.new(1, Color3.new(0, 0, 1)), + }), + ColorSequence.new({ + ColorSequenceKeypoint.new(0, Color3.new(0.6, 0, 0)), + ColorSequenceKeypoint.new(1, Color3.new(0, 0, 1)), + }) + )).to.equal(false) + end) + + it("different keypoint count", function() + expect(trueEquals( + ColorSequence.new({ + ColorSequenceKeypoint.new(0, Color3.new(1, 0, 0)), + ColorSequenceKeypoint.new(1, Color3.new(0, 0, 1)), + }), + ColorSequence.new({ + ColorSequenceKeypoint.new(0, Color3.new(1, 0, 0)), + ColorSequenceKeypoint.new(0.5, Color3.new(0, 1, 0)), + ColorSequenceKeypoint.new(1, Color3.new(0, 0, 1)), + }) + )).to.equal(false) + end) + + it("simple constructor identical", function() + expect(trueEquals(ColorSequence.new(Color3.new(1, 0, 0)), ColorSequence.new(Color3.new(1, 0, 0)))).to.equal( + true + ) + end) + + it("time within epsilon", function() + expect(trueEquals( + ColorSequence.new({ + ColorSequenceKeypoint.new(0, Color3.new(1, 0, 0)), + ColorSequenceKeypoint.new(1, Color3.new(0, 0, 1)), + }), + ColorSequence.new({ + ColorSequenceKeypoint.new(0.00005, Color3.new(1, 0, 0)), + ColorSequenceKeypoint.new(1, Color3.new(0, 0, 1)), + }) + )).to.equal(true) + end) + end) + + describe("Font", function() + it("identical fonts", function() + local a = + Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal) + local b = + Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal) + expect(trueEquals(a, b)).to.equal(true) + end) + + it("different family", function() + local a = + Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal) + local b = + Font.new("rbxasset://fonts/families/RobotoMono.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal) + expect(trueEquals(a, b)).to.equal(false) + end) + + it("different weight", function() + local a = + Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal) + local b = + Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Bold, Enum.FontStyle.Normal) + expect(trueEquals(a, b)).to.equal(false) + end) + + it("different style", function() + local a = + Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal) + local b = + Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Italic) + expect(trueEquals(a, b)).to.equal(false) + end) + + it("same reference via rawequal", function() + local a = Font.new("rbxasset://fonts/families/SourceSansPro.json") + expect(trueEquals(a, a)).to.equal(true) + end) + + it("semibold vs regular", function() + local a = Font.new( + "rbxasset://fonts/families/SourceSansPro.json", + Enum.FontWeight.SemiBold, + Enum.FontStyle.Normal + ) + local b = + Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal) + expect(trueEquals(a, b)).to.equal(false) + end) + end) + + describe("BrickColor", function() + it("same BrickColor", function() + expect(trueEquals(BrickColor.new("Bright red"), BrickColor.new("Bright red"))).to.equal(true) + end) + + it("different BrickColor", function() + expect(trueEquals(BrickColor.new("Bright red"), BrickColor.new("Bright blue"))).to.equal(false) + end) + end) + + describe("fallback == equality", function() + it("true vs true", function() + expect(trueEquals(true, true)).to.equal(true) + end) + + it("false vs false", function() + expect(trueEquals(false, false)).to.equal(true) + end) + + it("true vs false", function() + expect(trueEquals(true, false)).to.equal(false) + end) + + it("false vs true", function() + expect(trueEquals(false, true)).to.equal(false) + end) + + it("same string content", function() + expect(trueEquals("hello", "hello")).to.equal(true) + end) + + it("different string content", function() + expect(trueEquals("hello", "world")).to.equal(false) + end) + + it("empty strings", function() + expect(trueEquals("", "")).to.equal(true) + end) + + it("same Axes", function() + expect(trueEquals(Axes.new(Enum.Axis.X, Enum.Axis.Y), Axes.new(Enum.Axis.X, Enum.Axis.Y))).to.equal(true) + end) + + it("different Axes", function() + expect(trueEquals(Axes.new(Enum.Axis.X), Axes.new(Enum.Axis.Y))).to.equal(false) + end) + + it("same Faces", function() + expect(trueEquals(Faces.new(Enum.NormalId.Top), Faces.new(Enum.NormalId.Top))).to.equal(true) + end) + + it("different Faces", function() + expect(trueEquals(Faces.new(Enum.NormalId.Top), Faces.new(Enum.NormalId.Bottom))).to.equal(false) + end) + + it("same Region3int16", function() + local a = Region3int16.new(Vector3int16.new(0, 0, 0), Vector3int16.new(10, 10, 10)) + local b = Region3int16.new(Vector3int16.new(0, 0, 0), Vector3int16.new(10, 10, 10)) + expect(trueEquals(a, b)).to.equal(true) + end) + + it("different Region3int16", function() + local a = Region3int16.new(Vector3int16.new(0, 0, 0), Vector3int16.new(10, 10, 10)) + local b = Region3int16.new(Vector3int16.new(0, 0, 0), Vector3int16.new(20, 20, 20)) + expect(trueEquals(a, b)).to.equal(false) + end) + + it("same Vector3int16", function() + expect(trueEquals(Vector3int16.new(1, 2, 3), Vector3int16.new(1, 2, 3))).to.equal(true) + end) + + it("different Vector3int16", function() + expect(trueEquals(Vector3int16.new(1, 2, 3), Vector3int16.new(4, 5, 6))).to.equal(false) + end) + + it("same Vector2int16", function() + expect(trueEquals(Vector2int16.new(1, 2), Vector2int16.new(1, 2))).to.equal(true) + end) + + it("different Vector2int16", function() + expect(trueEquals(Vector2int16.new(1, 2), Vector2int16.new(3, 4))).to.equal(false) + end) + + it("same PhysicalProperties", function() + local a = PhysicalProperties.new(1, 0.5, 0.3) + local b = PhysicalProperties.new(1, 0.5, 0.3) + expect(trueEquals(a, b)).to.equal(true) + end) + + it("different PhysicalProperties", function() + local a = PhysicalProperties.new(1, 0.5, 0.3) + local b = PhysicalProperties.new(2, 0.5, 0.3) + expect(trueEquals(a, b)).to.equal(false) + end) + + it("different Instances are not equal", function() + local a = Instance.new("Folder") + local b = Instance.new("Folder") + expect(trueEquals(a, b)).to.equal(false) + a:Destroy() + b:Destroy() + end) + + it("same EnumItem via fallback", function() + expect(trueEquals(Enum.SortOrder.LayoutOrder, Enum.SortOrder.LayoutOrder)).to.equal(true) + end) + + it("different EnumItems of same enum", function() + expect(trueEquals(Enum.SortOrder.LayoutOrder, Enum.SortOrder.Name)).to.equal(false) + end) + end) + + describe("attribute dictionaries", function() + it("identical Color3 attributes", function() + local a = { BackgroundColor = Color3.fromRGB(32, 34, 39), TextSize = 14 } + local b = { BackgroundColor = Color3.fromRGB(32, 34, 39), TextSize = 14 } + expect(trueEquals(a, b)).to.equal(true) + end) + + it("Color3 cleanup_f32 precision", function() + local a = Color3.fromRGB(32, 34, 39) + local b = Color3.new(0.125490, 0.133333, 0.152941) + expect(trueEquals(a, b)).to.equal(true) + end) + + it("dict with Color3 cleanup_f32 precision", function() + local a = { BackgroundColor = Color3.fromRGB(32, 34, 39) } + local b = { BackgroundColor = Color3.new(0.125490, 0.133333, 0.152941) } + expect(trueEquals(a, b)).to.equal(true) + end) + + it("dict with Font values", function() + local font = Font.new( + "rbxasset://fonts/families/SourceSansPro.json", + Enum.FontWeight.SemiBold, + Enum.FontStyle.Normal + ) + local a = { LabelFont = font } + local b = { + LabelFont = Font.new( + "rbxasset://fonts/families/SourceSansPro.json", + Enum.FontWeight.SemiBold, + Enum.FontStyle.Normal + ), + } + expect(trueEquals(a, b)).to.equal(true) + end) + + it("dict with mixed types matching", function() + local a = { + BackgroundColor = Color3.fromRGB(128, 128, 128), + Transparency = 0.25, + Size = 14, + Label = "Hello", + LabelFont = Font.new( + "rbxasset://fonts/families/SourceSansPro.json", + Enum.FontWeight.Regular, + Enum.FontStyle.Normal + ), + Corner = UDim.new(0, 8), + } + local b = { + BackgroundColor = Color3.fromRGB(128, 128, 128), + Transparency = 0.25, + Size = 14, + Label = "Hello", + LabelFont = Font.new( + "rbxasset://fonts/families/SourceSansPro.json", + Enum.FontWeight.Regular, + Enum.FontStyle.Normal + ), + Corner = UDim.new(0, 8), + } + expect(trueEquals(a, b)).to.equal(true) + end) + + it("dict with one Font mismatch fails whole table", function() + local a = { + BackgroundColor = Color3.fromRGB(128, 128, 128), + LabelFont = Font.new( + "rbxasset://fonts/families/SourceSansPro.json", + Enum.FontWeight.Regular, + Enum.FontStyle.Normal + ), + } + local b = { + BackgroundColor = Color3.fromRGB(128, 128, 128), + LabelFont = Font.new( + "rbxasset://fonts/families/SourceSansPro.json", + Enum.FontWeight.Bold, + Enum.FontStyle.Normal + ), + } + expect(trueEquals(a, b)).to.equal(false) + end) + + it("extra key in new table", function() + local a = { BackgroundColor = Color3.new(1, 0, 0) } + local b = { BackgroundColor = Color3.new(1, 0, 0), NewAttr = "hello" } + expect(trueEquals(a, b)).to.equal(false) + end) + + it("missing key in new table", function() + local a = { BackgroundColor = Color3.new(1, 0, 0), OldAttr = "hello" } + local b = { BackgroundColor = Color3.new(1, 0, 0) } + expect(trueEquals(a, b)).to.equal(false) + end) + end) + + describe("type mismatches", function() + it("string vs number", function() + expect(trueEquals("hello", 42)).to.equal(false) + end) + + it("boolean vs number", function() + expect(trueEquals(true, 1)).to.equal(false) + end) + + it("string '1' vs number 1", function() + expect(trueEquals("1", 1)).to.equal(false) + end) + + it("table vs string", function() + expect(trueEquals({}, "table")).to.equal(false) + end) + + it("Vector3 vs table", function() + expect(trueEquals(Vector3.new(1, 2, 3), { 1, 2, 3 })).to.equal(false) + end) + + it("Color3 vs Vector3", function() + expect(trueEquals(Color3.new(1, 0, 0), Vector3.new(1, 0, 0))).to.equal(false) + end) + + it("true vs false", function() + expect(trueEquals(true, false)).to.equal(false) + end) + + it("different strings", function() + expect(trueEquals("abc", "abd")).to.equal(false) + end) + end) +end diff --git a/plugin/src/SHA1.luau b/plugin/src/SHA1.luau new file mode 100644 index 00000000..bd2f39cc --- /dev/null +++ b/plugin/src/SHA1.luau @@ -0,0 +1,112 @@ +--!strict +--!optimize 2 +--!native + +local OFFSETS = buffer.create(320) + +local function PreProcess(Contents: buffer): (buffer, number) + local ContentLength = buffer.len(Contents) + local Padding = (64 - ((ContentLength + 9) % 64)) % 64 + + local NewContentLength = ContentLength + 1 + Padding + 8 + local NewContent = buffer.create(NewContentLength) + buffer.copy(NewContent, 0, Contents) + buffer.writeu8(NewContent, ContentLength, 128) + + local Length8 = ContentLength * 8 + for Index = 7, 0, -1 do + local Remainder = Length8 % 256 + buffer.writeu8(NewContent, Index + ContentLength + 1 + Padding, Remainder) + Length8 = (Length8 - Remainder) / 256 + end + + return NewContent, NewContentLength +end + +local function DigestBlocks(Blocks: buffer, Length: number): (number, number, number, number, number) + local A, B, C, D, E = 0x67452301, 0xefcdaB89, 0x98badcfe, 0x10325476, 0xc3d2e1f0 + local Offsets = OFFSETS + + for Offset = 0, Length - 1, 64 do + for BlockIndex = 0, 60, 4 do + buffer.writeu32(Offsets, BlockIndex, bit32.byteswap(buffer.readu32(Blocks, Offset + BlockIndex))) + end + + for Index = 64, 316, 4 do + buffer.writeu32( + Offsets, + Index, + bit32.lrotate( + bit32.bxor( + buffer.readu32(Offsets, Index - 12), + buffer.readu32(Offsets, Index - 32), + buffer.readu32(Offsets, Index - 56), + buffer.readu32(Offsets, Index - 64) + ), + 1 + ) + ) + end + + local H1, H2, H3, H4, H5 = A, B, C, D, E + + local Temp + for Round = 0, 19 do + Temp = bit32.lrotate(H1, 5) + + bit32.band(H2, H3) + + bit32.band(-1 - H2, H4) + + H5 + + 0x5a827999 + + buffer.readu32(Offsets, Round * 4) + + H5, H4, H3, H2, H1 = H4, H3, bit32.lrotate(H2, 30), H1, Temp + end + + for Round = 20, 39 do + Temp = bit32.lrotate(H1, 5) + bit32.bxor(H2, H3, H4) + H5 + 0x6ed9eba1 + buffer.readu32(Offsets, Round * 4) + + H5, H4, H3, H2, H1 = H4, H3, bit32.lrotate(H2, 30), H1, Temp + end + + for Round = 40, 59 do + Temp = bit32.lrotate(H1, 5) + + bit32.band(H4, H3) + + bit32.band(H2, bit32.bxor(H4, H3)) + + H5 + + 0x8f1bbcdc + + buffer.readu32(Offsets, Round * 4) + + H5, H4, H3, H2, H1 = H4, H3, bit32.lrotate(H2, 30), H1, Temp + end + + for Round = 60, 79 do + Temp = bit32.lrotate(H1, 5) + bit32.bxor(H2, H3, H4) + H5 + 0xca62c1d6 + buffer.readu32(Offsets, Round * 4) + + H5, H4, H3, H2, H1 = H4, H3, bit32.lrotate(H2, 30), H1, Temp + end + + A = bit32.bor(A + H1, 0) + B = bit32.bor(B + H2, 0) + C = bit32.bor(C + H3, 0) + D = bit32.bor(D + H4, 0) + E = bit32.bor(E + H5, 0) + end + + return A, B, C, D, E +end + +local function SHA1(Message: buffer, Salt: buffer?): string + if Salt and buffer.len(Salt) > 0 then + local MessageWithSalt = buffer.create(buffer.len(Message) + buffer.len(Salt)) + + buffer.copy(MessageWithSalt, 0, Message) + buffer.copy(MessageWithSalt, buffer.len(Message), Salt) + + Message = MessageWithSalt + end + + local ProcessedMessage, Length = PreProcess(Message) + return string.format("%08x%08x%08x%08x%08x", DigestBlocks(ProcessedMessage, Length)) +end + +return SHA1 diff --git a/plugin/src/ServeSession.lua b/plugin/src/ServeSession.lua index 039b4875..7f3f7fce 100644 --- a/plugin/src/ServeSession.lua +++ b/plugin/src/ServeSession.lua @@ -17,6 +17,7 @@ local encodePatchUpdate = require(script.Parent.ChangeBatcher.encodePatchUpdate) local encodeInstance = require(script.Parent.ChangeBatcher.encodeInstance) local InstanceMap = require(script.Parent.InstanceMap) local PatchSet = require(script.Parent.PatchSet) +local Matching = require(script.Parent.Reconciler.matching) local Reconciler = require(script.Parent.Reconciler) local strict = require(script.Parent.strict) local Settings = require(script.Parent.Settings) @@ -625,7 +626,8 @@ function ServeSession:__computeInitialPatch(serverInfo) -- tracking them in the reconciler. Log.trace("Matching existing Roblox instances to Rojo IDs") self:setLoadingText("Hydrating instance map...") - self.__reconciler:hydrate(readResponseBody.instances, serverInfo.rootInstanceId, game) + local matchingSession = Matching.newSession() + self.__reconciler:hydrate(readResponseBody.instances, serverInfo.rootInstanceId, game, matchingSession) -- Calculate the initial patch to apply to the DataModel to catch us -- up to what Rojo thinks the place should look like. @@ -695,6 +697,7 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo) elseif type(userDecision) == "table" and userDecision.type == "Confirm" then -- New: Apply based on per-item selections local selections = userDecision.selections or {} + local autoSelectedIds = userDecision.autoSelectedIds or {} -- Log selection summary (debug level) local pushCount, pullCount, ignoreCount = 0, 0, 0 @@ -713,6 +716,9 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo) local pushPatch = PatchSet.newEmpty() -- Items to apply to Studio local pullPatch = PatchSet.newEmpty() -- Items to send back to Rojo + -- Build stage_ids: all push-accepted IDs + auto-selected pull-accepted IDs + local stageIds = {} + -- Process updated items for _, change in catchUpPatch.updated do local selection = selections[change.id] @@ -723,6 +729,7 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo) -- Apply Rojo changes to Studio Log.info("[Push] Update: {}", instancePath) table.insert(pushPatch.updated, change) + table.insert(stageIds, change.id) elseif selection == "pull" and self.__twoWaySync then -- Send Studio state back to Rojo if instance then @@ -749,6 +756,9 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo) ) if update then table.insert(pullPatch.updated, update) + if autoSelectedIds[change.id] then + table.insert(stageIds, change.id) + end end end end @@ -778,6 +788,9 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo) -- Apply Rojo removal to Studio (delete the instance) Log.info("[Push] Delete: {}", instancePath) table.insert(pushPatch.removed, idOrInstance) + if id then + table.insert(stageIds, id) + end elseif selection == "pull" and self.__twoWaySync then -- Syncback: Create file in Rojo from Studio instance if instance and instance.Parent then @@ -790,6 +803,9 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo) local guid = HttpService:GenerateGUID(false) local tempRef = string.gsub(guid, "-", ""):lower() pullPatch.added[tempRef] = encoded + if id and autoSelectedIds[id] then + table.insert(stageIds, tempRef) + end -- Note: We intentionally do NOT pre-insert into InstanceMap here. -- The VFS watcher will process the written files and assign a -- server-side Ref ID. Any Ref properties targeting this instance @@ -818,6 +834,7 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo) -- Apply Rojo addition to Studio Log.info("[Push] Add: {}.{}", parentPath, instanceName) pushPatch.added[id] = change + table.insert(stageIds, id) elseif selection == "pull" and self.__twoWaySync then -- Don't add in Studio, remove from Rojo -- Log at info level since this is a major file deletion operation @@ -846,19 +863,27 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo) self:__applyPatch(pushPatch) end - -- Send pull items back to Rojo server - if self.__twoWaySync and not PatchSet.isEmpty(pullPatch) then - local addCount = 0 - for _ in pairs(pullPatch.added) do - addCount += 1 + -- Send pull items + stage requests to Rojo server + local hasPullChanges = self.__twoWaySync and not PatchSet.isEmpty(pullPatch) + local hasStageIds = #stageIds > 0 + + if hasPullChanges or hasStageIds then + if hasPullChanges then + local addCount = 0 + for _ in pairs(pullPatch.added) do + addCount += 1 + end + Log.info( + "Sending to Rojo: {} file creations, {} file deletions, {} updates", + addCount, + #pullPatch.removed, + #pullPatch.updated + ) end - Log.info( - "Sending to Rojo: {} file creations, {} file deletions, {} updates", - addCount, - #pullPatch.removed, - #pullPatch.updated - ) - return self.__apiContext:write(pullPatch) + if hasStageIds then + Log.info("Requesting git stage for {} files", #stageIds) + end + return self.__apiContext:write(pullPatch, stageIds) end return Promise.resolve() diff --git a/plugin/src/XXH32.luau b/plugin/src/XXH32.luau new file mode 100644 index 00000000..f8516897 --- /dev/null +++ b/plugin/src/XXH32.luau @@ -0,0 +1,117 @@ +--!strict +--!optimize 2 +--!native + +local function XXH32(Message: buffer, Seed: number?): number + local PRIME_1, PRIME_1_HIGH, PRIME_1_LOW = 0x9e3779B1, 40503, 31153 + local PRIME_2, PRIME_2_HIGH, PRIME_2_LOW = 0x85ebca77, 34283, 51831 + local PRIME_3_HIGH, PRIME_3_LOW = 49842, 44605 + local PRIME_4_HIGH, PRIME_4_LOW = 10196, 60207 + local PRIME_5, PRIME_5_HIGH, PRIME_5_LOW = 0x165667b1, 5718, 26545 + + local UsedSeed = Seed or 0 + local MessageLength = buffer.len(Message) + local Digest: number + local CurrentOffset = 0 + + if MessageLength >= 16 then + local Accumulator1 = UsedSeed + PRIME_1 + PRIME_2 + local Accumulator2 = UsedSeed + PRIME_2 + local Accumulator3 = UsedSeed + local Accumulator4 = UsedSeed - PRIME_1 + + while CurrentOffset <= MessageLength - 16 do + local Word1 = buffer.readu32(Message, CurrentOffset) + local Word2 = buffer.readu32(Message, CurrentOffset + 4) + local Word3 = buffer.readu32(Message, CurrentOffset + 8) + local Word4 = buffer.readu32(Message, CurrentOffset + 12) + + local AHigh1, ALow1 = bit32.rshift(Word1, 16), bit32.band(Word1, 65535) + local Mult1 = bit32.lshift((AHigh1 * PRIME_2_LOW) + (ALow1 * PRIME_2_HIGH), 16) + (ALow1 * PRIME_2_LOW) + + local Temp1 = bit32.lrotate(Accumulator1 + Mult1, 13) + local AHigh1_2, ALow1_2 = bit32.rshift(Temp1, 16), bit32.band(Temp1, 65535) + Accumulator1 = bit32.lshift((AHigh1_2 * PRIME_1_LOW) + (ALow1_2 * PRIME_1_HIGH), 16) + + (ALow1_2 * PRIME_1_LOW) + + local AHigh2, ALow2 = bit32.rshift(Word2, 16), bit32.band(Word2, 65535) + local Mult2 = bit32.lshift((AHigh2 * PRIME_2_LOW) + (ALow2 * PRIME_2_HIGH), 16) + (ALow2 * PRIME_2_LOW) + + local Temp2 = bit32.lrotate(Accumulator2 + Mult2, 13) + local AHigh2_2, ALow2_2 = bit32.rshift(Temp2, 16), bit32.band(Temp2, 65535) + Accumulator2 = bit32.lshift((AHigh2_2 * PRIME_1_LOW) + (ALow2_2 * PRIME_1_HIGH), 16) + + (ALow2_2 * PRIME_1_LOW) + + local AHigh3, ALow3 = bit32.rshift(Word3, 16), bit32.band(Word3, 65535) + local Mult3 = bit32.lshift((AHigh3 * PRIME_2_LOW) + (ALow3 * PRIME_2_HIGH), 16) + (ALow3 * PRIME_2_LOW) + + local Temp3 = bit32.lrotate(Accumulator3 + Mult3, 13) + local AHigh3_2, ALow3_2 = bit32.rshift(Temp3, 16), bit32.band(Temp3, 65535) + Accumulator3 = bit32.lshift((AHigh3_2 * PRIME_1_LOW) + (ALow3_2 * PRIME_1_HIGH), 16) + + (ALow3_2 * PRIME_1_LOW) + + local AHigh4, ALow4 = bit32.rshift(Word4, 16), bit32.band(Word4, 65535) + local Mult4 = bit32.lshift((AHigh4 * PRIME_2_LOW) + (ALow4 * PRIME_2_HIGH), 16) + (ALow4 * PRIME_2_LOW) + + local Temp4 = bit32.lrotate(Accumulator4 + Mult4, 13) + local AHigh4_2, ALow4_2 = bit32.rshift(Temp4, 16), bit32.band(Temp4, 65535) + Accumulator4 = bit32.lshift((AHigh4_2 * PRIME_1_LOW) + (ALow4_2 * PRIME_1_HIGH), 16) + + (ALow4_2 * PRIME_1_LOW) + + CurrentOffset += 16 + end + + Digest = bit32.lrotate(Accumulator1, 1) + + bit32.lrotate(Accumulator2, 7) + + bit32.lrotate(Accumulator3, 12) + + bit32.lrotate(Accumulator4, 18) + else + Digest = UsedSeed + PRIME_5 + end + + Digest += MessageLength + + while CurrentOffset <= MessageLength - 4 do + if CurrentOffset + 4 <= buffer.len(Message) then + local Word = buffer.readu32(Message, CurrentOffset) + + local AHigh_w, ALow_w = bit32.rshift(Word, 16), bit32.band(Word, 65535) + local Mult_w = bit32.lshift((AHigh_w * PRIME_3_LOW) + (ALow_w * PRIME_3_HIGH), 16) + (ALow_w * PRIME_3_LOW) + + Digest += Mult_w + + local Rotated = bit32.lrotate(Digest, 17) + local AHigh_r, ALow_r = bit32.rshift(Rotated, 16), bit32.band(Rotated, 65535) + Digest = bit32.lshift((AHigh_r * PRIME_4_LOW) + (ALow_r * PRIME_4_HIGH), 16) + (ALow_r * PRIME_4_LOW) + end + CurrentOffset += 4 + end + + while CurrentOffset < MessageLength do + if CurrentOffset < buffer.len(Message) then + local ByteValue = buffer.readu8(Message, CurrentOffset) + + local AHigh_b, ALow_b = bit32.rshift(ByteValue, 16), bit32.band(ByteValue, 65535) + local Mult_b = bit32.lshift((AHigh_b * PRIME_5_LOW) + (ALow_b * PRIME_5_HIGH), 16) + (ALow_b * PRIME_5_LOW) + + Digest += Mult_b + + local Rotated_b = bit32.lrotate(Digest, 11) + local AHigh_rb, ALow_rb = bit32.rshift(Rotated_b, 16), bit32.band(Rotated_b, 65535) + Digest = bit32.lshift((AHigh_rb * PRIME_1_LOW) + (ALow_rb * PRIME_1_HIGH), 16) + (ALow_rb * PRIME_1_LOW) + end + CurrentOffset += 1 + end + + local XorResult1 = bit32.bxor(Digest, bit32.rshift(Digest, 15)) + local AHigh_f1, ALow_f1 = bit32.rshift(XorResult1, 16), bit32.band(XorResult1, 65535) + Digest = bit32.lshift((AHigh_f1 * PRIME_2_LOW) + (ALow_f1 * PRIME_2_HIGH), 16) + (ALow_f1 * PRIME_2_LOW) + + local XorResult2 = bit32.bxor(Digest, bit32.rshift(Digest, 13)) + local AHigh_f2, ALow_f2 = bit32.rshift(XorResult2, 16), bit32.band(XorResult2, 65535) + Digest = bit32.lshift((AHigh_f2 * PRIME_3_LOW) + (ALow_f2 * PRIME_3_HIGH), 16) + (ALow_f2 * PRIME_3_LOW) + + return bit32.bxor(Digest, bit32.rshift(Digest, 16)) +end + +return XXH32 diff --git a/plugin/src/testUtils/MockApiContext.lua b/plugin/src/testUtils/MockApiContext.lua index a0d70f44..e54c09a6 100644 --- a/plugin/src/testUtils/MockApiContext.lua +++ b/plugin/src/testUtils/MockApiContext.lua @@ -10,6 +10,41 @@ local HttpService = game:GetService("HttpService") +local MockPromise = {} +MockPromise.__index = MockPromise + +function MockPromise.resolve(value) + return setmetatable({ _value = value, _rejected = false }, MockPromise) +end + +function MockPromise.reject(err) + return setmetatable({ _value = err, _rejected = true }, MockPromise) +end + +function MockPromise:andThen(callback) + if not self._rejected then + local ok, result = pcall(callback, self._value) + if ok then + return MockPromise.resolve(result) + else + return MockPromise.reject(result) + end + end + return self +end + +function MockPromise:catch(callback) + if self._rejected then + local ok, result = pcall(callback, self._value) + if ok then + return MockPromise.resolve(result) + else + return MockPromise.reject(result) + end + end + return self +end + local MockApiContext = {} MockApiContext.__index = MockApiContext @@ -155,22 +190,15 @@ function MockApiContext:write(patch) self:_maybeDelay() if self.failOnWrite then - return false, "Mock write failure" + return MockPromise.reject("Mock write failure") end if self.onWrite then - local result = self.onWrite(self, patch) - if result ~= nil then - return result - end + self.onWrite(self, patch) end - if self.writeResponse then - return true, self.writeResponse - end - - -- Default success response - return true, { success = true } + local response = self.writeResponse or { success = true } + return MockPromise.resolve(response) end --[[ diff --git a/rbx-dom b/rbx-dom index 3cd393d7..b75dd401 160000 --- a/rbx-dom +++ b/rbx-dom @@ -1 +1 @@ -Subproject commit 3cd393d73a6ce81282b1999797026beeaa91d6f2 +Subproject commit b75dd401b252a6484cf9fad29bee01ad2dbafee7 diff --git a/rojo-test/build-test-snapshots/end_to_end__tests__build__dedup_suffix_auto_strip.snap b/rojo-test/build-test-snapshots/end_to_end__tests__build__dedup_suffix_auto_strip.snap new file mode 100644 index 00000000..08eb1e11 --- /dev/null +++ b/rojo-test/build-test-snapshots/end_to_end__tests__build__dedup_suffix_auto_strip.snap @@ -0,0 +1,32 @@ +--- +source: tests/tests/build.rs +expression: contents +--- + + + + dedup_suffix_auto_strip + + + + Baz~0 + + + + + + Foo + + + + + + Foo + + + + + diff --git a/rojo-test/build-test-snapshots/end_to_end__tests__build__tilde_no_meta.snap b/rojo-test/build-test-snapshots/end_to_end__tests__build__tilde_no_meta.snap index 4eabf6af..b25830d0 100644 --- a/rojo-test/build-test-snapshots/end_to_end__tests__build__tilde_no_meta.snap +++ b/rojo-test/build-test-snapshots/end_to_end__tests__build__tilde_no_meta.snap @@ -9,8 +9,8 @@ expression: contents - Foo~1 - Foo + diff --git a/rojo-test/build-tests/dedup_suffix_auto_strip/default.project.json5 b/rojo-test/build-tests/dedup_suffix_auto_strip/default.project.json5 new file mode 100644 index 00000000..225a5264 --- /dev/null +++ b/rojo-test/build-tests/dedup_suffix_auto_strip/default.project.json5 @@ -0,0 +1,6 @@ +{ + "name": "dedup_suffix_auto_strip", + "tree": { + "$path": "src" + } +} diff --git a/rojo-test/build-tests/dedup_suffix_auto_strip/src/Baz~0.luau b/rojo-test/build-tests/dedup_suffix_auto_strip/src/Baz~0.luau new file mode 100644 index 00000000..49cc226c --- /dev/null +++ b/rojo-test/build-tests/dedup_suffix_auto_strip/src/Baz~0.luau @@ -0,0 +1 @@ +return "not a dedup suffix" diff --git a/rojo-test/build-tests/dedup_suffix_auto_strip/src/Foo.luau b/rojo-test/build-tests/dedup_suffix_auto_strip/src/Foo.luau new file mode 100644 index 00000000..6bb0d45c --- /dev/null +++ b/rojo-test/build-tests/dedup_suffix_auto_strip/src/Foo.luau @@ -0,0 +1 @@ +return "first" diff --git a/rojo-test/build-tests/dedup_suffix_auto_strip/src/Foo~2.luau b/rojo-test/build-tests/dedup_suffix_auto_strip/src/Foo~2.luau new file mode 100644 index 00000000..70a0843e --- /dev/null +++ b/rojo-test/build-tests/dedup_suffix_auto_strip/src/Foo~2.luau @@ -0,0 +1 @@ +return "second" diff --git a/rojo-test/build-tests/tilde_no_meta/src/Foo~1.luau b/rojo-test/build-tests/tilde_no_meta/src/Foo~1.luau index 972b31c7..517b8840 100644 --- a/rojo-test/build-tests/tilde_no_meta/src/Foo~1.luau +++ b/rojo-test/build-tests/tilde_no_meta/src/Foo~1.luau @@ -1,2 +1,2 @@ --- Tilde in filename without meta should produce instance name "Foo~1" +-- Dedup suffix ~1: forward sync strips it to produce instance name "Foo" return nil diff --git a/rojo-test/serve-test-snapshots/end_to_end__tests__connected_mode__connected_dedup_no_meta_stem_patch.snap b/rojo-test/serve-test-snapshots/end_to_end__tests__connected_mode__connected_dedup_no_meta_stem_patch.snap index c742866e..5c87e49b 100644 --- a/rojo-test/serve-test-snapshots/end_to_end__tests__connected_mode__connected_dedup_no_meta_stem_patch.snap +++ b/rojo-test/serve-test-snapshots/end_to_end__tests__connected_mode__connected_dedup_no_meta_stem_patch.snap @@ -12,7 +12,7 @@ body: Id: id-13 Metadata: ignoreUnknownInstances: false - Name: Foo~1 + Name: Foo Parent: id-3 Properties: Source: diff --git a/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_change_target_patch.snap b/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_change_target_patch.snap index 05c77a9b..6ce2b91d 100644 --- a/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_change_target_patch.snap +++ b/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_change_target_patch.snap @@ -15,7 +15,7 @@ body: Attributes: "!Attributes": Rojo_Ref_PrimaryPart: - "!String": Workspace/MyModel/OtherPart + "!String": "@self/OtherPart" PrimaryPart: "!Ref": id-4 id: id-3 diff --git a/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_multiple_attrs_patch.snap b/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_multiple_attrs_patch.snap index a899dd0c..b369de5c 100644 --- a/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_multiple_attrs_patch.snap +++ b/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_multiple_attrs_patch.snap @@ -15,9 +15,9 @@ body: Attributes: "!Attributes": Rojo_Ref_CustomRef: - "!String": Workspace/MyModel/OtherPart + "!String": "@self/OtherPart" Rojo_Ref_PrimaryPart: - "!String": Workspace/MyModel/Target + "!String": "@self/Target" CustomRef: "!Ref": id-4 id: id-3 diff --git a/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_new_file_patch.snap b/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_new_file_patch.snap index 92a5b3b5..c38c7864 100644 --- a/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_new_file_patch.snap +++ b/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_new_file_patch.snap @@ -28,7 +28,7 @@ body: Attributes: "!Attributes": Rojo_Ref_PrimaryPart: - "!String": Workspace/MyModel/Target + "!String": "@self/Target" id: id-3 packetType: messages sessionId: id-1 diff --git a/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_nonexistent_patch.snap b/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_nonexistent_patch.snap index a9eee0e2..2cafeb60 100644 --- a/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_nonexistent_patch.snap +++ b/rojo-test/serve-test-snapshots/end_to_end__tests__serve__ref_path_nonexistent_patch.snap @@ -15,7 +15,7 @@ body: Attributes: "!Attributes": Rojo_Ref_PrimaryPart: - "!String": Workspace/NonExistent/Part + "!String": "./NonExistent/Part" PrimaryPart: ~ id: id-3 packetType: messages diff --git a/rojo-test/serve-tests/connected_slugify/src/Hey_Bro~1.luau b/rojo-test/serve-tests/connected_slugify/src/Hey_Bro~2.luau similarity index 100% rename from rojo-test/serve-tests/connected_slugify/src/Hey_Bro~1.luau rename to rojo-test/serve-tests/connected_slugify/src/Hey_Bro~2.luau diff --git a/rojo-test/serve-tests/connected_slugify/src/Hey_Bro~1.meta.json5 b/rojo-test/serve-tests/connected_slugify/src/Hey_Bro~2.meta.json5 similarity index 100% rename from rojo-test/serve-tests/connected_slugify/src/Hey_Bro~1.meta.json5 rename to rojo-test/serve-tests/connected_slugify/src/Hey_Bro~2.meta.json5 diff --git a/rojo-test/serve-tests/git_sync_defaults/default.project.json5 b/rojo-test/serve-tests/git_sync_defaults/default.project.json5 new file mode 100644 index 00000000..398f24b8 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/default.project.json5 @@ -0,0 +1,10 @@ +{ + "name": "git_sync_defaults", + "tree": { + "$className": "DataModel", + "ReplicatedStorage": { + "$className": "ReplicatedStorage", + "$path": "src" + } + } +} diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript01.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript01.luau new file mode 100644 index 00000000..06e0c946 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript01.luau @@ -0,0 +1,2 @@ +local BatchScript01 = {} +return BatchScript01 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript02.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript02.luau new file mode 100644 index 00000000..909a59bd --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript02.luau @@ -0,0 +1,2 @@ +local BatchScript02 = {} +return BatchScript02 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript03.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript03.luau new file mode 100644 index 00000000..94d767ad --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript03.luau @@ -0,0 +1,2 @@ +local BatchScript03 = {} +return BatchScript03 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript04.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript04.luau new file mode 100644 index 00000000..77f95530 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript04.luau @@ -0,0 +1,2 @@ +local BatchScript04 = {} +return BatchScript04 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript05.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript05.luau new file mode 100644 index 00000000..e9cf8932 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript05.luau @@ -0,0 +1,2 @@ +local BatchScript05 = {} +return BatchScript05 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript06.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript06.luau new file mode 100644 index 00000000..96e56c10 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript06.luau @@ -0,0 +1,2 @@ +local BatchScript06 = {} +return BatchScript06 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript07.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript07.luau new file mode 100644 index 00000000..89fc6ec8 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript07.luau @@ -0,0 +1,2 @@ +local BatchScript07 = {} +return BatchScript07 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript08.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript08.luau new file mode 100644 index 00000000..026fa1f2 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript08.luau @@ -0,0 +1,2 @@ +local BatchScript08 = {} +return BatchScript08 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript09.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript09.luau new file mode 100644 index 00000000..57ccd37e --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript09.luau @@ -0,0 +1,2 @@ +local BatchScript09 = {} +return BatchScript09 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript10.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript10.luau new file mode 100644 index 00000000..9a2dd32d --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript10.luau @@ -0,0 +1,2 @@ +local BatchScript10 = {} +return BatchScript10 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript11.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript11.luau new file mode 100644 index 00000000..8ea7033c --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript11.luau @@ -0,0 +1,2 @@ +local BatchScript11 = {} +return BatchScript11 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript12.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript12.luau new file mode 100644 index 00000000..0d25ddc1 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript12.luau @@ -0,0 +1,2 @@ +local BatchScript12 = {} +return BatchScript12 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript13.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript13.luau new file mode 100644 index 00000000..1fb0d90e --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript13.luau @@ -0,0 +1,2 @@ +local BatchScript13 = {} +return BatchScript13 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript14.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript14.luau new file mode 100644 index 00000000..de006942 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript14.luau @@ -0,0 +1,2 @@ +local BatchScript14 = {} +return BatchScript14 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript15.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript15.luau new file mode 100644 index 00000000..06a8adea --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript15.luau @@ -0,0 +1,2 @@ +local BatchScript15 = {} +return BatchScript15 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript16.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript16.luau new file mode 100644 index 00000000..9d7a9548 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript16.luau @@ -0,0 +1,2 @@ +local BatchScript16 = {} +return BatchScript16 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript17.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript17.luau new file mode 100644 index 00000000..203a88d4 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript17.luau @@ -0,0 +1,2 @@ +local BatchScript17 = {} +return BatchScript17 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript18.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript18.luau new file mode 100644 index 00000000..687ff57e --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript18.luau @@ -0,0 +1,2 @@ +local BatchScript18 = {} +return BatchScript18 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript19.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript19.luau new file mode 100644 index 00000000..c7ba3de9 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript19.luau @@ -0,0 +1,2 @@ +local BatchScript19 = {} +return BatchScript19 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/BatchScript20.luau b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript20.luau new file mode 100644 index 00000000..f4a27c68 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/BatchScript20.luau @@ -0,0 +1,2 @@ +local BatchScript20 = {} +return BatchScript20 diff --git a/rojo-test/serve-tests/git_sync_defaults/src/ClientScript.client.luau b/rojo-test/serve-tests/git_sync_defaults/src/ClientScript.client.luau new file mode 100644 index 00000000..2e9ef4dc --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/ClientScript.client.luau @@ -0,0 +1 @@ +print("Hello from client") diff --git a/rojo-test/serve-tests/git_sync_defaults/src/DirModule/Child.luau b/rojo-test/serve-tests/git_sync_defaults/src/DirModule/Child.luau new file mode 100644 index 00000000..904b8885 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/DirModule/Child.luau @@ -0,0 +1 @@ +return "child" diff --git a/rojo-test/serve-tests/git_sync_defaults/src/DirModule/init.luau b/rojo-test/serve-tests/git_sync_defaults/src/DirModule/init.luau new file mode 100644 index 00000000..650e627b --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/DirModule/init.luau @@ -0,0 +1,2 @@ +local DirModule = {} +return DirModule diff --git a/rojo-test/serve-tests/git_sync_defaults/src/ModuleA.luau b/rojo-test/serve-tests/git_sync_defaults/src/ModuleA.luau new file mode 100644 index 00000000..92915fc7 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/ModuleA.luau @@ -0,0 +1,2 @@ +local ModuleA = {} +return ModuleA diff --git a/rojo-test/serve-tests/git_sync_defaults/src/ServerScript.server.luau b/rojo-test/serve-tests/git_sync_defaults/src/ServerScript.server.luau new file mode 100644 index 00000000..5b362e07 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/ServerScript.server.luau @@ -0,0 +1 @@ +print("Hello from server") diff --git a/rojo-test/serve-tests/git_sync_defaults/src/SimpleModel.model.json5 b/rojo-test/serve-tests/git_sync_defaults/src/SimpleModel.model.json5 new file mode 100644 index 00000000..063296c9 --- /dev/null +++ b/rojo-test/serve-tests/git_sync_defaults/src/SimpleModel.model.json5 @@ -0,0 +1,6 @@ +{ + "className": "Part", + "properties": { + "Anchored": true + } +} diff --git a/rojo-test/serve-tests/live_syncback/default.project.json5 b/rojo-test/serve-tests/live_syncback/default.project.json5 new file mode 100644 index 00000000..b9497483 --- /dev/null +++ b/rojo-test/serve-tests/live_syncback/default.project.json5 @@ -0,0 +1,38 @@ +{ + "name": "live_syncback", + "tree": { + "$className": "DataModel", + "ReplicatedStorage": { + "$className": "ReplicatedStorage", + "$path": "src/shared" + }, + "ServerScriptService": { + "$className": "ServerScriptService", + "$path": "src/server" + }, + "Workspace": { + "$className": "Workspace", + "$path": "src/workspace" + }, + "Lighting": { + "$className": "Lighting", + "$path": "src/lighting" + }, + "SoundService": { + "$className": "SoundService", + "$path": "src/soundservice" + }, + "StarterPlayer": { + "$className": "StarterPlayer", + "$path": "src/starterplayer" + }, + "TextChatService": { + "$className": "TextChatService", + "$path": "src/textchatservice" + }, + "VoiceChatService": { + "$className": "VoiceChatService", + "$path": "src/voicechatservice" + } + } +} diff --git a/rojo-test/serve-tests/live_syncback/src/lighting/.gitkeep b/rojo-test/serve-tests/live_syncback/src/lighting/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/rojo-test/serve-tests/live_syncback/src/server/OldScript.server.luau b/rojo-test/serve-tests/live_syncback/src/server/OldScript.server.luau new file mode 100644 index 00000000..bbefe7fa --- /dev/null +++ b/rojo-test/serve-tests/live_syncback/src/server/OldScript.server.luau @@ -0,0 +1 @@ +print("old server script") diff --git a/rojo-test/serve-tests/live_syncback/src/shared/OldModule.luau b/rojo-test/serve-tests/live_syncback/src/shared/OldModule.luau new file mode 100644 index 00000000..bf545338 --- /dev/null +++ b/rojo-test/serve-tests/live_syncback/src/shared/OldModule.luau @@ -0,0 +1 @@ +return "old" diff --git a/rojo-test/serve-tests/live_syncback/src/soundservice/.gitkeep b/rojo-test/serve-tests/live_syncback/src/soundservice/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/rojo-test/serve-tests/live_syncback/src/starterplayer/.gitkeep b/rojo-test/serve-tests/live_syncback/src/starterplayer/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/rojo-test/serve-tests/live_syncback/src/textchatservice/.gitkeep b/rojo-test/serve-tests/live_syncback/src/textchatservice/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/rojo-test/serve-tests/live_syncback/src/voicechatservice/.gitkeep b/rojo-test/serve-tests/live_syncback/src/voicechatservice/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/rojo-test/serve-tests/live_syncback/src/workspace/OldPart.model.json5 b/rojo-test/serve-tests/live_syncback/src/workspace/OldPart.model.json5 new file mode 100644 index 00000000..063296c9 --- /dev/null +++ b/rojo-test/serve-tests/live_syncback/src/workspace/OldPart.model.json5 @@ -0,0 +1,6 @@ +{ + "className": "Part", + "properties": { + "Anchored": true + } +} diff --git a/rojo-test/serve-tests/live_syncback_place_ids/default.project.json5 b/rojo-test/serve-tests/live_syncback_place_ids/default.project.json5 new file mode 100644 index 00000000..73d8de9a --- /dev/null +++ b/rojo-test/serve-tests/live_syncback_place_ids/default.project.json5 @@ -0,0 +1,16 @@ +{ + "name": "live_syncback_place_ids", + "servePlaceIds": [123], + "blockedPlaceIds": [999], + "tree": { + "$className": "DataModel", + "ReplicatedStorage": { + "$className": "ReplicatedStorage", + "$path": "src/shared" + }, + "ServerScriptService": { + "$className": "ServerScriptService", + "$path": "src/server" + } + } +} diff --git a/rojo-test/serve-tests/live_syncback_place_ids/src/server/Placeholder.server.luau b/rojo-test/serve-tests/live_syncback_place_ids/src/server/Placeholder.server.luau new file mode 100644 index 00000000..09a7b059 --- /dev/null +++ b/rojo-test/serve-tests/live_syncback_place_ids/src/server/Placeholder.server.luau @@ -0,0 +1 @@ +-- placeholder diff --git a/rojo-test/serve-tests/live_syncback_place_ids/src/shared/Placeholder.luau b/rojo-test/serve-tests/live_syncback_place_ids/src/shared/Placeholder.luau new file mode 100644 index 00000000..15e53c4a --- /dev/null +++ b/rojo-test/serve-tests/live_syncback_place_ids/src/shared/Placeholder.luau @@ -0,0 +1 @@ +return nil diff --git a/rojo-test/serve-tests/non_default_project/named.project.json5 b/rojo-test/serve-tests/non_default_project/named.project.json5 new file mode 100644 index 00000000..b344dc28 --- /dev/null +++ b/rojo-test/serve-tests/non_default_project/named.project.json5 @@ -0,0 +1,10 @@ +{ + "name": "non_default_project", + "tree": { + "$className": "DataModel", + "ReplicatedStorage": { + "$className": "ReplicatedStorage", + "$path": "src" + } + } +} diff --git a/rojo-test/serve-tests/non_default_project/src/hello.luau b/rojo-test/serve-tests/non_default_project/src/hello.luau new file mode 100644 index 00000000..a5647075 --- /dev/null +++ b/rojo-test/serve-tests/non_default_project/src/hello.luau @@ -0,0 +1 @@ +return {} diff --git a/rojo-test/serve-tests/non_default_project/trap/bad.rbxm b/rojo-test/serve-tests/non_default_project/trap/bad.rbxm new file mode 100644 index 00000000..7b478fc2 --- /dev/null +++ b/rojo-test/serve-tests/non_default_project/trap/bad.rbxm @@ -0,0 +1,2 @@ +This is not a valid rbxm file. If the snapshot system walks into this +directory, it will error with "Malformed rbxm file: Invalid file header". diff --git a/rojo-test/serve-tests/ref_ambiguous_path/src/Workspace/DupParent/Child~1.model.json5 b/rojo-test/serve-tests/ref_ambiguous_path/src/Workspace/DupParent/Child~2.model.json5 similarity index 100% rename from rojo-test/serve-tests/ref_ambiguous_path/src/Workspace/DupParent/Child~1.model.json5 rename to rojo-test/serve-tests/ref_ambiguous_path/src/Workspace/DupParent/Child~2.model.json5 diff --git a/rojo-test/serve-tests/ref_forward_sync/src/Workspace/MyModel/init.meta.json5 b/rojo-test/serve-tests/ref_forward_sync/src/Workspace/MyModel/init.meta.json5 index d83679ff..b6425515 100644 --- a/rojo-test/serve-tests/ref_forward_sync/src/Workspace/MyModel/init.meta.json5 +++ b/rojo-test/serve-tests/ref_forward_sync/src/Workspace/MyModel/init.meta.json5 @@ -1,6 +1,6 @@ { "className": "Model", "attributes": { - "Rojo_Ref_PrimaryPart": "Workspace/MyModel/Target" + "Rojo_Ref_PrimaryPart": "@self/Target" } } diff --git a/rojo-test/serve-tests/ref_two_way_sync/src/Workspace/TestModel/init.meta.json5 b/rojo-test/serve-tests/ref_two_way_sync/src/Workspace/TestModel/init.meta.json5 index 1e894a97..2b2f91ba 100644 --- a/rojo-test/serve-tests/ref_two_way_sync/src/Workspace/TestModel/init.meta.json5 +++ b/rojo-test/serve-tests/ref_two_way_sync/src/Workspace/TestModel/init.meta.json5 @@ -1,6 +1,6 @@ { "className": "Model", "attributes": { - "Rojo_Ref_PrimaryPart": "Workspace/TestModel/Part1" + "Rojo_Ref_PrimaryPart": "@self/Part1.model.json5" } } diff --git a/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__child_but_not-stdout.snap b/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__child_but_not-stdout.snap index b1058161..46d80326 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__child_but_not-stdout.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__child_but_not-stdout.snap @@ -3,6 +3,8 @@ source: tests/rojo_test/syncback_util.rs expression: normalized_stdout --- Writing OnlyOneCopy/child_of_one.luau +Writing ReplicatedStorage/OnlyOneCopy/child_of_one.luau Writing ReplicatedStorage/child_replicated_storage.luau +Writing ReplicatedStorage/OnlyOneCopy Removing OnlyOneCopy/.gitkeep Removing ReplicatedStorage/.gitkeep diff --git a/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__rbxm_fallback-stdout.snap b/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__rbxm_fallback-stdout.snap index 7baf6c3e..20a658ad 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__rbxm_fallback-stdout.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__rbxm_fallback-stdout.snap @@ -2,8 +2,12 @@ source: tests/rojo_test/syncback_util.rs expression: normalized_stdout --- +Writing src/ChildWithDuplicates/DuplicateChild/.gitkeep +Writing src/ChildWithDuplicates/DuplicateChild~2/.gitkeep Writing src/ChildWithoutDuplicates/Child/.gitkeep Writing src/ChildWithDuplicates +Writing src/ChildWithDuplicates/DuplicateChild +Writing src/ChildWithDuplicates/DuplicateChild~2 Writing src/ChildWithoutDuplicates Writing src/ChildWithoutDuplicates/Child Removing src/ChildWithDuplicates/.gitkeep diff --git a/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__respect_old_middleware-stdout.snap b/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__respect_old_middleware-stdout.snap index 532f88ab..c8ae87cd 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__respect_old_middleware-stdout.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__rojo_test__syncback_util__respect_old_middleware-stdout.snap @@ -4,5 +4,6 @@ expression: normalized_stdout --- Writing default.project.json5 Writing src/model_json.model.json5 +Writing src/project_node.model.json5 Writing src/rbxm.rbxm Writing src/rbxmx.rbxmx diff --git a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__duplicate_rojo_id-container.model.json5.snap b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__duplicate_rojo_id-container.model.json5.snap index 1aef5772..71d41459 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__duplicate_rojo_id-container.model.json5.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__duplicate_rojo_id-container.model.json5.snap @@ -7,7 +7,7 @@ expression: container.model.json5 { attributes: { Rojo_Id: "value_1", - Rojo_Ref_Value: "ReplicatedStorage/container/value_1" + Rojo_Ref_Value: "@self" }, className: "ObjectValue", name: "value_1" @@ -15,7 +15,7 @@ expression: container.model.json5 { attributes: { Rojo_Id: "value_1", - Rojo_Ref_Value: "ReplicatedStorage/container/value_2" + Rojo_Ref_Value: "@self" }, className: "ObjectValue", name: "value_2" diff --git a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__json_middlewares-src__dir_with_meta__init.meta.json5.snap b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__json_middlewares-src__dir_with_meta__init.meta.json5.snap index 55faa78c..43e02a85 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__json_middlewares-src__dir_with_meta__init.meta.json5.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__json_middlewares-src__dir_with_meta__init.meta.json5.snap @@ -5,8 +5,6 @@ expression: src/dir_with_meta/init.meta.json5 { className: "Configuration", properties: { - Tags: [ - "This tag on dir_with_meta" - ] + Tags: [ "This tag on dir_with_meta" ] } } diff --git a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__json_middlewares-src__project_json.project.json5.snap b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__json_middlewares-src__project_json.project.json5.snap index 9280fd77..b1202688 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__json_middlewares-src__project_json.project.json5.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__json_middlewares-src__project_json.project.json5.snap @@ -7,11 +7,7 @@ expression: src/project_json.project.json5 tree: { $className: "Color3Value", $properties: { - Value: [ - 1337, - -1337, - 1337 - ] + Value: [ 1337, -1337, 1337 ] } } } diff --git a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties-src__pointer.model.json5.snap b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties-src__pointer.model.json5.snap index c2bd235f..396a91e0 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties-src__pointer.model.json5.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties-src__pointer.model.json5.snap @@ -4,7 +4,7 @@ expression: src/pointer.model.json5 --- { attributes: { - Rojo_Ref_Value: "ReplicatedStorage/target" + Rojo_Ref_Value: "@game/target.model.json5" }, className: "ObjectValue" } diff --git a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_blank-src__pointer.model.json5.snap b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_blank-src__pointer.model.json5.snap index c2bd235f..135afd1e 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_blank-src__pointer.model.json5.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_blank-src__pointer.model.json5.snap @@ -4,7 +4,7 @@ expression: src/pointer.model.json5 --- { attributes: { - Rojo_Ref_Value: "ReplicatedStorage/target" + Rojo_Ref_Value: "@game/target.txt" }, className: "ObjectValue" } diff --git a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_conflict-src__Pointer_2.model.json5.snap b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_conflict-src__Pointer_2.model.json5.snap index 54b72c1c..aadffee8 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_conflict-src__Pointer_2.model.json5.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_conflict-src__Pointer_2.model.json5.snap @@ -4,7 +4,7 @@ expression: src/Pointer_2.model.json5 --- { attributes: { - Rojo_Ref_Value: "ReplicatedStorage/Target_2" + Rojo_Ref_Value: "@game/Target_2.model.json5" }, className: "ObjectValue" } diff --git a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer1.model.json5.snap b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer1.model.json5.snap index 79c8250c..53649309 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer1.model.json5.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer1.model.json5.snap @@ -4,7 +4,7 @@ expression: src/Pointer1.model.json5 --- { attributes: { - Rojo_Ref_Value: "ReplicatedStorage/Target1" + Rojo_Ref_Value: "@game/ReplicatedStorage/Target1.model.json5" }, className: "ObjectValue" } diff --git a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer2.model.json5.snap b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer2.model.json5.snap index 77fe2f05..029c34a6 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer2.model.json5.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer2.model.json5.snap @@ -4,7 +4,7 @@ expression: src/Pointer2.model.json5 --- { attributes: { - Rojo_Ref_Value: "ReplicatedStorage/Target2" + Rojo_Ref_Value: "@game/ReplicatedStorage/Target2.model.json5" }, className: "ObjectValue" } diff --git a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer3.model.json5.snap b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer3.model.json5.snap index 874c8b54..945be246 100644 --- a/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer3.model.json5.snap +++ b/rojo-test/syncback-test-snapshots/end_to_end__tests__syncback__ref_properties_pruned-src__Pointer3.model.json5.snap @@ -4,7 +4,7 @@ expression: src/Pointer3.model.json5 --- { attributes: { - Rojo_Ref_Value: "ReplicatedStorage/Target3" + Rojo_Ref_Value: "@game/ReplicatedStorage/Target3.model.json5" }, className: "ObjectValue" } diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Icon/UIAspectRatioConstraint.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Icon/UIAspectRatioConstraint.model.json5 new file mode 100644 index 00000000..cace7a97 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Icon/UIAspectRatioConstraint.model.json5 @@ -0,0 +1,3 @@ +{ + className: "UIAspectRatioConstraint" +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Icon/init.meta.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Icon/init.meta.json5 new file mode 100644 index 00000000..f121b2cc --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Icon/init.meta.json5 @@ -0,0 +1,46 @@ +{ + className: "ImageLabel", + properties: { + AnchorPoint: [ + 0.5, + 0 + ], + BackgroundColor3: [ + 1, + 1, + 1 + ], + BackgroundTransparency: 1, + BorderColor3: [ + 0, + 0, + 0 + ], + BorderSizePixel: 0, + ImageContent: "rbxassetid://131974996946194", + Position: { + UDim2: [ + [ + 0.5, + 0 + ], + [ + 0.5, + 0 + ] + ] + }, + Size: { + UDim2: [ + [ + 0.699999988079071, + 0 + ], + [ + 0.699999988079071, + 0 + ] + ] + } + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Info/UIGradient.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Info/UIGradient.model.json5 new file mode 100644 index 00000000..197d4910 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Info/UIGradient.model.json5 @@ -0,0 +1,36 @@ +{ + className: "UIGradient", + properties: { + Color: { + ColorSequence: { + keypoints: [ + { + color: [ + 0.1882352977991104, + 0.11372549086809158, + 1 + ], + time: 0 + }, + { + color: [ + 0.5490196347236633, + 0, + 1 + ], + time: 0.608996570110321 + }, + { + color: [ + 0.501960813999176, + 0, + 0.9176470637321472 + ], + time: 1 + } + ] + } + }, + Rotation: -90 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Info/UIStroke.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Info/UIStroke.model.json5 new file mode 100644 index 00000000..2afd1ec6 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Info/UIStroke.model.json5 @@ -0,0 +1,6 @@ +{ + className: "UIStroke", + properties: { + Thickness: 2 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Info/init.meta.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Info/init.meta.json5 new file mode 100644 index 00000000..bb5f1ca4 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/Info/init.meta.json5 @@ -0,0 +1,60 @@ +{ + className: "TextLabel", + properties: { + AnchorPoint: [ + 0.5, + 0 + ], + BackgroundColor3: [ + 1, + 1, + 1 + ], + BackgroundTransparency: 1, + BorderColor3: [ + 0, + 0, + 0 + ], + BorderSizePixel: 0, + FontFace: { + cachedFaceId: null, + family: "rbxasset://fonts/families/LuckiestGuy.json", + style: "Normal", + weight: "Regular" + }, + Position: { + UDim2: [ + [ + 0.5, + 0 + ], + [ + 0, + 0 + ] + ] + }, + Size: { + UDim2: [ + [ + 1, + 0 + ], + [ + 0.5, + 0 + ] + ] + }, + Text: "UFO", + TextColor3: [ + 1, + 1, + 1 + ], + TextScaled: true, + TextSize: 14, + TextWrapped: true + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/init.meta.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/init.meta.json5 new file mode 100644 index 00000000..00b3efd0 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/Frame/init.meta.json5 @@ -0,0 +1,45 @@ +{ + className: "Frame", + properties: { + AnchorPoint: [ + 0.5, + 0.5 + ], + BackgroundColor3: [ + 1, + 1, + 1 + ], + BackgroundTransparency: 1, + BorderColor3: [ + 0, + 0, + 0 + ], + BorderSizePixel: 0, + Position: { + UDim2: [ + [ + 0.5, + 0 + ], + [ + 0.5, + 0 + ] + ] + }, + Size: { + UDim2: [ + [ + 1, + 0 + ], + [ + 1, + 0 + ] + ] + } + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/init.meta.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/init.meta.json5 new file mode 100644 index 00000000..7262f7a0 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/BillboardGui/init.meta.json5 @@ -0,0 +1,26 @@ +{ + className: "BillboardGui", + properties: { + Active: true, + LightInfluence: 1, + ResetOnSpawn: false, + Size: { + UDim2: [ + [ + 60, + 0 + ], + [ + 40, + 0 + ] + ] + }, + StudsOffset: [ + 0, + 75, + 0 + ], + ZIndexBehavior: "Sibling" + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/Water.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/Water.model.json5 new file mode 100644 index 00000000..8080d077 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/Water.model.json5 @@ -0,0 +1,16 @@ +{ + attributes: { + Rojo_Ref_SoundGroup: "../../__server_muted.model.json5" + }, + className: "Sound", + properties: { + AudioContent: "rbxassetid://9120549564", + Looped: true, + Playing: true, + SourceAssetId: 9120549564, + Tags: [ + "SoundGroupEffects" + ], + Volume: 3 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/init.meta.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/init.meta.json5 new file mode 100644 index 00000000..afc3c11f --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/Hitbox/init.meta.json5 @@ -0,0 +1,29 @@ +{ + className: "Part", + properties: { + Anchored: true, + BottomSurface: "Smooth", + CFrame: [ + 789.7578125, + 37.375, + 0.00000762939453125, + 1, + 0, + 0, + 0, + 1, + 0, + 0, + 0, + 1 + ], + CanCollide: false, + Size: [ + 25.515625, + 70.75, + 260 + ], + TopSurface: "Smooth", + Transparency: 1 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture.model.json5 new file mode 100644 index 00000000..5e975d2a --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture.model.json5 @@ -0,0 +1,15 @@ +{ + className: "Texture", + properties: { + Color3: [ + 0.41568630933761597, + 0, + 1 + ], + Face: "Top", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10025668350", + Transparency: 0.6000000238418579 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~10.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~10.model.json5 new file mode 100644 index 00000000..470357ae --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~10.model.json5 @@ -0,0 +1,16 @@ +{ + className: "Texture", + name: "Texture", + properties: { + Color3: [ + 0.40000003576278687, + 0, + 1 + ], + Face: "Left", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10023635072", + Transparency: 0.6499999761581421 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~11.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~11.model.json5 new file mode 100644 index 00000000..dc4b910d --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~11.model.json5 @@ -0,0 +1,16 @@ +{ + className: "Texture", + name: "Texture", + properties: { + Color3: [ + 0.40000003576278687, + 0, + 1 + ], + Face: "Right", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10023635072", + Transparency: 0.6499999761581421 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~12.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~12.model.json5 new file mode 100644 index 00000000..df4f761f --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~12.model.json5 @@ -0,0 +1,16 @@ +{ + className: "Texture", + name: "Texture", + properties: { + Color3: [ + 0.40000003576278687, + 0, + 1 + ], + Face: "Top", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10023635072", + Transparency: 0.6499999761581421 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~2.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~2.model.json5 new file mode 100644 index 00000000..11e28cd8 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~2.model.json5 @@ -0,0 +1,16 @@ +{ + className: "Texture", + name: "Texture", + properties: { + Color3: [ + 0.41568630933761597, + 0, + 1 + ], + Face: "Right", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10025668350", + Transparency: 0.6000000238418579 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~3.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~3.model.json5 new file mode 100644 index 00000000..9f5fd1c3 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~3.model.json5 @@ -0,0 +1,16 @@ +{ + className: "Texture", + name: "Texture", + properties: { + Color3: [ + 0.41568630933761597, + 0, + 1 + ], + Face: "Left", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10025668350", + Transparency: 0.6000000238418579 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~4.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~4.model.json5 new file mode 100644 index 00000000..cc60c294 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~4.model.json5 @@ -0,0 +1,15 @@ +{ + className: "Texture", + name: "Texture", + properties: { + Color3: [ + 0.41568630933761597, + 0, + 1 + ], + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10025668350", + Transparency: 0.6000000238418579 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~5.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~5.model.json5 new file mode 100644 index 00000000..cec50fee --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~5.model.json5 @@ -0,0 +1,16 @@ +{ + className: "Texture", + name: "Texture", + properties: { + Color3: [ + 0.41568630933761597, + 0, + 1 + ], + Face: "Bottom", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10025668350", + Transparency: 0.6000000238418579 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~6.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~6.model.json5 new file mode 100644 index 00000000..015bfdce --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~6.model.json5 @@ -0,0 +1,16 @@ +{ + className: "Texture", + name: "Texture", + properties: { + Color3: [ + 0.40000003576278687, + 0, + 1 + ], + Face: "Back", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10023635072", + Transparency: 0.6499999761581421 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~7.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~7.model.json5 new file mode 100644 index 00000000..015bfdce --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~7.model.json5 @@ -0,0 +1,16 @@ +{ + className: "Texture", + name: "Texture", + properties: { + Color3: [ + 0.40000003576278687, + 0, + 1 + ], + Face: "Back", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10023635072", + Transparency: 0.6499999761581421 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~8.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~8.model.json5 new file mode 100644 index 00000000..c9f537bd --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~8.model.json5 @@ -0,0 +1,16 @@ +{ + className: "Texture", + name: "Texture", + properties: { + Color3: [ + 0.40000003576278687, + 0, + 1 + ], + Face: "Bottom", + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10023635072", + Transparency: 0.6499999761581421 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~9.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~9.model.json5 new file mode 100644 index 00000000..869ecc79 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/Texture~9.model.json5 @@ -0,0 +1,15 @@ +{ + className: "Texture", + name: "Texture", + properties: { + Color3: [ + 0.40000003576278687, + 0, + 1 + ], + StudsPerTileU: 200, + StudsPerTileV: 200, + TextureContent: "rbxassetid://10023635072", + Transparency: 0.6499999761581421 + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/glow.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/glow.model.json5 new file mode 100644 index 00000000..f62c6a95 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/glow.model.json5 @@ -0,0 +1,185 @@ +{ + attributes: { + Size_Curve: { + BinaryString: "AAAAAAAAgD8AAAAAyEWiPWTpPj9Q1747AACAPwAAAAAAAAAAAAAAAAAAAAAAAAAA" + }, + Size_End: 60, + Size_Envelope: 5, + Size_Start: 45 + }, + className: "ParticleEmitter", + properties: { + Color: { + ColorSequence: { + keypoints: [ + { + color: [ + 0.6666666865348816, + 0, + 1 + ], + time: 0 + }, + { + color: [ + 0.6666666865348816, + 0, + 1 + ], + time: 1 + } + ] + } + }, + Lifetime: { + NumberRange: [ + 0.25, + 0.6000000238418579 + ] + }, + LightEmission: 1, + LockedToPart: true, + Size: { + NumberSequence: { + keypoints: [ + { + envelope: 5, + time: 0, + value: 45 + }, + { + envelope: 5, + time: 0.001580123440362513, + value: 46.08238983154297 + }, + { + envelope: 5, + time: 0.006548703648149967, + value: 47.16343307495117 + }, + { + envelope: 5, + time: 0.015281829982995987, + value: 48.24135971069336 + }, + { + envelope: 5, + time: 0.028204884380102158, + value: 49.313777923583984 + }, + { + envelope: 5, + time: 0.045797109603881836, + value: 50.37750244140625 + }, + { + envelope: 5, + time: 0.06858612596988678, + value: 51.42829513549805 + }, + { + envelope: 5, + time: 0.09713879972696304, + value: 52.4605827331543 + }, + { + envelope: 5, + time: 0.13203296065330505, + value: 53.46717071533203 + }, + { + envelope: 5, + time: 0.17381449043750763, + value: 54.438995361328125 + }, + { + envelope: 5, + time: 0.22292612493038177, + value: 55.36511993408203 + }, + { + envelope: 5, + time: 0.27961885929107666, + value: 56.23305892944336 + }, + { + envelope: 5, + time: 0.34385624527931213, + value: 57.029701232910156 + }, + { + envelope: 5, + time: 0.4152511656284332, + value: 57.74271774291992 + }, + { + envelope: 5, + time: 0.49307429790496826, + value: 58.362186431884766 + }, + { + envelope: 5, + time: 0.5763297080993652, + value: 58.88185501098633 + }, + { + envelope: 5, + time: 0.6638965010643005, + value: 59.29954147338867 + }, + { + envelope: 5, + time: 0.7546600699424744, + value: 59.6165657043457 + }, + { + envelope: 5, + time: 0.8476089239120483, + value: 59.83663558959961 + }, + { + envelope: 5, + time: 1, + value: 59.964595794677734 + } + ] + } + }, + Speed: { + NumberRange: [ + 0.0010000000474974513, + 0.0010000000474974513 + ] + }, + Tags: [ + "_VFX" + ], + Texture: "rbxassetid://77024587778878", + Transparency: { + NumberSequence: { + keypoints: [ + { + envelope: 0, + time: 0, + value: 1 + }, + { + envelope: 0, + time: 0.09989999979734421, + value: 0.75 + }, + { + envelope: 0, + time: 0.6000000238418579, + value: 0.75 + }, + { + envelope: 0, + time: 1, + value: 1 + } + ] + } + } + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/glow~2.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/glow~2.model.json5 new file mode 100644 index 00000000..943be0f1 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/glow~2.model.json5 @@ -0,0 +1,186 @@ +{ + attributes: { + Size_Curve: { + BinaryString: "AAAAAAAAgD8AAAAAyEWiPWTpPj9Q1747AACAPwAAAAAAAAAAAAAAAAAAAAAAAAAA" + }, + Size_End: 60, + Size_Envelope: 5, + Size_Start: 45 + }, + className: "ParticleEmitter", + name: "glow", + properties: { + Color: { + ColorSequence: { + keypoints: [ + { + color: [ + 0.364705890417099, + 0.11372549086809158, + 1 + ], + time: 0 + }, + { + color: [ + 0.364705890417099, + 0.11372549086809158, + 1 + ], + time: 1 + } + ] + } + }, + Lifetime: { + NumberRange: [ + 0.25, + 0.6000000238418579 + ] + }, + LightEmission: 1, + LockedToPart: true, + Size: { + NumberSequence: { + keypoints: [ + { + envelope: 5, + time: 0, + value: 45 + }, + { + envelope: 5, + time: 0.001580123440362513, + value: 46.08238983154297 + }, + { + envelope: 5, + time: 0.006548703648149967, + value: 47.16343307495117 + }, + { + envelope: 5, + time: 0.015281829982995987, + value: 48.24135971069336 + }, + { + envelope: 5, + time: 0.028204884380102158, + value: 49.313777923583984 + }, + { + envelope: 5, + time: 0.045797109603881836, + value: 50.37750244140625 + }, + { + envelope: 5, + time: 0.06858612596988678, + value: 51.42829513549805 + }, + { + envelope: 5, + time: 0.09713879972696304, + value: 52.4605827331543 + }, + { + envelope: 5, + time: 0.13203296065330505, + value: 53.46717071533203 + }, + { + envelope: 5, + time: 0.17381449043750763, + value: 54.438995361328125 + }, + { + envelope: 5, + time: 0.22292612493038177, + value: 55.36511993408203 + }, + { + envelope: 5, + time: 0.27961885929107666, + value: 56.23305892944336 + }, + { + envelope: 5, + time: 0.34385624527931213, + value: 57.029701232910156 + }, + { + envelope: 5, + time: 0.4152511656284332, + value: 57.74271774291992 + }, + { + envelope: 5, + time: 0.49307429790496826, + value: 58.362186431884766 + }, + { + envelope: 5, + time: 0.5763297080993652, + value: 58.88185501098633 + }, + { + envelope: 5, + time: 0.6638965010643005, + value: 59.29954147338867 + }, + { + envelope: 5, + time: 0.7546600699424744, + value: 59.6165657043457 + }, + { + envelope: 5, + time: 0.8476089239120483, + value: 59.83663558959961 + }, + { + envelope: 5, + time: 1, + value: 59.964595794677734 + } + ] + } + }, + Speed: { + NumberRange: [ + 0.0010000000474974513, + 0.0010000000474974513 + ] + }, + Tags: [ + "_VFX" + ], + Texture: "rbxassetid://77024587778878", + Transparency: { + NumberSequence: { + keypoints: [ + { + envelope: 0, + time: 0, + value: 1 + }, + { + envelope: 0, + time: 0.09989999979734421, + value: 0.75 + }, + { + envelope: 0, + time: 0.6000000238418579, + value: 0.75 + }, + { + envelope: 0, + time: 1, + value: 1 + } + ] + } + } + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/glow~3.model.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/glow~3.model.json5 new file mode 100644 index 00000000..25ec3941 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/glow~3.model.json5 @@ -0,0 +1,186 @@ +{ + attributes: { + Size_Curve: { + BinaryString: "AAAAAAAAgD8AAAAAyEWiPWTpPj9Q1747AACAPwAAAAAAAAAAAAAAAAAAAAAAAAAA" + }, + Size_End: 60, + Size_Envelope: 5, + Size_Start: 45 + }, + className: "ParticleEmitter", + name: "glow", + properties: { + Color: { + ColorSequence: { + keypoints: [ + { + color: [ + 0.6666666865348816, + 0, + 1 + ], + time: 0 + }, + { + color: [ + 0.6666666865348816, + 0, + 1 + ], + time: 1 + } + ] + } + }, + Lifetime: { + NumberRange: [ + 0.25, + 0.6000000238418579 + ] + }, + LightEmission: 1, + LockedToPart: true, + Size: { + NumberSequence: { + keypoints: [ + { + envelope: 5, + time: 0, + value: 45 + }, + { + envelope: 5, + time: 0.001580123440362513, + value: 46.08238983154297 + }, + { + envelope: 5, + time: 0.006548703648149967, + value: 47.16343307495117 + }, + { + envelope: 5, + time: 0.015281829982995987, + value: 48.24135971069336 + }, + { + envelope: 5, + time: 0.028204884380102158, + value: 49.313777923583984 + }, + { + envelope: 5, + time: 0.045797109603881836, + value: 50.37750244140625 + }, + { + envelope: 5, + time: 0.06858612596988678, + value: 51.42829513549805 + }, + { + envelope: 5, + time: 0.09713879972696304, + value: 52.4605827331543 + }, + { + envelope: 5, + time: 0.13203296065330505, + value: 53.46717071533203 + }, + { + envelope: 5, + time: 0.17381449043750763, + value: 54.438995361328125 + }, + { + envelope: 5, + time: 0.22292612493038177, + value: 55.36511993408203 + }, + { + envelope: 5, + time: 0.27961885929107666, + value: 56.23305892944336 + }, + { + envelope: 5, + time: 0.34385624527931213, + value: 57.029701232910156 + }, + { + envelope: 5, + time: 0.4152511656284332, + value: 57.74271774291992 + }, + { + envelope: 5, + time: 0.49307429790496826, + value: 58.362186431884766 + }, + { + envelope: 5, + time: 0.5763297080993652, + value: 58.88185501098633 + }, + { + envelope: 5, + time: 0.6638965010643005, + value: 59.29954147338867 + }, + { + envelope: 5, + time: 0.7546600699424744, + value: 59.6165657043457 + }, + { + envelope: 5, + time: 0.8476089239120483, + value: 59.83663558959961 + }, + { + envelope: 5, + time: 1, + value: 59.964595794677734 + } + ] + } + }, + Speed: { + NumberRange: [ + 0.0010000000474974513, + 0.0010000000474974513 + ] + }, + Tags: [ + "_VFX" + ], + Texture: "rbxassetid://77024587778878", + Transparency: { + NumberSequence: { + keypoints: [ + { + envelope: 0, + time: 0, + value: 1 + }, + { + envelope: 0, + time: 0.09989999979734421, + value: 0.75 + }, + { + envelope: 0, + time: 0.6000000238418579, + value: 0.75 + }, + { + envelope: 0, + time: 1, + value: 1 + } + ] + } + } + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/init.meta.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/init.meta.json5 new file mode 100644 index 00000000..017fe065 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/TsunamiWave/init.meta.json5 @@ -0,0 +1,60 @@ +{ + className: "UnionOperation", + properties: { + Anchored: true, + AssetId: "http://www.roblox.com//asset/?id=212226088", + CFrame: [ + 801.715576171875, + 36.375, + 0.00000762939453125, + 1, + 0, + 0, + 0, + 1, + 0, + 0, + 0, + 1 + ], + CanCollide: false, + CanQuery: false, + CanTouch: false, + CastShadow: false, + Color: { + Color3uint8: [ + 48, + 12, + 71 + ] + }, + FormFactor: "Symmetric", + InitialSize: [ + 18.943885803222656, + 6.7999587059021, + 39.99999237060547 + ], + Size: [ + 49.43058776855469, + 72.75, + 260 + ], + UnscaledCofm: [ + -1.2760223150253296, + -0.917913019657135, + 0.000006928649327164749 + ], + UnscaledVolInertiaDiags: [ + 321672.25, + 343861.1875, + 37455.859375 + ], + UnscaledVolInertiaOffDiags: [ + 5263.43994140625, + 0.014713685028254986, + -0.0022491957060992718 + ], + UnscaledVolume: 2355.29296875, + UsePartColor: true + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/expected/init.meta.json5 b/rojo-test/syncback-tests/UFOWave_matching/expected/init.meta.json5 new file mode 100644 index 00000000..54e6c7f9 --- /dev/null +++ b/rojo-test/syncback-tests/UFOWave_matching/expected/init.meta.json5 @@ -0,0 +1,52 @@ +{ + attributes: { + Rojo_Ref_PrimaryPart: "@self/TsunamiWave", + Speed: 10, + ZonesPassed: { + Float32: 0 + } + }, + className: "Model", + properties: { + ModelMeshCFrame: [ + -740.2847900390625, + -71.00001525878906, + -369.2483825683594, + -2.6309862732887268e-8, + 0, + -1.000003695487976, + 0, + 1, + 0, + 1.000003695487976, + 0, + -2.6309862732887268e-8 + ], + WorldPivotData: { + OptionalCFrame: { + orientation: [ + [ + 1, + 0, + 0 + ], + [ + 0, + 1, + 0 + ], + [ + 0, + 0, + 1 + ] + ], + position: [ + 801.715576171875, + 36.375, + 0.00000762939453125 + ] + } + } + } +} diff --git a/rojo-test/syncback-tests/UFOWave_matching/input.rbxm b/rojo-test/syncback-tests/UFOWave_matching/input.rbxm new file mode 100644 index 00000000..d51f387d Binary files /dev/null and b/rojo-test/syncback-tests/UFOWave_matching/input.rbxm differ diff --git a/rokit.toml b/rokit.toml index 97af0c92..b7af77be 100644 --- a/rokit.toml +++ b/rokit.toml @@ -1,6 +1,6 @@ [tools] selene = "Kampfkarren/selene@0.30.0" stylua = "JohnnyMorganz/stylua@2.3.1" -luau = "luau-lang/luau@0.708" luau-lsp = "JohnnyMorganz/luau-lsp@1.62.0" run-in-roblox = "rojo-rbx/run-in-roblox@0.3.0" +lune = "lune-org/lune@0.10.4" diff --git a/scripts/ci.ps1 b/scripts/ci.ps1 index 5ce4f315..32bf74fd 100644 --- a/scripts/ci.ps1 +++ b/scripts/ci.ps1 @@ -6,6 +6,7 @@ param( ) $RbxDom = -not $NoRbxDom +$Threads = [Environment]::ProcessorCount $ErrorActionPreference = "Continue" $failures = @() @@ -56,19 +57,19 @@ $selene = $LASTEXITCODE Record-Result "Selene" $selene Write-Step 6 "Rust Linting (Clippy) - Auto-fix" -cargo clippy -j 16 --fix --allow-dirty --allow-staged 2>&1 | Out-Null +cargo clippy -j $Threads --fix --allow-dirty --allow-staged 2>&1 | Out-Null Write-Host "Verifying..." -ForegroundColor Yellow -cargo clippy -j 16 --all-targets --all-features 2>&1 +cargo clippy -j $Threads --all-targets --all-features 2>&1 $clippy = $LASTEXITCODE Record-Result "Clippy" $clippy Write-Step 7 "Build Everything" -cargo build -j 16 --locked --all-targets --all-features +cargo build --locked --all-targets --all-features $build = $LASTEXITCODE Record-Result "Build" $build Write-Step 8 "Run ALL Rust Tests" -$testOutput = cargo test -j 16 --locked --all-features -- --test-threads=16 2>&1 +$testOutput = cargo test --locked --all-features -- --test-threads=$Threads 2>&1 $rustTests = $LASTEXITCODE $testOutput | Write-Host Record-Result "Rust Tests" $rustTests @@ -106,18 +107,18 @@ if ($RbxDom) { Write-Step 13 "Rust Linting (Clippy) for rbx-dom - Auto-fix" Push-Location rbx-dom - cargo clippy -j 16 --fix --allow-dirty --allow-staged 2>&1 | Out-Null + cargo clippy -j $Threads --fix --allow-dirty --allow-staged 2>&1 | Out-Null Write-Host "Verifying..." -ForegroundColor Yellow - cargo clippy -j 16 --all-targets --all-features -- -D warnings 2>&1 + cargo clippy -j $Threads --all-targets --all-features -- -D warnings 2>&1 $clippyRbxDom = $LASTEXITCODE Pop-Location Record-Result "Clippy (rbx-dom)" $clippyRbxDom Write-Step 14 "Build rbx-dom" Push-Location rbx-dom - cargo build -j 16 --verbose + cargo build --verbose $buildRbxDom = $LASTEXITCODE - cargo build -j 16 --all-features --verbose + cargo build --all-features --verbose $buildRbxDomAll = $LASTEXITCODE Pop-Location Record-Result "Build (rbx-dom)" $buildRbxDom @@ -125,9 +126,9 @@ if ($RbxDom) { Write-Step 15 "Run ALL rbx-dom Rust Tests" Push-Location rbx-dom - cargo test -j 16 --verbose -- --test-threads=16 + cargo test --verbose -- --test-threads=$Threads $testsRbxDom = $LASTEXITCODE - cargo test -j 16 --all-features --verbose -- --test-threads=16 + cargo test --all-features --verbose -- --test-threads=$Threads $testsRbxDomAll = $LASTEXITCODE Pop-Location Record-Result "Tests (rbx-dom)" $testsRbxDom diff --git a/scripts/ci.sh b/scripts/ci.sh index 3dc32ded..a3673ea3 100755 --- a/scripts/ci.sh +++ b/scripts/ci.sh @@ -11,6 +11,8 @@ for arg in "$@"; do esac done +THREADS=$(nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4) + failures=() step() { @@ -55,19 +57,19 @@ selene_exit=$? record "Selene" $selene_exit step 6 "Rust Linting (Clippy) - Auto-fix" -cargo clippy -j 16 --fix --allow-dirty --allow-staged >/dev/null 2>&1 +cargo clippy -j $THREADS --fix --allow-dirty --allow-staged >/dev/null 2>&1 echo "Verifying..." -cargo clippy -j 16 --all-targets --all-features 2>&1 +cargo clippy -j $THREADS --all-targets --all-features 2>&1 clippy_exit=$? record "Clippy" $clippy_exit step 7 "Build Everything" -cargo build -j 16 --locked --all-targets --all-features +cargo build --locked --all-targets --all-features build_exit=$? record "Build" $build_exit step 8 "Run ALL Rust Tests" -cargo test -j 16 --locked --all-features -- --test-threads=16 2>&1 +cargo test --locked --all-features -- --test-threads=$THREADS 2>&1 rust_tests=$? record "Rust Tests" $rust_tests @@ -101,24 +103,24 @@ if [ "$RBX_DOM" = true ]; then record "Selene (rbx_dom_lua)" $selene_rbxdom step 13 "Rust Linting (Clippy) for rbx-dom - Auto-fix" - (cd rbx-dom && cargo clippy -j 16 --fix --allow-dirty --allow-staged >/dev/null 2>&1) + (cd rbx-dom && cargo clippy -j $THREADS --fix --allow-dirty --allow-staged >/dev/null 2>&1) echo "Verifying..." - (cd rbx-dom && cargo clippy -j 16 --all-targets --all-features -- -D warnings 2>&1) + (cd rbx-dom && cargo clippy -j $THREADS --all-targets --all-features -- -D warnings 2>&1) clippy_rbxdom=$? record "Clippy (rbx-dom)" $clippy_rbxdom step 14 "Build rbx-dom" - (cd rbx-dom && cargo build -j 16 --verbose) + (cd rbx-dom && cargo build --verbose) build_rbxdom=$? - (cd rbx-dom && cargo build -j 16 --all-features --verbose) + (cd rbx-dom && cargo build --all-features --verbose) build_rbxdom_all=$? record "Build (rbx-dom)" $build_rbxdom record "Build (rbx-dom all-features)" $build_rbxdom_all step 15 "Run ALL rbx-dom Rust Tests" - (cd rbx-dom && cargo test -j 16 --verbose -- --test-threads=16) + (cd rbx-dom && cargo test --verbose -- --test-threads=$THREADS) tests_rbxdom=$? - (cd rbx-dom && cargo test -j 16 --all-features --verbose -- --test-threads=16) + (cd rbx-dom && cargo test --all-features --verbose -- --test-threads=$THREADS) tests_rbxdom_all=$? record "Tests (rbx-dom)" $tests_rbxdom record "Tests (rbx-dom all-features)" $tests_rbxdom_all diff --git a/scripts/format.ps1 b/scripts/format.ps1 new file mode 100644 index 00000000..28f53b17 --- /dev/null +++ b/scripts/format.ps1 @@ -0,0 +1,126 @@ +# Auto-fix Formatting & Static Analysis +# Usage: .\scripts\format.ps1 [-NoRbxDom] + +param( + [switch]$NoRbxDom +) + +$RbxDom = -not $NoRbxDom +$Threads = [Environment]::ProcessorCount + +$ErrorActionPreference = "Continue" +$failures = @() + +function Write-Step($number, $name) { + Write-Host "" + Write-Host "=== Step ${number}: $name ===" -ForegroundColor Cyan +} + +function Record-Result($step, $exitCode) { + if ($exitCode -ne 0) { + $script:failures += $step + Write-Host "FAIL" -ForegroundColor Red + } else { + Write-Host "PASS" -ForegroundColor Green + } +} + +# ─── Rojo ──────────────────────────────────────────────────────────────────── + +Write-Step 1 "Auto-fix Lua Formatting (Stylua)" +stylua plugin/src +$styluaRojo = $LASTEXITCODE +Record-Result "Stylua (Rojo)" $styluaRojo + +Write-Step 2 "Auto-fix Rust Formatting" +cargo fmt +$rustfmt = $LASTEXITCODE +Record-Result "Rustfmt" $rustfmt + +Write-Step 3 "Auto-fix TypeScript Formatting (Prettier)" +Push-Location vscode-rojo +npx prettier --write "src/**/*.ts" +$prettier = $LASTEXITCODE +Pop-Location +Record-Result "Prettier" $prettier + +Write-Step 4 "Lua Static Analysis (Selene)" +selene plugin/src +$selene = $LASTEXITCODE +Record-Result "Selene" $selene + +Write-Step 5 "Rust Linting (Clippy) - Auto-fix" +cargo clippy -j $Threads --fix --allow-dirty --allow-staged 2>&1 | Out-Null +Write-Host "Verifying..." -ForegroundColor Yellow +cargo clippy -j $Threads --all-targets --all-features 2>&1 +$clippy = $LASTEXITCODE +Record-Result "Clippy" $clippy + +# ─── rbx-dom ───────────────────────────────────────────────────────────────── + +if ($RbxDom) { + Write-Step 6 "Auto-fix Lua Formatting (Stylua) for rbx_dom_lua" + stylua rbx-dom/rbx_dom_lua/src + $styluaRbxDom = $LASTEXITCODE + Record-Result "Stylua (rbx_dom_lua)" $styluaRbxDom + + Write-Step 7 "Auto-fix Rust Formatting (rbx-dom)" + Push-Location rbx-dom + cargo fmt + $rustfmtRbxDom = $LASTEXITCODE + Pop-Location + Record-Result "Rustfmt (rbx-dom)" $rustfmtRbxDom + + Write-Step 8 "Lua Static Analysis (Selene) for rbx_dom_lua" + selene rbx-dom/rbx_dom_lua/src + $seleneRbxDom = $LASTEXITCODE + Record-Result "Selene (rbx_dom_lua)" $seleneRbxDom + + Write-Step 9 "Rust Linting (Clippy) for rbx-dom - Auto-fix" + Push-Location rbx-dom + cargo clippy -j $Threads --fix --allow-dirty --allow-staged 2>&1 | Out-Null + Write-Host "Verifying..." -ForegroundColor Yellow + cargo clippy -j $Threads --all-targets --all-features -- -D warnings 2>&1 + $clippyRbxDom = $LASTEXITCODE + Pop-Location + Record-Result "Clippy (rbx-dom)" $clippyRbxDom +} + +# ─── Report ────────────────────────────────────────────────────────────────── + +Write-Host "" +Write-Host "=== FORMAT COMPLETE ===" -ForegroundColor Magenta +Write-Host "" +Write-Host "--- Rojo ---" -ForegroundColor White +Write-Host "" +Write-Host "Formatting:" +Write-Host " - Stylua: $(if ($styluaRojo -eq 0) {'PASS'} else {'FAIL'})" +Write-Host " - Rustfmt: $(if ($rustfmt -eq 0) {'PASS'} else {'FAIL'})" +Write-Host " - Prettier (vscode-rojo): $(if ($prettier -eq 0) {'PASS'} else {'FAIL'})" +Write-Host "" +Write-Host "Linting:" +Write-Host " - Selene: $(if ($selene -eq 0) {'PASS'} else {'FAIL'})" +Write-Host " - Clippy: $(if ($clippy -eq 0) {'PASS'} else {'FAIL'})" + +if ($RbxDom) { + Write-Host "" + Write-Host "--- rbx-dom ---" -ForegroundColor White + Write-Host "" + Write-Host "Formatting:" + Write-Host " - Stylua (rbx_dom_lua): $(if ($styluaRbxDom -eq 0) {'PASS'} else {'FAIL'})" + Write-Host " - Rustfmt: $(if ($rustfmtRbxDom -eq 0) {'PASS'} else {'FAIL'})" + Write-Host "" + Write-Host "Linting:" + Write-Host " - Selene (rbx_dom_lua): $(if ($seleneRbxDom -eq 0) {'PASS'} else {'FAIL'})" + Write-Host " - Clippy: $(if ($clippyRbxDom -eq 0) {'PASS'} else {'FAIL'})" +} + +Write-Host "" +if ($failures.Count -eq 0) { + Write-Host "--- Overall: PASS ---" -ForegroundColor Green + exit 0 +} else { + Write-Host "--- Overall: FAIL ---" -ForegroundColor Red + Write-Host "Failed steps: $($failures -join ', ')" -ForegroundColor Red + exit 1 +} diff --git a/scripts/format.sh b/scripts/format.sh new file mode 100644 index 00000000..38fcceff --- /dev/null +++ b/scripts/format.sh @@ -0,0 +1,126 @@ +#!/usr/bin/env bash +# Auto-fix Formatting & Static Analysis +# Usage: bash scripts/format.sh [--no-rbx-dom] + +set -o pipefail + +RBX_DOM=true +for arg in "$@"; do + case "$arg" in + --no-rbx-dom) RBX_DOM=false ;; + esac +done + +THREADS=$(nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4) + +failures=() + +step() { + echo "" + echo "=== Step $1: $2 ===" +} + +record() { + if [ "$2" -ne 0 ]; then + failures+=("$1") + echo "FAIL" + else + echo "PASS" + fi +} + +# ─── Rojo ──────────────────────────────────────────────────────────────────── + +step 1 "Auto-fix Lua Formatting (Stylua)" +stylua plugin/src +stylua_rojo=$? +record "Stylua (Rojo)" $stylua_rojo + +step 2 "Auto-fix Rust Formatting" +cargo fmt +rustfmt_exit=$? +record "Rustfmt" $rustfmt_exit + +step 3 "Auto-fix TypeScript Formatting (Prettier)" +(cd vscode-rojo && npx prettier --write "src/**/*.ts") +prettier_exit=$? +record "Prettier" $prettier_exit + +step 4 "Lua Static Analysis (Selene)" +selene plugin/src +selene_exit=$? +record "Selene" $selene_exit + +step 5 "Rust Linting (Clippy) - Auto-fix" +cargo clippy -j $THREADS --fix --allow-dirty --allow-staged >/dev/null 2>&1 +echo "Verifying..." +cargo clippy -j $THREADS --all-targets --all-features 2>&1 +clippy_exit=$? +record "Clippy" $clippy_exit + +# ─── rbx-dom ───────────────────────────────────────────────────────────────── + +if [ "$RBX_DOM" = true ]; then + step 6 "Auto-fix Lua Formatting (Stylua) for rbx_dom_lua" + (cd rbx-dom && stylua rbx_dom_lua/src) + stylua_rbxdom=$? + record "Stylua (rbx_dom_lua)" $stylua_rbxdom + + step 7 "Auto-fix Rust Formatting (rbx-dom)" + (cd rbx-dom && cargo fmt) + rustfmt_rbxdom=$? + record "Rustfmt (rbx-dom)" $rustfmt_rbxdom + + step 8 "Lua Static Analysis (Selene) for rbx_dom_lua" + (cd rbx-dom && selene rbx_dom_lua/src) + selene_rbxdom=$? + record "Selene (rbx_dom_lua)" $selene_rbxdom + + step 9 "Rust Linting (Clippy) for rbx-dom - Auto-fix" + (cd rbx-dom && cargo clippy -j $THREADS --fix --allow-dirty --allow-staged >/dev/null 2>&1) + echo "Verifying..." + (cd rbx-dom && cargo clippy -j $THREADS --all-targets --all-features -- -D warnings 2>&1) + clippy_rbxdom=$? + record "Clippy (rbx-dom)" $clippy_rbxdom +fi + +# ─── Report ────────────────────────────────────────────────────────────────── + +pass_or_fail() { [ "$1" -eq 0 ] && echo "PASS" || echo "FAIL"; } + +echo "" +echo "=== FORMAT COMPLETE ===" +echo "" +echo "--- Rojo ---" +echo "" +echo "Formatting:" +echo " - Stylua: $(pass_or_fail $stylua_rojo)" +echo " - Rustfmt: $(pass_or_fail $rustfmt_exit)" +echo " - Prettier (vscode-rojo): $(pass_or_fail $prettier_exit)" +echo "" +echo "Linting:" +echo " - Selene: $(pass_or_fail $selene_exit)" +echo " - Clippy: $(pass_or_fail $clippy_exit)" + +if [ "$RBX_DOM" = true ]; then + echo "" + echo "--- rbx-dom ---" + echo "" + echo "Formatting:" + echo " - Stylua (rbx_dom_lua): $(pass_or_fail $stylua_rbxdom)" + echo " - Rustfmt: $(pass_or_fail $rustfmt_rbxdom)" + echo "" + echo "Linting:" + echo " - Selene (rbx_dom_lua): $(pass_or_fail $selene_rbxdom)" + echo " - Clippy: $(pass_or_fail $clippy_rbxdom)" +fi + +echo "" +if [ ${#failures[@]} -eq 0 ]; then + echo "--- Overall: PASS ---" + exit 0 +else + echo "--- Overall: FAIL ---" + echo "Failed steps: $(IFS=', '; echo "${failures[*]}")" + exit 1 +fi diff --git a/scripts/publish-plugin.ps1 b/scripts/publish-plugin.ps1 index 6c974936..8fdaa55e 100644 --- a/scripts/publish-plugin.ps1 +++ b/scripts/publish-plugin.ps1 @@ -1,14 +1,9 @@ -# Build and publish the Rojo plugin to the Creator Store -# Loads API key from .env file, ROBLOX_API_KEY env var, or -ApiKey argument. -# Get an API key at https://create.roblox.com/credentials (assets read+write). - -param( - [string]$ApiKey -) +# Build and publish the Atlas plugin to the Creator Store via OpenCloud. +# Loads env vars from .env file in the repo root. +# Required .env vars: PLUGIN_UPLOAD_TOKEN, PLUGIN_CI_UNIVERSE_ID, PLUGIN_CI_PLACE_ID $ErrorActionPreference = "Stop" -# Load .env file if it exists $envFile = Join-Path $PSScriptRoot "..\.env" if (Test-Path $envFile) { foreach ($line in Get-Content $envFile) { @@ -22,16 +17,14 @@ if (Test-Path $envFile) { } } -# Resolve API key: argument > env var > .env (already loaded above) -if (-not $ApiKey) { - $ApiKey = $env:ROBLOX_API_KEY -} - -if (-not $ApiKey -or $ApiKey -eq "your-key-here") { - Write-Host "Error: No API key provided." -ForegroundColor Red - Write-Host "Set ROBLOX_API_KEY in .env, env var, or pass -ApiKey " -ForegroundColor Yellow +if (-not $env:PLUGIN_UPLOAD_TOKEN) { + Write-Host "Error: PLUGIN_UPLOAD_TOKEN not set. Add it to .env or set as env var." -ForegroundColor Red exit 1 } +$env:RBX_API_KEY = $env:PLUGIN_UPLOAD_TOKEN +$env:RBX_UNIVERSE_ID = $env:PLUGIN_CI_UNIVERSE_ID +$env:RBX_PLACE_ID = $env:PLUGIN_CI_PLACE_ID + Write-Host "Building and uploading plugin..." -ForegroundColor Cyan -cargo run -- upload plugin.project.json --asset_id 111151139098227 +lune run upload-plugin Atlas.rbxm diff --git a/scripts/publish-plugin.sh b/scripts/publish-plugin.sh new file mode 100755 index 00000000..e29fad4f --- /dev/null +++ b/scripts/publish-plugin.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# Build and publish the Atlas plugin to the Creator Store via OpenCloud. +# Loads env vars from .env file in the repo root. +# Required .env vars: PLUGIN_UPLOAD_TOKEN, PLUGIN_CI_UNIVERSE_ID, PLUGIN_CI_PLACE_ID + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ENV_FILE="$SCRIPT_DIR/../.env" + +if [ -f "$ENV_FILE" ]; then + set -a + source "$ENV_FILE" + set +a +fi + +if [ -z "${PLUGIN_UPLOAD_TOKEN:-}" ]; then + echo "Error: PLUGIN_UPLOAD_TOKEN not set. Add it to .env or set as env var." >&2 + exit 1 +fi + +export RBX_API_KEY="$PLUGIN_UPLOAD_TOKEN" +export RBX_UNIVERSE_ID="${PLUGIN_CI_UNIVERSE_ID:-}" +export RBX_PLACE_ID="${PLUGIN_CI_PLACE_ID:-}" + +echo "Building and uploading plugin..." +lune run upload-plugin Atlas.rbxm diff --git a/scripts/update-reflection.ps1 b/scripts/update-reflection.ps1 new file mode 100644 index 00000000..cb3b5261 --- /dev/null +++ b/scripts/update-reflection.ps1 @@ -0,0 +1,44 @@ +# Update Reflection Database +# Regenerates database.msgpack (Rust) and database.json (Lua) from Roblox Studio. +# Requires: Roblox Studio installed locally. +# Usage: .\scripts\update-reflection.ps1 [--dry-run] + +param( + [switch]$DryRun +) + +$ErrorActionPreference = "Stop" + +Push-Location rbx-dom + +try { + if ($DryRun) { + Write-Host "Dry run: generating reflection database (no output files)" -ForegroundColor Yellow + cargo run --bin rbx_reflector -- generate --patches patches + } else { + Write-Host "Generating reflection database..." -ForegroundColor Cyan + cargo run --bin rbx_reflector -- generate --patches patches rbx_reflection_database/database.msgpack rbx_dom_lua/src/database.json + + Write-Host "Generating allValues.json..." -ForegroundColor Cyan + cargo run --bin rbx_reflector -- values rbx_dom_lua/src/allValues.json + + if ($LASTEXITCODE -ne 0) { + Write-Host "Failed with exit code $LASTEXITCODE" -ForegroundColor Red + exit $LASTEXITCODE + } + + Write-Host "Copying to plugin/rbx_dom_lua/..." -ForegroundColor Cyan + Copy-Item rbx_dom_lua/src/database.json ../plugin/rbx_dom_lua/database.json + Copy-Item rbx_dom_lua/src/allValues.json ../plugin/rbx_dom_lua/allValues.json + Copy-Item rbx_dom_lua/src/Error.lua ../plugin/rbx_dom_lua/Error.lua + Copy-Item rbx_dom_lua/src/base64.lua ../plugin/rbx_dom_lua/base64.lua + Copy-Item rbx_dom_lua/src/PropertyDescriptor.lua ../plugin/rbx_dom_lua/PropertyDescriptor.lua + Copy-Item rbx_dom_lua/src/customProperties.lua ../plugin/rbx_dom_lua/customProperties.lua + # NOTE: init.lua and EncodedValue.lua are intentionally different in the plugin + # (findClassDescriptor, Int64/msgpack handling, float serialization) — do NOT copy them. + } + + Write-Host "Done." -ForegroundColor Green +} finally { + Pop-Location +} diff --git a/scripts/update-reflection.sh b/scripts/update-reflection.sh new file mode 100644 index 00000000..121a1ba3 --- /dev/null +++ b/scripts/update-reflection.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +# Update Reflection Database +# Regenerates database.msgpack (Rust) and database.json (Lua) from Roblox Studio. +# Requires: Roblox Studio installed locally. +# Usage: bash scripts/update-reflection.sh [--dry-run] + +set -e + +cd "$(dirname "$0")/../rbx-dom" + +if [ "$1" = "--dry-run" ]; then + echo "Dry run: generating reflection database (no output files)" + cargo run --bin rbx_reflector -- generate --patches patches +else + echo "Generating reflection database..." + cargo run --bin rbx_reflector -- generate --patches patches rbx_reflection_database/database.msgpack rbx_dom_lua/src/database.json + + echo "Generating allValues.json..." + cargo run --bin rbx_reflector -- values rbx_dom_lua/src/allValues.json + + echo "Copying to plugin/rbx_dom_lua/..." + cp rbx_dom_lua/src/database.json ../plugin/rbx_dom_lua/database.json + cp rbx_dom_lua/src/allValues.json ../plugin/rbx_dom_lua/allValues.json + cp rbx_dom_lua/src/Error.lua ../plugin/rbx_dom_lua/Error.lua + cp rbx_dom_lua/src/base64.lua ../plugin/rbx_dom_lua/base64.lua + cp rbx_dom_lua/src/PropertyDescriptor.lua ../plugin/rbx_dom_lua/PropertyDescriptor.lua + cp rbx_dom_lua/src/customProperties.lua ../plugin/rbx_dom_lua/customProperties.lua + # NOTE: init.lua and EncodedValue.lua are intentionally different in the plugin + # (findClassDescriptor, Int64/msgpack handling, float serialization) — do NOT copy them. +fi + +echo "Done." diff --git a/src/change_processor.rs b/src/change_processor.rs index 86176325..8c0e7a70 100644 --- a/src/change_processor.rs +++ b/src/change_processor.rs @@ -2,6 +2,7 @@ use crossbeam_channel::{select, Receiver, RecvError, Sender}; use jod_thread::JoinHandle; use memofs::{IoResultExt, Vfs, VfsEvent}; use rbx_dom_weak::types::{Ref, Variant}; +use std::collections::HashSet; use std::fmt; use std::path::{Path, PathBuf}; use std::time::{Duration, Instant}; @@ -16,7 +17,10 @@ use crate::{ apply_patch_set, compute_patch_set, AppliedPatchSet, InstigatingSource, PatchSet, RojoTree, }, snapshot_middleware::{snapshot_from_vfs, snapshot_project_node}, - syncback::{deduplicate_name, name_needs_slugify, slugify_name, strip_script_suffix}, + syncback::{ + dedup_suffix::{compute_cleanup_action, parse_dedup_suffix, DedupCleanupAction}, + deduplicate_name, name_needs_slugify, slugify_name, strip_script_suffix, + }, }; /// Wrapper that displays a path relative to a project root directory. @@ -70,6 +74,7 @@ pub struct ChangeProcessor { impl ChangeProcessor { /// Spin up the ChangeProcessor, connecting it to the given tree, VFS, and /// outbound message queue. + #[allow(clippy::too_many_arguments)] pub fn start( tree: Arc>, vfs: Arc, @@ -78,7 +83,9 @@ impl ChangeProcessor { suppressed_paths: Arc>>, ref_path_index: Arc>, project_root: PathBuf, + project_file_path: PathBuf, critical_error_receiver: Option>, + git_repo_root: Option, ) -> Self { let (shutdown_sender, shutdown_receiver) = crossbeam_channel::bounded(1); let vfs_receiver = vfs.event_receiver(); @@ -89,6 +96,8 @@ impl ChangeProcessor { // Canonicalize project_root so path comparisons work with the // \\?\ prefix that std::fs::canonicalize adds on Windows. let project_root = std::fs::canonicalize(&project_root).unwrap_or(project_root); + let project_file_path = + std::fs::canonicalize(&project_file_path).unwrap_or(project_file_path); let task = JobThreadContext { tree, vfs, @@ -96,7 +105,9 @@ impl ChangeProcessor { pending_recovery: Mutex::new(Vec::new()), suppressed_paths, project_root, + project_file_path, ref_path_index, + git_repo_root, }; let job_thread = jod_thread::Builder::new() @@ -104,11 +115,6 @@ impl ChangeProcessor { .spawn(move || { log::trace!("ChangeProcessor thread started"); - // Populate the RefPathIndex from existing meta/model files - // so that renames of instances with pre-existing Rojo_Ref_* - // attributes are handled correctly from the first rename. - task.populate_initial_ref_path_index(); - // Tracks when to run the next reconciliation pass. Set to // Some(future_instant) after VFS events arrive, cleared // after reconcile_tree() runs. This ensures we only do the @@ -238,9 +244,18 @@ struct JobThreadContext { /// Root directory of the project, used to display relative paths in logs. project_root: PathBuf, + /// Path to the project file (e.g. `plugin.project.json`). Used by + /// `reconcile_tree` so that re-snapshotting goes through the project + /// middleware instead of walking the directory as a generic Folder. + project_file_path: PathBuf, + /// Index of meta/model files that contain `Rojo_Ref_*` attributes. /// Shared with ApiService for efficient rename path updates. ref_path_index: Arc>, + + /// Git repository root, if the project is in a git repo. + /// Used for auto-staging Source writes. + git_repo_root: Option, } impl JobThreadContext { @@ -249,61 +264,6 @@ impl JobThreadContext { rel_path(path, &self.project_root) } - /// Scan all `.meta.json5`, `.model.json5`, `.meta.json`, `.model.json` - /// files under the project root for existing `Rojo_Ref_*` attributes - /// and populate the `RefPathIndex`. This ensures pre-existing ref - /// attributes (from CLI syncback, manual edits, or previous sessions) - /// are indexed so rename path updates work from the first rename. - fn populate_initial_ref_path_index(&self) { - let mut count = 0; - let mut dirs_to_visit = vec![self.project_root.clone()]; - while let Some(dir) = dirs_to_visit.pop() { - let entries = match fs::read_dir(&dir) { - Ok(entries) => entries, - Err(_) => continue, - }; - for entry in entries.filter_map(|e| e.ok()) { - let path = entry.path(); - if path.is_dir() { - dirs_to_visit.push(path); - continue; - } - let name = match path.file_name().and_then(|n| n.to_str()) { - Some(n) => n, - None => continue, - }; - if !(name.ends_with(".meta.json5") - || name.ends_with(".model.json5") - || name.ends_with(".meta.json") - || name.ends_with(".model.json")) - { - continue; - } - if let Ok(bytes) = fs::read(&path) { - if let Ok(val) = crate::json::from_slice::(&bytes) { - if let Some(attrs) = val.get("attributes").and_then(|a| a.as_object()) { - let mut index = self.ref_path_index.lock().unwrap(); - for (key, value) in attrs { - if key.starts_with(crate::REF_PATH_ATTRIBUTE_PREFIX) { - if let Some(path_str) = value.as_str() { - index.add(path_str, &path); - count += 1; - } - } - } - } - } - } - } - } - if count > 0 { - log::info!( - "RefPathIndex: populated {} Rojo_Ref_* entries from existing meta/model files", - count - ); - } - } - /// Find the init file inside a directory-format script folder. /// Returns the path to the first `init.*.luau` or `init.*.lua` found. fn find_init_file(dir: &Path) -> Option { @@ -466,28 +426,25 @@ impl JobThreadContext { /// Uses the `RefPathIndex` for O(affected_files) lookup instead of /// scanning the full tree. After updating files, also updates the index /// keys and filesystem paths so future renames remain efficient. - fn update_ref_paths_after_rename(&self, old_path: &str, new_path: &str) { + fn update_ref_paths_after_rename( + &self, + old_path: &str, + new_path: &str, + tree: &crate::snapshot::RojoTree, + ) { if old_path == new_path { return; } - // Look up affected files via the pre-computed index. let files_from_index = self.ref_path_index.lock().unwrap().find_by_prefix(old_path); if files_from_index.is_empty() { return; } - // The directory may have already been renamed on disk by the time - // this function is called. The index still stores old filesystem - // paths. Remap them: replace the old directory segment with the - // new one so we can find the actual files. - let old_segments = crate::split_ref_path(old_path); - let old_segment = old_segments.last().map(|s| s.as_str()).unwrap_or(old_path); - let new_segments = crate::split_ref_path(new_path); - let new_segment = new_segments.last().map(|s| s.as_str()).unwrap_or(new_path); + let old_segment = old_path.rsplit('/').next().unwrap_or(old_path); + let new_segment = new_path.rsplit('/').next().unwrap_or(new_path); - // Pre-compute slugified forms for fallback comparison. let slugified_old = if name_needs_slugify(old_segment) { Some(slugify_name(old_segment)) } else { @@ -508,7 +465,6 @@ impl JobThreadContext { if file_path.exists() { return file_path; } - // File doesn't exist at old path — remap directory component. let mut result = PathBuf::new(); let mut replaced = false; for comp in file_path.components() { @@ -542,13 +498,23 @@ impl JobThreadContext { let mut updated_count = 0; for file_path in &files_to_check { + let source_abs = tree + .get_ids_at_path(file_path) + .first() + .map(|&id| crate::ref_target_path_from_tree(tree, id)) + .unwrap_or_default(); + self.suppress_path(file_path); - match crate::syncback::meta::update_ref_paths_in_file(file_path, old_path, new_path) { + match crate::syncback::meta::update_ref_paths_in_file( + file_path, + old_path, + new_path, + &source_abs, + ) { Ok(true) => { updated_count += 1; } Ok(false) => { - // No stale refs in this file; undo suppression self.unsuppress_path(file_path); } Err(err) => { @@ -971,13 +937,14 @@ impl JobThreadContext { let start = Instant::now(); let instance_context = InstanceContext::new(); - let snapshot = match snapshot_from_vfs(&instance_context, &self.vfs, &self.project_root) { - Ok(s) => s, - Err(e) => { - log::error!("Tree reconciliation snapshot error: {:?}", e); - return; - } - }; + let snapshot = + match snapshot_from_vfs(&instance_context, &self.vfs, &self.project_file_path) { + Ok(s) => s, + Err(e) => { + log::error!("Tree reconciliation snapshot error: {:?}", e); + return; + } + }; let mut tree = self.tree.lock().unwrap(); let root_id = tree.get_root_id(); @@ -1003,6 +970,8 @@ impl JobThreadContext { added_instances: patch_set.added_instances, removed_instances: patch_set.removed_instances, updated_instances: Vec::new(), + stage_ids: HashSet::new(), + stage_paths: Vec::new(), }; let applied = apply_patch_set(&mut tree, structural_patch); @@ -1017,7 +986,7 @@ impl JobThreadContext { ); } - fn handle_tree_event(&self, patch_set: PatchSet) { + fn handle_tree_event(&self, mut patch_set: PatchSet) { // Log incoming patch summary at debug level log::debug!( "Processing client patch: {} removed, {} added, {} updated", @@ -1069,21 +1038,255 @@ impl JobThreadContext { } } + // Dedup suffix cleanup: after removals, check if any dedup + // groups need suffix renames (base-name promotion, group-to-1). + // Must run BEFORE apply_patch_set removes the instances from + // the tree, because we need parent/sibling relationships. + // + // Build a set of ALL removed Refs so the sibling enumeration + // skips co-removed instances (Fix 2: batch-removal awareness). + let removed_set: std::collections::HashSet = + patch_set.removed_instances.iter().copied().collect(); + // Collect metadata updates for survivors whose filesystem path + // changed due to dedup cleanup renames (Fix 1: metadata drift). + let mut dedup_metadata_updates: Vec<(Ref, PathBuf)> = Vec::new(); + + for &removed_id in &patch_set.removed_instances { + let (parent_ref, removed_fs_name, removed_file_dir) = { + let Some(inst) = tree.get_instance(removed_id) else { + continue; + }; + let Some(source) = &inst.metadata().instigating_source else { + continue; + }; + if matches!(source, InstigatingSource::ProjectNode { .. }) { + continue; + } + let parent = inst.parent(); + let fs_name = tree.filesystem_name_for(removed_id); + // Derive the directory containing this file from its own + // instigating source path. This works even when the parent + // is a ProjectNode (where instigating_source is not a Path). + let file_dir = match source { + InstigatingSource::Path(p) => { + let fname = p.file_name().and_then(|f| f.to_str()).unwrap_or(""); + if fname.starts_with("init.") { + // Directory-format: path is dir/init.luau, + // the containing dir is the grandparent + p.parent().and_then(|d| d.parent()).map(|d| d.to_path_buf()) + } else { + p.parent().map(|d| d.to_path_buf()) + } + } + _ => None, + }; + (parent, fs_name, file_dir) + }; + + if parent_ref.is_none() { + continue; + } + + // Parse the dedup suffix from the removed instance's FS name + let removed_stem = removed_fs_name + .split('.') + .next() + .unwrap_or(&removed_fs_name); + let (base_stem, _) = match parse_dedup_suffix(removed_stem) { + Some((base, n)) => (base.to_string(), Some(n)), + None => (removed_stem.to_string(), None), + }; + + // Find siblings that share the same dedup base stem AND extension. + // Matching by stem alone would incorrectly group across different + // middleware types (e.g., Foo.server.luau and Foo.luau have the + // same stem "Foo" but different dedup keys). + let removed_extension = + removed_fs_name.find('.').map(|i| &removed_fs_name[i + 1..]); + + let Some(parent_inst) = tree.get_instance(parent_ref) else { + continue; + }; + let mut remaining_stems: Vec = Vec::new(); + let deleted_was_base = parse_dedup_suffix(removed_stem).is_none(); + + for &sibling_ref in parent_inst.children() { + // Skip ALL instances being removed in this patch, not just + // the current one. This prevents co-removed siblings from + // inflating the remaining count and defeating cleanup rules. + if removed_set.contains(&sibling_ref) { + continue; + } + let sibling_fs = tree.filesystem_name_for(sibling_ref); + let sibling_stem = sibling_fs.split('.').next().unwrap_or(&sibling_fs); + let sibling_ext = sibling_fs.find('.').map(|i| &sibling_fs[i + 1..]); + + // Only group siblings with the same extension (same dedup key space) + if sibling_ext != removed_extension { + continue; + } + + let sibling_base = match parse_dedup_suffix(sibling_stem) { + Some((b, _)) => b, + None => sibling_stem, + }; + if sibling_base.eq_ignore_ascii_case(&base_stem) { + remaining_stems.push(sibling_stem.to_string()); + } + } + + if remaining_stems.is_empty() { + continue; + } + + let extension = removed_extension; + + // Get the parent directory path (derived from the removed + // instance's own path, not the parent's instigating source, + // because the parent may be a ProjectNode). + let Some(parent_dir) = removed_file_dir.clone() else { + continue; + }; + + let action = compute_cleanup_action( + &base_stem, + extension, + &remaining_stems, + deleted_was_base, + &parent_dir, + ); + + match action { + DedupCleanupAction::None => {} + DedupCleanupAction::RemoveSuffix { from, to } + | DedupCleanupAction::PromoteLowest { from, to } => { + log::info!( + "Dedup cleanup: renaming {} -> {}", + self.display_path(&from), + self.display_path(&to), + ); + self.suppress_path_any(&from); + self.suppress_path(&to); + if let Err(e) = fs::rename(&from, &to) { + log::warn!( + "Dedup cleanup rename failed: {} -> {}: {}", + from.display(), + to.display(), + e + ); + self.unsuppress_path(&to); + } else { + // Also rename the adjacent meta file if it exists + // (standalone files only; directory-format instances + // have init.meta.json5 inside the directory which + // moves automatically with the dir rename). + if from.is_file() || !from.exists() { + if let (Some(from_parent), Some(from_name), Some(to_name)) = ( + from.parent(), + from.file_stem().and_then(|s| s.to_str()), + to.file_stem().and_then(|s| s.to_str()), + ) { + let from_base = strip_script_suffix(from_name); + let to_base = strip_script_suffix(to_name); + let old_meta = + from_parent.join(format!("{}.meta.json5", from_base)); + if old_meta.exists() { + let new_meta = + from_parent.join(format!("{}.meta.json5", to_base)); + self.suppress_path_any(&old_meta); + self.suppress_path(&new_meta); + if fs::rename(&old_meta, &new_meta).is_err() { + self.unsuppress_path_any(&old_meta); + self.unsuppress_path(&new_meta); + } + } + } + } + + // Update ref paths if the renamed instance has refs + let old_ref_segment = + from.file_name().and_then(|f| f.to_str()).unwrap_or(""); + let new_ref_segment = + to.file_name().and_then(|f| f.to_str()).unwrap_or(""); + if old_ref_segment != new_ref_segment { + let parent_path = + crate::ref_target_path_from_tree(&tree, parent_ref); + let old_prefix = format!("{}/{}", parent_path, old_ref_segment); + let new_prefix = format!("{}/{}", parent_path, new_ref_segment); + self.update_ref_paths_after_rename(&old_prefix, &new_prefix, &tree); + } + + // Fix 1: Update the renamed survivor's in-memory + // metadata so InstigatingSource::Path and + // relevant_paths point to the new filesystem path. + let old_from_name = + from.file_name().and_then(|f| f.to_str()).unwrap_or(""); + for &sibling_ref in parent_inst.children() { + if removed_set.contains(&sibling_ref) { + continue; + } + let sibling_fs = tree.filesystem_name_for(sibling_ref); + if sibling_fs == old_from_name { + // This is the survivor being renamed. + // Compute its new instigating source path. + if let Some(meta) = tree.get_metadata(sibling_ref) { + if let Some(InstigatingSource::Path(old_path)) = + &meta.instigating_source + { + let new_source = if old_path + .file_name() + .and_then(|f| f.to_str()) + .map(|f| f.starts_with("init.")) + .unwrap_or(false) + { + // Directory-format: old source + // is dir/init.luau; the dir was + // renamed so update the dir + // portion. + let init_name = + old_path.file_name().unwrap().to_str().unwrap(); + to.join(init_name) + } else { + // Standalone file: new source + // IS the `to` path. + to.clone() + }; + dedup_metadata_updates.push((sibling_ref, new_source)); + } + } + break; + } + } + } + } + } + } + // Collect (Ref, new_instigating_source) for instances whose // filesystem path changed (rename / ClassName transition). // Applied after the PatchSet to keep metadata in sync. let mut metadata_updates: Vec<(Ref, PathBuf)> = Vec::new(); + // Paths to stage via git add after all writes complete. + // Starts with pre-resolved paths from api.rs, then Source writes are appended. + let mut pending_stage_paths = std::mem::take(&mut patch_set.stage_paths); + for update in &patch_set.updated_instances { let id = update.id; // Capture the old path BEFORE rename for Rojo_Ref_* path updates. // Must be computed before the `tree.get_instance(id)` borrow. - let old_ref_path = if update.changed_name.is_some() { - Some(crate::ref_target_path(tree.inner(), id)) - } else { - None - }; + let old_ref_path = + if update.changed_name.is_some() || update.changed_class_name.is_some() { + Some(crate::ref_target_path_from_tree(&tree, id)) + } else { + None + }; + + // The new filesystem name segment after a rename. Set during + // rename handling and used by the ref path update code to build + // the correct filesystem-name-based ref path. + let mut new_ref_segment: Option = None; if let Some(instance) = tree.get_instance(id) { // Track the current source file path — rename and ClassName @@ -1155,6 +1358,15 @@ impl JobThreadContext { // The init file moved with the directory. overridden_source_path = Some(new_dir_path.join(file_name)); + new_ref_segment = + Some(slugified_new_name.clone()); + if patch_set.stage_ids.contains(&id) { + let old_init = dir_path.join(file_name); + pending_stage_paths + .retain(|p| *p != old_init); + pending_stage_paths + .push(new_dir_path.join(file_name)); + } let old_meta = grandparent.join(format!( "{}.meta.json5", dir_name @@ -1315,6 +1527,11 @@ impl JobThreadContext { effective_meta_base = old_base; } else { overridden_source_path = Some(new_path.clone()); + new_ref_segment = Some(new_file_name.clone()); + if patch_set.stage_ids.contains(&id) { + pending_stage_paths.retain(|p| *p != *path); + pending_stage_paths.push(new_path.clone()); + } let old_meta = parent .join(format!("{}.meta.json5", old_base)); let new_meta = parent.join(format!( @@ -1339,8 +1556,9 @@ impl JobThreadContext { // Always update the name field when the // instance name changed, even if the path didn't // change (e.g. "Foo/Bar" → "Foo|Bar" both slugify - // to "Foo_Bar"). Use deduped_new_name because - // dedup may have appended ~N. + // to "Foo_Bar"). Only slugification requires a + // meta name -- dedup suffixes (~N) are stripped + // automatically by forward sync. // // For .model.json5/.model.json files, the name // field lives INSIDE the model file, not in @@ -1349,7 +1567,7 @@ impl JobThreadContext { let model_file = overridden_source_path .as_deref() .unwrap_or(path.as_path()); - if deduped_new_name != *new_name { + if slugified_new_name != *new_name { self.upsert_model_name_field( model_file, new_name, ); @@ -1361,7 +1579,7 @@ impl JobThreadContext { "{}.meta.json5", effective_meta_base )); - if deduped_new_name != *new_name { + if slugified_new_name != *new_name { self.upsert_meta_name_field( ¤t_meta, new_name, @@ -1493,6 +1711,19 @@ impl JobThreadContext { ); } else { overridden_source_path = Some(new_path.clone()); + if patch_set.stage_ids.contains(&id) { + pending_stage_paths + .retain(|p| *p != actual_file); + pending_stage_paths.push(new_path.clone()); + } + // For standalone files, the ref path + // segment changes with the extension. + // For init files, the directory name + // (ref segment) is unchanged. + if !is_init { + new_ref_segment = + Some(new_file_name.clone()); + } } } } else { @@ -1569,6 +1800,8 @@ impl JobThreadContext { id, err ); + } else if patch_set.stage_ids.contains(&id) { + pending_stage_paths.push(write_path.clone()); } } else { log::warn!("Cannot change Source to non-string value."); @@ -1594,31 +1827,30 @@ impl JobThreadContext { // // NOTE: The tree name hasn't been updated yet (apply_patch_set // runs after this loop), so we can't use full_path_of for the - // new path. Construct it by replacing the last path segment. - // Use split_ref_path for escape-aware splitting (handles - // instance names containing "/"). + // new path. Construct it by replacing the last path segment + // with the NEW filesystem name (set during rename handling). if let Some(ref old_ref_path) = old_ref_path { - if let Some(ref new_name) = update.changed_name { - let segments = crate::split_ref_path(old_ref_path); + if let Some(ref segment) = new_ref_segment { + // Use the filesystem name computed during rename handling + let segments: Vec<&str> = old_ref_path.split('/').collect(); let new_ref_path = if segments.len() > 1 { - // Rebuild parent path from all segments except the last, - // re-escaping each segment to preserve the original format. - let parent_parts: Vec<_> = segments[..segments.len() - 1] - .iter() - .map(|s| crate::escape_ref_path_segment(s).into_owned()) - .collect(); - format!( - "{}/{}", - parent_parts.join("/"), - crate::escape_ref_path_segment(new_name) - ) + let parent = segments[..segments.len() - 1].join("/"); + format!("{}/{}", parent, segment) } else { - // Root-level instance (no parent in path) - crate::escape_ref_path_segment(new_name).into_owned() + segment.clone() }; if *old_ref_path != new_ref_path { - self.update_ref_paths_after_rename(old_ref_path, &new_ref_path); + self.update_ref_paths_after_rename(old_ref_path, &new_ref_path, &tree); } + } else if update.changed_name.is_some() || update.changed_class_name.is_some() { + // Rename or class change was requested but no filesystem + // rename happened (e.g., ProjectNode, init-file class + // change where directory name stays the same). No ref + // path update needed. + log::trace!( + "Skipping ref path update for {:?}: no filesystem rename", + id + ); } } } @@ -1629,7 +1861,9 @@ impl JobThreadContext { // This keeps instigating_source and path_to_ids in sync so that // subsequent VFS events (and future renames) target the correct // instance and path. - for (id, new_instigating_source) in metadata_updates { + for (id, new_instigating_source) in + metadata_updates.into_iter().chain(dedup_metadata_updates) + { if let Some(old_metadata) = tree.get_metadata(id).cloned() { let mut new_metadata = old_metadata; new_metadata.instigating_source = @@ -1639,6 +1873,14 @@ impl JobThreadContext { } } + // Consolidated git staging: one git_add call for all paths + // (pre-resolved from api.rs + Source writes from this function). + if !pending_stage_paths.is_empty() { + if let Some(ref repo_root) = self.git_repo_root { + crate::git::git_add(repo_root, &pending_stage_paths); + } + } + applied }; diff --git a/src/cli/init.rs b/src/cli/init.rs index 1fa25e58..792518fb 100644 --- a/src/cli/init.rs +++ b/src/cli/init.rs @@ -131,6 +131,10 @@ impl InitCommand { ("core.autocrlf", "false"), ("core.eol", "lf"), ("core.safecrlf", "false"), + // ("core.splitIndex", "true"), + // ("core.fsmonitor", "true"), + // ("core.untrackedcache", "true"), + // ("feature.manyFiles", "true"), ] { let _ = Command::new("git") .args(["config", "--local", key, value]) @@ -159,8 +163,10 @@ impl InitCommand { ".cursor", ]) .current_dir(&base_path) + .stdin(Stdio::null()) .stdout(Stdio::null()) .stderr(Stdio::null()) + .env("GIT_TERMINAL_PROMPT", "0") .status(); match result { diff --git a/src/cli/plugin.rs b/src/cli/plugin.rs index b91320f5..99d72973 100644 --- a/src/cli/plugin.rs +++ b/src/cli/plugin.rs @@ -99,5 +99,5 @@ fn uninstall_plugin() -> anyhow::Result<()> { #[test] fn plugin_initialize() { - assert!(initialize_plugin().is_ok()) + let _ = initialize_plugin().unwrap(); } diff --git a/src/cli/serve.rs b/src/cli/serve.rs index df91220a..1efe19a5 100644 --- a/src/cli/serve.rs +++ b/src/cli/serve.rs @@ -1,13 +1,25 @@ use std::{ - net::{IpAddr, Ipv4Addr}, - path::PathBuf, + collections::HashMap, + io::Cursor, + net::{IpAddr, Ipv4Addr, SocketAddr}, + path::{Path, PathBuf}, sync::Arc, }; +use anyhow::Context; use clap::Parser; use memofs::Vfs; +use rbx_dom_weak::{types::Ref, types::Variant, InstanceBuilder, WeakDom}; -use crate::{serve_session::ServeSession, web::LiveServer}; +use crate::{ + project::Project, + serve_session::ServeSession, + syncback::syncback_loop, + web::{ + interface::{ServerExitReason, SyncbackPayload}, + LiveServer, + }, +}; use super::resolve_path; @@ -35,30 +47,261 @@ impl ServeCommand { pub fn run(self) -> anyhow::Result<()> { let project_path = resolve_path(&self.project); - let (vfs, critical_errors) = Vfs::new_default_with_errors(); - - let session = Arc::new(ServeSession::new(vfs, project_path, Some(critical_errors))?); - - let ip = self - .address - .or_else(|| session.serve_address()) - .unwrap_or(DEFAULT_BIND_ADDRESS.into()); - - let port = self - .port - .or_else(|| session.project_port()) - .unwrap_or(DEFAULT_PORT); - - let server = LiveServer::new(session); + let (ip, port) = { + let vfs = Vfs::new_oneshot(); + let project = Project::load_initial_project(&vfs, &project_path)?; + let ip = self + .address + .or(project.serve_address) + .unwrap_or(DEFAULT_BIND_ADDRESS.into()); + let port = self.port.or(project.serve_port).unwrap_or(DEFAULT_PORT); + (ip, port) + }; + let addr: SocketAddr = (ip, port).into(); let host = if ip.is_loopback() { "localhost".to_owned() } else { ip.to_string() }; - log::info!("Listening: http://{}:{}", host, port); - server.start((ip, port).into()); - Ok(()) + loop { + let (vfs, critical_errors) = Vfs::new_default_with_errors(); + let session = Arc::new(ServeSession::new( + vfs, + project_path.clone(), + Some(critical_errors), + )?); + let server = LiveServer::new(session); + + log::info!("Listening: http://{}:{}", host, port); + + match server.start(addr) { + ServerExitReason::SyncbackRequested(payload) => { + log::info!("Live syncback requested, running..."); + match run_live_syncback(&project_path, payload) { + Ok(()) => log::info!("Syncback complete, restarting serve..."), + Err(err) => { + log::error!("Live syncback failed: {err:#}. Restarting serve...") + } + } + continue; + } + } + } + } +} + +fn run_live_syncback(project_path: &Path, payload: SyncbackPayload) -> anyhow::Result<()> { + let new_dom = build_dom_from_chunks(payload)?; + + let vfs = Vfs::new_oneshot(); + let session_old = ServeSession::new_oneshot(vfs, project_path)?; + + let mut dom_old = session_old.tree(); + + let syncback_timer = std::time::Instant::now(); + log::info!("Beginning live syncback (clean mode)..."); + + let result = syncback_loop( + session_old.vfs(), + &mut dom_old, + new_dom, + session_old.root_project(), + false, // incremental = false → clean mode + )?; + + log::debug!( + "Syncback finished in {:.02}s", + syncback_timer.elapsed().as_secs_f32() + ); + + let base_path = session_old.root_project().folder_location(); + drop(dom_old); + + log::info!("Writing to the file system..."); + result + .fs_snapshot + .write_to_vfs_parallel(base_path, session_old.vfs())?; + + log::info!( + "Finished live syncback: wrote {} files/folders, removed {}.", + result.fs_snapshot.added_paths().len(), + result.fs_snapshot.removed_paths().len() + ); + + crate::git::refresh_git_index(base_path); + + drop(session_old); + + Ok(()) +} + +fn build_dom_from_chunks(payload: SyncbackPayload) -> anyhow::Result { + use crate::syncback::VISIBLE_SERVICES; + + let mut dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let root_ref = dom.root_ref(); + let mut global_ref_map: HashMap = HashMap::new(); + let mut created_services: std::collections::HashSet = std::collections::HashSet::new(); + + let cloned_children: Vec = if !payload.data.is_empty() { + let chunk_dom = rbx_binary::from_reader(Cursor::new(&payload.data)) + .context("Failed to parse rbxm data blob")?; + + let mut cloned = Vec::new(); + for &child_ref in chunk_dom.root().children() { + deep_clone_into( + &chunk_dom, + &mut dom, + child_ref, + root_ref, + &mut global_ref_map, + ); + let new_ref = *global_ref_map.get(&child_ref).unwrap(); + cloned.push(new_ref); + } + cloned + } else { + Vec::new() + }; + + let mut cursor = 0usize; + let mut all_carriers: Vec = Vec::new(); + let mut deferred_refs: Vec<(Ref, Vec<(String, Ref)>)> = Vec::new(); + + for chunk in &payload.services { + let mut builder = InstanceBuilder::new(&chunk.class_name); + for (key, value) in &chunk.properties { + if !crate::syncback::should_property_serialize(&chunk.class_name, key) { + continue; + } + builder = builder.with_property(key.as_str(), value.clone()); + } + let service_ref = dom.insert(root_ref, builder); + created_services.insert(chunk.class_name.clone()); + + let child_count = chunk.child_count as usize; + let ref_count = chunk.ref_target_count as usize; + let total = child_count + ref_count; + let end = cursor + total; + anyhow::ensure!( + end <= cloned_children.len(), + "Service '{}' claims {} children + {} carriers but only {} instances remain in blob", + chunk.class_name, + child_count, + ref_count, + cloned_children.len().saturating_sub(cursor) + ); + let service_range: Vec = cloned_children[cursor..end].to_vec(); + + for &child_ref in &service_range[..child_count.min(service_range.len())] { + dom.transfer_within(child_ref, service_ref); + } + + let carrier_start = child_count; + let carrier_end = service_range.len(); + let carriers = &service_range[carrier_start..carrier_end]; + + let mut ref_entries: Vec<(String, Ref)> = Vec::new(); + for (prop_name, &idx) in &chunk.refs { + if idx == 0 || (idx as usize) > carriers.len() { + continue; + } + ref_entries.push((prop_name.clone(), carriers[idx as usize - 1])); + } + if !ref_entries.is_empty() { + deferred_refs.push((service_ref, ref_entries)); + } + + all_carriers.extend_from_slice(carriers); + cursor = end; + } + + anyhow::ensure!( + cursor == cloned_children.len(), + "Service chunks account for {} instances but rbxm blob contains {}", + cursor, + cloned_children.len() + ); + + for &service_name in VISIBLE_SERVICES { + if !created_services.contains(service_name) { + dom.insert(root_ref, InstanceBuilder::new(service_name)); + } + } + + fixup_ref_properties(&mut dom, &global_ref_map); + + for (service_ref, ref_entries) in deferred_refs { + for (prop_name, carrier_ref) in ref_entries { + if let Some(Variant::Ref(actual_target)) = dom + .get_by_ref(carrier_ref) + .and_then(|inst| inst.properties.get(&rbx_dom_weak::ustr("Value")).cloned()) + { + let service = dom.get_by_ref_mut(service_ref).unwrap(); + service + .properties + .insert(prop_name.as_str().into(), Variant::Ref(actual_target)); + } + } + } + + for carrier_ref in all_carriers { + dom.destroy(carrier_ref); + } + + Ok(dom) +} + +fn deep_clone_into( + source: &WeakDom, + target: &mut WeakDom, + source_ref: Ref, + target_parent: Ref, + ref_map: &mut HashMap, +) { + let inst = source.get_by_ref(source_ref).unwrap(); + let mut builder = InstanceBuilder::new(inst.class.as_str()).with_name(inst.name.as_str()); + + for (key, value) in &inst.properties { + builder = builder.with_property(key.as_str(), value.clone()); + } + + let new_ref = target.insert(target_parent, builder); + ref_map.insert(source_ref, new_ref); + + for &child_ref in inst.children() { + deep_clone_into(source, target, child_ref, new_ref, ref_map); + } +} + +fn fixup_ref_properties(dom: &mut WeakDom, ref_map: &HashMap) { + let all_refs: Vec = ref_map.values().copied().collect(); + for inst_ref in all_refs { + let props_to_fix: Vec<(String, Ref)> = { + let inst = dom.get_by_ref(inst_ref).unwrap(); + inst.properties + .iter() + .filter_map(|(key, value)| { + if let Variant::Ref(r) = value { + if let Some(&mapped) = ref_map.get(r) { + Some((key.to_string(), mapped)) + } else { + None + } + } else { + None + } + }) + .collect() + }; + + if !props_to_fix.is_empty() { + let inst = dom.get_by_ref_mut(inst_ref).unwrap(); + for (key, new_ref) in props_to_fix { + inst.properties.insert(key.into(), Variant::Ref(new_ref)); + } + } } } diff --git a/src/cli/sourcemap.rs b/src/cli/sourcemap.rs index 3c103dd7..0ec05686 100644 --- a/src/cli/sourcemap.rs +++ b/src/cli/sourcemap.rs @@ -182,22 +182,28 @@ fn patch_set_affects_sourcemap( fn recurse_create_node<'a>( tree: &'a RojoTree, referent: Ref, - project_dir: &Path, + canonical_project_dir: &Path, filter: fn(&InstanceWithMeta) -> bool, use_absolute_paths: bool, ) -> Option> { let instance = tree.get_instance(referent).expect("instance did not exist"); - let children: Vec<_> = instance + let results: Vec>> = instance .children() .par_iter() - .filter_map(|&child_id| { - recurse_create_node(tree, child_id, project_dir, filter, use_absolute_paths) + .map(|&child_id| { + recurse_create_node( + tree, + child_id, + canonical_project_dir, + filter, + use_absolute_paths, + ) }) .collect(); - // If this object has no children and doesn't pass the filter, it doesn't - // contain any information we're looking for. + let children: Vec<_> = results.into_iter().flatten().collect(); + if children.is_empty() && !filter(&instance) { return None; } @@ -206,25 +212,19 @@ fn recurse_create_node<'a>( .metadata() .relevant_paths .iter() - // Not all paths listed as relevant are guaranteed to exist. .filter(|path| path.is_file()) .map(|path| path.as_path()); let mut output_file_paths: Vec> = Vec::with_capacity(instance.metadata().relevant_paths.len()); - // Canonicalize project_dir once to normalize Windows \\?\ prefixes - let canonical_project_dir = - std::fs::canonicalize(project_dir).unwrap_or_else(|_| project_dir.to_path_buf()); - for val in file_paths { if use_absolute_paths { let abs_path = path::absolute(val).expect(ABSOLUTE_PATH_FAILED_ERR); output_file_paths.push(Cow::Owned(abs_path)); } else { - let canonical_val = std::fs::canonicalize(val).unwrap_or_else(|_| val.to_path_buf()); output_file_paths.push(Cow::Owned( - pathdiff::diff_paths(&canonical_val, &canonical_project_dir) + pathdiff::diff_paths(val, canonical_project_dir) .expect("Failed to compute relative path from project dir"), )); } @@ -247,10 +247,13 @@ pub(crate) fn write_sourcemap( ) -> anyhow::Result<()> { let tree = session.tree(); + let canonical_project_dir = std::fs::canonicalize(session.root_dir()) + .unwrap_or_else(|_| session.root_dir().to_path_buf()); + let root_node = recurse_create_node( &tree, tree.get_root_id(), - session.root_dir(), + &canonical_project_dir, filter, use_absolute_paths, ); @@ -324,10 +327,7 @@ fn recurse_create_node_from_dom<'a>( .map(|paths| { paths .iter() - .filter_map(|p| { - let canonical = std::fs::canonicalize(p).unwrap_or_else(|_| p.clone()); - pathdiff::diff_paths(&canonical, project_dir).map(Cow::Owned) - }) + .filter_map(|p| pathdiff::diff_paths(p, project_dir).map(Cow::Owned)) .collect() }) .unwrap_or_default(); diff --git a/src/cli/syncback.rs b/src/cli/syncback.rs index 557e3788..c33c9628 100644 --- a/src/cli/syncback.rs +++ b/src/cli/syncback.rs @@ -2,7 +2,6 @@ use std::{ io::{self, BufReader, Write as _}, mem::forget, path::{Path, PathBuf}, - process::{Command, Stdio}, time::Instant, }; @@ -280,8 +279,7 @@ impl SyncbackCommand { result.fs_snapshot.removed_paths().len() ); - // Refresh git index if in a git repository - refresh_git_index(base_path); + crate::git::refresh_git_index(base_path); // Delete input file if using default Project.rbxl location if let Some(input_path) = &delete_input_after_syncback { @@ -386,42 +384,6 @@ fn download_place(place_id: u64) -> anyhow::Result { Ok(temp_file) } -/// Refreshes the git index if the project is in a git repository. -/// -/// This is useful because syncback may rewrite files with identical content, -/// which can cause git to report them as modified due to timestamp changes. -fn refresh_git_index(project_dir: &Path) { - // Check if .git exists in project dir or any parent - let mut check_dir = Some(project_dir); - let mut is_git_repo = false; - while let Some(dir) = check_dir { - if dir.join(".git").exists() { - is_git_repo = true; - break; - } - check_dir = dir.parent(); - } - - if is_git_repo { - log::info!("Refreshing git index..."); - // Exit code 1 is normal - it just means files were refreshed. - // We only warn if git itself fails to run. - match Command::new("git") - .args(["update-index", "--refresh", "-q"]) - .current_dir(project_dir) - .stdin(Stdio::null()) - .stdout(Stdio::null()) - .stderr(Stdio::null()) - .status() - { - Ok(_) => log::info!("Git index refreshed."), - Err(e) => log::warn!("Failed to run git update-index --refresh: {}", e), - } - } else { - log::debug!("Not a git repository, skipping index refresh."); - } -} - fn read_dom(path: &Path, file_kind: FileKind) -> anyhow::Result { let content = BufReader::new(File::open(path)?); match file_kind { diff --git a/src/git.rs b/src/git.rs new file mode 100644 index 00000000..9c19ad49 --- /dev/null +++ b/src/git.rs @@ -0,0 +1,1011 @@ +use std::{ + collections::{HashMap, HashSet}, + io::{BufRead, BufReader, Write}, + path::{Path, PathBuf}, + process::{Command, Stdio}, + sync::{Arc, Mutex}, + thread, +}; + +use rbx_dom_weak::types::Ref; +use sha1::{Digest, Sha1}; + +use crate::{snapshot::RojoTree, web::interface::GitMetadata}; + +const SCRIPT_CLASSES: &[&str] = &["Script", "LocalScript", "ModuleScript"]; + +fn is_script_class(class: &str) -> bool { + SCRIPT_CLASSES.contains(&class) +} + +pub fn git_repo_root(project_root: &Path) -> Option { + let output = Command::new("git") + .args([ + "-C", + &project_root.to_string_lossy(), + "rev-parse", + "--show-toplevel", + ]) + .output() + .ok()?; + + if !output.status.success() { + return None; + } + + let stdout = String::from_utf8_lossy(&output.stdout); + let path_str = stdout.trim(); + if path_str.is_empty() { + return None; + } + + Some(PathBuf::from(path_str)) +} + +fn git_changed_files(repo_root: &Path) -> Option> { + let mut changed = HashSet::new(); + + let diff_output = Command::new("git") + .args([ + "-C", + &repo_root.to_string_lossy(), + "diff", + "HEAD", + "--name-only", + ]) + .output() + .ok()?; + + if diff_output.status.success() { + for line in String::from_utf8_lossy(&diff_output.stdout).lines() { + let trimmed = line.trim(); + if !trimmed.is_empty() { + changed.insert(PathBuf::from(trimmed)); + } + } + } else { + log::debug!("git diff HEAD failed (possibly no commits yet), skipping diff"); + } + + let untracked_output = Command::new("git") + .args([ + "-C", + &repo_root.to_string_lossy(), + "ls-files", + "--others", + "--exclude-standard", + ]) + .output() + .ok()?; + + if untracked_output.status.success() { + for line in String::from_utf8_lossy(&untracked_output.stdout).lines() { + let trimmed = line.trim(); + if !trimmed.is_empty() { + changed.insert(PathBuf::from(trimmed)); + } + } + } + + if changed.is_empty() && !diff_output.status.success() { + return None; + } + + Some(changed) +} + +#[cfg(test)] +fn git_show(repo_root: &Path, object_ref: &str) -> Option { + let output = Command::new("git") + .args(["-C", &repo_root.to_string_lossy(), "show", object_ref]) + .output() + .ok()?; + + if !output.status.success() { + return None; + } + + String::from_utf8(output.stdout).ok() +} + +#[cfg(test)] +fn git_show_head(repo_root: &Path, rel_path: &Path) -> Option { + let object_ref = format!("HEAD:{}", rel_path.to_string_lossy().replace('\\', "/")); + git_show(repo_root, &object_ref) +} + +#[cfg(test)] +fn git_show_staged(repo_root: &Path, rel_path: &Path) -> Option { + let object_ref = format!(":0:{}", rel_path.to_string_lossy().replace('\\', "/")); + git_show(repo_root, &object_ref) +} + +/// Retrieves git blob hashes for multiple object refs in a single subprocess. +/// Returns a Vec with the same length as `object_refs`, where each element is +/// `Some(sha1_hex)` if the object exists, or `None` if missing. +/// +/// Uses `git cat-file --batch-check` which outputs the stored blob hash +/// directly, avoiding the need to download and re-hash file contents. +fn git_batch_check_hashes(repo_root: &Path, object_refs: &[String]) -> Vec> { + if object_refs.is_empty() { + return Vec::new(); + } + + let mut child = match Command::new("git") + .args([ + "-C", + &repo_root.to_string_lossy(), + "cat-file", + "--batch-check", + ]) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()) + .spawn() + { + Ok(child) => child, + Err(err) => { + log::warn!("Failed to spawn git cat-file --batch-check: {}", err); + return vec![None; object_refs.len()]; + } + }; + + let stdin = child.stdin.take().unwrap(); + let refs_for_writer: Vec = object_refs.to_vec(); + let writer_thread = thread::spawn(move || { + let mut stdin = stdin; + for object_ref in &refs_for_writer { + if writeln!(stdin, "{}", object_ref).is_err() { + break; + } + } + }); + + let expected = object_refs.len(); + let mut results = Vec::with_capacity(expected); + + if let Some(stdout) = child.stdout.take() { + let reader = BufReader::new(stdout); + for line in reader.lines() { + let line = match line { + Ok(l) => l, + Err(_) => break, + }; + + if line.ends_with(" missing") { + results.push(None); + } else if let Some(sha1) = line.split(' ').next() { + if sha1.len() == 40 && sha1.bytes().all(|b| b.is_ascii_hexdigit()) { + results.push(Some(sha1.to_string())); + } else { + results.push(None); + } + } else { + results.push(None); + } + + if results.len() >= expected { + break; + } + } + } + + while results.len() < expected { + results.push(None); + } + + let _ = writer_thread.join(); + let _ = child.wait(); + results +} + +pub fn compute_blob_sha1(content: &str) -> String { + let header = format!("blob {}\0", content.len()); + let mut hasher = Sha1::new(); + hasher.update(header.as_bytes()); + hasher.update(content.as_bytes()); + format!("{:x}", hasher.finalize()) +} + +pub fn git_add(repo_root: &Path, paths: &[PathBuf]) { + if paths.is_empty() { + return; + } + + let mut cmd = Command::new("git"); + cmd.args(["-C", &repo_root.to_string_lossy(), "add", "--"]); + for path in paths { + cmd.arg(path); + } + + match cmd.output() { + Ok(output) => { + if !output.status.success() { + let stderr = String::from_utf8_lossy(&output.stderr); + log::warn!("git add failed: {}", stderr.trim()); + } + } + Err(err) => { + log::warn!("Failed to run git add: {}", err); + } + } +} + +struct ResolvedInstance { + id: Ref, + class_name: String, + rel_path: PathBuf, +} + +/// Computes git metadata for the two-way sync confirmation UI. +/// +/// `repo_root` must be the git repository root (from `git_repo_root()`), +/// cached at session start by `ServeSession`. +/// +/// Uses a three-phase approach to avoid holding the tree lock during I/O: +/// 1. Run git commands to get changed files (no lock) +/// 2. Briefly lock tree to resolve file paths to instance Refs and class names +/// 3. Single `git cat-file --batch-check` call to get blob hashes (no lock) +pub fn compute_git_metadata(tree_handle: &Arc>, repo_root: &Path) -> GitMetadata { + let changed_files = match git_changed_files(repo_root) { + Some(files) => files, + None => { + return GitMetadata { + changed_ids: Vec::new(), + script_committed_hashes: HashMap::new(), + }; + } + }; + + if changed_files.is_empty() { + return GitMetadata { + changed_ids: Vec::new(), + script_committed_hashes: HashMap::new(), + }; + } + + // Brief lock: resolve changed file paths to instance Refs and class names + let resolved: Vec = { + let tree = tree_handle.lock().unwrap(); + let mut result = Vec::new(); + + for rel_path in &changed_files { + let abs_path = repo_root.join(rel_path); + let canonical_path = std::fs::canonicalize(&abs_path).ok(); + + let first = tree.get_ids_at_path(&abs_path); + let ids = if !first.is_empty() { + first + } else if let Some(ref canon) = canonical_path { + tree.get_ids_at_path(canon) + } else { + first + }; + + for &id in ids { + if let Some(instance) = tree.get_instance(id) { + result.push(ResolvedInstance { + id, + class_name: instance.class_name().to_string(), + rel_path: rel_path.clone(), + }); + } + } + } + + result + }; // Lock released + + let changed_ids: Vec = resolved.iter().map(|ri| ri.id).collect(); + let mut script_committed_hashes: HashMap> = HashMap::new(); + + struct ScriptRef { + resolved_idx: usize, + is_staged: bool, + } + + let mut object_refs = Vec::new(); + let mut script_refs: Vec = Vec::new(); + + for (idx, ri) in resolved.iter().enumerate() { + if !is_script_class(&ri.class_name) { + continue; + } + let path_str = ri.rel_path.to_string_lossy().replace('\\', "/"); + + object_refs.push(format!("HEAD:{}", path_str)); + script_refs.push(ScriptRef { + resolved_idx: idx, + is_staged: false, + }); + + object_refs.push(format!(":0:{}", path_str)); + script_refs.push(ScriptRef { + resolved_idx: idx, + is_staged: true, + }); + } + + let batch_hashes = git_batch_check_hashes(repo_root, &object_refs); + + for (i, info) in script_refs.iter().enumerate() { + let sha1 = match batch_hashes.get(i).and_then(|h| h.as_ref()) { + Some(h) => h, + None => continue, + }; + let ri = &resolved[info.resolved_idx]; + let hashes = script_committed_hashes + .entry(ri.id) + .or_insert_with(|| Vec::with_capacity(2)); + + if info.is_staged { + if hashes.is_empty() || hashes[0] != *sha1 { + hashes.push(sha1.clone()); + } + } else { + hashes.push(sha1.clone()); + } + } + + GitMetadata { + changed_ids, + script_committed_hashes, + } +} + +/// Refreshes the git index if the project is in a git repository. +/// +/// This is useful because syncback may rewrite files with identical content, +/// which can cause git to report them as modified due to timestamp changes. +pub fn refresh_git_index(project_dir: &Path) { + let mut check_dir = Some(project_dir); + let mut is_git_repo = false; + while let Some(dir) = check_dir { + if dir.join(".git").exists() { + is_git_repo = true; + break; + } + check_dir = dir.parent(); + } + + if is_git_repo { + log::info!("Refreshing git index..."); + match Command::new("git") + .args(["update-index", "--refresh", "-q"]) + .current_dir(project_dir) + .stdin(Stdio::null()) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .status() + { + Ok(_) => log::info!("Git index refreshed."), + Err(e) => log::warn!("Failed to run git update-index --refresh: {}", e), + } + } else { + log::debug!("Not a git repository, skipping index refresh."); + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::fs; + use tempfile::tempdir; + + fn git_init(dir: &Path) { + Command::new("git") + .args(["-C", &dir.to_string_lossy(), "init"]) + .output() + .expect("git init failed"); + Command::new("git") + .args([ + "-C", + &dir.to_string_lossy(), + "config", + "user.email", + "test@test.com", + ]) + .output() + .unwrap(); + Command::new("git") + .args(["-C", &dir.to_string_lossy(), "config", "user.name", "Test"]) + .output() + .unwrap(); + } + + fn git_commit_all(dir: &Path, msg: &str) { + Command::new("git") + .args(["-C", &dir.to_string_lossy(), "add", "-A"]) + .output() + .unwrap(); + Command::new("git") + .args(["-C", &dir.to_string_lossy(), "commit", "-m", msg]) + .output() + .unwrap(); + } + + fn git_stage(dir: &Path, file: &str) { + Command::new("git") + .args(["-C", &dir.to_string_lossy(), "add", file]) + .output() + .unwrap(); + } + + fn git_is_staged(dir: &Path, file: &str) -> bool { + let output = Command::new("git") + .args([ + "-C", + &dir.to_string_lossy(), + "diff", + "--cached", + "--name-only", + ]) + .output() + .unwrap(); + let stdout = String::from_utf8_lossy(&output.stdout); + stdout.lines().any(|l| l.trim() == file) + } + + // ----------------------------------------------------------------------- + // compute_blob_sha1 + // ----------------------------------------------------------------------- + + #[test] + fn blob_sha1_empty_content() { + let hash = compute_blob_sha1(""); + // git hash-object -t blob --stdin <<< '' produces the hash for empty blob + // "blob 0\0" -> SHA1 = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 + assert_eq!(hash, "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"); + } + + #[test] + fn blob_sha1_hello_world() { + // Verify against `echo -n "hello world" | git hash-object --stdin` + // "blob 11\0hello world" -> SHA1 = 95d09f2b10159347eece71399a7e2e907ea3df4f + let hash = compute_blob_sha1("hello world"); + assert_eq!(hash, "95d09f2b10159347eece71399a7e2e907ea3df4f"); + } + + #[test] + fn blob_sha1_multiline_script() { + let content = "local foo = 1\nlocal bar = 2\nreturn foo + bar\n"; + let hash = compute_blob_sha1(content); + assert_eq!(hash.len(), 40); + // Same content must produce same hash + assert_eq!(hash, compute_blob_sha1(content)); + } + + #[test] + fn blob_sha1_different_content_different_hash() { + let h1 = compute_blob_sha1("version 1"); + let h2 = compute_blob_sha1("version 2"); + assert_ne!(h1, h2); + } + + #[test] + fn blob_sha1_matches_git_hash_object() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + + let content = "print('test script')\n"; + fs::write(dir.path().join("test.luau"), content).unwrap(); + + let output = Command::new("git") + .args([ + "-C", + &dir.path().to_string_lossy(), + "hash-object", + "test.luau", + ]) + .output() + .unwrap(); + let git_hash = String::from_utf8_lossy(&output.stdout).trim().to_string(); + + let our_hash = compute_blob_sha1(content); + assert_eq!(our_hash, git_hash); + } + + // ----------------------------------------------------------------------- + // git_repo_root + // ----------------------------------------------------------------------- + + #[test] + fn repo_root_not_a_repo() { + let dir = tempdir().unwrap(); + assert!(git_repo_root(dir.path()).is_none()); + } + + #[test] + fn repo_root_valid_repo() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + let root = git_repo_root(dir.path()).unwrap(); + assert!(root.exists()); + } + + #[test] + fn repo_root_from_subdirectory() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + let sub = dir.path().join("sub").join("deep"); + fs::create_dir_all(&sub).unwrap(); + let root = git_repo_root(&sub).unwrap(); + // Subdirectory should resolve to the same repo root + assert_eq!( + root.canonicalize().unwrap(), + git_repo_root(dir.path()).unwrap().canonicalize().unwrap() + ); + } + + // ----------------------------------------------------------------------- + // git_changed_files + // ----------------------------------------------------------------------- + + #[test] + fn changed_files_no_changes() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("file.luau"), "content").unwrap(); + git_commit_all(dir.path(), "init"); + + let changed = git_changed_files(dir.path()); + assert!(changed.is_some()); + assert!(changed.unwrap().is_empty()); + } + + #[test] + fn changed_files_unstaged_modification() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("script.luau"), "original").unwrap(); + git_commit_all(dir.path(), "init"); + fs::write(dir.path().join("script.luau"), "modified").unwrap(); + + let changed = git_changed_files(dir.path()).unwrap(); + assert!(changed.contains(&PathBuf::from("script.luau"))); + } + + #[test] + fn changed_files_staged_modification() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("script.luau"), "original").unwrap(); + git_commit_all(dir.path(), "init"); + fs::write(dir.path().join("script.luau"), "staged version").unwrap(); + git_stage(dir.path(), "script.luau"); + + let changed = git_changed_files(dir.path()).unwrap(); + assert!(changed.contains(&PathBuf::from("script.luau"))); + } + + #[test] + fn changed_files_untracked_file() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("existing.luau"), "content").unwrap(); + git_commit_all(dir.path(), "init"); + fs::write(dir.path().join("new_file.luau"), "new").unwrap(); + + let changed = git_changed_files(dir.path()).unwrap(); + assert!(changed.contains(&PathBuf::from("new_file.luau"))); + } + + #[test] + fn changed_files_deleted_file() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("to_delete.luau"), "content").unwrap(); + git_commit_all(dir.path(), "init"); + fs::remove_file(dir.path().join("to_delete.luau")).unwrap(); + + let changed = git_changed_files(dir.path()).unwrap(); + assert!(changed.contains(&PathBuf::from("to_delete.luau"))); + } + + #[test] + fn changed_files_multiple_types() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("committed.luau"), "ok").unwrap(); + fs::write(dir.path().join("will_modify.luau"), "old").unwrap(); + git_commit_all(dir.path(), "init"); + + fs::write(dir.path().join("will_modify.luau"), "new").unwrap(); + fs::write(dir.path().join("untracked.luau"), "brand new").unwrap(); + + let changed = git_changed_files(dir.path()).unwrap(); + assert!(changed.contains(&PathBuf::from("will_modify.luau"))); + assert!(changed.contains(&PathBuf::from("untracked.luau"))); + assert!(!changed.contains(&PathBuf::from("committed.luau"))); + } + + #[test] + fn changed_files_no_head_returns_none() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + // No commits at all, but a file exists + fs::write(dir.path().join("new.luau"), "content").unwrap(); + + let changed = git_changed_files(dir.path()); + // Should still return Some with the untracked file + assert!(changed.is_some()); + assert!(changed.unwrap().contains(&PathBuf::from("new.luau"))); + } + + #[test] + fn changed_files_subdirectory() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::create_dir_all(dir.path().join("src")).unwrap(); + fs::write(dir.path().join("src/module.luau"), "original").unwrap(); + git_commit_all(dir.path(), "init"); + fs::write(dir.path().join("src/module.luau"), "modified").unwrap(); + + let changed = git_changed_files(dir.path()).unwrap(); + assert!(changed.contains(&PathBuf::from("src/module.luau"))); + } + + // ----------------------------------------------------------------------- + // git_show_head / git_show_staged + // ----------------------------------------------------------------------- + + #[test] + fn show_head_committed_file() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("script.luau"), "committed content").unwrap(); + git_commit_all(dir.path(), "init"); + + let content = git_show_head(dir.path(), Path::new("script.luau")); + assert_eq!(content.as_deref(), Some("committed content")); + } + + #[test] + fn show_head_new_file_returns_none() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("dummy.luau"), "x").unwrap(); + git_commit_all(dir.path(), "init"); + + let content = git_show_head(dir.path(), Path::new("nonexistent.luau")); + assert!(content.is_none()); + } + + #[test] + fn show_head_returns_committed_not_working_tree() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("script.luau"), "v1").unwrap(); + git_commit_all(dir.path(), "v1"); + fs::write(dir.path().join("script.luau"), "v2 modified").unwrap(); + + let content = git_show_head(dir.path(), Path::new("script.luau")); + assert_eq!(content.as_deref(), Some("v1")); + } + + #[test] + fn show_staged_returns_staged_content() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("script.luau"), "v1").unwrap(); + git_commit_all(dir.path(), "v1"); + fs::write(dir.path().join("script.luau"), "v2 staged").unwrap(); + git_stage(dir.path(), "script.luau"); + + let content = git_show_staged(dir.path(), Path::new("script.luau")); + assert_eq!(content.as_deref(), Some("v2 staged")); + } + + #[test] + fn show_staged_returns_none_for_unstaged() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("script.luau"), "v1").unwrap(); + git_commit_all(dir.path(), "init"); + + // File not staged, show :0: should return committed version (same as HEAD) + let staged = git_show_staged(dir.path(), Path::new("script.luau")); + let head = git_show_head(dir.path(), Path::new("script.luau")); + assert_eq!(staged, head); + } + + #[test] + fn show_staged_differs_from_head_when_staged() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("script.luau"), "v1").unwrap(); + git_commit_all(dir.path(), "v1"); + fs::write(dir.path().join("script.luau"), "v2 staged").unwrap(); + git_stage(dir.path(), "script.luau"); + + let head = git_show_head(dir.path(), Path::new("script.luau")).unwrap(); + let staged = git_show_staged(dir.path(), Path::new("script.luau")).unwrap(); + assert_ne!(head, staged); + assert_eq!(head, "v1"); + assert_eq!(staged, "v2 staged"); + } + + #[test] + fn show_head_subdirectory_path() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::create_dir_all(dir.path().join("src")).unwrap(); + fs::write(dir.path().join("src/init.luau"), "init content").unwrap(); + git_commit_all(dir.path(), "init"); + + let content = git_show_head(dir.path(), Path::new("src/init.luau")); + assert_eq!(content.as_deref(), Some("init content")); + } + + // ----------------------------------------------------------------------- + // git_batch_check_hashes + // ----------------------------------------------------------------------- + + #[test] + fn batch_check_empty_refs() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + let results = git_batch_check_hashes(dir.path(), &[]); + assert!(results.is_empty()); + } + + #[test] + fn batch_check_single_ref() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("script.luau"), "local x = 42\nreturn x").unwrap(); + git_commit_all(dir.path(), "init"); + + let refs = vec!["HEAD:script.luau".to_string()]; + let results = git_batch_check_hashes(dir.path(), &refs); + assert_eq!(results.len(), 1); + + let batch_hash = results[0].as_ref().unwrap(); + let content = git_show_head(dir.path(), Path::new("script.luau")).unwrap(); + let expected_hash = compute_blob_sha1(&content); + assert_eq!(*batch_hash, expected_hash); + } + + #[test] + fn batch_check_missing_ref() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("dummy.luau"), "x").unwrap(); + git_commit_all(dir.path(), "init"); + + let refs = vec!["HEAD:nonexistent.luau".to_string()]; + let results = git_batch_check_hashes(dir.path(), &refs); + assert_eq!(results.len(), 1); + assert!(results[0].is_none()); + } + + #[test] + fn batch_check_head_and_staged_same() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("a.luau"), "content a").unwrap(); + git_commit_all(dir.path(), "init"); + + let refs = vec!["HEAD:a.luau".to_string(), ":0:a.luau".to_string()]; + let results = git_batch_check_hashes(dir.path(), &refs); + assert_eq!(results.len(), 2); + assert!(results[0].is_some()); + assert_eq!(results[0], results[1]); + assert_eq!( + results[0].as_ref().unwrap(), + &compute_blob_sha1("content a") + ); + } + + #[test] + fn batch_check_head_and_staged_differ() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("b.luau"), "original").unwrap(); + git_commit_all(dir.path(), "init"); + fs::write(dir.path().join("b.luau"), "staged version").unwrap(); + git_stage(dir.path(), "b.luau"); + + let refs = vec!["HEAD:b.luau".to_string(), ":0:b.luau".to_string()]; + let results = git_batch_check_hashes(dir.path(), &refs); + assert_eq!(results.len(), 2); + assert!(results[0].is_some()); + assert!(results[1].is_some()); + assert_ne!(results[0], results[1]); + assert_eq!(results[0].as_ref().unwrap(), &compute_blob_sha1("original")); + assert_eq!( + results[1].as_ref().unwrap(), + &compute_blob_sha1("staged version") + ); + } + + #[test] + fn batch_check_multiple_files_mixed() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("a.luau"), "content a").unwrap(); + fs::write(dir.path().join("b.luau"), "content b").unwrap(); + git_commit_all(dir.path(), "init"); + fs::write(dir.path().join("b.luau"), "content b modified").unwrap(); + git_stage(dir.path(), "b.luau"); + + let refs = vec![ + "HEAD:a.luau".to_string(), + ":0:a.luau".to_string(), + "HEAD:b.luau".to_string(), + ":0:b.luau".to_string(), + "HEAD:nonexistent.luau".to_string(), + ]; + let results = git_batch_check_hashes(dir.path(), &refs); + assert_eq!(results.len(), 5); + + assert_eq!(results[0], results[1]); // a: HEAD == staged + assert_ne!(results[2], results[3]); // b: HEAD != staged + assert!(results[4].is_none()); // missing + + assert_eq!( + results[0].as_ref().unwrap(), + &compute_blob_sha1("content a") + ); + assert_eq!( + results[2].as_ref().unwrap(), + &compute_blob_sha1("content b") + ); + assert_eq!( + results[3].as_ref().unwrap(), + &compute_blob_sha1("content b modified") + ); + } + + #[test] + fn batch_check_subdirectory_path() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::create_dir_all(dir.path().join("src")).unwrap(); + fs::write(dir.path().join("src/init.luau"), "init content").unwrap(); + git_commit_all(dir.path(), "init"); + + let refs = vec!["HEAD:src/init.luau".to_string()]; + let results = git_batch_check_hashes(dir.path(), &refs); + assert_eq!(results.len(), 1); + assert_eq!( + results[0].as_ref().unwrap(), + &compute_blob_sha1("init content") + ); + } + + // ----------------------------------------------------------------------- + // git_add + // ----------------------------------------------------------------------- + + #[test] + fn git_add_stages_file() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("file.luau"), "content").unwrap(); + git_commit_all(dir.path(), "init"); + fs::write(dir.path().join("file.luau"), "modified").unwrap(); + + assert!(!git_is_staged(dir.path(), "file.luau")); + git_add(dir.path(), &[PathBuf::from("file.luau")]); + assert!(git_is_staged(dir.path(), "file.luau")); + } + + #[test] + fn git_add_stages_multiple_files() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("a.luau"), "a").unwrap(); + fs::write(dir.path().join("b.luau"), "b").unwrap(); + git_commit_all(dir.path(), "init"); + fs::write(dir.path().join("a.luau"), "a modified").unwrap(); + fs::write(dir.path().join("b.luau"), "b modified").unwrap(); + + git_add( + dir.path(), + &[PathBuf::from("a.luau"), PathBuf::from("b.luau")], + ); + assert!(git_is_staged(dir.path(), "a.luau")); + assert!(git_is_staged(dir.path(), "b.luau")); + } + + #[test] + fn git_add_empty_paths_is_noop() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + // Should not panic or error + git_add(dir.path(), &[]); + } + + #[test] + fn git_add_stages_new_file() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("existing.luau"), "x").unwrap(); + git_commit_all(dir.path(), "init"); + fs::write(dir.path().join("new.luau"), "new content").unwrap(); + + git_add(dir.path(), &[PathBuf::from("new.luau")]); + assert!(git_is_staged(dir.path(), "new.luau")); + } + + // ----------------------------------------------------------------------- + // is_script_class + // ----------------------------------------------------------------------- + + #[test] + fn script_class_detection() { + assert!(is_script_class("Script")); + assert!(is_script_class("LocalScript")); + assert!(is_script_class("ModuleScript")); + assert!(!is_script_class("Folder")); + assert!(!is_script_class("Part")); + assert!(!is_script_class("Model")); + assert!(!is_script_class("StringValue")); + } + + // ----------------------------------------------------------------------- + // Hash consistency: our SHA1 matches git hash-object for various content + // ----------------------------------------------------------------------- + + #[test] + fn hash_consistency_empty_file() { + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("empty.luau"), "").unwrap(); + + let output = Command::new("git") + .args([ + "-C", + &dir.path().to_string_lossy(), + "hash-object", + "empty.luau", + ]) + .output() + .unwrap(); + let git_hash = String::from_utf8_lossy(&output.stdout).trim().to_string(); + assert_eq!(compute_blob_sha1(""), git_hash); + } + + #[test] + fn hash_consistency_unicode_content() { + let content = "-- Unicode: 日本語テスト\nlocal x = '🎮'\n"; + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("unicode.luau"), content).unwrap(); + + let output = Command::new("git") + .args([ + "-C", + &dir.path().to_string_lossy(), + "hash-object", + "unicode.luau", + ]) + .output() + .unwrap(); + let git_hash = String::from_utf8_lossy(&output.stdout).trim().to_string(); + assert_eq!(compute_blob_sha1(content), git_hash); + } + + #[test] + fn hash_consistency_large_file() { + let content: String = (0..10000) + .map(|i| format!("local var_{} = {}\n", i, i)) + .collect(); + let dir = tempdir().unwrap(); + git_init(dir.path()); + fs::write(dir.path().join("large.luau"), &content).unwrap(); + + let output = Command::new("git") + .args([ + "-C", + &dir.path().to_string_lossy(), + "hash-object", + "large.luau", + ]) + .output() + .unwrap(); + let git_hash = String::from_utf8_lossy(&output.stdout).trim().to_string(); + assert_eq!(compute_blob_sha1(&content), git_hash); + } +} diff --git a/src/json.rs b/src/json.rs index 058c6278..3abeccfd 100644 --- a/src/json.rs +++ b/src/json.rs @@ -4,7 +4,9 @@ //! to reduce boilerplate and improve ergonomics when working with JSONC files. use anyhow::Context as _; +use lexical_write_float::{format::STANDARD, Options, RoundMode, ToLexicalWithOptions}; use serde::{de::DeserializeOwned, Serialize}; +use std::num::{NonZeroI32, NonZeroUsize}; /// Parse JSONC text into a `serde_json::Value`. /// @@ -126,6 +128,52 @@ pub fn from_slice_with_context( from_str_with_context(text, context) } +const SCI_POSITIVE_BREAK: Option = NonZeroI32::new(15); +const SCI_NEGATIVE_BREAK: Option = NonZeroI32::new(-6); + +const F32_FLOAT_OPTIONS: Options = Options::builder() + .trim_floats(true) + .nan_string(Some(b"NaN")) + .inf_string(Some(b"Infinity")) + .positive_exponent_break(SCI_POSITIVE_BREAK) + .negative_exponent_break(SCI_NEGATIVE_BREAK) + .max_significant_digits(NonZeroUsize::new(6)) + .round_mode(RoundMode::Round) + .build_strict(); + +const F64_FLOAT_OPTIONS: Options = Options::builder() + .trim_floats(true) + .nan_string(Some(b"NaN")) + .inf_string(Some(b"Infinity")) + .positive_exponent_break(SCI_POSITIVE_BREAK) + .negative_exponent_break(SCI_NEGATIVE_BREAK) + .round_mode(RoundMode::Round) + .build_strict(); + +const F32_BUF_SIZE: usize = F32_FLOAT_OPTIONS.buffer_size_const::(); +const F64_BUF_SIZE: usize = F64_FLOAT_OPTIONS.buffer_size_const::(); + +pub(crate) const F32_DISK_OPTIONS: &Options = &F32_FLOAT_OPTIONS; +pub(crate) const F64_DISK_OPTIONS: &Options = &F64_FLOAT_OPTIONS; +pub(crate) const F32_DISK_BUF_SIZE: usize = F32_BUF_SIZE; +pub(crate) const F64_DISK_BUF_SIZE: usize = F64_BUF_SIZE; + +pub(crate) fn format_f32(v: f32) -> String { + let mut buffer = [0u8; F32_BUF_SIZE]; + let digits = v.to_lexical_with_options::(&mut buffer, &F32_FLOAT_OPTIONS); + std::str::from_utf8(digits) + .expect("lexical-write-float produced invalid utf-8") + .into() +} + +pub(crate) fn format_f64(v: f64) -> String { + let mut buffer = [0u8; F64_BUF_SIZE]; + let digits = v.to_lexical_with_options::(&mut buffer, &F64_FLOAT_OPTIONS); + std::str::from_utf8(digits) + .expect("lexical-write-float produced invalid utf-8") + .into() +} + /// A JSON5 value that uses BTreeMap for sorted keys and supports NaN/Infinity. /// Uses String for numbers to preserve exact representation (including scientific notation). #[derive(Debug, Clone, PartialEq)] @@ -154,6 +202,25 @@ impl Json5Value { Json5Value::Array(arr) => { if arr.is_empty() { output.push_str("[]"); + } else if arr.len() <= 20 + && arr.iter().all(|v| { + matches!( + v, + Json5Value::Null + | Json5Value::Bool(_) + | Json5Value::Number(_) + | Json5Value::String(_) + ) + }) + { + output.push_str("[ "); + for (i, item) in arr.iter().enumerate() { + if i > 0 { + output.push_str(", "); + } + item.write_to(output, 0); + } + output.push_str(" ]"); } else { output.push_str("[\n"); for (i, item) in arr.iter().enumerate() { @@ -341,27 +408,12 @@ impl<'a> serde::Serializer for &mut CompactJson5Serializer<'a> { } fn serialize_f32(self, v: f32) -> Result<(), Self::Error> { - self.serialize_f64(v as f64) + self.output.push_str(&format_f32(v)); + Ok(()) } fn serialize_f64(self, v: f64) -> Result<(), Self::Error> { - if v.is_nan() { - self.output.push_str("NaN"); - } else if v.is_infinite() { - if v.is_sign_positive() { - self.output.push_str("Infinity"); - } else { - self.output.push_str("-Infinity"); - } - } else { - // Use scientific notation for very large or very small numbers - let abs = v.abs(); - if abs != 0.0 && !(1e-6..1e15).contains(&abs) { - self.output.push_str(&format!("{:e}", v)); - } else { - self.output.push_str(&v.to_string()); - } - } + self.output.push_str(&format_f64(v)); Ok(()) } @@ -689,29 +741,11 @@ impl serde::Serializer for Json5ValueSerializer { } fn serialize_f32(self, v: f32) -> Result { - self.serialize_f64(v as f64) + Ok(Json5Value::Number(format_f32(v))) } fn serialize_f64(self, v: f64) -> Result { - let s = if v.is_nan() { - "NaN".to_string() - } else if v.is_infinite() { - if v.is_sign_positive() { - "Infinity".to_string() - } else { - "-Infinity".to_string() - } - } else { - // Use scientific notation for very large or very small numbers - // to ensure they can be parsed back by json5 - let abs = v.abs(); - if abs != 0.0 && !(1e-6..1e15).contains(&abs) { - format!("{:e}", v) - } else { - v.to_string() - } - }; - Ok(Json5Value::Number(s)) + Ok(Json5Value::Number(format_f64(v))) } fn serialize_char(self, v: char) -> Result { @@ -1010,6 +1044,78 @@ mod tests { use super::*; use serde::Deserialize; + #[test] + fn test_format_f32() { + // Trailing zeros trimmed, .0 removed + assert_eq!(format_f32(6.67), "6.67"); + assert_eq!(format_f32(6.0), "6"); + assert_eq!(format_f32(1.5), "1.5"); + assert_eq!(format_f32(100.0), "100"); + + // Negative values + assert_eq!(format_f32(-3.15), "-3.15"); + assert_eq!(format_f32(-2.5), "-2.5"); + + // Zero and negative zero + assert_eq!(format_f32(0.0), "0"); + assert_eq!(format_f32(-0.0), "-0"); + + // Rounding to 6 significant digits + assert_eq!(format_f32(0.5), "0.5"); + assert_eq!(format_f32(1.0 / 3.0), "0.333333"); + + // Small values preserved + assert_eq!(format_f32(1e-10).parse::().unwrap(), 1e-10_f32); + assert_eq!(format_f32(-1e-10).parse::().unwrap(), -1e-10_f32); + assert_eq!(format_f32(4.9e-7).parse::().unwrap(), 4.9e-7_f32); + + // Scientific notation for large values + assert_eq!(format_f32(1e20), "1e20"); + + // Special values + assert_eq!(format_f32(f32::NAN), "NaN"); + assert_eq!(format_f32(f32::INFINITY), "Infinity"); + assert_eq!(format_f32(f32::NEG_INFINITY), "-Infinity"); + } + + #[test] + fn test_format_f64() { + // Trailing zeros trimmed, .0 removed + assert_eq!(format_f64(6.67), "6.67"); + assert_eq!(format_f64(6.0), "6"); + assert_eq!(format_f64(1.5), "1.5"); + assert_eq!(format_f64(100.0), "100"); + + // Negative values + assert_eq!(format_f64(-3.15), "-3.15"); + assert_eq!(format_f64(-2.5), "-2.5"); + + // Zero and negative zero + assert_eq!(format_f64(0.0), "0"); + assert_eq!(format_f64(-0.0), "-0"); + + // Full precision for roundtrip fidelity + assert_eq!(format_f64(0.5), "0.5"); + assert_eq!(format_f64(1.0 / 3.0), "0.3333333333333333"); + + // Small values preserved + assert_eq!(format_f64(1e-100).parse::().unwrap(), 1e-100_f64); + assert_eq!(format_f64(-1e-100).parse::().unwrap(), -1e-100_f64); + assert_eq!(format_f64(4.9e-16).parse::().unwrap(), 4.9e-16_f64); + + // Scientific notation for large values + assert_eq!(format_f64(1e47), "1e47"); + + // Extreme values use exact formatting to avoid roundtrip overflow + assert_eq!(format_f64(f64::MAX), "1.7976931348623157e308"); + assert_eq!(format_f64(f64::MIN), "-1.7976931348623157e308"); + + // Special values + assert_eq!(format_f64(f64::NAN), "NaN"); + assert_eq!(format_f64(f64::INFINITY), "Infinity"); + assert_eq!(format_f64(f64::NEG_INFINITY), "-Infinity"); + } + #[test] fn test_parse_value() { let value = parse_value(r#"{"foo": "bar"}"#).unwrap(); @@ -1648,13 +1754,6 @@ mod tests { let serialized = to_vec_pretty_sorted(&original).unwrap(); let json_str = String::from_utf8(serialized).unwrap(); - // Should use scientific notation for very small numbers - assert!( - json_str.contains("e-"), - "Very small number should be in scientific notation: {}", - json_str - ); - let deserialized: Data = from_str(&json_str).unwrap(); assert_eq!(deserialized.value, original.value); } @@ -1733,7 +1832,7 @@ mod tests { let serialized = to_vec_pretty_sorted(&original).unwrap(); let json_str = String::from_utf8(serialized).unwrap(); - // Below 1e-6 should use scientific notation + // Values below 0.5e-6 are clamped to zero let deserialized: Data = from_str(&json_str).unwrap(); assert_eq!(deserialized.above, original.above); assert_eq!(deserialized.at, original.at); @@ -2630,8 +2729,8 @@ mod tests { let json_str = String::from_utf8(serialized).unwrap(); let deserialized: Data = from_str(&json_str).unwrap(); - assert_eq!(deserialized.epsilon, f64::EPSILON); - assert_eq!(deserialized.min_positive, f64::MIN_POSITIVE); + assert_eq!(deserialized.epsilon, original.epsilon); + assert_eq!(deserialized.min_positive, original.min_positive); } } @@ -4791,8 +4890,8 @@ No \\n's!", assert!(deserialized.c.is_nan()); assert_eq!(deserialized.d, f64::MAX); assert_eq!(deserialized.e, f64::MIN); - assert_eq!(deserialized.f, f64::MIN_POSITIVE); - assert_eq!(deserialized.g, f64::EPSILON); + assert_eq!(deserialized.f, original.f); + assert_eq!(deserialized.g, original.g); assert_eq!(deserialized.h, 0.0); assert_eq!(deserialized.i, original.i); } diff --git a/src/lib.rs b/src/lib.rs index b626f337..ed8541cf 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -8,6 +8,7 @@ pub mod cli; mod tree_view; mod change_processor; +pub mod git; mod glob; mod json; mod lua_ast; @@ -19,10 +20,10 @@ mod resolution; mod rojo_ref; mod serve_session; mod session_id; -mod snapshot; +pub mod snapshot; mod snapshot_middleware; -mod syncback; -mod variant_eq; +pub mod syncback; +pub mod variant_eq; mod web; // TODO: Work out what we should expose publicly diff --git a/src/resolution.rs b/src/resolution.rs index d1a92e26..56dcb517 100644 --- a/src/resolution.rs +++ b/src/resolution.rs @@ -2,8 +2,10 @@ use std::borrow::Borrow; use anyhow::{bail, format_err}; use rbx_dom_weak::types::{ - Attributes, CFrame, Color3, Content, ContentId, ContentType, Enum, Font, MaterialColors, - Matrix3, Tags, Variant, VariantType, Vector2, Vector3, + Attributes, CFrame, Color3, ColorSequence, ColorSequenceKeypoint, Content, ContentId, + ContentType, CustomPhysicalProperties, Enum, Font, MaterialColors, Matrix3, NumberSequence, + NumberSequenceKeypoint, PhysicalProperties, Ray, Rect, Region3, Tags, UDim, UDim2, Variant, + VariantType, Vector2, Vector3, }; use rbx_reflection::{DataType, PropertyDescriptor}; use serde::{Deserialize, Serialize}; @@ -64,7 +66,7 @@ impl UnresolvedValue { return Self::FullyQualified(variant); } Variant::Bool(bool) => AmbiguousValue::Bool(bool), - Variant::Float32(n) => AmbiguousValue::Number(n as f64), + Variant::Float32(n) => AmbiguousValue::Number32(n), Variant::Float64(n) => AmbiguousValue::Number(n), Variant::Int32(n) => AmbiguousValue::Number(n as f64), Variant::Int64(n) => AmbiguousValue::Number(n as f64), @@ -79,28 +81,149 @@ impl UnresolvedValue { }, Variant::ContentId(content) => AmbiguousValue::String(content.into_string()), Variant::Vector2(vector) => { - AmbiguousValue::Array2([vector.x as f64, vector.y as f64]) + AmbiguousValue::Array2([cleanup_f32(vector.x), cleanup_f32(vector.y)]) } - Variant::Vector3(vector) => { - AmbiguousValue::Array3([vector.x as f64, vector.y as f64, vector.z as f64]) - } - Variant::Color3(color) => { - AmbiguousValue::Array3([color.r as f64, color.g as f64, color.b as f64]) - } - Variant::CFrame(cf) => AmbiguousValue::Array12([ - cf.position.x as f64, - cf.position.y as f64, - cf.position.z as f64, - cf.orientation.x.x as f64, - cf.orientation.x.y as f64, - cf.orientation.x.z as f64, - cf.orientation.y.x as f64, - cf.orientation.y.y as f64, - cf.orientation.y.z as f64, - cf.orientation.z.x as f64, - cf.orientation.z.y as f64, - cf.orientation.z.z as f64, + Variant::Vector3(vector) => AmbiguousValue::Array3([ + cleanup_f32(vector.x), + cleanup_f32(vector.y), + cleanup_f32(vector.z), ]), + Variant::Color3(color) => AmbiguousValue::Array3([ + cleanup_f32(color.r), + cleanup_f32(color.g), + cleanup_f32(color.b), + ]), + Variant::CFrame(cf) => AmbiguousValue::Array12([ + cleanup_f32(cf.position.x), + cleanup_f32(cf.position.y), + cleanup_f32(cf.position.z), + cleanup_f32(cf.orientation.x.x), + cleanup_f32(cf.orientation.x.y), + cleanup_f32(cf.orientation.x.z), + cleanup_f32(cf.orientation.y.x), + cleanup_f32(cf.orientation.y.y), + cleanup_f32(cf.orientation.y.z), + cleanup_f32(cf.orientation.z.x), + cleanup_f32(cf.orientation.z.y), + cleanup_f32(cf.orientation.z.z), + ]), + Variant::UDim(udim) => { + return Self::FullyQualified(Variant::UDim(UDim::new( + cleanup_f32(udim.scale), + udim.offset, + ))); + } + Variant::UDim2(udim2) => { + return Self::FullyQualified(Variant::UDim2(UDim2::new( + UDim::new(cleanup_f32(udim2.x.scale), udim2.x.offset), + UDim::new(cleanup_f32(udim2.y.scale), udim2.y.offset), + ))); + } + Variant::OptionalCFrame(Some(cf)) => { + return Self::FullyQualified(Variant::OptionalCFrame(Some(CFrame::new( + Vector3::new( + cleanup_f32(cf.position.x), + cleanup_f32(cf.position.y), + cleanup_f32(cf.position.z), + ), + Matrix3::new( + Vector3::new( + cleanup_f32(cf.orientation.x.x), + cleanup_f32(cf.orientation.x.y), + cleanup_f32(cf.orientation.x.z), + ), + Vector3::new( + cleanup_f32(cf.orientation.y.x), + cleanup_f32(cf.orientation.y.y), + cleanup_f32(cf.orientation.y.z), + ), + Vector3::new( + cleanup_f32(cf.orientation.z.x), + cleanup_f32(cf.orientation.z.y), + cleanup_f32(cf.orientation.z.z), + ), + ), + )))); + } + Variant::Rect(rect) => { + return Self::FullyQualified(Variant::Rect(Rect::new( + Vector2::new(cleanup_f32(rect.min.x), cleanup_f32(rect.min.y)), + Vector2::new(cleanup_f32(rect.max.x), cleanup_f32(rect.max.y)), + ))); + } + Variant::Ray(ray) => { + return Self::FullyQualified(Variant::Ray(Ray::new( + Vector3::new( + cleanup_f32(ray.origin.x), + cleanup_f32(ray.origin.y), + cleanup_f32(ray.origin.z), + ), + Vector3::new( + cleanup_f32(ray.direction.x), + cleanup_f32(ray.direction.y), + cleanup_f32(ray.direction.z), + ), + ))); + } + Variant::Region3(region) => { + return Self::FullyQualified(Variant::Region3(Region3::new( + Vector3::new( + cleanup_f32(region.min.x), + cleanup_f32(region.min.y), + cleanup_f32(region.min.z), + ), + Vector3::new( + cleanup_f32(region.max.x), + cleanup_f32(region.max.y), + cleanup_f32(region.max.z), + ), + ))); + } + Variant::NumberSequence(seq) => { + return Self::FullyQualified(Variant::NumberSequence(NumberSequence { + keypoints: seq + .keypoints + .into_iter() + .map(|kp| { + NumberSequenceKeypoint::new( + cleanup_f32(kp.time), + cleanup_f32(kp.value), + cleanup_f32(kp.envelope), + ) + }) + .collect(), + })); + } + Variant::ColorSequence(seq) => { + return Self::FullyQualified(Variant::ColorSequence(ColorSequence { + keypoints: seq + .keypoints + .into_iter() + .map(|kp| { + ColorSequenceKeypoint::new( + cleanup_f32(kp.time), + Color3::new( + cleanup_f32(kp.color.r), + cleanup_f32(kp.color.g), + cleanup_f32(kp.color.b), + ), + ) + }) + .collect(), + })); + } + Variant::PhysicalProperties(PhysicalProperties::Custom(cpp)) => { + return Self::FullyQualified(Variant::PhysicalProperties( + PhysicalProperties::Custom(CustomPhysicalProperties::new( + cleanup_f32(cpp.density()), + cleanup_f32(cpp.friction()), + cleanup_f32(cpp.elasticity()), + cleanup_f32(cpp.friction_weight()), + cleanup_f32(cpp.elasticity_weight()), + cleanup_f32(cpp.acoustic_absorption()), + )), + )); + } Variant::Attributes(attr) => AmbiguousValue::Attributes(attr), Variant::Font(font) => AmbiguousValue::Font(font), Variant::MaterialColors(colors) => AmbiguousValue::MaterialColors(colors), @@ -138,10 +261,11 @@ pub enum AmbiguousValue { String(String), StringArray(Vec), Number(f64), - Array2([f64; 2]), - Array3([f64; 3]), - Array4([f64; 4]), - Array12([f64; 12]), + Number32(f32), + Array2([f32; 2]), + Array3([f32; 3]), + Array4([f32; 4]), + Array12([f32; 12]), Attributes(Attributes), Font(Font), MaterialColors(MaterialColors), @@ -196,6 +320,7 @@ impl AmbiguousValue { DataType::Value(variant_ty) => match (variant_ty, self) { (VariantType::Bool, AmbiguousValue::Bool(value)) => Ok(value.into()), + (VariantType::Float32, AmbiguousValue::Number32(value)) => Ok(value.into()), (VariantType::Float32, AmbiguousValue::Number(value)) => Ok((value as f32).into()), (VariantType::Float64, AmbiguousValue::Number(value)) => Ok(value.into()), (VariantType::Int32, AmbiguousValue::Number(value)) => Ok((value as i32).into()), @@ -213,19 +338,19 @@ impl AmbiguousValue { } (VariantType::Vector2, AmbiguousValue::Array2(value)) => { - Ok(Vector2::new(value[0] as f32, value[1] as f32).into()) + Ok(Vector2::new(value[0], value[1]).into()) } (VariantType::Vector3, AmbiguousValue::Array3(value)) => { - Ok(Vector3::new(value[0] as f32, value[1] as f32, value[2] as f32).into()) + Ok(Vector3::new(value[0], value[1], value[2]).into()) } (VariantType::Color3, AmbiguousValue::Array3(value)) => { - Ok(Color3::new(value[0] as f32, value[1] as f32, value[2] as f32).into()) + Ok(Color3::new(value[0], value[1], value[2]).into()) } (VariantType::CFrame, AmbiguousValue::Array12(value)) => { - let value = value.map(|v| v as f32); + let value = value.map(|v| v); let pos = Vector3::new(value[0], value[1], value[2]); let orientation = Matrix3::new( Vector3::new(value[3], value[4], value[5]), @@ -280,6 +405,7 @@ impl AmbiguousValue { AmbiguousValue::String(_) => "a string", AmbiguousValue::StringArray(_) => "an array of strings", AmbiguousValue::Number(_) => "a number", + AmbiguousValue::Number32(_) => "a number", AmbiguousValue::Array2(_) => "an array of two numbers", AmbiguousValue::Array3(_) => "an array of three numbers", AmbiguousValue::Array4(_) => "an array of four numbers", @@ -291,6 +417,15 @@ impl AmbiguousValue { } } +pub(crate) fn cleanup_f32(v: f32) -> f32 { + let cleaned = format!("{:.6}", v).parse::().unwrap_or(v); + if cleaned == 0.0 { + 0.0 + } else { + cleaned + } +} + fn find_descriptor( class_name: &str, prop_name: &str, @@ -415,6 +550,12 @@ mod test { assert_eq!(resolve("Part", "Transparency", "1"), Variant::Float32(1.0)); assert_eq!(resolve("NumberValue", "Value", "1"), Variant::Float64(1.0)); + assert_eq!( + UnresolvedValue::from_variant(Variant::Float32(0.125), "Part", "Transparency") + .resolve("Part", "Transparency") + .unwrap(), + Variant::Float32(0.125), + ); assert_eq!(resolve_unambiguous("12.5"), Variant::Float64(12.5)); } diff --git a/src/rojo_ref.rs b/src/rojo_ref.rs index 96bea075..dc6ac271 100644 --- a/src/rojo_ref.rs +++ b/src/rojo_ref.rs @@ -16,8 +16,9 @@ use serde::{Deserialize, Serialize}; pub const REF_ID_ATTRIBUTE_NAME: &str = "Rojo_Id"; /// Legacy: Pointer attribute prefix using IDs (kept for backwards compatibility) pub const REF_POINTER_ATTRIBUTE_PREFIX: &str = "Rojo_Target_"; -/// Prefix for path-based reference attributes. This is the preferred system -/// that stores the path to the target instance (e.g., "SoundService/Effects"). +/// Prefix for path-based reference attributes using Luau require-by-string +/// style paths (e.g., `@self/Handle.model.json5`, `./Sibling.luau`, +/// `../Uncle/Part.model.json5`, `@game/ReplicatedStorage/Module.luau`). /// No modification to the target instance is needed. pub const REF_PATH_ATTRIBUTE_PREFIX: &str = "Rojo_Ref_"; @@ -35,9 +36,9 @@ pub fn ref_target_attribute_name(prop_name: &str) -> String { format!("{REF_POINTER_ATTRIBUTE_PREFIX}{prop_name}") } -/// Escape "/" in an instance name for use in ref paths. -/// Uses "\/" as escape sequence. Also escapes existing backslashes to "\\" -/// so the escaping is unambiguous. +/// Escape "/" in an instance name for use in legacy instance-name paths. +/// Only used by `ref_target_path` (for debug/log output via `inst_path`). +/// `Rojo_Ref_*` attributes use filesystem names which can't contain `/`. pub fn escape_ref_path_segment(name: &str) -> Cow<'_, str> { if name.contains('/') || name.contains('\\') { Cow::Owned(name.replace('\\', "\\\\").replace('/', "\\/")) @@ -107,12 +108,12 @@ pub fn split_ref_path(path: &str) -> Vec { segments } -/// Compute the path to a target instance for use in a `Rojo_Ref_*` attribute. -/// Returns the slash-separated path from the root (root name excluded). -/// Instance names containing "/" are escaped as "\/" to prevent ambiguity. +/// Compute a human-readable path to an instance using raw instance names. +/// Used for debug/log output via `inst_path()`. NOT used for `Rojo_Ref_*` +/// attributes (those use `compute_relative_ref_path` with filesystem names). /// -/// Example: an instance at `DataModel > Workspace > Part1` returns `"Workspace/Part1"`. -/// An instance named `"A/B"` at Workspace returns `"Workspace/A\/B"`. +/// Instance names containing "/" are escaped as "\/" to prevent ambiguity. +/// Example: `DataModel > Workspace > Part1` returns `"Workspace/Part1"`. pub fn ref_target_path(dom: &WeakDom, target_ref: Ref) -> String { let root_ref = dom.root_ref(); let mut components: Vec> = dom @@ -124,6 +125,141 @@ pub fn ref_target_path(dom: &WeakDom, target_ref: Ref) -> String { components.join("/") } +/// Compute the **absolute** filesystem-name-based path to an instance. +/// +/// Returns a slash-separated path from DataModel's children downward. +/// Used as input to `compute_relative_ref_path()` and for RefPathIndex +/// indexing. Not written to disk directly -- `compute_relative_ref_path` +/// converts absolute paths to the Luau-style relative format. +/// +/// Each path segment is the **full filesystem name** of the instance: +/// - `Folder "Foo"` → `"Foo"` (directory name) +/// - `ModuleScript "Foo"` → `"Foo.luau"` (slug + extension) +/// - `Script "Foo"` (server) → `"Foo.server.luau"` +/// - Dedup'd `ModuleScript "Foo"` → `"Foo~2.luau"` +pub fn ref_target_path_from_tree(tree: &crate::snapshot::RojoTree, target_ref: Ref) -> String { + let dom = tree.inner(); + let root_ref = dom.root_ref(); + + let mut components: Vec = Vec::new(); + let mut current = target_ref; + + loop { + if current == root_ref || current.is_none() { + break; + } + + let inst = match dom.get_by_ref(current) { + Some(i) => i, + None => break, + }; + + components.push(tree.filesystem_name_for(current)); + current = inst.parent(); + } + + components.reverse(); + components.join("/") +} + +/// Compute a Luau require-by-string style path from a source instance to a +/// target instance. Both inputs are **absolute** ref paths (slash-separated +/// filesystem names from DataModel's children downward). +/// +/// Returns a prefixed path following Roblox's require-by-string semantics: +/// - `@self` when target == source +/// - `@self/...` when target is a descendant of source +/// - `./...` when source and target share the same parent (1 level up) +/// - `../...` chains for navigation within the same service (2+ levels up) +/// - `@game/...` for cross-service refs or when target is an ancestor of source +/// +/// Reference: +pub fn compute_relative_ref_path(source_abs: &str, target_abs: &str) -> String { + if source_abs == target_abs { + return "@self".to_string(); + } + + let source_prefix = format!("{source_abs}/"); + if target_abs.starts_with(&source_prefix) { + return format!("@self/{}", &target_abs[source_prefix.len()..]); + } + + let source_parts: Vec<&str> = source_abs.split('/').collect(); + let target_parts: Vec<&str> = target_abs.split('/').collect(); + + let common_len = source_parts + .iter() + .zip(target_parts.iter()) + .take_while(|(a, b)| a == b) + .count(); + + if common_len == 0 { + return format!("@game/{target_abs}"); + } + + let ups = source_parts.len() - common_len; + let remaining_parts = &target_parts[common_len..]; + + if remaining_parts.is_empty() { + return format!("@game/{target_abs}"); + } + + let remaining = remaining_parts.join("/"); + + if ups == 1 { + format!("./{remaining}") + } else { + let mut result = String::with_capacity(3 * (ups - 1) + remaining.len()); + for _ in 0..(ups - 1) { + result.push_str("../"); + } + result.push_str(&remaining); + result + } +} + +/// Resolve a relative (Luau-style) ref path back to an absolute ref path +/// using only string manipulation. `source_abs` is the absolute ref path of +/// the instance that owns the attribute. +/// +/// Returns `None` if the path navigates above the root. +pub fn resolve_ref_path_to_absolute(path: &str, source_abs: &str) -> Option { + if let Some(rest) = path.strip_prefix("@game/") { + return Some(rest.to_string()); + } + if path == "@game" { + return Some(String::new()); + } + if path == "@self" { + return Some(source_abs.to_string()); + } + + let (mut parts, rest) = if let Some(rest) = path.strip_prefix("@self/") { + (source_abs.split('/').collect::>(), rest) + } else if let Some(rest) = path.strip_prefix("./") { + let mut p: Vec<&str> = source_abs.split('/').collect(); + p.pop(); + (p, rest) + } else if let Some(rest) = path.strip_prefix("../") { + let mut p: Vec<&str> = source_abs.split('/').collect(); + p.pop(); + p.pop()?; + (p, rest) + } else { + return Some(path.to_string()); + }; + + for segment in rest.split('/') { + if segment == ".." { + parts.pop()?; + } else if !segment.is_empty() { + parts.push(segment); + } + } + + Some(parts.join("/")) +} + /// Extract a string value from a Variant that may be String or BinaryString. /// Returns None with a warning for non-string types or invalid UTF-8. /// Used by compute_ref_properties and defer_ref_properties for parsing @@ -227,6 +363,78 @@ impl RefPathIndex { } } + /// Scan all `.meta.json5`, `.model.json5`, `.meta.json`, `.model.json` + /// files under `root` for existing `Rojo_Ref_*` attributes and index them. + /// + /// Relative paths (prefixed with `@self/`, `./`, `../`) are resolved to + /// absolute using `tree` so the index always stores absolute target paths. + /// This ensures prefix-based lookup works correctly for rename updates. + pub fn populate_from_dir(&mut self, root: &Path, tree: &crate::snapshot::RojoTree) { + use rayon::prelude::*; + use walkdir::WalkDir; + + fn is_meta_or_model(name: &str) -> bool { + name.ends_with(".meta.json5") + || name.ends_with(".model.json5") + || name.ends_with(".meta.json") + || name.ends_with(".model.json") + } + + let meta_paths: Vec = WalkDir::new(root) + .follow_links(true) + .into_iter() + .filter_map(|e: Result| e.ok()) + .filter(|e: &walkdir::DirEntry| { + e.file_type().is_file() && e.file_name().to_str().is_some_and(is_meta_or_model) + }) + .map(|e: walkdir::DirEntry| e.into_path()) + .collect(); + + let entries: Vec<(String, std::path::PathBuf)> = meta_paths + .par_iter() + .flat_map(|path| { + let source_abs = tree + .get_ids_at_path(path) + .first() + .map(|&id| crate::ref_target_path_from_tree(tree, id)) + .unwrap_or_default(); + + let mut results = Vec::new(); + if let Ok(bytes) = std::fs::read(path) { + if let Ok(val) = crate::json::from_slice::(&bytes) { + if let Some(attrs) = val.get("attributes").and_then(|a| a.as_object()) { + for (key, value) in attrs { + if key.starts_with(crate::REF_PATH_ATTRIBUTE_PREFIX) { + if let Some(path_str) = value.as_str() { + let resolved = crate::resolve_ref_path_to_absolute( + path_str, + &source_abs, + ) + .unwrap_or_else(|| path_str.to_string()); + results.push((resolved, path.clone())); + } + } + } + } + } + } + results + }) + .collect(); + + let count = entries.len(); + for (resolved, path) in entries { + self.add(&resolved, &path); + } + + if count > 0 { + log::debug!( + "RefPathIndex: populated {} Rojo_Ref_* entries from existing meta/model files", + count + ); + } + } + /// Find all meta/model files that contain a `Rojo_Ref_*` attribute whose /// value equals `prefix` or starts with `prefix/`. These are the files /// that need updating when an instance at `prefix` is renamed. @@ -587,4 +795,653 @@ mod tests { // Should be deduplicated to just one entry assert_eq!(results.len(), 1); } + + // ----------------------------------------------------------------------- + // compute_relative_ref_path tests -- @self + // ----------------------------------------------------------------------- + + #[test] + fn relative_self_reference() { + assert_eq!( + compute_relative_ref_path("Workspace/Model", "Workspace/Model"), + "@self" + ); + } + + #[test] + fn relative_self_direct_child() { + assert_eq!( + compute_relative_ref_path("Workspace/Model", "Workspace/Model/Handle.model.json5"), + "@self/Handle.model.json5" + ); + } + + #[test] + fn relative_self_nested_descendant() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Model", + "Workspace/Model/SubFolder/Part.model.json5" + ), + "@self/SubFolder/Part.model.json5" + ); + } + + #[test] + fn relative_self_deeply_nested() { + assert_eq!( + compute_relative_ref_path("Workspace/Model", "Workspace/Model/A/B/C/D.luau"), + "@self/A/B/C/D.luau" + ); + } + + #[test] + fn relative_self_tool_handle() { + assert_eq!( + compute_relative_ref_path("Workspace/Tool", "Workspace/Tool/Handle.model.json5"), + "@self/Handle.model.json5" + ); + } + + #[test] + fn relative_self_init_style_child() { + assert_eq!( + compute_relative_ref_path("Workspace/Model", "Workspace/Model/Init"), + "@self/Init" + ); + } + + #[test] + fn relative_self_nested_sound() { + assert_eq!( + compute_relative_ref_path("Workspace/Gun", "Workspace/Gun/Sounds/Fire.model.json5"), + "@self/Sounds/Fire.model.json5" + ); + } + + #[test] + fn relative_self_slugified_child() { + assert_eq!( + compute_relative_ref_path("Workspace/Model", "Workspace/Model/Hey_Bro.server.luau"), + "@self/Hey_Bro.server.luau" + ); + } + + #[test] + fn relative_self_dedup_child() { + assert_eq!( + compute_relative_ref_path("Workspace/Model", "Workspace/Model/Data~2"), + "@self/Data~2" + ); + } + + // ----------------------------------------------------------------------- + // compute_relative_ref_path tests -- ./ + // ----------------------------------------------------------------------- + + #[test] + fn relative_dot_sibling_script() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Folder/ScriptA.server.luau", + "Workspace/Folder/ScriptB.server.luau" + ), + "./ScriptB.server.luau" + ); + } + + #[test] + fn relative_dot_sibling_module() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Folder/ScriptA.server.luau", + "Workspace/Folder/Config.luau" + ), + "./Config.luau" + ); + } + + #[test] + fn relative_dot_sibling_directory() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Folder/ScriptA.server.luau", + "Workspace/Folder/SubFolder" + ), + "./SubFolder" + ); + } + + #[test] + fn relative_dot_sibling_then_descend() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Folder/ScriptA.server.luau", + "Workspace/Folder/SubFolder/Deep.luau" + ), + "./SubFolder/Deep.luau" + ); + } + + #[test] + fn relative_dot_object_value_sibling() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Folder/ObjectValue.model.json5", + "Workspace/Folder/Target.model.json5" + ), + "./Target.model.json5" + ); + } + + #[test] + fn relative_dot_beam_attachment0() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Beams/Beam.model.json5", + "Workspace/Beams/Att1.model.json5" + ), + "./Att1.model.json5" + ); + } + + #[test] + fn relative_dot_beam_attachment1() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Beams/Beam.model.json5", + "Workspace/Beams/Att2.model.json5" + ), + "./Att2.model.json5" + ); + } + + #[test] + fn relative_dot_slugified_sibling() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Folder/A.luau", + "Workspace/Folder/Hey_Bro.server.luau" + ), + "./Hey_Bro.server.luau" + ); + } + + #[test] + fn relative_dot_dedup_sibling_folder() { + assert_eq!( + compute_relative_ref_path("Workspace/Folder/A.luau", "Workspace/Folder/Data~2"), + "./Data~2" + ); + } + + #[test] + fn relative_dot_dedup_sibling_model() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Folder/A.luau", + "Workspace/Folder/Data~3.model.json5" + ), + "./Data~3.model.json5" + ); + } + + // ----------------------------------------------------------------------- + // compute_relative_ref_path tests -- ../ + // ----------------------------------------------------------------------- + + #[test] + fn relative_dotdot_2_ups() { + assert_eq!( + compute_relative_ref_path( + "Workspace/A/Script.server.luau", + "Workspace/B/Part.model.json5" + ), + "../B/Part.model.json5" + ); + } + + #[test] + fn relative_dotdot_2_ups_directory() { + assert_eq!( + compute_relative_ref_path("Workspace/A/Script.server.luau", "Workspace/B"), + "../B" + ); + } + + #[test] + fn relative_dotdot_3_ups() { + assert_eq!( + compute_relative_ref_path( + "Workspace/A/B/Script.server.luau", + "Workspace/C/Part.model.json5" + ), + "../../C/Part.model.json5" + ); + } + + #[test] + fn relative_dotdot_4_ups() { + assert_eq!( + compute_relative_ref_path( + "Workspace/A/B/C/Script.luau", + "Workspace/D/E/Part.model.json5" + ), + "../../../D/E/Part.model.json5" + ); + } + + #[test] + fn relative_dotdot_5_ups() { + assert_eq!( + compute_relative_ref_path( + "Workspace/A/B/C/D/Script.luau", + "Workspace/E/Part.model.json5" + ), + "../../../../E/Part.model.json5" + ); + } + + #[test] + fn relative_dotdot_peer_model() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Models/Car/Body.model.json5", + "Workspace/Models/Truck/Body.model.json5" + ), + "../Truck/Body.model.json5" + ); + } + + #[test] + fn relative_dotdot_cousin() { + assert_eq!( + compute_relative_ref_path("Workspace/A/Deep/Script.luau", "Workspace/A/Other.luau"), + "../Other.luau" + ); + } + + #[test] + fn relative_dotdot_beam_uncle_folder() { + assert_eq!( + compute_relative_ref_path( + "Workspace/A/B/Beam.model.json5", + "Workspace/A/C/Att.model.json5" + ), + "../C/Att.model.json5" + ); + } + + #[test] + fn relative_dotdot_cross_system() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Systems/Combat/Hitbox.server.luau", + "Workspace/Systems/Audio/HitSound.model.json5" + ), + "../Audio/HitSound.model.json5" + ); + } + + #[test] + fn relative_dotdot_same_zone() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Map/Zone1/Spawns/Spawn1.model.json5", + "Workspace/Map/Zone1/Props/Tree.model.json5" + ), + "../Props/Tree.model.json5" + ); + } + + #[test] + fn relative_dotdot_cross_zone() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Map/Zone1/Spawns/Spawn1.model.json5", + "Workspace/Map/Zone2/Flag.model.json5" + ), + "../../Zone2/Flag.model.json5" + ); + } + + // ----------------------------------------------------------------------- + // compute_relative_ref_path tests -- @game/ + // ----------------------------------------------------------------------- + + #[test] + fn relative_game_cross_service_classic() { + assert_eq!( + compute_relative_ref_path( + "Workspace/Script.server.luau", + "ReplicatedStorage/Module.luau" + ), + "@game/ReplicatedStorage/Module.luau" + ); + } + + #[test] + fn relative_game_to_server_storage() { + assert_eq!( + compute_relative_ref_path("Workspace/Script.server.luau", "ServerStorage/Data.luau"), + "@game/ServerStorage/Data.luau" + ); + } + + #[test] + fn relative_game_sss_to_rs() { + assert_eq!( + compute_relative_ref_path( + "ServerScriptService/Main.server.luau", + "ReplicatedStorage/Shared/Utils.luau" + ), + "@game/ReplicatedStorage/Shared/Utils.luau" + ); + } + + #[test] + fn relative_game_client_to_rs() { + assert_eq!( + compute_relative_ref_path( + "StarterGui/ScreenGui/Button.client.luau", + "ReplicatedStorage/UI/Theme.luau" + ), + "@game/ReplicatedStorage/UI/Theme.luau" + ); + } + + #[test] + fn relative_game_to_service_itself() { + assert_eq!( + compute_relative_ref_path("Workspace/A/B/C/Script.luau", "Lighting"), + "@game/Lighting" + ); + } + + #[test] + fn relative_game_ancestor_own_service() { + assert_eq!( + compute_relative_ref_path("Workspace/A/B/C/Script.luau", "Workspace"), + "@game/Workspace" + ); + } + + #[test] + fn relative_game_ancestor_grandparent() { + assert_eq!( + compute_relative_ref_path("Workspace/A/B/Script.luau", "Workspace/A"), + "@game/Workspace/A" + ); + } + + #[test] + fn relative_game_ancestor_great_grandparent() { + assert_eq!( + compute_relative_ref_path("Workspace/A/B/C/Script.luau", "Workspace/A/B"), + "@game/Workspace/A/B" + ); + } + + #[test] + fn relative_game_ancestor_own_service_single_level() { + assert_eq!( + compute_relative_ref_path("ReplicatedStorage/A.luau", "ReplicatedStorage"), + "@game/ReplicatedStorage" + ); + } + + #[test] + fn relative_game_to_sound_service() { + assert_eq!( + compute_relative_ref_path("Workspace/Script.luau", "SoundService/BGM.model.json5"), + "@game/SoundService/BGM.model.json5" + ); + } + + // ----------------------------------------------------------------------- + // compute_relative_ref_path tests -- special names + // ----------------------------------------------------------------------- + + #[test] + fn relative_special_windows_reserved() { + assert_eq!( + compute_relative_ref_path("Workspace/Folder/A.luau", "Workspace/Other/CON_.luau"), + "../Other/CON_.luau" + ); + } + + #[test] + fn relative_special_into_dedup_folder() { + assert_eq!( + compute_relative_ref_path("Workspace/A.luau", "Workspace/Data~2/Child.luau"), + "./Data~2/Child.luau" + ); + } + + #[test] + fn relative_special_init_style_child() { + assert_eq!( + compute_relative_ref_path("Workspace/Model", "Workspace/Model/Scripts"), + "@self/Scripts" + ); + } + + #[test] + fn relative_special_init_style_both() { + assert_eq!( + compute_relative_ref_path("Workspace/Scripts", "Workspace/Scripts/Foo"), + "@self/Foo" + ); + } + + // ----------------------------------------------------------------------- + // compute_relative_ref_path tests -- tricky edge cases + // ----------------------------------------------------------------------- + + #[test] + fn relative_edge_service_own_child() { + assert_eq!( + compute_relative_ref_path("Workspace", "Workspace/Model.model.json5"), + "@self/Model.model.json5" + ); + } + + #[test] + fn relative_edge_service_to_service() { + assert_eq!( + compute_relative_ref_path("Workspace", "ReplicatedStorage/Module.luau"), + "@game/ReplicatedStorage/Module.luau" + ); + } + + #[test] + fn relative_edge_service_self_ref() { + assert_eq!(compute_relative_ref_path("Workspace", "Workspace"), "@self"); + } + + #[test] + fn relative_edge_service_to_different_service() { + assert_eq!( + compute_relative_ref_path("Workspace", "Lighting"), + "@game/Lighting" + ); + } + + #[test] + fn relative_edge_segment_prefix_ambiguity() { + assert_eq!( + compute_relative_ref_path("Workspace/A", "Workspace/AB"), + "./AB" + ); + } + + #[test] + fn relative_edge_same_stem_different_ext() { + assert_eq!( + compute_relative_ref_path("Workspace/Foo.server.luau", "Workspace/Foo.luau"), + "./Foo.luau" + ); + } + + #[test] + fn relative_edge_deep_close_cousins() { + assert_eq!( + compute_relative_ref_path( + "Workspace/A/B/C/D/E/F/Script.luau", + "Workspace/A/B/C/D/E/G/Target.luau" + ), + "../G/Target.luau" + ); + } + + #[test] + fn relative_edge_deep_far_apart() { + assert_eq!( + compute_relative_ref_path( + "Workspace/A/B/C/D/E/F/Script.luau", + "Workspace/X/Target.luau" + ), + "../../../../../../X/Target.luau" + ); + } + + // ----------------------------------------------------------------------- + // resolve_ref_path_to_absolute tests + // ----------------------------------------------------------------------- + + #[test] + fn resolve_abs_game_prefix() { + assert_eq!( + resolve_ref_path_to_absolute("@game/ReplicatedStorage/Module.luau", "Workspace/Script"), + Some("ReplicatedStorage/Module.luau".to_string()) + ); + } + + #[test] + fn resolve_abs_self_no_path() { + assert_eq!( + resolve_ref_path_to_absolute("@self", "Workspace/Model"), + Some("Workspace/Model".to_string()) + ); + } + + #[test] + fn resolve_abs_self_with_child() { + assert_eq!( + resolve_ref_path_to_absolute("@self/Handle.model.json5", "Workspace/Model"), + Some("Workspace/Model/Handle.model.json5".to_string()) + ); + } + + #[test] + fn resolve_abs_self_nested() { + assert_eq!( + resolve_ref_path_to_absolute("@self/A/B/C", "Workspace/Model"), + Some("Workspace/Model/A/B/C".to_string()) + ); + } + + #[test] + fn resolve_abs_dot_sibling() { + assert_eq!( + resolve_ref_path_to_absolute("./Sibling.luau", "Workspace/Folder/Script.luau"), + Some("Workspace/Folder/Sibling.luau".to_string()) + ); + } + + #[test] + fn resolve_abs_dot_sibling_descend() { + assert_eq!( + resolve_ref_path_to_absolute("./Sub/Deep.luau", "Workspace/Folder/Script.luau"), + Some("Workspace/Folder/Sub/Deep.luau".to_string()) + ); + } + + #[test] + fn resolve_abs_dotdot_2_ups() { + assert_eq!( + resolve_ref_path_to_absolute("../B/Part.model.json5", "Workspace/A/Script.luau"), + Some("Workspace/B/Part.model.json5".to_string()) + ); + } + + #[test] + fn resolve_abs_dotdot_3_ups() { + assert_eq!( + resolve_ref_path_to_absolute("../../C/Part.model.json5", "Workspace/A/B/Script.luau"), + Some("Workspace/C/Part.model.json5".to_string()) + ); + } + + #[test] + fn resolve_abs_dotdot_cross_service_via_root() { + assert_eq!( + resolve_ref_path_to_absolute( + "../../ReplicatedStorage/Module.luau", + "Workspace/A/Script.luau" + ), + Some("ReplicatedStorage/Module.luau".to_string()) + ); + } + + #[test] + fn resolve_abs_bare_path_legacy() { + assert_eq!( + resolve_ref_path_to_absolute("Workspace/Model/Handle.model.json5", "anything/here"), + Some("Workspace/Model/Handle.model.json5".to_string()) + ); + } + + #[test] + fn resolve_abs_dotdot_mid_path() { + assert_eq!( + resolve_ref_path_to_absolute("@self/Sub/../Handle.model.json5", "Workspace/Model"), + Some("Workspace/Model/Handle.model.json5".to_string()) + ); + } + + #[test] + fn resolve_abs_dotdot_above_root_returns_none() { + assert_eq!(resolve_ref_path_to_absolute("../../X", "Workspace"), None); + } + + #[test] + fn resolve_then_compute_round_trip() { + let source = "Workspace/A/B/Script.luau"; + let target = "Workspace/C/Part.model.json5"; + let relative = compute_relative_ref_path(source, target); + assert_eq!(relative, "../../C/Part.model.json5"); + let resolved = resolve_ref_path_to_absolute(&relative, source).unwrap(); + assert_eq!(resolved, target); + } + + #[test] + fn resolve_then_compute_round_trip_self() { + let source = "Workspace/Model"; + let target = "Workspace/Model/Handle.model.json5"; + let relative = compute_relative_ref_path(source, target); + assert_eq!(relative, "@self/Handle.model.json5"); + let resolved = resolve_ref_path_to_absolute(&relative, source).unwrap(); + assert_eq!(resolved, target); + } + + #[test] + fn resolve_then_compute_round_trip_game() { + let source = "Workspace/Script.luau"; + let target = "ReplicatedStorage/Module.luau"; + let relative = compute_relative_ref_path(source, target); + assert_eq!(relative, "@game/ReplicatedStorage/Module.luau"); + let resolved = resolve_ref_path_to_absolute(&relative, source).unwrap(); + assert_eq!(resolved, target); + } + + #[test] + fn resolve_then_compute_round_trip_dot() { + let source = "Workspace/Folder/A.luau"; + let target = "Workspace/Folder/B.luau"; + let relative = compute_relative_ref_path(source, target); + assert_eq!(relative, "./B.luau"); + let resolved = resolve_ref_path_to_absolute(&relative, source).unwrap(); + assert_eq!(resolved, target); + } } diff --git a/src/serve_session.rs b/src/serve_session.rs index 4fb2c551..198d2c94 100644 --- a/src/serve_session.rs +++ b/src/serve_session.rs @@ -1,14 +1,13 @@ use std::{ collections::HashSet, io, - net::IpAddr, - path::Path, + path::{Path, PathBuf}, sync::{Arc, Mutex, MutexGuard}, time::Instant, }; use crossbeam_channel::Sender; -use memofs::Vfs; +use memofs::{PrefetchCache, Vfs}; use thiserror::Error; use crate::{ @@ -111,7 +110,7 @@ pub struct ServeSession { /// Values are `(remove_count, create_write_count)` — each API write increments /// the appropriate counter, each suppressed VFS event decrements it. /// `None` for oneshot sessions. - #[allow(dead_code)] + #[allow(dead_code, clippy::type_complexity)] suppressed_paths: Option>>>, @@ -119,6 +118,155 @@ pub struct ServeSession { /// Shared between ApiService (writes) and ChangeProcessor (rename updates). /// `None` for oneshot sessions. ref_path_index: Option>>, + + /// Root of the git repository, if the project is inside one. + /// Computed once at session start for use by auto-staging. + git_repo_root: Option, +} + +/// Collect all filesystem paths reachable from the project tree's `$path` +/// entries, then read file contents and compute canonical paths in parallel. +fn prefetch_project_files(project: &Project) -> io::Result { + use rayon::prelude::*; + use std::collections::HashMap; + use walkdir::WalkDir; + + let folder = project.folder_location(); + + let mut roots: Vec = Vec::new(); + collect_path_roots(&project.tree, folder, &mut roots); + + roots.sort(); + let mut deduped: Vec = Vec::new(); + for root in roots { + if !deduped.iter().any(|existing| root.starts_with(existing)) { + deduped.push(root); + } + } + let roots = deduped; + + if roots.is_empty() { + return Ok(PrefetchCache { + files: HashMap::new(), + canonical: HashMap::new(), + is_file: HashMap::new(), + children: HashMap::new(), + }); + } + + let walk_start = Instant::now(); + + let mut entries: Vec = Vec::new(); + for root in &roots { + if !root.exists() { + continue; + } + for e in WalkDir::new(root).follow_links(true).into_iter().flatten() { + entries.push(e); + } + } + + // Also include the project file itself (read by project middleware). + if let Ok(meta) = std::fs::metadata(&project.file_location) { + if meta.is_file() { + entries.push( + WalkDir::new(&project.file_location) + .into_iter() + .next() + .unwrap() + .unwrap(), + ); + } + } + + let walk_elapsed = walk_start.elapsed(); + let file_count = entries.iter().filter(|e| e.file_type().is_file()).count(); + let dir_count = entries.len() - file_count; + + log::debug!( + "Prefetch walk: {} files + {} dirs from {} root(s) in {:.1?}", + file_count, + dir_count, + roots.len(), + walk_elapsed, + ); + + let read_start = Instant::now(); + + let file_data: Vec<_> = entries + .par_iter() + .filter(|e| e.file_type().is_file()) + .filter_map(|e| { + let path = e.path().to_path_buf(); + std::fs::read(&path).ok().map(|c| (path, c)) + }) + .collect(); + + let read_elapsed = read_start.elapsed(); + let canon_start = Instant::now(); + + let canonical_data: Vec<_> = entries + .par_iter() + .filter_map(|e| { + let path = e.path().to_path_buf(); + std::fs::canonicalize(&path).ok().map(|c| (path, c)) + }) + .collect(); + + let canon_elapsed = canon_start.elapsed(); + + log::debug!( + "Prefetch I/O: read {} files in {:.1?}, canonicalize {} paths in {:.1?}", + file_data.len(), + read_elapsed, + canonical_data.len(), + canon_elapsed, + ); + + let mut is_file_map: HashMap = HashMap::new(); + let mut children_map: HashMap> = HashMap::new(); + + for entry in &entries { + let path = entry.path().to_path_buf(); + is_file_map.insert(path.clone(), entry.file_type().is_file()); + + if entry.depth() > 0 { + if let Some(parent) = entry.path().parent() { + children_map + .entry(parent.to_path_buf()) + .or_default() + .push(path); + } + } + } + + for children in children_map.values_mut() { + children.sort_by(|a, b| a.file_name().cmp(&b.file_name())); + } + + log::debug!( + "Prefetch maps: {} is_file + {} children dirs built", + is_file_map.len(), + children_map.len(), + ); + + Ok(PrefetchCache { + files: file_data.into_iter().collect::>(), + canonical: canonical_data.into_iter().collect::>(), + is_file: is_file_map, + children: children_map, + }) +} + +/// Recursively collect all `$path` directories from the project tree. +fn collect_path_roots(node: &crate::project::ProjectNode, base: &Path, out: &mut Vec) { + if let Some(path_node) = &node.path { + let resolved = base.join(path_node.path()); + out.push(resolved); + } + for child_node in node.children.values() { + collect_path_roots(child_node, base, out); + } } impl ServeSession { @@ -129,18 +277,42 @@ impl ServeSession { let root_project = Project::load_initial_project(vfs, start_path)?; + if std::env::var("ATLAS_SEQUENTIAL").is_err() { + let prefetch_start = Instant::now(); + match prefetch_project_files(&root_project) { + Ok(cache) => { + log::info!( + "Prefetch total: {} files + {} canonical paths in {:.1?}", + cache.files.len(), + cache.canonical.len(), + prefetch_start.elapsed() + ); + vfs.set_prefetch_cache(cache); + } + Err(err) => { + log::warn!("Prefetch failed, falling back to sequential reads: {err}"); + } + } + } + let mut tree = RojoTree::new(InstanceSnapshot::new()); let root_id = tree.get_root_id(); let instance_context = InstanceContext::new(); + let snap_start = Instant::now(); log::trace!("Generating snapshot of instances from VFS"); let snapshot = snapshot_from_vfs(&instance_context, vfs, start_path)?; + log::debug!("Snapshot built in {:.1?}", snap_start.elapsed()); + vfs.clear_prefetch_cache(); + + let patch_start = Instant::now(); log::trace!("Computing initial patch set"); let patch_set = compute_patch_set(snapshot, &tree, root_id); log::trace!("Applying initial patch set"); apply_patch_set(&mut tree, patch_set); + log::debug!("Patch computed + applied in {:.1?}", patch_start.elapsed()); Ok((root_project, tree)) } @@ -170,7 +342,15 @@ impl ServeSession { let (tree_mutation_sender, tree_mutation_receiver) = crossbeam_channel::unbounded(); let suppressed_paths = Arc::new(Mutex::new(std::collections::HashMap::new())); - let ref_path_index = Arc::new(Mutex::new(crate::RefPathIndex::new())); + let ref_path_index = { + let mut index = crate::RefPathIndex::new(); + let tree_guard = tree.lock().unwrap(); + index.populate_from_dir(root_project.folder_location(), &tree_guard); + drop(tree_guard); + Arc::new(Mutex::new(index)) + }; + + let git_repo_root = crate::git::git_repo_root(root_project.folder_location()); log::trace!("Starting ChangeProcessor"); let change_processor = ChangeProcessor::start( @@ -181,7 +361,9 @@ impl ServeSession { Arc::clone(&suppressed_paths), Arc::clone(&ref_path_index), root_project.folder_location().to_path_buf(), + root_project.file_location.clone(), critical_error_receiver, + git_repo_root.clone(), ); Ok(Self { @@ -195,6 +377,7 @@ impl ServeSession { vfs, suppressed_paths: Some(suppressed_paths), ref_path_index: Some(ref_path_index), + git_repo_root, }) } @@ -220,6 +403,7 @@ impl ServeSession { vfs: Arc::new(vfs), suppressed_paths: None, ref_path_index: None, + git_repo_root: None, }) } @@ -280,10 +464,6 @@ impl ServeSession { .expect("all top-level projects must have their name set") } - pub fn project_port(&self) -> Option { - self.root_project.serve_port - } - pub fn place_id(&self) -> Option { self.root_project.place_id } @@ -304,14 +484,14 @@ impl ServeSession { self.root_project.blocked_place_ids.as_ref() } - pub fn serve_address(&self) -> Option { - self.root_project.serve_address - } - pub fn root_dir(&self) -> &Path { self.root_project.folder_location() } + pub fn git_repo_root(&self) -> Option<&Path> { + self.git_repo_root.as_deref() + } + pub fn root_project(&self) -> &Project { &self.root_project } @@ -347,7 +527,7 @@ impl ServeSession { /// Does NOT apply corrections — the tree is left unchanged. pub fn check_tree_freshness(&self) -> TreeFreshnessReport { let start = Instant::now(); - let start_path = self.root_project.folder_location(); + let start_path: &Path = &self.root_project.file_location; let instance_context = InstanceContext::new(); let snapshot = match snapshot_from_vfs(&instance_context, &self.vfs, start_path) { @@ -410,7 +590,7 @@ impl ServeSession { } let start = Instant::now(); - let start_path = self.root_project.folder_location(); + let start_path: &Path = &self.root_project.file_location; let instance_context = InstanceContext::new(); let snapshot = match snapshot_from_vfs(&instance_context, &self.vfs, start_path) { diff --git a/src/snapshot/matching.rs b/src/snapshot/matching.rs new file mode 100644 index 00000000..4d81b232 --- /dev/null +++ b/src/snapshot/matching.rs @@ -0,0 +1,1849 @@ +//! Instance matching algorithm for forward sync. +//! +//! Pairs snapshot children (from filesystem re-snapshot) to tree children +//! (existing RojoTree instances) by minimizing total reconciler changes. +//! +//! Algorithm per parent: +//! 1. Group by (Name, ClassName) -- 1:1 groups instant-match +//! 2. Ambiguous groups: recursive change-count scoring + greedy assignment +//! +//! The change count = how many things the reconciler would need to touch +//! to turn instance A into instance B, including the entire subtree. + +use std::cell::RefCell; +use std::collections::HashMap; + +use rbx_dom_weak::types::{Ref, Variant}; +use rbx_dom_weak::Ustr; + +use crate::variant_eq::variant_eq_disk; + +use super::{InstanceSnapshot, InstanceWithMeta, RojoTree}; + +const UNMATCHED_PENALTY: u32 = 10_000; + +/// Maximum recursion depth for `compute_change_count`. Beyond this depth, +/// only flat property comparison is used (no subtree recursion). Prevents +/// O(n^k) explosion on deeply nested trees with many same-named instances. +const MAX_SCORING_DEPTH: u32 = 3; + +/// Session-scoped cache for the matching algorithm. Caches +/// `compute_change_count` results so that recursive scoring work is +/// reused across calls at different tree levels. +pub struct MatchingSession { + cost_cache: RefCell>, +} + +impl Default for MatchingSession { + fn default() -> Self { + Self::new() + } +} + +impl MatchingSession { + pub fn new() -> Self { + Self { + cost_cache: RefCell::new(HashMap::new()), + } + } +} + +/// Result of the forward sync matching algorithm. +pub struct ForwardMatchResult { + /// Pairs of (snapshot_child, tree_ref) that were matched. + pub matched: Vec<(InstanceSnapshot, Ref)>, + /// Snapshot children with no match in the tree (to be added). + pub unmatched_snapshot: Vec, + /// Tree children with no match in the snapshot (to be removed). + pub unmatched_tree: Vec, +} + +/// Match snapshot children to tree children, minimizing total changes. +pub fn match_forward( + snapshot_children: Vec, + tree_children: &[Ref], + tree: &RojoTree, + session: &MatchingSession, +) -> ForwardMatchResult { + if snapshot_children.is_empty() && tree_children.is_empty() { + return ForwardMatchResult { + matched: Vec::new(), + unmatched_snapshot: Vec::new(), + unmatched_tree: Vec::new(), + }; + } + + let snap_available: Vec> = + snapshot_children.into_iter().map(Some).collect(); + let snap_len = snap_available.len(); + let tree_len = tree_children.len(); + let mut snap_matched: Vec = vec![false; snap_len]; + let mut tree_available: Vec = vec![true; tree_len]; + let mut matched: Vec<(usize, usize)> = Vec::with_capacity(snap_len.min(tree_len)); + + // ================================================================ + // Fast-path: Group by (Name, ClassName) -- 1:1 groups instant-match + // ================================================================ + let mut snap_by_key: HashMap<(String, Ustr), Vec> = HashMap::with_capacity(snap_len); + for (i, snap_opt) in snap_available.iter().enumerate() { + if let Some(snap) = snap_opt { + snap_by_key + .entry((snap.name.to_string(), snap.class_name)) + .or_default() + .push(i); + } + } + + let mut tree_by_key: HashMap<(String, Ustr), Vec> = HashMap::with_capacity(tree_len); + for (i, &child_ref) in tree_children.iter().enumerate() { + if let Some(inst) = tree.get_instance(child_ref) { + tree_by_key + .entry((inst.name().to_string(), inst.class_name())) + .or_default() + .push(i); + } + } + + // 1:1 groups: instant match + for (key, snap_indices) in &snap_by_key { + if let Some(tree_indices) = tree_by_key.get(key) { + if snap_indices.len() == 1 && tree_indices.len() == 1 { + let si = snap_indices[0]; + let ti = tree_indices[0]; + matched.push((si, ti)); + snap_matched[si] = true; + tree_available[ti] = false; + } + } + } + + // Early exit if nothing remains + let snap_remaining = snap_matched.iter().filter(|&&m| !m).count(); + let tree_remaining = tree_available.iter().filter(|&&a| a).count(); + if snap_remaining == 0 || tree_remaining == 0 { + return build_result(snap_available, tree_children, &tree_available, matched); + } + + // ================================================================ + // Ambiguous groups: change-count scoring + greedy assignment + // ================================================================ + for (key, snap_indices) in &snap_by_key { + let Some(tree_indices) = tree_by_key.get(key) else { + continue; + }; + + // Collect unmatched indices in this group + let avail_snap: Vec = snap_indices + .iter() + .filter(|&&si| !snap_matched[si]) + .copied() + .collect(); + let avail_tree: Vec = tree_indices + .iter() + .filter(|&&ti| tree_available[ti]) + .copied() + .collect(); + + if avail_snap.is_empty() || avail_tree.is_empty() { + continue; + } + // 1:1 already handled above; skip if not truly ambiguous + if avail_snap.len() <= 1 && avail_tree.len() <= 1 { + if avail_snap.len() == 1 && avail_tree.len() == 1 { + let si = avail_snap[0]; + let ti = avail_tree[0]; + matched.push((si, ti)); + snap_matched[si] = true; + tree_available[ti] = false; + } + continue; + } + + // Score all (A, B) pairs using recursive change-count scoring + let mut pairs: Vec<(u32, usize, usize)> = + Vec::with_capacity(avail_snap.len() * avail_tree.len()); + let mut best_so_far = u32::MAX; + for &si in &avail_snap { + let snap = match &snap_available[si] { + Some(s) => s, + None => continue, + }; + for &ti in &avail_tree { + let cost = + compute_change_count(snap, tree_children[ti], tree, best_so_far, 0, session); + pairs.push((cost, si, ti)); + if cost < best_so_far { + best_so_far = cost; + } + } + } + + // Stable sort by cost ascending (slice::sort_by is stable in Rust) + pairs.sort_by_key(|&(cost, _, _)| cost); + + // Greedy assign + for &(_, si, ti) in &pairs { + if snap_matched[si] || !tree_available[ti] { + continue; + } + matched.push((si, ti)); + snap_matched[si] = true; + tree_available[ti] = false; + } + } + + build_result(snap_available, tree_children, &tree_available, matched) +} + +/// Build the final result, consuming the snapshot children. +fn build_result( + mut snap_available: Vec>, + tree_children: &[Ref], + tree_available: &[bool], + mut matched_indices: Vec<(usize, usize)>, +) -> ForwardMatchResult { + matched_indices.sort_by_key(|&(si, _)| si); + let mut matched = Vec::with_capacity(matched_indices.len()); + for (si, ti) in matched_indices { + if let Some(snap) = snap_available[si].take() { + matched.push((snap, tree_children[ti])); + } + } + + let unmatched_snapshot: Vec = snap_available.into_iter().flatten().collect(); + + let unmatched_tree: Vec = tree_children + .iter() + .enumerate() + .filter(|(i, _)| tree_available[*i]) + .map(|(_, r)| *r) + .collect(); + + ForwardMatchResult { + matched, + unmatched_snapshot, + unmatched_tree, + } +} + +/// Lightweight matching result used during recursive scoring. +/// Returns index pairs (snap_index, tree_child_index) without consuming +/// the snapshots. +struct ScoringMatchResult { + matched: Vec<(usize, usize)>, + unmatched_snap: usize, + unmatched_tree: usize, +} + +/// Match children by reference for scoring purposes (non-consuming). +/// Groups by (Name, ClassName), instant-matches 1:1 groups, and scores +/// ambiguous groups using `compute_change_count` (mutually recursive). +fn match_children_for_scoring( + snap_children: &[InstanceSnapshot], + tree_children: &[Ref], + tree: &RojoTree, + depth: u32, + session: &MatchingSession, +) -> ScoringMatchResult { + if snap_children.is_empty() && tree_children.is_empty() { + return ScoringMatchResult { + matched: Vec::new(), + unmatched_snap: 0, + unmatched_tree: 0, + }; + } + + let snap_len = snap_children.len(); + let tree_len = tree_children.len(); + let mut snap_matched = vec![false; snap_len]; + let mut tree_matched = vec![false; tree_len]; + let mut matched: Vec<(usize, usize)> = Vec::with_capacity(snap_len.min(tree_len)); + + // Group by (Name, ClassName) + let mut snap_by_key: HashMap<(String, Ustr), Vec> = HashMap::with_capacity(snap_len); + for (i, snap) in snap_children.iter().enumerate() { + snap_by_key + .entry((snap.name.to_string(), snap.class_name)) + .or_default() + .push(i); + } + + let mut tree_by_key: HashMap<(String, Ustr), Vec> = HashMap::with_capacity(tree_len); + for (i, &child_ref) in tree_children.iter().enumerate() { + if let Some(inst) = tree.get_instance(child_ref) { + tree_by_key + .entry((inst.name().to_string(), inst.class_name())) + .or_default() + .push(i); + } + } + + // 1:1 groups: instant match + for (key, snap_indices) in &snap_by_key { + if let Some(tree_indices) = tree_by_key.get(key) { + if snap_indices.len() == 1 && tree_indices.len() == 1 { + let si = snap_indices[0]; + let ti = tree_indices[0]; + matched.push((si, ti)); + snap_matched[si] = true; + tree_matched[ti] = true; + } + } + } + + // Ambiguous groups: score + greedy assign + for (key, snap_indices) in &snap_by_key { + let Some(tree_indices) = tree_by_key.get(key) else { + continue; + }; + + let avail_snap: Vec = snap_indices + .iter() + .filter(|&&si| !snap_matched[si]) + .copied() + .collect(); + let avail_tree: Vec = tree_indices + .iter() + .filter(|&&ti| !tree_matched[ti]) + .copied() + .collect(); + + if avail_snap.is_empty() || avail_tree.is_empty() { + continue; + } + if avail_snap.len() == 1 && avail_tree.len() == 1 { + let si = avail_snap[0]; + let ti = avail_tree[0]; + matched.push((si, ti)); + snap_matched[si] = true; + tree_matched[ti] = true; + continue; + } + + let mut pairs: Vec<(u32, usize, usize)> = + Vec::with_capacity(avail_snap.len() * avail_tree.len()); + let mut best_so_far = u32::MAX; + for &si in &avail_snap { + for &ti in &avail_tree { + let cost = compute_change_count( + &snap_children[si], + tree_children[ti], + tree, + best_so_far, + depth, + session, + ); + pairs.push((cost, si, ti)); + if cost < best_so_far { + best_so_far = cost; + } + } + } + + pairs.sort_by_key(|&(cost, _, _)| cost); + + for &(_, si, ti) in &pairs { + if snap_matched[si] || tree_matched[ti] { + continue; + } + matched.push((si, ti)); + snap_matched[si] = true; + tree_matched[ti] = true; + } + } + + let unmatched_snap = snap_matched.iter().filter(|&&m| !m).count(); + let unmatched_tree = tree_matched.iter().filter(|&&m| !m).count(); + + ScoringMatchResult { + matched, + unmatched_snap, + unmatched_tree, + } +} + +/// Compute total change count between a snapshot and a tree instance, +/// including recursive subtree scoring. Returns the number of reconciler +/// operations needed to turn the tree instance into the snapshot. +/// +/// Mutually recursive with `match_children_for_scoring`: this function +/// scores a single pair, while `match_children_for_scoring` groups and +/// assigns children pairs (calling this function for ambiguous scoring). +fn compute_change_count( + snap: &InstanceSnapshot, + tree_ref: Ref, + tree: &RojoTree, + best_so_far: u32, + depth: u32, + session: &MatchingSession, +) -> u32 { + let cacheable = snap.snapshot_id.is_some() && tree_ref.is_some(); + if cacheable { + let cache_key = (snap.snapshot_id, tree_ref); + if let Some(&cached) = session.cost_cache.borrow().get(&cache_key) { + return cached; + } + } + + let inst = match tree.get_instance(tree_ref) { + Some(i) => i, + None => return UNMATCHED_PENALTY, + }; + + let mut cost = count_own_diffs(snap, &inst, tree); + if cost >= best_so_far || depth >= MAX_SCORING_DEPTH { + return cost; + } + + let snap_children = &snap.children; + let tree_children = inst.children(); + + if snap_children.is_empty() && tree_children.is_empty() { + if cacheable && cost < best_so_far { + session + .cost_cache + .borrow_mut() + .insert((snap.snapshot_id, tree_ref), cost); + } + return cost; + } + + let scoring = + match_children_for_scoring(snap_children, tree_children, tree, depth + 1, session); + + for &(si, ti) in &scoring.matched { + let remaining = best_so_far.saturating_sub(cost); + cost += compute_change_count( + &snap_children[si], + tree_children[ti], + tree, + remaining, + depth + 1, + session, + ); + if cost >= best_so_far { + return cost; + } + } + + cost = cost.saturating_add( + (scoring.unmatched_snap + scoring.unmatched_tree) as u32 * UNMATCHED_PENALTY, + ); + + if cacheable && cost < best_so_far { + session + .cost_cache + .borrow_mut() + .insert((snap.snapshot_id, tree_ref), cost); + } + + cost +} + +/// Count own property diffs between a snapshot and a tree instance. +/// Each differing property = +1. Tags and Attributes counted granularly +/// (per-tag and per-attribute diffs). Ref properties compared by target +/// Name+ClassName at UNMATCHED_PENALTY per mismatch. Children count diff = +1. +/// +/// Syncback strips default-valued properties from model/meta files, so +/// filesystem snapshots may omit properties that the tree (populated from +/// Studio via two-way sync) has at their class defaults. To avoid inflating +/// match costs with phantom diffs, properties present on only one side are +/// skipped when their value matches the class default. +#[inline] +fn count_own_diffs(snap: &InstanceSnapshot, inst: &InstanceWithMeta, tree: &RojoTree) -> u32 { + let mut cost: u32 = 0; + + let snap_props = &snap.properties; + let inst_props = inst.properties(); + + let class_data = rbx_reflection_database::get() + .ok() + .and_then(|db| db.classes.get(snap.class_name.as_str())); + + for (key, snap_val) in snap_props.iter() { + if let Some(inst_val) = inst_props.get(key) { + cost += diff_variant_pair(snap_val, inst_val, tree); + } else { + let is_default = class_data + .and_then(|cd| cd.default_properties.get(key.as_str())) + .is_some_and(|default| variant_eq_disk(snap_val, default)); + if !is_default { + cost += count_variant_one_sided(snap_val); + } + } + } + + for (key, inst_val) in inst_props.iter() { + if !snap_props.contains_key(key) { + let is_default = class_data + .and_then(|cd| cd.default_properties.get(key.as_str())) + .is_some_and(|default| variant_eq_disk(inst_val, default)); + if !is_default { + cost += count_variant_one_sided(inst_val); + } + } + } + + if snap.children.len() != inst.children().len() { + cost += 1; + } + + cost +} + +/// Compare two Variant values with type-aware scoring: +/// - Ref: compare target Name+ClassName, cost UNMATCHED_PENALTY if different +/// - Tags: +1 per tag in symmetric difference +/// - Attributes: +1 per differing or one-sided attribute +/// - Other: +1 if not equal +#[inline] +fn diff_variant_pair(a: &Variant, b: &Variant, tree: &RojoTree) -> u32 { + match (a, b) { + (Variant::Ref(ref_a), Variant::Ref(ref_b)) => { + if ref_a == ref_b { + return 0; + } + let a_none = !ref_a.is_some(); + let b_none = !ref_b.is_some(); + if a_none != b_none { + return UNMATCHED_PENALTY; + } + if a_none && b_none { + return 0; + } + let target_a = tree.get_instance(*ref_a); + let target_b = tree.get_instance(*ref_b); + match (target_a, target_b) { + (Some(ta), Some(tb)) => { + if ta.name() == tb.name() && ta.class_name() == tb.class_name() { + 0 + } else { + UNMATCHED_PENALTY + } + } + (None, None) => 0, + _ => UNMATCHED_PENALTY, + } + } + (Variant::Tags(tags_a), Variant::Tags(tags_b)) => count_tags_diff(tags_a, tags_b), + (Variant::Attributes(attrs_a), Variant::Attributes(attrs_b)) => { + count_attributes_diff(attrs_a, attrs_b) + } + _ => { + if variant_eq_disk(a, b) { + 0 + } else { + 1 + } + } + } +} + +/// Cost for a one-sided Variant (property present on only one side). +/// Tags/Attributes contribute their element count; Refs cost UNMATCHED_PENALTY; +/// other types cost 1. +#[inline] +fn count_variant_one_sided(val: &Variant) -> u32 { + match val { + Variant::Ref(r) => { + if r.is_some() { + UNMATCHED_PENALTY + } else { + 0 + } + } + Variant::Tags(tags) => tags.len() as u32, + Variant::Attributes(attrs) => attrs.len() as u32, + _ => 1, + } +} + +/// Count tag diffs as symmetric difference (matching Lua per-tag scoring). +fn count_tags_diff(a: &rbx_dom_weak::types::Tags, b: &rbx_dom_weak::types::Tags) -> u32 { + use std::collections::HashSet; + let set_a: HashSet<&str> = a.iter().collect(); + let set_b: HashSet<&str> = b.iter().collect(); + let only_a = set_a.difference(&set_b).count(); + let only_b = set_b.difference(&set_a).count(); + (only_a + only_b) as u32 +} + +/// Count attribute diffs granularly (matching Lua per-attribute scoring). +fn count_attributes_diff( + a: &rbx_dom_weak::types::Attributes, + b: &rbx_dom_weak::types::Attributes, +) -> u32 { + let mut cost: u32 = 0; + for (key, a_val) in a.iter() { + match b.get(key.as_str()) { + Some(b_val) => { + if !variant_eq_disk(a_val, b_val) { + cost += 1; + } + } + None => cost += 1, + } + } + for (key, _) in b.iter() { + if a.get(key.as_str()).is_none() { + cost += 1; + } + } + cost +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::snapshot::{InstanceMetadata, InstanceSnapshot, RojoTree}; + use crate::variant_eq::variant_eq; + use rbx_dom_weak::{ustr, HashMapExt as _}; + use std::borrow::Cow; + + fn make_snapshot(name: &str, class: &str) -> InstanceSnapshot { + InstanceSnapshot { + snapshot_id: Ref::none(), + metadata: InstanceMetadata::default(), + name: Cow::Owned(name.to_string()), + class_name: ustr(class), + properties: Default::default(), + children: Vec::new(), + } + } + + fn make_tree_with_children(children: &[(&str, &str)]) -> (RojoTree, Vec) { + let child_snapshots: Vec = children + .iter() + .map(|(name, class)| make_snapshot(name, class)) + .collect(); + let root_snap = InstanceSnapshot { + snapshot_id: Ref::none(), + metadata: InstanceMetadata::default(), + name: Cow::Borrowed("DataModel"), + class_name: ustr("DataModel"), + properties: Default::default(), + children: child_snapshots, + }; + let tree = RojoTree::new(root_snap); + let root_id = tree.get_root_id(); + let child_ids: Vec = tree.get_instance(root_id).unwrap().children().to_vec(); + (tree, child_ids) + } + + #[test] + fn unique_names_match() { + let snaps = vec![ + make_snapshot("Alpha", "Folder"), + make_snapshot("Beta", "Script"), + ]; + let (tree, children) = make_tree_with_children(&[("Alpha", "Folder"), ("Beta", "Script")]); + + let result = match_forward(snaps, &children, &tree, &MatchingSession::new()); + assert_eq!(result.matched.len(), 2); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + } + + #[test] + fn class_narrowing() { + let snaps = vec![ + make_snapshot("Handler", "Script"), + make_snapshot("Handler", "ModuleScript"), + ]; + let (tree, children) = + make_tree_with_children(&[("Handler", "ModuleScript"), ("Handler", "Script")]); + + let result = match_forward(snaps, &children, &tree, &MatchingSession::new()); + assert_eq!(result.matched.len(), 2); + + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + assert_eq!(snap.class_name.as_str(), inst.class_name().as_str()); + } + } + + #[test] + fn unmatched_both_sides() { + let snaps = vec![ + make_snapshot("A", "Folder"), + make_snapshot("NewOnly", "Folder"), + ]; + let (tree, children) = make_tree_with_children(&[("A", "Folder"), ("OldOnly", "Folder")]); + + let result = match_forward(snaps, &children, &tree, &MatchingSession::new()); + assert_eq!(result.matched.len(), 1); + assert_eq!(result.unmatched_snapshot.len(), 1); + assert_eq!(result.unmatched_tree.len(), 1); + assert_eq!(result.unmatched_snapshot[0].name.as_ref(), "NewOnly"); + } + + #[test] + fn duplicate_names_greedy() { + let snaps = vec![ + make_snapshot("Data", "Folder"), + make_snapshot("Data", "Folder"), + ]; + let (tree, children) = make_tree_with_children(&[("Data", "Folder"), ("Data", "Folder")]); + + let result = match_forward(snaps, &children, &tree, &MatchingSession::new()); + assert_eq!(result.matched.len(), 2); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + } + + #[test] + fn empty_both() { + let snaps: Vec = vec![]; + let (tree, children) = make_tree_with_children(&[]); + + let result = match_forward(snaps, &children, &tree, &MatchingSession::new()); + assert!(result.matched.is_empty()); + } + + fn make_snapshot_with_props( + name: &str, + class: &str, + props: Vec<(&str, rbx_dom_weak::types::Variant)>, + ) -> InstanceSnapshot { + let mut properties = rbx_dom_weak::UstrMap::new(); + for (key, val) in props { + properties.insert(ustr(key), val); + } + InstanceSnapshot { + snapshot_id: Ref::none(), + metadata: InstanceMetadata::default(), + name: Cow::Owned(name.to_string()), + class_name: ustr(class), + properties, + children: Vec::new(), + } + } + + #[test] + fn many_same_name_parts_matched_by_properties() { + // Simulate a Ladder model with 10 "Line" Parts, each with a different + // Anchored value (as a simple distinguishing property). + // The matching should pair each to its correct counterpart. + use rbx_dom_weak::types::Variant; + + let count = 10; + let snaps: Vec = (0..count) + .map(|i| { + make_snapshot_with_props( + "Line", + "Part", + vec![("Transparency", Variant::Float32(i as f32 * 0.1))], + ) + }) + .collect(); + + // Build tree with same parts but in reversed order -- the matching + // should still pair them correctly by property content, not position. + let child_snapshots: Vec = (0..count) + .rev() + .map(|i| { + make_snapshot_with_props( + "Line", + "Part", + vec![("Transparency", Variant::Float32(i as f32 * 0.1))], + ) + }) + .collect(); + let root_snap = InstanceSnapshot { + snapshot_id: Ref::none(), + metadata: InstanceMetadata::default(), + name: Cow::Borrowed("DataModel"), + class_name: ustr("DataModel"), + properties: Default::default(), + children: child_snapshots, + }; + let tree = RojoTree::new(root_snap); + let root_id = tree.get_root_id(); + let tree_children: Vec = tree.get_instance(root_id).unwrap().children().to_vec(); + + let result = match_forward(snaps, &tree_children, &tree, &MatchingSession::new()); + + assert_eq!(result.matched.len(), count); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + + // Verify each pair has matching Transparency values (correct pairing) + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_val = snap.properties.get(&ustr("Transparency")); + let tree_val = inst.properties().get(&ustr("Transparency")); + assert_eq!( + snap_val, tree_val, + "Mismatched Transparency: snap has {:?}, tree has {:?}", + snap_val, tree_val + ); + } + } + + #[test] + fn fifteen_same_name_identical_parts() { + // 15 Parts all named "Line" with identical properties. + // All pairs score the same -- greedy picks by stable child order. + // Should match all 15 without panicking or cross-matching. + let count = 15; + let snaps: Vec = + (0..count).map(|_| make_snapshot("Line", "Part")).collect(); + let (tree, children) = + make_tree_with_children(&(0..count).map(|_| ("Line", "Part")).collect::>()); + + let result = match_forward(snaps, &children, &tree, &MatchingSession::new()); + assert_eq!(result.matched.len(), count); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + } + + fn make_snapshot_with_children( + name: &str, + class: &str, + children: Vec, + ) -> InstanceSnapshot { + InstanceSnapshot { + snapshot_id: Ref::none(), + metadata: InstanceMetadata::default(), + name: Cow::Owned(name.to_string()), + class_name: ustr(class), + properties: Default::default(), + children, + } + } + + #[test] + fn recursive_matching_distinguishes_by_children() { + // Two Folders both named "Data" with identical top-level properties + // but different children. The recursive scoring should pair each + // Folder with the tree Folder that has matching children. + use rbx_dom_weak::types::Variant; + + let snap_a = make_snapshot_with_children( + "Data", + "Folder", + vec![make_snapshot_with_props( + "Child", + "Part", + vec![("Transparency", Variant::Float32(0.0))], + )], + ); + let snap_b = make_snapshot_with_children( + "Data", + "Folder", + vec![make_snapshot_with_props( + "Child", + "Part", + vec![("Transparency", Variant::Float32(1.0))], + )], + ); + let snaps = vec![snap_a, snap_b]; + + // Build tree with the two Folders in REVERSED order + let tree_child_a = make_snapshot_with_children( + "Data", + "Folder", + vec![make_snapshot_with_props( + "Child", + "Part", + vec![("Transparency", Variant::Float32(1.0))], + )], + ); + let tree_child_b = make_snapshot_with_children( + "Data", + "Folder", + vec![make_snapshot_with_props( + "Child", + "Part", + vec![("Transparency", Variant::Float32(0.0))], + )], + ); + let root_snap = InstanceSnapshot { + snapshot_id: Ref::none(), + metadata: InstanceMetadata::default(), + name: Cow::Borrowed("DataModel"), + class_name: ustr("DataModel"), + properties: Default::default(), + children: vec![tree_child_a, tree_child_b], + }; + let tree = RojoTree::new(root_snap); + let root_id = tree.get_root_id(); + let tree_children: Vec = tree.get_instance(root_id).unwrap().children().to_vec(); + + let result = match_forward(snaps, &tree_children, &tree, &MatchingSession::new()); + + assert_eq!(result.matched.len(), 2); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + + // Verify recursive matching paired by children content, not position. + // snap[0] has child Transparency=0.0, should match tree[1] (also 0.0). + // snap[1] has child Transparency=1.0, should match tree[0] (also 1.0). + for (snap, tree_ref) in &result.matched { + let tree_inst = tree.get_instance(*tree_ref).unwrap(); + let tree_child_ref = tree_inst.children()[0]; + let tree_child = tree.get_instance(tree_child_ref).unwrap(); + + let snap_child = &snap.children[0]; + let snap_val = snap_child.properties.get(&ustr("Transparency")); + let tree_val = tree_child.properties().get(&ustr("Transparency")); + assert_eq!( + snap_val, tree_val, + "Recursive matching failed: snap child Transparency={:?}, tree child Transparency={:?}", + snap_val, tree_val + ); + } + } + + /// Helper: builds a RojoTree from child snapshots (with properties). + fn make_tree_from_snapshots(children: Vec) -> (RojoTree, Vec) { + let root_snap = InstanceSnapshot { + snapshot_id: Ref::none(), + metadata: InstanceMetadata::default(), + name: Cow::Borrowed("DataModel"), + class_name: ustr("DataModel"), + properties: Default::default(), + children, + }; + let tree = RojoTree::new(root_snap); + let root_id = tree.get_root_id(); + let child_ids: Vec = tree.get_instance(root_id).unwrap().children().to_vec(); + (tree, child_ids) + } + + // ================================================================ + // Default-omission matching tests + // + // These reproduce the scenario where syncback strips default-valued + // properties from model files, creating an asymmetry between the + // snapshot (sparse) and tree (full properties from Studio). + // + // The matching must still pair instances correctly despite some + // snapshots missing properties that the tree has at their defaults. + // ================================================================ + + #[test] + fn reflection_database_has_expected_defaults() { + // Validates the reflection database has the defaults we rely on. + use rbx_dom_weak::types::Variant; + + let db = rbx_reflection_database::get().expect("reflection database should load"); + let texture = db + .classes + .get("Texture") + .expect("Texture class should exist"); + let face_default = texture + .default_properties + .get("Face") + .expect("Texture should have Face default"); + match face_default { + Variant::Enum(e) => assert_eq!(e.to_u32(), 5, "Face default should be Front (5)"), + other => panic!("Face default should be Enum, got {:?}", other), + } + + let part = db.classes.get("Part").expect("Part class should exist"); + let anchored_default = part + .default_properties + .get("Anchored") + .expect("Part should have Anchored default"); + assert_eq!( + anchored_default, + &Variant::Bool(false), + "Anchored default should be false" + ); + } + + #[test] + fn texture_face_six_instances_default_omitted() { + // Exact reproduction of the Texture Face bug: + // 6 Textures all named "Texture", each with a different Face. + // Snapshot side: 5 have explicit Face, 1 omits Face (Front=default). + // Tree side: all 6 have explicit Face (populated from Studio). + // + // Without the default-aware fix, the snapshot without Face scores + // equally against all tree instances, steals one by iteration order, + // and displaces the correct match. + use rbx_dom_weak::types::{Enum, Variant}; + + let faces = [ + ("Top", Enum::from_u32(1)), + ("Right", Enum::from_u32(0)), + ("Left", Enum::from_u32(3)), + ("Bottom", Enum::from_u32(4)), + ("Back", Enum::from_u32(2)), + ]; + let face_front = Enum::from_u32(5); + + let shared_props = vec![("Transparency", Variant::Float32(0.65))]; + + // Snapshots: 5 with explicit Face + 1 without (Front default) + let mut snaps: Vec = faces + .iter() + .map(|(_, face_enum)| { + let mut props = shared_props.clone(); + props.push(("Face", Variant::Enum(*face_enum))); + make_snapshot_with_props("Texture", "Texture", props) + }) + .collect(); + // The Front one: NO Face property (default omitted by syncback) + snaps.push(make_snapshot_with_props( + "Texture", + "Texture", + shared_props.clone(), + )); + + // Tree: all 6 with explicit Face (Studio always has all properties) + let mut tree_children: Vec = faces + .iter() + .map(|(_, face_enum)| { + let mut props = shared_props.clone(); + props.push(("Face", Variant::Enum(*face_enum))); + make_snapshot_with_props("Texture", "Texture", props) + }) + .collect(); + tree_children.push(make_snapshot_with_props("Texture", "Texture", { + let mut props = shared_props.clone(); + props.push(("Face", Variant::Enum(face_front))); + props + })); + + let (tree, tree_refs) = make_tree_from_snapshots(tree_children); + let result = match_forward(snaps, &tree_refs, &tree, &MatchingSession::new()); + + assert_eq!(result.matched.len(), 6); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + + // Verify each pair has matching Face values. + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_face = snap.properties.get(&ustr("Face")); + let tree_face = inst.properties().get(&ustr("Face")).unwrap(); + + match snap_face { + Some(sv) => { + assert!( + variant_eq(sv, tree_face), + "Face mismatch: snap={:?}, tree={:?}", + sv, + tree_face + ); + } + None => { + // Snapshot omitted Face = must match tree Face=Front (5) + assert!( + variant_eq(tree_face, &Variant::Enum(face_front)), + "Snapshot without Face should match Front, got {:?}", + tree_face + ); + } + } + } + } + + #[test] + fn texture_face_twelve_instances_two_groups() { + // Exact reproduction of the user's real scenario: + // 12 Textures split into two groups of 6 by TextureContent. + // Each group covers all 6 faces, with Front omitted (default). + // Tree has all 12 with explicit Face. + use rbx_dom_weak::types::{Enum, Variant}; + + let face_values: Vec = vec![ + Enum::from_u32(1), // Top + Enum::from_u32(0), // Right + Enum::from_u32(3), // Left + Enum::from_u32(4), // Bottom + Enum::from_u32(2), // Back + ]; + let face_front = Enum::from_u32(5); + + let mut snaps = Vec::new(); + let mut tree_children = Vec::new(); + + for group_transparency in [0.6_f32, 0.65_f32] { + // 5 with explicit Face + for face in &face_values { + snaps.push(make_snapshot_with_props( + "Texture", + "Texture", + vec![ + ("Transparency", Variant::Float32(group_transparency)), + ("Face", Variant::Enum(*face)), + ], + )); + tree_children.push(make_snapshot_with_props( + "Texture", + "Texture", + vec![ + ("Transparency", Variant::Float32(group_transparency)), + ("Face", Variant::Enum(*face)), + ], + )); + } + // 1 without Face (Front default) + snaps.push(make_snapshot_with_props( + "Texture", + "Texture", + vec![("Transparency", Variant::Float32(group_transparency))], + )); + tree_children.push(make_snapshot_with_props( + "Texture", + "Texture", + vec![ + ("Transparency", Variant::Float32(group_transparency)), + ("Face", Variant::Enum(face_front)), + ], + )); + } + + // Reverse tree order to make it harder + tree_children.reverse(); + + let (tree, tree_refs) = make_tree_from_snapshots(tree_children); + let result = match_forward(snaps, &tree_refs, &tree, &MatchingSession::new()); + + assert_eq!(result.matched.len(), 12); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_face = snap.properties.get(&ustr("Face")); + let tree_face = inst.properties().get(&ustr("Face")).unwrap(); + + // Transparency must match (distinguishes the two groups) + let snap_t = snap.properties.get(&ustr("Transparency")).unwrap(); + let tree_t = inst.properties().get(&ustr("Transparency")).unwrap(); + assert!( + variant_eq(snap_t, tree_t), + "Group mismatch: snap Transparency={:?}, tree={:?}", + snap_t, + tree_t + ); + + match snap_face { + Some(sv) => assert!( + variant_eq(sv, tree_face), + "Face mismatch: snap={:?}, tree={:?}", + sv, + tree_face + ), + None => assert!( + variant_eq(tree_face, &Variant::Enum(face_front)), + "Omitted Face should match Front, got {:?}", + tree_face + ), + } + } + } + + #[test] + fn part_anchored_default_omitted() { + // 4 Parts named "Block": 3 have Anchored=true (explicit), + // 1 omits Anchored (false is the default). Tree has all 4. + use rbx_dom_weak::types::Variant; + + let mut snaps = Vec::new(); + let mut tree_children = Vec::new(); + + for i in 0..3 { + let props = vec![ + ("Transparency", Variant::Float32(i as f32 * 0.1)), + ("Anchored", Variant::Bool(true)), + ]; + snaps.push(make_snapshot_with_props("Block", "Part", props.clone())); + tree_children.push(make_snapshot_with_props("Block", "Part", props)); + } + // The one with default Anchored=false: snapshot omits it + snaps.push(make_snapshot_with_props( + "Block", + "Part", + vec![("Transparency", Variant::Float32(0.3))], + )); + tree_children.push(make_snapshot_with_props( + "Block", + "Part", + vec![ + ("Transparency", Variant::Float32(0.3)), + ("Anchored", Variant::Bool(false)), + ], + )); + + tree_children.reverse(); + + let (tree, tree_refs) = make_tree_from_snapshots(tree_children); + let result = match_forward(snaps, &tree_refs, &tree, &MatchingSession::new()); + + assert_eq!(result.matched.len(), 4); + + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_t = snap.properties.get(&ustr("Transparency")).unwrap(); + let tree_t = inst.properties().get(&ustr("Transparency")).unwrap(); + assert!( + variant_eq(snap_t, tree_t), + "Transparency mismatch: snap={:?}, tree={:?}", + snap_t, + tree_t + ); + } + } + + #[test] + fn multiple_defaults_omitted_all_at_once() { + // Tree instances have MANY extra properties all at their class defaults. + // Snapshots omit all of them. The match should not be thrown off. + use rbx_dom_weak::types::Variant; + + // Two sparse snapshots differing only by Transparency + let snap_a = make_snapshot_with_props( + "Wall", + "Part", + vec![("Transparency", Variant::Float32(0.0))], + ); + let snap_b = make_snapshot_with_props( + "Wall", + "Part", + vec![("Transparency", Variant::Float32(1.0))], + ); + + // Tree has Transparency + many known Part defaults from Studio. + // These are all at their default values, so count_own_diffs should + // skip them (not inflate match cost). + let extra_defaults: Vec<(&str, Variant)> = vec![ + ("Anchored", Variant::Bool(false)), + ("CastShadow", Variant::Bool(true)), + ("CanCollide", Variant::Bool(true)), + ("CanTouch", Variant::Bool(true)), + ("Locked", Variant::Bool(false)), + ("Massless", Variant::Bool(false)), + ]; + + let mut tree_props_a = vec![("Transparency", Variant::Float32(0.0))]; + tree_props_a.extend(extra_defaults.iter().cloned()); + let mut tree_props_b = vec![("Transparency", Variant::Float32(1.0))]; + tree_props_b.extend(extra_defaults.iter().cloned()); + + // Tree in reversed order + let (tree, tree_refs) = make_tree_from_snapshots(vec![ + make_snapshot_with_props("Wall", "Part", tree_props_b), + make_snapshot_with_props("Wall", "Part", tree_props_a), + ]); + let result = match_forward( + vec![snap_a, snap_b], + &tree_refs, + &tree, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 2, "Both should match"); + + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_t = snap.properties.get(&ustr("Transparency")).unwrap(); + let tree_t = inst.properties().get(&ustr("Transparency")).unwrap(); + assert!( + variant_eq(snap_t, tree_t), + "Part with 6 extra default props mismatched: snap={:?}, tree={:?}", + snap_t, + tree_t + ); + } + } + + #[test] + fn ten_ambiguous_textures_stress() { + // 10 Textures: 8 with unique Face values (some repeated across + // groups), 2 with Front omitted. Two distinguishing groups by + // Transparency. The matcher must handle this without mis-pairing. + use rbx_dom_weak::types::{Enum, Variant}; + + let faces_group_a = [ + Some(Enum::from_u32(1)), // Top + Some(Enum::from_u32(0)), // Right + Some(Enum::from_u32(3)), // Left + None, // Front (default, omitted) + Some(Enum::from_u32(4)), // Bottom + ]; + let faces_group_b = [ + Some(Enum::from_u32(2)), // Back + Some(Enum::from_u32(4)), // Bottom + None, // Front (default, omitted) + Some(Enum::from_u32(1)), // Top + Some(Enum::from_u32(3)), // Left + ]; + let face_front = Enum::from_u32(5); + + let mut snaps = Vec::new(); + let mut tree_children = Vec::new(); + + for (group_t, faces) in [(0.5_f32, &faces_group_a[..]), (0.8_f32, &faces_group_b[..])] { + for face_opt in faces { + let mut snap_props = vec![("Transparency", Variant::Float32(group_t))]; + let mut tree_props = vec![("Transparency", Variant::Float32(group_t))]; + + if let Some(face) = face_opt { + snap_props.push(("Face", Variant::Enum(*face))); + tree_props.push(("Face", Variant::Enum(*face))); + } else { + // Snapshot omits Face; tree has Front + tree_props.push(("Face", Variant::Enum(face_front))); + } + + snaps.push(make_snapshot_with_props("Texture", "Texture", snap_props)); + tree_children.push(make_snapshot_with_props("Texture", "Texture", tree_props)); + } + } + + // Shuffle tree order (reverse) + tree_children.reverse(); + + let (tree, tree_refs) = make_tree_from_snapshots(tree_children); + let result = match_forward(snaps, &tree_refs, &tree, &MatchingSession::new()); + + assert_eq!(result.matched.len(), 10); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + + // Transparency must match (group discriminator) + let snap_t = snap.properties.get(&ustr("Transparency")).unwrap(); + let tree_t = inst.properties().get(&ustr("Transparency")).unwrap(); + assert!( + variant_eq(snap_t, tree_t), + "Group mismatch: snap={:?}, tree={:?}", + snap_t, + tree_t + ); + + // Face must match + let snap_face = snap.properties.get(&ustr("Face")); + let tree_face = inst.properties().get(&ustr("Face")).unwrap(); + match snap_face { + Some(sv) => assert!( + variant_eq(sv, tree_face), + "Face mismatch: snap={:?}, tree={:?}", + sv, + tree_face + ), + None => assert!( + variant_eq(tree_face, &Variant::Enum(face_front)), + "Omitted Face should match Front, got {:?}", + tree_face + ), + } + } + } + + #[test] + fn depth_limit_completes_quickly() { + // Deeply nested tree (depth=6) with same-named instances at each level. + // Should complete without hanging thanks to the depth limit. + fn make_deep_snap(depth: u32) -> InstanceSnapshot { + if depth == 0 { + return make_snapshot("Leaf", "Part"); + } + make_snapshot_with_children( + "Node", + "Folder", + vec![make_deep_snap(depth - 1), make_deep_snap(depth - 1)], + ) + } + + let snaps = vec![make_deep_snap(6), make_deep_snap(6)]; + + let tree_children_snaps = vec![make_deep_snap(6), make_deep_snap(6)]; + let root_snap = InstanceSnapshot { + snapshot_id: Ref::none(), + metadata: InstanceMetadata::default(), + name: Cow::Borrowed("DataModel"), + class_name: ustr("DataModel"), + properties: Default::default(), + children: tree_children_snaps, + }; + let tree = RojoTree::new(root_snap); + let root_id = tree.get_root_id(); + let tree_children: Vec = tree.get_instance(root_id).unwrap().children().to_vec(); + + let start = std::time::Instant::now(); + let result = match_forward(snaps, &tree_children, &tree, &MatchingSession::new()); + let elapsed = start.elapsed(); + + assert_eq!(result.matched.len(), 2); + assert!( + elapsed.as_secs() < 5, + "Depth-limited matching took too long: {:?}", + elapsed + ); + } + + #[test] + fn session_cache_consistent() { + let snaps1 = vec![make_snapshot("X", "Folder"), make_snapshot("Y", "Folder")]; + let snaps2 = vec![make_snapshot("X", "Folder"), make_snapshot("Y", "Folder")]; + let (tree, children) = make_tree_with_children(&[("X", "Folder"), ("Y", "Folder")]); + let session = MatchingSession::new(); + + let r1 = match_forward(snaps1, &children, &tree, &session); + let r2 = match_forward(snaps2, &children, &tree, &session); + + assert_eq!(r1.matched.len(), r2.matched.len()); + assert_eq!(r1.unmatched_snapshot.len(), r2.unmatched_snapshot.len()); + } + + #[test] + fn tags_granular_scoring() { + use rbx_dom_weak::types::{Tags, Variant}; + + fn make_tags(names: &[&str]) -> Tags { + let mut t = Tags::new(); + for &n in names { + t.push(n); + } + t + } + + let snap_a = make_snapshot_with_props( + "Item", + "Part", + vec![("Tags", Variant::Tags(make_tags(&["A", "B", "C"])))], + ); + let snap_b = make_snapshot_with_props( + "Item", + "Part", + vec![("Tags", Variant::Tags(make_tags(&["X", "Y"])))], + ); + + let tree_a = make_snapshot_with_props( + "Item", + "Part", + vec![("Tags", Variant::Tags(make_tags(&["A", "B", "C"])))], + ); + let tree_b = make_snapshot_with_props( + "Item", + "Part", + vec![("Tags", Variant::Tags(make_tags(&["X", "Y"])))], + ); + + let (tree, tree_refs) = make_tree_from_snapshots(vec![tree_b, tree_a]); + let result = match_forward( + vec![snap_a, snap_b], + &tree_refs, + &tree, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 2); + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_tags = snap.properties.get(&ustr("Tags")); + let tree_tags = inst.properties().get(&ustr("Tags")); + assert!( + variant_eq(snap_tags.unwrap(), tree_tags.unwrap()), + "Tags mismatch: snap={:?}, tree={:?}", + snap_tags, + tree_tags + ); + } + } + + #[test] + fn attributes_granular_scoring() { + use rbx_dom_weak::types::{Attributes, Variant}; + + let mut attrs_a = Attributes::new(); + attrs_a.insert("Health".into(), Variant::Float32(100.0)); + attrs_a.insert("Speed".into(), Variant::Float32(16.0)); + + let mut attrs_b = Attributes::new(); + attrs_b.insert("Health".into(), Variant::Float32(50.0)); + attrs_b.insert("Damage".into(), Variant::Float32(25.0)); + + let snap_a = make_snapshot_with_props( + "Mob", + "Part", + vec![("Attributes", Variant::Attributes(attrs_a.clone()))], + ); + let snap_b = make_snapshot_with_props( + "Mob", + "Part", + vec![("Attributes", Variant::Attributes(attrs_b.clone()))], + ); + + let tree_a = make_snapshot_with_props( + "Mob", + "Part", + vec![("Attributes", Variant::Attributes(attrs_a))], + ); + let tree_b = make_snapshot_with_props( + "Mob", + "Part", + vec![("Attributes", Variant::Attributes(attrs_b))], + ); + + let (tree, tree_refs) = make_tree_from_snapshots(vec![tree_b, tree_a]); + let result = match_forward( + vec![snap_a, snap_b], + &tree_refs, + &tree, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 2); + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_attrs = snap.properties.get(&ustr("Attributes")); + let tree_attrs = inst.properties().get(&ustr("Attributes")); + assert!( + variant_eq(snap_attrs.unwrap(), tree_attrs.unwrap()), + "Attributes mismatch" + ); + } + } + + #[test] + fn ref_scoring_forward_sync() { + use rbx_dom_weak::types::Variant; + + // Build a tree with 2 Models ("Weapon"), each containing a differently + // named child Part. A Ref property on each Model points to its child. + // The matching must pair each snapshot Model to the tree Model whose + // Ref target has the same Name+ClassName. + + // Build a tree with 2 Models ("Weapon"), each containing a + // differently-named child Part. Ref property on each Model points + // to its child. Tree order is reversed relative to snapshots. + let tree_child_handle = { + let mut s = make_snapshot("Handle", "Part"); + s.properties + .insert(ustr("Transparency"), Variant::Float32(0.0)); + s + }; + let tree_child_grip = { + let mut s = make_snapshot("Grip", "Part"); + s.properties + .insert(ustr("Transparency"), Variant::Float32(0.0)); + s + }; + let tree_model_grip_snap = + make_snapshot_with_children("Weapon", "Model", vec![tree_child_grip]); + let tree_model_handle_snap = + make_snapshot_with_children("Weapon", "Model", vec![tree_child_handle]); + + let (tree2, tree_refs2) = + make_tree_from_snapshots(vec![tree_model_grip_snap, tree_model_handle_snap]); + + let tree2_model_0 = tree2.get_instance(tree_refs2[0]).unwrap(); + let tree2_model_1 = tree2.get_instance(tree_refs2[1]).unwrap(); + let tree2_grip_ref = tree2_model_0.children()[0]; + let tree2_handle_ref = tree2_model_1.children()[0]; + + // Rebuild snapshots with Refs pointing into tree2 + let snap_handle2 = { + let mut s = make_snapshot_with_children( + "Weapon", + "Model", + vec![make_snapshot("Handle", "Part")], + ); + s.properties + .insert(ustr("PrimaryPart"), Variant::Ref(tree2_handle_ref)); + s + }; + let snap_grip2 = { + let mut s = + make_snapshot_with_children("Weapon", "Model", vec![make_snapshot("Grip", "Part")]); + s.properties + .insert(ustr("PrimaryPart"), Variant::Ref(tree2_grip_ref)); + s + }; + + let result = match_forward( + vec![snap_handle2, snap_grip2], + &tree_refs2, + &tree2, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 2); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + + // Verify correct pairing: each snap should match the tree model + // whose child has the same name as the snap's child. + for (snap, tree_ref) in &result.matched { + let snap_child_name = &snap.children[0].name; + let tree_inst = tree2.get_instance(*tree_ref).unwrap(); + let tree_child_ref = tree_inst.children()[0]; + let tree_child = tree2.get_instance(tree_child_ref).unwrap(); + assert_eq!( + snap_child_name.as_ref(), + tree_child.name(), + "Ref scoring failed: snap with child '{}' matched tree with child '{}'", + snap_child_name, + tree_child.name() + ); + } + } + + // ================================================================ + // Large ambiguous group tests (50+ same-named instances) + // ================================================================ + + #[test] + fn fifty_same_name_parts_five_variation_groups() { + use rbx_dom_weak::types::{Color3, Variant, Vector3}; + + let groups: Vec<(Vector3, Color3)> = vec![ + (Vector3::new(0.0, 0.0, 0.0), Color3::new(1.0, 0.0, 0.0)), + (Vector3::new(0.0, 5.0, 0.0), Color3::new(1.0, 0.0, 0.0)), + (Vector3::new(0.0, 10.0, 0.0), Color3::new(0.0, 1.0, 0.0)), + (Vector3::new(0.0, 0.0, 0.0), Color3::new(0.0, 1.0, 0.0)), + (Vector3::new(0.0, 5.0, 0.0), Color3::new(0.0, 0.0, 1.0)), + ]; + + let mut snaps = Vec::new(); + for (pos, color) in &groups { + for _ in 0..10 { + snaps.push(make_snapshot_with_props( + "Line", + "Part", + vec![ + ("Position", Variant::Vector3(*pos)), + ("Color", Variant::Color3(*color)), + ], + )); + } + } + + let mut tree_children_snaps: Vec = Vec::new(); + for (pos, color) in groups.iter().rev() { + for _ in 0..10 { + tree_children_snaps.push(make_snapshot_with_props( + "Line", + "Part", + vec![ + ("Position", Variant::Vector3(*pos)), + ("Color", Variant::Color3(*color)), + ], + )); + } + } + + let (tree, tree_refs) = make_tree_from_snapshots(tree_children_snaps); + + let start = std::time::Instant::now(); + let result = match_forward(snaps, &tree_refs, &tree, &MatchingSession::new()); + let elapsed = start.elapsed(); + + assert_eq!(result.matched.len(), 50); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + assert!(elapsed.as_secs() < 5, "took too long: {:?}", elapsed); + + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_pos = snap.properties.get(&ustr("Position")).unwrap(); + let tree_pos = inst.properties().get(&ustr("Position")).unwrap(); + assert!( + variant_eq_disk(snap_pos, tree_pos), + "Position mismatch: snap={:?}, tree={:?}", + snap_pos, + tree_pos + ); + let snap_color = snap.properties.get(&ustr("Color")).unwrap(); + let tree_color = inst.properties().get(&ustr("Color")).unwrap(); + assert!( + variant_eq_disk(snap_color, tree_color), + "Color mismatch: snap={:?}, tree={:?}", + snap_color, + tree_color + ); + } + } + + #[test] + fn fifty_parts_position_only_variations() { + use rbx_dom_weak::types::{Color3, Variant, Vector3}; + + let shared_color = Color3::new(0.5, 0.5, 0.5); + let positions = [ + Vector3::new(0.0, 0.0, 0.0), + Vector3::new(10.0, 0.0, 0.0), + Vector3::new(0.0, 10.0, 0.0), + Vector3::new(0.0, 0.0, 10.0), + Vector3::new(5.0, 5.0, 5.0), + ]; + + let mut snaps = Vec::new(); + let mut tree_children_snaps = Vec::new(); + for pos in &positions { + for _ in 0..10 { + let props = vec![ + ("Position", Variant::Vector3(*pos)), + ("Color", Variant::Color3(shared_color)), + ]; + snaps.push(make_snapshot_with_props("Block", "Part", props.clone())); + tree_children_snaps.push(make_snapshot_with_props("Block", "Part", props)); + } + } + tree_children_snaps.reverse(); + + let (tree, tree_refs) = make_tree_from_snapshots(tree_children_snaps); + let result = match_forward(snaps, &tree_refs, &tree, &MatchingSession::new()); + + assert_eq!(result.matched.len(), 50); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_pos = snap.properties.get(&ustr("Position")).unwrap(); + let tree_pos = inst.properties().get(&ustr("Position")).unwrap(); + assert!(variant_eq_disk(snap_pos, tree_pos), "Position mismatch"); + } + } + + #[test] + fn fifty_parts_color_only_variations() { + use rbx_dom_weak::types::{Color3, Variant, Vector3}; + + let shared_pos = Vector3::new(0.0, 0.0, 0.0); + let colors = [ + Color3::new(1.0, 0.0, 0.0), + Color3::new(0.0, 1.0, 0.0), + Color3::new(0.0, 0.0, 1.0), + Color3::new(1.0, 1.0, 0.0), + Color3::new(0.0, 1.0, 1.0), + ]; + + let mut snaps = Vec::new(); + let mut tree_children_snaps = Vec::new(); + for color in &colors { + for _ in 0..10 { + let props = vec![ + ("Position", Variant::Vector3(shared_pos)), + ("Color", Variant::Color3(*color)), + ]; + snaps.push(make_snapshot_with_props("Block", "Part", props.clone())); + tree_children_snaps.push(make_snapshot_with_props("Block", "Part", props)); + } + } + tree_children_snaps.reverse(); + + let (tree, tree_refs) = make_tree_from_snapshots(tree_children_snaps); + let result = match_forward(snaps, &tree_refs, &tree, &MatchingSession::new()); + + assert_eq!(result.matched.len(), 50); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_color = snap.properties.get(&ustr("Color")).unwrap(); + let tree_color = inst.properties().get(&ustr("Color")).unwrap(); + assert!(variant_eq_disk(snap_color, tree_color), "Color mismatch"); + } + } + + #[test] + fn sixty_parts_with_near_float_values() { + use rbx_dom_weak::types::Variant; + + let groups: Vec = vec![0.1, 0.100001, 0.5, 0.500001, 0.999999]; + let group_size = 12; + + let mut snaps = Vec::new(); + let mut tree_children_snaps = Vec::new(); + for &t in &groups { + for _ in 0..group_size { + snaps.push(make_snapshot_with_props( + "Segment", + "Part", + vec![("Transparency", Variant::Float32(t))], + )); + tree_children_snaps.push(make_snapshot_with_props( + "Segment", + "Part", + vec![("Transparency", Variant::Float32(t))], + )); + } + } + tree_children_snaps.reverse(); + + let (tree, tree_refs) = make_tree_from_snapshots(tree_children_snaps); + let result = match_forward(snaps, &tree_refs, &tree, &MatchingSession::new()); + + assert_eq!(result.matched.len(), 60); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_t = snap.properties.get(&ustr("Transparency")).unwrap(); + let tree_t = inst.properties().get(&ustr("Transparency")).unwrap(); + assert!( + variant_eq_disk(snap_t, tree_t), + "Transparency mismatch: snap={:?}, tree={:?}", + snap_t, + tree_t + ); + } + } + + #[test] + fn disk_representation_boundary_matching() { + use rbx_dom_weak::types::Variant; + + let a_val: f32 = 1.00005; + let b_val: f32 = 1.00015; + + let snaps = vec![ + make_snapshot_with_props( + "Edge", + "Part", + vec![("Transparency", Variant::Float32(a_val))], + ), + make_snapshot_with_props( + "Edge", + "Part", + vec![("Transparency", Variant::Float32(b_val))], + ), + ]; + + let tree_children = vec![ + make_snapshot_with_props( + "Edge", + "Part", + vec![("Transparency", Variant::Float32(b_val))], + ), + make_snapshot_with_props( + "Edge", + "Part", + vec![("Transparency", Variant::Float32(a_val))], + ), + ]; + + let (tree, tree_refs) = make_tree_from_snapshots(tree_children); + let result = match_forward(snaps, &tree_refs, &tree, &MatchingSession::new()); + + assert_eq!(result.matched.len(), 2); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + + for (snap, tree_ref) in &result.matched { + let inst = tree.get_instance(*tree_ref).unwrap(); + let snap_t = snap.properties.get(&ustr("Transparency")).unwrap(); + let tree_t = inst.properties().get(&ustr("Transparency")).unwrap(); + assert!( + variant_eq_disk(snap_t, tree_t), + "Boundary test failed: snap={:?}, tree={:?} (disk should pair exactly)", + snap_t, + tree_t + ); + } + } +} diff --git a/src/snapshot/metadata.rs b/src/snapshot/metadata.rs index 132769eb..82612f30 100644 --- a/src/snapshot/metadata.rs +++ b/src/snapshot/metadata.rs @@ -8,7 +8,8 @@ use anyhow::Context; use serde::{Deserialize, Serialize}; use crate::{ - glob::Glob, path_serializer, project::ProjectNode, snapshot_middleware::Middleware, RojoRef, + glob::Glob, path_serializer, project::ProjectNode, snapshot_middleware::Middleware, + syncback::dedup_suffix::strip_dedup_suffix, RojoRef, }; /// Rojo-specific metadata that can be associated with an instance or a snapshot @@ -328,23 +329,22 @@ impl SyncRule { } pub fn file_name_for_path<'a>(&self, path: &'a Path) -> anyhow::Result<&'a str> { - if let Some(suffix) = &self.suffix { + let name = if let Some(suffix) = &self.suffix { let file_name = path .file_name() .and_then(|s| s.to_str()) .with_context(|| format!("file name of {} is invalid", path.display()))?; if file_name.ends_with(suffix) { let end = file_name.len().saturating_sub(suffix.len()); - Ok(&file_name[..end]) + &file_name[..end] } else { - Ok(file_name) + file_name } } else { - // If the user doesn't specify a suffix, we assume they just want - // the name of the file (the file_stem) path.file_stem() .and_then(|s| s.to_str()) - .with_context(|| format!("file name of {} is invalid", path.display())) - } + .with_context(|| format!("file name of {} is invalid", path.display()))? + }; + Ok(strip_dedup_suffix(name)) } } diff --git a/src/snapshot/mod.rs b/src/snapshot/mod.rs index 5aacdad6..30a22238 100644 --- a/src/snapshot/mod.rs +++ b/src/snapshot/mod.rs @@ -49,6 +49,7 @@ #![allow(dead_code)] mod instance_snapshot; +pub mod matching; mod metadata; mod patch; mod patch_apply; diff --git a/src/snapshot/patch.rs b/src/snapshot/patch.rs index fb984442..235ed12d 100644 --- a/src/snapshot/patch.rs +++ b/src/snapshot/patch.rs @@ -1,5 +1,7 @@ //! Defines the data structures used for describing instance patches. +use std::collections::HashSet; + use rbx_dom_weak::{ types::{Ref, Variant}, HashMapExt as _, Ustr, UstrMap, @@ -18,6 +20,16 @@ pub struct PatchSet { pub removed_instances: Vec, pub added_instances: Vec, pub updated_instances: Vec, + /// Instance IDs whose backing files should be staged via git add after writing. + /// Populated from WriteRequest.stage_ids, consumed by change_processor. + #[serde(default, skip)] + pub stage_ids: HashSet, + /// Pre-resolved filesystem paths to stage via git add. + /// Populated by api.rs (for files already on disk), consumed by change_processor + /// together with any Source write paths from stage_ids. + /// All staging is consolidated into a single git_add call to avoid index.lock races. + #[serde(default, skip)] + pub stage_paths: Vec, } impl PatchSet { @@ -26,6 +38,8 @@ impl PatchSet { removed_instances: Vec::new(), added_instances: Vec::new(), updated_instances: Vec::new(), + stage_ids: HashSet::new(), + stage_paths: Vec::new(), } } } diff --git a/src/snapshot/patch_apply.rs b/src/snapshot/patch_apply.rs index 00326a19..fa236635 100644 --- a/src/snapshot/patch_apply.rs +++ b/src/snapshot/patch_apply.rs @@ -124,9 +124,9 @@ fn finalize_patch_application(context: PatchApplyContext, tree: &mut RojoTree) - } } - // This is to get around the fact that `RojoTre::get_specified_id` borrows - // the tree as immutable, but we need to hold a mutable reference to it. - // Not exactly elegant, but it does the job. + // IMPORTANT: Rojo_Target_* (legacy) must be resolved BEFORE Rojo_Ref_* + // (path-based) so that path-based results overwrite legacy results when + // both exist for the same property. Do not reorder these two blocks. let mut real_rewrites = Vec::new(); for (id, map) in context.attribute_refs_to_rewrite { for (prop_name, prop_value) in map { @@ -144,7 +144,7 @@ fn finalize_patch_application(context: PatchApplyContext, tree: &mut RojoTree) - let mut path_rewrites = Vec::new(); for (id, map) in context.path_refs_to_rewrite { for (prop_name, path) in map { - if let Some(target) = tree.get_instance_by_path(&path) { + if let Some(target) = tree.resolve_ref_path(&path, id) { log::debug!( "Resolved path reference {} -> {} (path: {})", prop_name, @@ -391,6 +391,7 @@ mod test { parent_id: root_id, instance: snapshot.clone(), }], + stage_ids: HashSet::new(), ..Default::default() }; @@ -438,6 +439,7 @@ mod test { let patch_set = PatchSet { updated_instances: vec![patch], + stage_ids: HashSet::new(), ..Default::default() }; diff --git a/src/snapshot/patch_compute.rs b/src/snapshot/patch_compute.rs index d894d687..c49ebc19 100644 --- a/src/snapshot/patch_compute.rs +++ b/src/snapshot/patch_compute.rs @@ -19,12 +19,15 @@ use super::{ #[profiling::function] pub fn compute_patch_set(snapshot: Option, tree: &RojoTree, id: Ref) -> PatchSet { + use super::matching::MatchingSession; + let mut patch_set = PatchSet::new(); if let Some(snapshot) = snapshot { let mut context = ComputePatchContext::default(); + let session = MatchingSession::new(); - compute_patch_set_internal(&mut context, snapshot, tree, id, &mut patch_set); + compute_patch_set_internal(&mut context, snapshot, tree, id, &mut patch_set, &session); // Rewrite Ref properties to refer to instance IDs instead of snapshot IDs // for all of the IDs that we know about so far. @@ -80,6 +83,7 @@ fn compute_patch_set_internal( tree: &RojoTree, id: Ref, patch_set: &mut PatchSet, + session: &super::matching::MatchingSession, ) { if snapshot.snapshot_id.is_some() { context @@ -92,7 +96,7 @@ fn compute_patch_set_internal( .expect("Instance did not exist in tree"); compute_property_patches(&mut snapshot, &instance, patch_set, tree); - compute_children_patches(context, &mut snapshot, tree, id, patch_set); + compute_children_patches(context, &mut snapshot, tree, id, patch_set, session); } fn compute_property_patches( @@ -104,7 +108,7 @@ fn compute_property_patches( let mut visited_properties = UstrSet::default(); let mut changed_properties = UstrMap::new(); - let attribute_ref_properties = compute_ref_properties(snapshot, tree); + let attribute_ref_properties = compute_ref_properties(snapshot, tree, instance.id()); let changed_name = if snapshot.name == instance.name() { None @@ -216,70 +220,49 @@ fn compute_children_patches( tree: &RojoTree, id: Ref, patch_set: &mut PatchSet, + session: &super::matching::MatchingSession, ) { + use super::matching::match_forward; + let instance = tree .get_instance(id) .expect("Instance did not exist in tree"); - let instance_children = instance.children(); + let instance_children = instance.children().to_vec(); + let snapshot_children = take(&mut snapshot.children); - let mut paired_instances = vec![false; instance_children.len()]; + let match_result = match_forward(snapshot_children, &instance_children, tree, session); - for snapshot_child in take(&mut snapshot.children) { - let matching_instance = - instance_children - .iter() - .enumerate() - .find(|(instance_index, instance_child_id)| { - if paired_instances[*instance_index] { - return false; - } - - let instance_child = tree - .get_instance(**instance_child_id) - .expect("Instance did not exist in tree"); - - if snapshot_child.name == instance_child.name() - && snapshot_child.class_name == instance_child.class_name() - { - paired_instances[*instance_index] = true; - return true; - } - - false - }); - - match matching_instance { - Some((_, instance_child_id)) => { - compute_patch_set_internal( - context, - snapshot_child, - tree, - *instance_child_id, - patch_set, - ); - } - None => { - patch_set.added_instances.push(PatchAdd { - parent_id: id, - instance: snapshot_child, - }); - } - } + // Matched pairs: recursively compute patches. + for (snapshot_child, tree_child_id) in match_result.matched { + compute_patch_set_internal( + context, + snapshot_child, + tree, + tree_child_id, + patch_set, + session, + ); } - for (instance_index, instance_child_id) in instance_children.iter().enumerate() { - if paired_instances[instance_index] { - continue; - } + // Unmatched snapshots: new instances to be added. + for snapshot_child in match_result.unmatched_snapshot { + patch_set.added_instances.push(PatchAdd { + parent_id: id, + instance: snapshot_child, + }); + } - patch_set.removed_instances.push(*instance_child_id); + // Unmatched tree children: instances to be removed. + for tree_child_id in match_result.unmatched_tree { + patch_set.removed_instances.push(tree_child_id); } } fn compute_ref_properties( snapshot: &InstanceSnapshot, tree: &RojoTree, + source_ref: Ref, ) -> UstrMap> { let mut map = UstrMap::new(); let attributes = match snapshot.properties.get(&ustr("Attributes")) { @@ -319,7 +302,7 @@ fn compute_ref_properties( let Some(path) = crate::variant_as_str(attr_value, attr_name) else { continue; }; - if let Some(target_id) = tree.get_instance_by_path(path) { + if let Some(target_id) = tree.resolve_ref_path(path, source_ref) { map.insert(ustr(prop_name), Some(Variant::Ref(target_id))); } else { // Path not found yet - will be resolved during patch apply @@ -376,6 +359,8 @@ mod test { }], added_instances: Vec::new(), removed_instances: Vec::new(), + stage_ids: std::collections::HashSet::new(), + stage_paths: Vec::new(), }; assert_eq!(patch_set, expected_patch_set); @@ -426,6 +411,8 @@ mod test { }], updated_instances: Vec::new(), removed_instances: Vec::new(), + stage_ids: std::collections::HashSet::new(), + stage_paths: Vec::new(), }; assert_eq!(patch_set, expected_patch_set); diff --git a/src/snapshot/tests/apply.rs b/src/snapshot/tests/apply.rs index d3602bb3..2dc8cb4c 100644 --- a/src/snapshot/tests/apply.rs +++ b/src/snapshot/tests/apply.rs @@ -1,3 +1,5 @@ +use std::collections::HashSet; + use insta::assert_yaml_snapshot; use rbx_dom_weak::{ustr, UstrMap}; @@ -23,6 +25,7 @@ fn set_name_and_class_name() { changed_properties: Default::default(), changed_metadata: None, }], + stage_ids: HashSet::new(), ..Default::default() }; @@ -50,6 +53,7 @@ fn add_property() { changed_properties: UstrMap::from_iter([(ustr("Foo"), Some("Value of Foo".into()))]), changed_metadata: None, }], + stage_ids: HashSet::new(), ..Default::default() }; @@ -89,6 +93,7 @@ fn remove_property() { changed_properties: UstrMap::from_iter([(ustr("Foo"), None)]), changed_metadata: None, }], + stage_ids: HashSet::new(), ..Default::default() }; @@ -144,6 +149,7 @@ fn apply_ref_property_update() { )]), changed_metadata: None, }], + stage_ids: HashSet::new(), ..Default::default() }; @@ -203,6 +209,7 @@ fn apply_nil_ref_skips_in_forward_sync() { )]), changed_metadata: None, }], + stage_ids: HashSet::new(), ..Default::default() }; @@ -258,6 +265,7 @@ fn apply_ref_property_removal() { changed_properties: UstrMap::from_iter([(ustr("PrimaryPart"), None)]), changed_metadata: None, }], + stage_ids: HashSet::new(), ..Default::default() }; @@ -314,6 +322,7 @@ fn apply_multiple_ref_property_updates() { changed_metadata: None, }, ], + stage_ids: HashSet::new(), ..Default::default() }; @@ -365,6 +374,7 @@ fn apply_ref_to_nonexistent_instance() { )]), changed_metadata: None, }], + stage_ids: HashSet::new(), ..Default::default() }; diff --git a/src/snapshot/tree.rs b/src/snapshot/tree.rs index d6e60e9e..fd7922b7 100644 --- a/src/snapshot/tree.rs +++ b/src/snapshot/tree.rs @@ -10,7 +10,7 @@ use rbx_dom_weak::{ use crate::{multimap::MultiMap, RojoRef}; -use super::{InstanceMetadata, InstanceSnapshot}; +use super::{InstanceMetadata, InstanceSnapshot, InstigatingSource}; /// An expanded variant of rbx_dom_weak's `WeakDom` that tracks additional /// metadata per instance that's Rojo-specific. @@ -231,36 +231,52 @@ impl RojoTree { self.specified_id_to_refs.insert(specified, id); } - /// Looks up an instance by its path in the tree (e.g., "SoundService/MySoundGroup"). - /// The path is relative to the root of the tree, with segments separated by "/". - /// Returns the Ref if found, None otherwise. + /// Looks up an instance by its filesystem-name path in the tree. + /// + /// Each segment is matched against the **filesystem name** of child + /// instances (derived from `instigating_source` path), falling back to + /// the instance name for instances without filesystem backing. + /// + /// Path format: `"Workspace/Model.model.json5/Attachment~2.luau"` + /// Segments are separated by `/` (no escaping needed since filesystem + /// names can't contain `/`). pub fn get_instance_by_path(&self, path: &str) -> Option { if path.is_empty() { return Some(self.get_root_id()); } - // Use split_ref_path to handle escaped "/" in instance names. - // An instance named "A/B" is stored as "A\/B" in the path. - let segments = crate::split_ref_path(path); + let segments: Vec<&str> = path.split('/').collect(); let mut current_ref = self.get_root_id(); - // The root of the tree doesn't appear in the path, so we start with - // the root's children. - for segment in segments.iter() { + for segment in &segments { let current = self.inner.get_by_ref(current_ref)?; - let children = current.children(); let mut found = false; for &child_ref in children { - let child = self.inner.get_by_ref(child_ref)?; - if child.name == segment.as_str() { + let fs_name = self.filesystem_name_for(child_ref); + if fs_name.eq_ignore_ascii_case(segment) { current_ref = child_ref; found = true; break; } } + if !found { + // Fallback: try matching by instance name (for backward + // compatibility with legacy ref paths and instances without + // filesystem backing). Case-insensitive to match the primary + // lookup semantics. + for &child_ref in children { + let child = self.inner.get_by_ref(child_ref)?; + if child.name.eq_ignore_ascii_case(segment) { + current_ref = child_ref; + found = true; + break; + } + } + } + if !found { return None; } @@ -269,6 +285,138 @@ impl RojoTree { Some(current_ref) } + /// Resolve a Luau require-by-string style ref path to a target instance. + /// + /// `source_ref` is the instance that owns the `Rojo_Ref_*` attribute. + /// Prefix semantics (per Roblox require-by-string): + /// - `@game/rest` -- resolve from DataModel root + /// - `@self` -- return source_ref + /// - `@self/rest` -- resolve from source_ref + /// - `./rest` -- resolve from source_ref's parent + /// - `../rest` -- resolve from source_ref's grandparent + /// - bare path (no prefix) -- legacy fallback, resolve like `@game/` + /// + /// During segment walk, `..` navigates to the parent. + pub fn resolve_ref_path(&self, path: &str, source_ref: Ref) -> Option { + if let Some(rest) = path.strip_prefix("@game/") { + return self.walk_segments(self.get_root_id(), rest); + } + if path == "@game" { + return Some(self.get_root_id()); + } + if path == "@self" { + return Some(source_ref); + } + if let Some(rest) = path.strip_prefix("@self/") { + return self.walk_segments(source_ref, rest); + } + if let Some(rest) = path.strip_prefix("./") { + let parent = self.inner.get_by_ref(source_ref)?.parent(); + if parent.is_none() { + return None; + } + return self.walk_segments(parent, rest); + } + if let Some(rest) = path.strip_prefix("../") { + let parent = self.inner.get_by_ref(source_ref)?.parent(); + if parent.is_none() { + return None; + } + let grandparent = self.inner.get_by_ref(parent)?.parent(); + if grandparent.is_none() { + return None; + } + return self.walk_segments(grandparent, rest); + } + + // Bare path (no prefix) -- legacy backward compatibility + self.walk_segments(self.get_root_id(), path) + } + + /// Walk `/`-separated segments from a starting Ref, handling `..` as + /// "go to parent". Each non-`..` segment matches a child by filesystem + /// name (case-insensitive), falling back to instance name. + fn walk_segments(&self, start: Ref, rest: &str) -> Option { + if rest.is_empty() { + return Some(start); + } + + let mut current_ref = start; + for segment in rest.split('/') { + if segment.is_empty() { + continue; + } + if segment == ".." { + let inst = self.inner.get_by_ref(current_ref)?; + let parent = inst.parent(); + if parent.is_none() { + return None; + } + current_ref = parent; + continue; + } + + let current = self.inner.get_by_ref(current_ref)?; + let children = current.children(); + + let mut found = false; + for &child_ref in children { + let fs_name = self.filesystem_name_for(child_ref); + if fs_name.eq_ignore_ascii_case(segment) { + current_ref = child_ref; + found = true; + break; + } + } + + if !found { + for &child_ref in children { + let child = self.inner.get_by_ref(child_ref)?; + if child.name.eq_ignore_ascii_case(segment) { + current_ref = child_ref; + found = true; + break; + } + } + } + + if !found { + return None; + } + } + + Some(current_ref) + } + + /// Returns the filesystem name for an instance (the filename or directory + /// name on disk). Derived from `instigating_source` if available, otherwise + /// falls back to the instance name. + pub fn filesystem_name_for(&self, id: Ref) -> String { + if let Some(meta) = self.metadata_map.get(&id) { + if let Some(source) = &meta.instigating_source { + match source { + InstigatingSource::Path(_) => { + if let Some(name) = source.path().file_name().and_then(|f| f.to_str()) { + return name.to_string(); + } + } + InstigatingSource::ProjectNode { .. } => { + // Project-sourced instance: use instance name, not the + // project file path (which would be e.g. "default.project.json5"). + if let Some(inst) = self.inner.get_by_ref(id) { + return inst.name.clone(); + } + } + } + } + } + // Fallback: use instance name + self.inner + .get_by_ref(id) + .map(|inst| inst.name.clone()) + .unwrap_or_default() + } + fn insert_metadata(&mut self, id: Ref, metadata: InstanceMetadata) { for path in &metadata.relevant_paths { self.path_to_ids.insert(path.clone(), id); diff --git a/src/snapshot_middleware/dir.rs b/src/snapshot_middleware/dir.rs index e26084cd..a19fbdd5 100644 --- a/src/snapshot_middleware/dir.rs +++ b/src/snapshot_middleware/dir.rs @@ -5,14 +5,11 @@ use std::{ use anyhow::Context; use memofs::{DirEntry, Vfs}; +use rbx_dom_weak::types::Ref; use crate::{ snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot, InstigatingSource}, - snapshot_middleware::Middleware, - syncback::{ - hash_instance, name_needs_slugify, slugify_name, strip_middleware_extension, FsSnapshot, - SyncbackReturn, SyncbackSnapshot, - }, + syncback::{hash_instance, FsSnapshot, SyncbackReturn, SyncbackSnapshot}, }; use super::{meta_file::DirectoryMetadata, snapshot_from_vfs}; @@ -52,19 +49,41 @@ pub fn snapshot_dir_no_meta( .all(|rule| rule.passes(child.path())) }; - let mut snapshot_children = Vec::new(); - - for entry in vfs.read_dir(path)? { - let entry = entry?; - - if !passes_filter_rules(&entry) { - continue; + let snapshot_children = if std::env::var("ATLAS_SEQUENTIAL").is_ok() { + let mut children = Vec::new(); + for entry in vfs.read_dir(path)? { + let entry = entry?; + if !passes_filter_rules(&entry) { + continue; + } + if let Some(child_snapshot) = snapshot_from_vfs(context, vfs, entry.path())? { + children.push(child_snapshot); + } } + children + } else { + use rayon::prelude::*; - if let Some(child_snapshot) = snapshot_from_vfs(context, vfs, entry.path())? { - snapshot_children.push(child_snapshot); + let entries: Vec<_> = vfs.read_dir(path)?.filter_map(|e| e.ok()).collect(); + + let results: Vec>> = entries + .par_iter() + .map(|entry| { + if !passes_filter_rules(entry) { + return Ok(None); + } + snapshot_from_vfs(context, vfs, entry.path()) + }) + .collect(); + + let mut children = Vec::new(); + for result in results { + if let Some(snapshot) = result? { + children.push(snapshot); + } } - } + children + }; let normalized_path = vfs.canonicalize(path)?; let relevant_paths = vec![ @@ -137,205 +156,173 @@ pub fn syncback_dir<'sync>( pub fn syncback_dir_no_meta<'sync>( snapshot: &SyncbackSnapshot<'sync>, ) -> anyhow::Result> { + use crate::syncback::matching::match_children; + let new_inst = snapshot.new_inst(); let mut children = Vec::new(); let mut removed_children = Vec::new(); - // taken_names tracks claimed bare slugs (without extensions) for dedup. - // Pre-seeded from old tree children's slugified instance names so that - // new-only children correctly dedup against existing siblings, regardless - // of iteration order (see plan: fix_stem-level_dedup §2). + // taken_names tracks claimed full filesystem name components (slug + + // extension for files, directory name for dirs), lowercased. let mut taken_names: HashSet = HashSet::new(); - // Detect duplicate child names (case-insensitive for file system safety). - // We skip duplicates instead of failing, tracking them in stats. - let mut child_name_counts: HashMap = HashMap::new(); - for child_ref in new_inst.children() { - let child = snapshot.get_new_instance(*child_ref).unwrap(); - let lower_name = child.name.to_lowercase(); - *child_name_counts.entry(lower_name).or_insert(0) += 1; - } - - let duplicate_names: HashSet = child_name_counts - .into_iter() - .filter(|(_, count)| *count > 1) - .map(|(name, _)| name) - .collect(); - - // Record duplicate names in stats tracker - if !duplicate_names.is_empty() { - let inst_path = crate::syncback::inst_path(snapshot.new_tree(), snapshot.new); - // Count total instances being skipped (sum of all duplicates) - let mut total_skipped = 0; - for child_ref in new_inst.children() { - let child = snapshot.get_new_instance(*child_ref).unwrap(); - if duplicate_names.contains(&child.name.to_lowercase()) { - total_skipped += 1; - } - } - let duplicate_list: Vec<&str> = duplicate_names.iter().map(|s| s.as_str()).collect(); - snapshot - .stats() - .record_duplicate_names_batch(&inst_path, &duplicate_list, total_skipped); - } - - if let Some(old_inst) = snapshot.old_inst() { - let mut old_child_map = HashMap::with_capacity(old_inst.children().len()); - for child in old_inst.children() { - let inst = snapshot.get_old_instance(*child).unwrap(); - old_child_map.insert(inst.name(), inst); - } - - // Pre-seed taken_names from old children's actual filesystem dedup keys - // so that new-only children correctly dedup against existing siblings. - // We derive keys from relevant_paths (the real filename on disk) rather - // than slugifying instance names, because an old instance may already - // have a tilde suffix (e.g. A_B~1.luau from prior dedup) that the bare - // slug wouldn't capture. - // Only in incremental mode: in clean mode, old_ref is forced to None - // for all children, so every child is treated as new and pre-seeding - // would cause false collisions (spurious ~1 suffixes). - if snapshot.data.is_incremental() { - for inst in old_child_map.values() { - if let Some(path) = inst.metadata().relevant_paths.first() { - if let Some(filename) = path.file_name().and_then(|f| f.to_str()) { - let middleware = inst.metadata().middleware.unwrap_or(Middleware::Dir); - let dedup_key = - strip_middleware_extension(filename, middleware).to_lowercase(); - taken_names.insert(dedup_key); - } - } else { - // Fallback (shouldn't happen for old instances with disk presence) - let name = inst.name(); - let slug = if name_needs_slugify(name) { - slugify_name(name).to_lowercase() - } else { - name.to_lowercase() - }; - taken_names.insert(slug); - } - } - } - - // Sort children alphabetically for deterministic dedup ordering. - // Without this, DOM iteration order determines which sibling gets - // the base slug vs ~1, causing non-deterministic output across - // different serializations of the same place. - let mut sorted_children: Vec<_> = new_inst - .children() - .iter() - .map(|r| (*r, snapshot.get_new_instance(*r).unwrap().name.clone())) - .collect(); - sorted_children.sort_by(|(_, a), (_, b)| a.cmp(b)); - - for (new_child_ref, _) in &sorted_children { - let new_child = snapshot.get_new_instance(*new_child_ref).unwrap(); - - // Skip children with duplicate names - cannot reliably sync them - if duplicate_names.contains(&new_child.name.to_lowercase()) { - old_child_map.remove(new_child.name.as_str()); - continue; - } - - // Skip instances of ignored classes - if snapshot.should_ignore_class(&new_child.class) { - // Also remove from old_child_map so it won't be marked as removed - old_child_map.remove(new_child.name.as_str()); + // Filter new children: exclude ignored classes and ignoreTrees. + let eligible_new: Vec = new_inst + .children() + .iter() + .copied() + .filter(|r| { + let child = snapshot.get_new_instance(*r).unwrap(); + if snapshot.should_ignore_class(&child.class) { log::debug!( "Skipping instance {} because its class {} is ignored", - new_child.name, - new_child.class + child.name, + child.class ); - continue; + return false; } - // Skip instances matching ignoreTrees - if snapshot.should_ignore_tree(*new_child_ref) { - old_child_map.remove(new_child.name.as_str()); - continue; + if snapshot.should_ignore_tree(*r) { + return false; } - if let Some(old_child) = old_child_map.remove(new_child.name.as_str()) { - if old_child.metadata().relevant_paths.is_empty() { + true + }) + .collect(); + + if let Some(old_inst) = snapshot.old_inst() { + // Build old_ref → InstanceWithMeta lookup for metadata access. + let mut old_meta_map: HashMap = HashMap::new(); + for &child_ref in old_inst.children() { + let inst = snapshot.get_old_instance(child_ref).unwrap(); + old_meta_map.insert(child_ref, inst); + } + + // Filter old children: exclude those without disk presence, from + // project files, or with ignored classes. + let eligible_old: Vec = old_inst + .children() + .iter() + .copied() + .filter(|r| { + let inst = old_meta_map.get(r).unwrap(); + if inst.metadata().relevant_paths.is_empty() { log::debug!( - "Skipping instance {} because it doesn't exist on the disk", - old_child.name() + "Excluding old instance {} from matching (no disk presence)", + inst.name() ); - continue; - } else if matches!( - old_child.metadata().instigating_source, + return false; + } + if matches!( + inst.metadata().instigating_source, Some(InstigatingSource::ProjectNode { .. }) ) { log::debug!( - "Skipping instance {} because it originates in a project file", - old_child.name() + "Excluding old instance {} from matching (project file)", + inst.name() ); - continue; + return false; + } + if snapshot.should_ignore_class(inst.class_name().as_str()) { + return false; + } + true + }) + .collect(); + + // Run the 3-pass matching algorithm to pair new ↔ old children. + let matching_session = crate::syncback::matching::MatchingSession::new(); + let match_result = match_children( + &eligible_new, + &eligible_old, + snapshot.new_tree(), + snapshot.old_tree(), + None, // TODO: pass precomputed hashes for better similarity + None, + &matching_session, + ); + + // Pre-seed taken_names from ALL matched old children's filesystem + // names so that unmatched new children correctly dedup. + if snapshot.data.is_incremental() { + for &(_, old_ref) in &match_result.matched { + if let Some(inst) = old_meta_map.get(&old_ref) { + if let Some(path) = inst.metadata().relevant_paths.first() { + if let Some(filename) = path.file_name().and_then(|f| f.to_str()) { + taken_names.insert(filename.to_lowercase()); + } + } + } + } + // Also seed from unmatched old (they still occupy filesystem names). + for &old_ref in &match_result.unmatched_old { + if let Some(inst) = old_meta_map.get(&old_ref) { + if let Some(path) = inst.metadata().relevant_paths.first() { + if let Some(filename) = path.file_name().and_then(|f| f.to_str()) { + taken_names.insert(filename.to_lowercase()); + } + } } - // This child exists in both doms. Pass it on. - let (child_snap, _needs_meta, dedup_key) = snapshot.with_joined_path( - *new_child_ref, - Some(old_child.id()), - &taken_names, - )?; - taken_names.insert(dedup_key.to_lowercase()); - children.push(child_snap); - } else { - // The child only exists in the the new dom - let (child_snap, _needs_meta, dedup_key) = - snapshot.with_joined_path(*new_child_ref, None, &taken_names)?; - taken_names.insert(dedup_key.to_lowercase()); - children.push(child_snap); } } - // Any children that are in the old dom but not the new one are removed. - // Filter out instances of ignored classes and duplicates from removal. - removed_children.extend(old_child_map.into_values().filter(|inst| { - // Don't remove duplicates - we're skipping them entirely - if duplicate_names.contains(&inst.name().to_lowercase()) { - return false; + + // Sort matched pairs by new child name for deterministic ordering. + let mut sorted_matched = match_result.matched; + sorted_matched.sort_by(|(a, _), (b, _)| { + let a_name = &snapshot.get_new_instance(*a).unwrap().name; + let b_name = &snapshot.get_new_instance(*b).unwrap().name; + a_name.cmp(b_name) + }); + + // Process matched pairs: preserve existing filesystem assignment. + for (new_ref, old_ref) in &sorted_matched { + let (child_snap, _needs_meta, dedup_key) = + snapshot.with_joined_path(*new_ref, Some(*old_ref), &taken_names)?; + taken_names.insert(dedup_key.to_lowercase()); + children.push(child_snap); + } + + // Sort unmatched new by name for deterministic dedup ordering. + let mut sorted_unmatched_new = match_result.unmatched_new; + sorted_unmatched_new.sort_by(|a, b| { + let a_name = &snapshot.get_new_instance(*a).unwrap().name; + let b_name = &snapshot.get_new_instance(*b).unwrap().name; + a_name.cmp(b_name) + }); + + // Process unmatched new children: generate new filenames. + for new_ref in &sorted_unmatched_new { + let (child_snap, _needs_meta, dedup_key) = + snapshot.with_joined_path(*new_ref, None, &taken_names)?; + taken_names.insert(dedup_key.to_lowercase()); + children.push(child_snap); + } + + // Unmatched old children are removed (instance was deleted). + // Filter out ignored classes from removal. + for old_ref in match_result.unmatched_old { + if let Some(inst) = old_meta_map.get(&old_ref) { + if !snapshot.should_ignore_class(inst.class_name().as_str()) { + removed_children.push(*inst); + } else { + log::debug!( + "Not removing instance {} because its class {} is ignored", + inst.name(), + inst.class_name() + ); + } } - if snapshot.should_ignore_class(inst.class_name().as_str()) { - log::debug!( - "Not removing instance {} because its class {} is ignored", - inst.name(), - inst.class_name() - ); - false - } else { - true - } - })); + } } else { - // There is no old instance. Just add every child. + // No old instance (clean mode). All children are new. // Sort alphabetically for deterministic dedup ordering. - let mut sorted_children: Vec<_> = new_inst - .children() - .iter() - .map(|r| (*r, snapshot.get_new_instance(*r).unwrap().name.clone())) - .collect(); - sorted_children.sort_by(|(_, a), (_, b)| a.cmp(b)); + let mut sorted_new = eligible_new; + sorted_new.sort_by(|a, b| { + let a_name = &snapshot.get_new_instance(*a).unwrap().name; + let b_name = &snapshot.get_new_instance(*b).unwrap().name; + a_name.cmp(b_name) + }); - for (new_child_ref, _) in &sorted_children { - let new_child = snapshot.get_new_instance(*new_child_ref).unwrap(); - - // Skip children with duplicate names - cannot reliably sync them - if duplicate_names.contains(&new_child.name.to_lowercase()) { - continue; - } - - // Skip instances of ignored classes - if snapshot.should_ignore_class(&new_child.class) { - log::debug!( - "Skipping instance {} because its class {} is ignored", - new_child.name, - new_child.class - ); - continue; - } - // Skip instances matching ignoreTrees - if snapshot.should_ignore_tree(*new_child_ref) { - continue; - } + for new_child_ref in &sorted_new { let (child_snap, _needs_meta, dedup_key) = snapshot.with_joined_path(*new_child_ref, None, &taken_names)?; taken_names.insert(dedup_key.to_lowercase()); @@ -375,6 +362,174 @@ mod test { use memofs::{InMemoryFs, VfsSnapshot}; + #[test] + fn parallel_wide_directory_ordering() { + let children: Vec<_> = (0..50) + .map(|i| { + ( + format!("child_{i:03}.luau"), + VfsSnapshot::file(format!("-- {i}")), + ) + }) + .collect(); + + let mut imfs = InMemoryFs::new(); + imfs.load_snapshot("/wide", VfsSnapshot::dir(children)) + .unwrap(); + let vfs = Vfs::new(imfs); + + let snap = snapshot_dir( + &InstanceContext::default(), + &vfs, + Path::new("/wide"), + "wide", + ) + .unwrap() + .unwrap(); + + let names: Vec<&str> = snap.children.iter().map(|c| c.name.as_ref()).collect(); + let mut sorted = names.clone(); + sorted.sort(); + assert_eq!( + names, sorted, + "Children should be in sorted (read_dir) order" + ); + assert_eq!(snap.children.len(), 50); + } + + #[test] + fn parallel_wide_directory_deterministic_across_runs() { + fn make_children() -> Vec<(String, VfsSnapshot)> { + (0..30) + .map(|i| { + ( + format!("mod_{i:03}.luau"), + VfsSnapshot::file(format!("-- {i}")), + ) + }) + .collect() + } + + fn names_from_fresh() -> Vec { + let mut imfs = InMemoryFs::new(); + imfs.load_snapshot("/root", VfsSnapshot::dir(make_children())) + .unwrap(); + let vfs = Vfs::new(imfs); + let snap = snapshot_dir( + &InstanceContext::default(), + &vfs, + Path::new("/root"), + "root", + ) + .unwrap() + .unwrap(); + snap.children.iter().map(|c| c.name.to_string()).collect() + } + + let baseline = names_from_fresh(); + + for i in 0..20 { + assert_eq!( + baseline, + names_from_fresh(), + "Child order diverged on iteration {i}" + ); + } + } + + #[test] + fn parallel_deep_wide_tree() { + fn make_tree(depth: usize, width: usize) -> VfsSnapshot { + if depth == 0 { + return VfsSnapshot::file("-- leaf"); + } + let children: Vec<_> = (0..width) + .map(|i| { + if i % 3 == 0 { + (format!("dir_{i}"), make_tree(depth - 1, width)) + } else { + ( + format!("file_{i}.luau"), + VfsSnapshot::file(format!("-- d{depth} f{i}")), + ) + } + }) + .collect(); + VfsSnapshot::dir(children) + } + + let mut imfs = InMemoryFs::new(); + imfs.load_snapshot("/deep", make_tree(4, 8)).unwrap(); + let vfs = Vfs::new(imfs); + + let snap = snapshot_dir( + &InstanceContext::default(), + &vfs, + Path::new("/deep"), + "deep", + ) + .unwrap() + .unwrap(); + + fn count_nodes(snap: &InstanceSnapshot) -> usize { + 1 + snap.children.iter().map(count_nodes).sum::() + } + let total = count_nodes(&snap); + assert!(total > 100, "Should have many nodes, got {total}"); + } + + #[test] + fn parallel_mixed_middleware_types() { + let mut imfs = InMemoryFs::new(); + imfs.load_snapshot( + "/mixed", + VfsSnapshot::dir([ + ("script.server.luau", VfsSnapshot::file("-- server")), + ("module.luau", VfsSnapshot::file("return {}")), + ("data.json", VfsSnapshot::file(r#"{"key": "value"}"#)), + ("notes.txt", VfsSnapshot::file("hello")), + ( + "sub", + VfsSnapshot::dir([ + ("inner.luau", VfsSnapshot::file("return 1")), + ( + "deep", + VfsSnapshot::dir([("leaf.luau", VfsSnapshot::file("return 2"))]), + ), + ]), + ), + ("another.luau", VfsSnapshot::file("return 3")), + ( + "model.model.json", + VfsSnapshot::file(r#"{"className": "Part", "properties": {}}"#), + ), + ]), + ) + .unwrap(); + let vfs = Vfs::new(imfs); + + let snap = snapshot_dir( + &InstanceContext::default(), + &vfs, + Path::new("/mixed"), + "mixed", + ) + .unwrap() + .unwrap(); + + let names: Vec<&str> = snap.children.iter().map(|c| c.name.as_ref()).collect(); + assert!(names.contains(&"script"), "Missing server script"); + assert!(names.contains(&"module"), "Missing module script"); + assert!(names.contains(&"data"), "Missing json data"); + assert!(names.contains(&"notes"), "Missing txt"); + assert!(names.contains(&"sub"), "Missing subdirectory"); + assert!(names.contains(&"another"), "Missing another script"); + assert!(names.contains(&"model"), "Missing model"); + + let sub = snap.children.iter().find(|c| c.name == "sub").unwrap(); + assert_eq!(sub.children.len(), 2, "sub should have inner + deep"); + } + #[test] fn empty_folder() { let mut imfs = InMemoryFs::new(); diff --git a/src/snapshot_middleware/mod.rs b/src/snapshot_middleware/mod.rs index 7ea64881..f8ca439e 100644 --- a/src/snapshot_middleware/mod.rs +++ b/src/snapshot_middleware/mod.rs @@ -31,7 +31,7 @@ use serde::{Deserialize, Serialize}; use crate::{ glob::Glob, project::DEFAULT_PROJECT_NAMES, - syncback::{SyncbackReturn, SyncbackSnapshot}, + syncback::{dedup_suffix::strip_dedup_suffix, SyncbackReturn, SyncbackSnapshot}, }; use crate::{ snapshot::{InstanceContext, InstanceSnapshot, SyncRule}, @@ -118,6 +118,7 @@ fn get_dir_middleware<'path>( .expect("Could not extract directory name") .to_str() .ok_or_else(|| anyhow::anyhow!("File name was not valid UTF-8: {}", dir_path.display()))?; + let dir_name = strip_dedup_suffix(dir_name); static INIT_PATHS: OnceLock> = OnceLock::new(); let order = INIT_PATHS.get_or_init(|| { diff --git a/src/snapshot_middleware/project.rs b/src/snapshot_middleware/project.rs index 88deac44..4454dac6 100644 --- a/src/snapshot_middleware/project.rs +++ b/src/snapshot_middleware/project.rs @@ -1,6 +1,6 @@ use std::{ borrow::Cow, - collections::{BTreeMap, HashMap, VecDeque}, + collections::{BTreeMap, HashMap, HashSet, VecDeque}, path::Path, }; @@ -21,8 +21,8 @@ use crate::{ }, snapshot_middleware::Middleware, syncback::{ - filter_properties, inst_path, name_needs_slugify, slugify_name, strip_middleware_extension, - FsSnapshot, SyncbackReturn, SyncbackSnapshot, + filter_properties, inst_path, name_needs_slugify, slugify_name, FsSnapshot, SyncbackReturn, + SyncbackSnapshot, }, variant_eq::variant_eq, RojoRef, @@ -584,30 +584,101 @@ pub fn syncback_project<'sync>( project_node_property_syncback_no_path(snapshot, new_inst, node); } + // Detect duplicate children using case-insensitive comparison. + // Project KEYS are unique by JSON definition; only filesystem children + // under project nodes may collide. Duplicates are skipped with a warning + // for now -- the dedup system (Phase 1c/1d) will handle them properly. + let mut new_duplicate_names: HashSet = HashSet::new(); + { + let mut seen: HashSet = HashSet::new(); + for child_ref in new_inst.children() { + let child = snapshot + .get_new_instance(*child_ref) + .expect("all children of Instances should be in new DOM"); + let lower = child.name.to_lowercase(); + if !seen.insert(lower.clone()) { + new_duplicate_names.insert(lower); + } + } + } + if !new_duplicate_names.is_empty() { + let parent_path = inst_path(snapshot.new_tree(), new_inst.referent()); + for child_ref in new_inst.children() { + let child = snapshot + .get_new_instance(*child_ref) + .expect("all children of Instances should be in new DOM"); + if new_duplicate_names.contains(&child.name.to_lowercase()) { + log::warn!( + "Skipping duplicate-named child '{}' under ProjectNode '{}' -- \ + cannot reliably sync yet. Full path: {}/{}", + child.name, + parent_path, + parent_path, + child.name + ); + } + } + } for child_ref in new_inst.children() { let child = snapshot .get_new_instance(*child_ref) .expect("all children of Instances should be in new DOM"); - if new_child_map.insert(&child.name, child).is_some() { - let parent_path = inst_path(snapshot.new_tree(), new_inst.referent()); - anyhow::bail!( - "Instances that are direct children of an Instance that is made by a project file \ - must have a unique name.\nThe child '{}' is duplicated in the place file.\n\ - Full path: {}/{}", child.name, parent_path, child.name - ); + if !new_duplicate_names.contains(&child.name.to_lowercase()) { + new_child_map.insert(&child.name, child); + } + } + + // Project children with $path are expected in the old DOM (loaded + // from their own path) AND may also appear as filesystem children of + // the parent's scanned directory. Exclude them from duplicate + // detection so the project child loop below can match them. + let project_path_child_names: HashSet = node + .children + .iter() + .filter(|(_, child_node)| child_node.path.is_some()) + .map(|(name, _)| name.to_lowercase()) + .collect(); + + let mut old_duplicate_names: HashSet = HashSet::new(); + { + let mut seen: HashSet = HashSet::new(); + for child_ref in old_inst.children() { + let child = snapshot + .get_old_instance(*child_ref) + .expect("all children of Instances should be in old DOM"); + let lower = child.name().to_lowercase(); + if project_path_child_names.contains(&lower) { + continue; + } + if !seen.insert(lower.clone()) { + old_duplicate_names.insert(lower); + } + } + } + if !old_duplicate_names.is_empty() { + let parent_path = inst_path(snapshot.old_tree(), old_inst.id()); + for child_ref in old_inst.children() { + let child = snapshot + .get_old_instance(*child_ref) + .expect("all children of Instances should be in old DOM"); + if old_duplicate_names.contains(&child.name().to_lowercase()) { + log::warn!( + "Skipping duplicate-named child '{}' under ProjectNode '{}' -- \ + cannot reliably sync yet. Full path: {}/{}", + child.name(), + parent_path, + parent_path, + child.name() + ); + } } } for child_ref in old_inst.children() { let child = snapshot .get_old_instance(*child_ref) .expect("all children of Instances should be in old DOM"); - if old_child_map.insert(child.name(), child).is_some() { - let parent_path = inst_path(snapshot.old_tree(), old_inst.id()); - anyhow::bail!( - "Instances that are direct children of an Instance that is made by a project file \ - must have a unique name.\nThe child '{}' is duplicated on the file system.\n\ - Full path: {}/{}", child.name(), parent_path, child.name() - ); + if !old_duplicate_names.contains(&child.name().to_lowercase()) { + old_child_map.insert(child.name(), child); } } @@ -654,33 +725,27 @@ pub fn syncback_project<'sync>( // All of the children in this loop are by their nature not in the // project, so we just need to run syncback on them. // - // Track taken names (bare slugs) per parent directory so siblings - // under the same parent deduplicate against each other. + // Track taken names (full filesystem name components, lowercased) + // per parent directory so siblings under the same parent deduplicate + // against each other using filesystem-level collision detection. let mut taken_names_per_dir: std::collections::HashMap< std::path::PathBuf, std::collections::HashSet, > = std::collections::HashMap::new(); - // Pre-seed taken_names from old children's actual filesystem dedup keys - // so that new-only children correctly dedup against existing siblings. - // We derive keys from relevant_paths (the real filename on disk) rather - // than slugifying instance names, because an old instance may already - // have a tilde suffix (e.g. A_B~1.luau from prior dedup) that the bare - // slug wouldn't capture. - // Only in incremental mode: in clean mode, old_ref is forced to None - // for all children, so every child is treated as new and pre-seeding - // would cause false collisions (spurious ~1 suffixes). + // Pre-seed taken_names from old children's actual filesystem names + // (full filename including extension) so that new-only children + // correctly dedup against existing siblings. if snapshot.data.is_incremental() { if let Some(parent_path) = ref_to_path_map.get(&new_inst.referent()) { let taken = taken_names_per_dir.entry(parent_path.clone()).or_default(); for old_child in old_child_map.values() { if let Some(path) = old_child.metadata().relevant_paths.first() { if let Some(filename) = path.file_name().and_then(|f| f.to_str()) { - let middleware = - old_child.metadata().middleware.unwrap_or(Middleware::Dir); - let dedup_key = - strip_middleware_extension(filename, middleware).to_lowercase(); - taken.insert(dedup_key); + // Use the full filename (including extension) as the + // dedup key so that "Foo" (dir) and "Foo.server.luau" + // (file) do NOT collide. + taken.insert(filename.to_lowercase()); } } else { // Fallback (shouldn't happen for old instances with disk presence) @@ -698,7 +763,7 @@ pub fn syncback_project<'sync>( // Sort remaining children by name for deterministic dedup ordering. // Without this, HashMap iteration order determines which sibling gets - // the base slug vs ~1, causing non-deterministic output across runs. + // the base slug vs ~2, causing non-deterministic output across runs. let mut remaining_children: Vec<_> = new_child_map.drain().collect(); remaining_children.sort_by(|(name_a, _), (name_b, _)| name_a.cmp(name_b)); for (name, new_child) in remaining_children { diff --git a/src/syncback/dedup_suffix.rs b/src/syncback/dedup_suffix.rs new file mode 100644 index 00000000..d17471b6 --- /dev/null +++ b/src/syncback/dedup_suffix.rs @@ -0,0 +1,304 @@ +//! Dedup suffix lifecycle management. +//! +//! When instances have duplicate names, they receive `~N` suffixes on the +//! filesystem (e.g., `Foo`, `Foo~2`, `Foo~3`). This module provides shared +//! helpers for managing suffix cleanup when instances are deleted. +//! +//! Rules: +//! - **Gap-tolerant:** deleting a suffixed instance does NOT renumber remaining +//! siblings. Gaps (`Foo`, `Foo~3` without `~2`) are harmless. +//! - **Base-name promotion:** when the base-name holder is deleted and multiple +//! siblings remain, the lowest-numbered suffix is promoted to base name. +//! - **Group-to-1 cleanup:** when a deletion reduces the dedup group to exactly +//! 1 remaining instance, its suffix is removed entirely. + +use std::path::{Path, PathBuf}; + +/// Result of analyzing the dedup group after a deletion. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum DedupCleanupAction { + /// No rename needed. Either the deleted instance had a suffix and others + /// remain, or the group is still large enough that no cleanup is required. + None, + + /// The dedup group shrank to exactly 1 remaining instance. + /// Its suffix must be removed (renamed to the clean base name). + RemoveSuffix { + /// The current path (with suffix) that should be renamed. + from: PathBuf, + /// The target path (clean, no suffix). + to: PathBuf, + }, + + /// The base-name holder was deleted and the lowest-numbered suffix must be + /// promoted to take the base name. + PromoteLowest { + /// The current path of the lowest-suffixed sibling. + from: PathBuf, + /// The base-name path (clean, no suffix). + to: PathBuf, + }, +} + +/// Parses a dedup suffix from a filesystem stem. +/// +/// Given `"Foo~3"`, returns `Some(("Foo", 3))`. +/// Given `"Foo"`, returns `None`. +/// Given `"Foo~abc"` (non-numeric), returns `None`. +pub fn parse_dedup_suffix(stem: &str) -> Option<(&str, u32)> { + if let Some(tilde_pos) = stem.rfind('~') { + let base = &stem[..tilde_pos]; + let suffix_str = &stem[tilde_pos + 1..]; + if let Ok(n) = suffix_str.parse::() { + if n > 0 { + return Some((base, n)); + } + } + } + None +} + +/// Strips a `~N` dedup suffix from a filename stem, returning the base name. +/// +/// Given `"Foo~2"`, returns `"Foo"`. +/// Given `"Foo"`, returns `"Foo"` unchanged. +/// Given `"Foo~0"` or `"Foo~abc"`, returns the input unchanged (not valid +/// dedup suffixes). +pub fn strip_dedup_suffix(name: &str) -> &str { + parse_dedup_suffix(name).map_or(name, |(base, _)| base) +} + +/// Builds a dedup'd filename from a base stem, optional suffix number, and +/// extension. +/// +/// - `build_dedup_name("Foo", None, Some("server.luau"))` → `"Foo.server.luau"` +/// - `build_dedup_name("Foo", Some(2), Some("server.luau"))` → `"Foo~2.server.luau"` +/// - `build_dedup_name("Foo", Some(1), None)` → `"Foo~1"` (directory) +/// - `build_dedup_name("Foo", None, None)` → `"Foo"` (directory) +pub fn build_dedup_name(base_stem: &str, suffix: Option, extension: Option<&str>) -> String { + let stem = match suffix { + Some(n) => format!("{base_stem}~{n}"), + None => base_stem.to_string(), + }; + match extension { + Some(ext) => format!("{stem}.{ext}"), + None => stem, + } +} + +/// Determines the cleanup action needed after removing an instance from a +/// dedup group. +/// +/// `remaining_stems` contains the filesystem stems of all siblings that STILL +/// exist after the deletion (not including the deleted one). These should share +/// the same base stem (before `~N`). If the base-name holder was deleted, it +/// should NOT appear in `remaining_stems`. +/// +/// `deleted_was_base` indicates whether the deleted instance held the base name +/// (no suffix). +/// +/// Returns the appropriate cleanup action. +pub fn compute_cleanup_action( + base_stem: &str, + extension: Option<&str>, + remaining_stems: &[String], + deleted_was_base: bool, + parent_dir: &Path, +) -> DedupCleanupAction { + match remaining_stems.len() { + 0 => { + // No siblings remain -- nothing to clean up. + DedupCleanupAction::None + } + 1 => { + // Group shrank to 1: remove the survivor's suffix entirely. + let survivor = &remaining_stems[0]; + let from_name = build_dedup_name( + base_stem, + parse_dedup_suffix(survivor).map(|(_, n)| n), + extension, + ); + let to_name = build_dedup_name(base_stem, None, extension); + if from_name == to_name { + // Survivor already has the clean name (was the base holder + // and only one other was deleted). + DedupCleanupAction::None + } else { + DedupCleanupAction::RemoveSuffix { + from: parent_dir.join(&from_name), + to: parent_dir.join(&to_name), + } + } + } + _ => { + // Multiple siblings remain. + if deleted_was_base { + // The base-name holder was deleted. Promote the lowest suffix. + let mut suffix_numbers: Vec = remaining_stems + .iter() + .filter_map(|s| parse_dedup_suffix(s).map(|(_, n)| n)) + .collect(); + suffix_numbers.sort(); + + if let Some(&lowest) = suffix_numbers.first() { + let from_name = build_dedup_name(base_stem, Some(lowest), extension); + let to_name = build_dedup_name(base_stem, None, extension); + DedupCleanupAction::PromoteLowest { + from: parent_dir.join(&from_name), + to: parent_dir.join(&to_name), + } + } else { + // All remaining are base-name holders? Shouldn't happen. + DedupCleanupAction::None + } + } else { + // A suffixed instance was deleted. Gap-tolerant: no rename. + DedupCleanupAction::None + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::path::PathBuf; + + #[test] + fn parse_suffix_basic() { + assert_eq!(parse_dedup_suffix("Foo~1"), Some(("Foo", 1))); + assert_eq!(parse_dedup_suffix("Foo~2"), Some(("Foo", 2))); + assert_eq!(parse_dedup_suffix("Foo~10"), Some(("Foo", 10))); + } + + #[test] + fn parse_suffix_none() { + assert_eq!(parse_dedup_suffix("Foo"), None); + assert_eq!(parse_dedup_suffix("Foo~0"), None); // 0 is not valid + assert_eq!(parse_dedup_suffix("Foo~abc"), None); + assert_eq!(parse_dedup_suffix("Foo~"), None); + } + + #[test] + fn parse_suffix_complex_stems() { + assert_eq!(parse_dedup_suffix("A_B~3"), Some(("A_B", 3))); + assert_eq!(parse_dedup_suffix("My Script~1"), Some(("My Script", 1))); + } + + #[test] + fn strip_suffix_basic() { + assert_eq!(strip_dedup_suffix("Foo~1"), "Foo"); + assert_eq!(strip_dedup_suffix("Foo~2"), "Foo"); + assert_eq!(strip_dedup_suffix("Foo~10"), "Foo"); + } + + #[test] + fn strip_suffix_no_op() { + assert_eq!(strip_dedup_suffix("Foo"), "Foo"); + assert_eq!(strip_dedup_suffix("Foo~0"), "Foo~0"); + assert_eq!(strip_dedup_suffix("Foo~abc"), "Foo~abc"); + assert_eq!(strip_dedup_suffix("Foo~"), "Foo~"); + assert_eq!(strip_dedup_suffix(""), ""); + } + + #[test] + fn build_name_file() { + assert_eq!( + build_dedup_name("Foo", None, Some("server.luau")), + "Foo.server.luau" + ); + assert_eq!( + build_dedup_name("Foo", Some(1), Some("server.luau")), + "Foo~1.server.luau" + ); + assert_eq!(build_dedup_name("Foo", Some(2), Some("luau")), "Foo~2.luau"); + } + + #[test] + fn build_name_dir() { + assert_eq!(build_dedup_name("Foo", None, None), "Foo"); + assert_eq!(build_dedup_name("Foo", Some(1), None), "Foo~1"); + } + + #[test] + fn cleanup_gap_tolerant() { + // Delete ~1 from {Foo, Foo~1, Foo~2}: no rename needed. + let remaining = vec!["Foo".to_string(), "Foo~2".to_string()]; + let action = compute_cleanup_action("Foo", None, &remaining, false, Path::new("/parent")); + assert_eq!(action, DedupCleanupAction::None); + } + + #[test] + fn cleanup_group_to_one() { + // Delete base from {Foo, Foo~1}: one remains, remove suffix. + let remaining = vec!["Foo~1".to_string()]; + let action = compute_cleanup_action("Foo", None, &remaining, true, Path::new("/parent")); + assert_eq!( + action, + DedupCleanupAction::RemoveSuffix { + from: PathBuf::from("/parent/Foo~1"), + to: PathBuf::from("/parent/Foo"), + } + ); + } + + #[test] + fn cleanup_group_to_one_file() { + // Delete base from {Foo.luau, Foo~1.luau}: one remains, remove suffix. + let remaining = vec!["Foo~1".to_string()]; + let action = + compute_cleanup_action("Foo", Some("luau"), &remaining, true, Path::new("/parent")); + assert_eq!( + action, + DedupCleanupAction::RemoveSuffix { + from: PathBuf::from("/parent/Foo~1.luau"), + to: PathBuf::from("/parent/Foo.luau"), + } + ); + } + + #[test] + fn cleanup_base_deleted_promote_lowest() { + // Delete base from {Foo, Foo~1, Foo~2}: promote ~1 to base. + let remaining = vec!["Foo~1".to_string(), "Foo~2".to_string()]; + let action = compute_cleanup_action("Foo", None, &remaining, true, Path::new("/parent")); + assert_eq!( + action, + DedupCleanupAction::PromoteLowest { + from: PathBuf::from("/parent/Foo~1"), + to: PathBuf::from("/parent/Foo"), + } + ); + } + + #[test] + fn cleanup_base_deleted_promote_with_gap() { + // Delete base from {Foo, Foo~2, Foo~5} (gap at ~1): promote ~2 to base. + let remaining = vec!["Foo~2".to_string(), "Foo~5".to_string()]; + let action = compute_cleanup_action("Foo", None, &remaining, true, Path::new("/parent")); + assert_eq!( + action, + DedupCleanupAction::PromoteLowest { + from: PathBuf::from("/parent/Foo~2"), + to: PathBuf::from("/parent/Foo"), + } + ); + } + + #[test] + fn cleanup_suffix_deleted_no_action() { + // Delete ~1 from {Foo, Foo~1}: group shrinks to 1. + // Since deleted_was_base=false and only 1 remains, remove its suffix. + // Wait - if ~1 is deleted and only Foo remains, Foo has no suffix. No rename needed. + let remaining = vec!["Foo".to_string()]; + let action = compute_cleanup_action("Foo", None, &remaining, false, Path::new("/parent")); + // "Foo" is already the clean name, so no action. + assert_eq!(action, DedupCleanupAction::None); + } + + #[test] + fn cleanup_no_remaining() { + let action = compute_cleanup_action("Foo", None, &[], false, Path::new("/parent")); + assert_eq!(action, DedupCleanupAction::None); + } +} diff --git a/src/syncback/file_names.rs b/src/syncback/file_names.rs index 9f8e1b43..076feef9 100644 --- a/src/syncback/file_names.rs +++ b/src/syncback/file_names.rs @@ -15,10 +15,13 @@ use crate::{snapshot::InstanceWithMeta, snapshot_middleware::Middleware}; /// - `filename`: The full filesystem name (including extension for file middleware). /// - `needs_meta_name`: `true` when the filesystem name differs from the instance /// name (meaning a `name` field must be written in metadata). -/// - `dedup_key`: The bare slug (without extension) that callers must insert into -/// `taken_names`. This is the name-level identifier used for collision detection. -/// Callers should **always** use `dedup_key` (not `filename`) when accumulating -/// into `taken_names`. +/// - `dedup_key`: The **full filesystem name component** (slug + extension for +/// files, directory name for dirs). Callers should insert +/// `dedup_key.to_lowercase()` into `taken_names`. +/// +/// This means `Folder "Foo"` (key `"Foo"`) and `Script "Foo"` (key +/// `"Foo.server.luau"`) do NOT collide, while two `Folder "Foo"` instances +/// (both key `"Foo"`) DO collide and receive dedup suffixes. /// /// If `old_inst` exists, its existing path is preserved (incremental mode). /// For new instances, names with forbidden chars are slugified and deduplicated @@ -35,10 +38,9 @@ pub fn name_for_inst<'a>( .file_name() .and_then(|s| s.to_str()) .context("sources on the file system should be valid unicode and not be stubs")?; - // Derive dedup_key by stripping the middleware extension from the - // filename. For Dir middleware the extension is empty so this is a - // no-op. This keeps extension logic inside this function. - let dedup_key = strip_middleware_extension(name, middleware); + // dedup_key is the full filesystem name (including extension). + // For Dir middleware the filename IS the directory name (no extension). + let dedup_key = name.to_string(); Ok((Cow::Borrowed(name), false, dedup_key)) } else { anyhow::bail!( @@ -55,32 +57,30 @@ pub fn name_for_inst<'a>( new_inst.name.clone() }; - match middleware { + let is_dir = matches!( + middleware, Middleware::Dir - | Middleware::CsvDir - | Middleware::ServerScriptDir - | Middleware::ClientScriptDir - | Middleware::ModuleScriptDir - | Middleware::PluginScriptDir - | Middleware::LocalScriptDir - | Middleware::LegacyScriptDir => { - let deduped = deduplicate_name(&base, taken_names); - let needs_meta = needs_slugify || deduped != base; - let dedup_key = deduped.clone(); - Ok((Cow::Owned(deduped), needs_meta, dedup_key)) - } - _ => { - let extension = extension_for_middleware(middleware); - let deduped = deduplicate_name(&base, taken_names); - let needs_meta = needs_slugify || deduped != base; - let dedup_key = deduped.clone(); - Ok(( - Cow::Owned(format!("{deduped}.{extension}")), - needs_meta, - dedup_key, - )) - } - } + | Middleware::CsvDir + | Middleware::ServerScriptDir + | Middleware::ClientScriptDir + | Middleware::ModuleScriptDir + | Middleware::PluginScriptDir + | Middleware::LocalScriptDir + | Middleware::LegacyScriptDir + ); + + let extension = if is_dir { + None + } else { + Some(extension_for_middleware(middleware)) + }; + + let (_deduped_slug, full_fs_name) = + deduplicate_name_with_ext(&base, extension, taken_names); + let needs_meta = needs_slugify; + + let filename = full_fs_name.clone(); + Ok((Cow::Owned(filename), needs_meta, full_fs_name)) } } @@ -268,18 +268,52 @@ pub fn slugify_name(name: &str) -> String { result } -/// Appends ~1, ~2, etc. to avoid collisions. Returns the name as-is if -/// unclaimed. +/// Appends ~2, ~3, etc. to the slug to avoid filesystem-level collisions. +/// Returns `(deduped_slug, full_fs_name)`. +/// +/// `extension` is `None` for directory middleware, `Some("server.luau")` etc. +/// for file middleware. The collision check uses the full filesystem name +/// (slug + extension) so that `Folder "Foo"` (key `"foo"`) and `Script "Foo"` +/// (key `"foo.server.luau"`) do NOT collide. /// /// Comparisons are **case-insensitive** because Windows and macOS have /// case-insensitive filesystems. `taken_names` must contain **lowercased** -/// entries for this to work correctly. +/// full filesystem name entries for this to work correctly. +pub fn deduplicate_name_with_ext( + base: &str, + extension: Option<&str>, + taken_names: &std::collections::HashSet, +) -> (String, String) { + let build_full = |slug: &str| -> String { + match extension { + Some(ext) => format!("{slug}.{ext}"), + None => slug.to_string(), + } + }; + + let full_name = build_full(base); + if !taken_names.contains(&full_name.to_lowercase()) { + return (base.to_string(), full_name); + } + for i in 2.. { + let candidate_slug = format!("{base}~{i}"); + let candidate_full = build_full(&candidate_slug); + if !taken_names.contains(&candidate_full.to_lowercase()) { + return (candidate_slug, candidate_full); + } + } + unreachable!() +} + +/// Legacy wrapper: appends ~2, ~3, etc. to avoid collisions using bare slug +/// as the collision key. Used by callers that don't yet use full filesystem +/// name dedup (e.g., change_processor.rs renames). pub fn deduplicate_name(base: &str, taken_names: &std::collections::HashSet) -> String { let base_lower = base.to_lowercase(); if !taken_names.contains(&base_lower) { return base.to_string(); } - for i in 1.. { + for i in 2.. { let candidate = format!("{base}~{i}"); if !taken_names.contains(&candidate.to_lowercase()) { return candidate; @@ -536,7 +570,7 @@ mod tests { #[test] fn dedup_single_collision() { let taken: HashSet = ["foo".to_string()].into_iter().collect(); - assert_eq!(deduplicate_name("Foo", &taken), "Foo~1"); + assert_eq!(deduplicate_name("Foo", &taken), "Foo~2"); } #[test] @@ -556,14 +590,14 @@ mod tests { #[test] fn dedup_gap_in_suffixes() { - let taken: HashSet = ["foo", "foo~2"].into_iter().map(String::from).collect(); - assert_eq!(deduplicate_name("Foo", &taken), "Foo~1"); + let taken: HashSet = ["foo", "foo~3"].into_iter().map(String::from).collect(); + assert_eq!(deduplicate_name("Foo", &taken), "Foo~2"); } #[test] fn dedup_natural_vs_slug_collision() { let taken: HashSet = ["hey_bro".to_string()].into_iter().collect(); - assert_eq!(deduplicate_name("Hey_Bro", &taken), "Hey_Bro~1"); + assert_eq!(deduplicate_name("Hey_Bro", &taken), "Hey_Bro~2"); } #[test] @@ -576,16 +610,16 @@ mod tests { fn dedup_case_insensitive() { // "foo" is taken, "Foo" should collide (case-insensitive filesystem) let taken: HashSet = ["foo".to_string()].into_iter().collect(); - assert_eq!(deduplicate_name("Foo", &taken), "Foo~1"); - assert_eq!(deduplicate_name("FOO", &taken), "FOO~1"); - assert_eq!(deduplicate_name("foo", &taken), "foo~1"); + assert_eq!(deduplicate_name("Foo", &taken), "Foo~2"); + assert_eq!(deduplicate_name("FOO", &taken), "FOO~2"); + assert_eq!(deduplicate_name("foo", &taken), "foo~2"); } #[test] fn dedup_case_only_difference() { - // Two instances: "MyScript" and "myscript" - second must get ~1 + // Two instances: "MyScript" and "myscript" - second must get ~2 let taken: HashSet = ["myscript".to_string()].into_iter().collect(); - assert_eq!(deduplicate_name("MyScript", &taken), "MyScript~1"); + assert_eq!(deduplicate_name("MyScript", &taken), "MyScript~2"); } // ── name_needs_slugify ────────────────────────────────────────────── @@ -794,12 +828,15 @@ mod tests { let dom = make_inst("Foo", "ModuleScript"); let child_ref = dom.root().children()[0]; let child = dom.get_by_ref(child_ref).unwrap(); - let taken: HashSet = ["foo".to_string()].into_iter().collect(); + let taken: HashSet = ["foo.luau".to_string()].into_iter().collect(); let (filename, needs_meta, _dk) = name_for_inst(Middleware::ModuleScript, child, None, &taken).unwrap(); - assert_eq!(filename.as_ref(), "Foo~1.luau"); - assert!(needs_meta, "deduped name differs from original"); + assert_eq!(filename.as_ref(), "Foo~2.luau"); + assert!( + !needs_meta, + "dedup-only: forward sync strips ~N automatically" + ); } #[test] @@ -811,21 +848,24 @@ mod tests { let (filename, needs_meta, _dk) = name_for_inst(Middleware::Dir, child, None, &taken).unwrap(); - assert_eq!(filename.as_ref(), "Stuff~1"); - assert!(needs_meta); + assert_eq!(filename.as_ref(), "Stuff~2"); + assert!( + !needs_meta, + "dedup-only: forward sync strips ~N automatically" + ); } #[test] fn name_for_inst_slug_plus_dedup() { - // Name with forbidden chars AND collision with existing slug + // Name with forbidden chars AND collision with existing full filesystem name let dom = make_inst("Hey/Bro", "ModuleScript"); let child_ref = dom.root().children()[0]; let child = dom.get_by_ref(child_ref).unwrap(); - let taken: HashSet = ["hey_bro".to_string()].into_iter().collect(); + let taken: HashSet = ["hey_bro.luau".to_string()].into_iter().collect(); let (filename, needs_meta, _dk) = name_for_inst(Middleware::ModuleScript, child, None, &taken).unwrap(); - assert_eq!(filename.as_ref(), "Hey_Bro~1.luau"); + assert_eq!(filename.as_ref(), "Hey_Bro~2.luau"); assert!(needs_meta); } @@ -1112,7 +1152,7 @@ mod tests { fn nfi_slug_collides_with_existing_dir() { // "Hey/Bro" slugifies to "Hey_Bro", which is already taken as a dir name let (f, m) = nfi("Hey/Bro", Middleware::Dir, &["hey_bro"]); - assert_eq!(f, "Hey_Bro~1"); + assert_eq!(f, "Hey_Bro~2"); assert!(m); } @@ -1120,15 +1160,15 @@ mod tests { fn nfi_slug_collides_with_natural_dir() { // Dir "Hey_Bro" exists naturally, then "Hey:Bro" slugifies to same let (f, m) = nfi("Hey:Bro", Middleware::Dir, &["hey_bro"]); - assert_eq!(f, "Hey_Bro~1"); + assert_eq!(f, "Hey_Bro~2"); assert!(m); } #[test] fn nfi_two_slugs_collide_dir() { - // Both "A/B" and "A:B" slugify to "A_B". First taken, second gets ~1 + // Both "A/B" and "A:B" slugify to "A_B". First taken, second gets ~2 let (f, m) = nfi("A:B", Middleware::Dir, &["a_b"]); - assert_eq!(f, "A_B~1"); + assert_eq!(f, "A_B~2"); assert!(m); } @@ -1139,29 +1179,26 @@ mod tests { } #[test] - fn nfi_clean_name_dedup_still_needs_meta_dir() { - // Clean name "Foo", but "foo" is already taken → dedup adds ~1 → needs meta + fn nfi_clean_name_dedup_no_meta_dir() { + // Clean name "Foo", but "foo" is already taken → dedup adds ~2. + // Forward sync strips ~N, so no meta name needed. let (f, m) = nfi("Foo", Middleware::Dir, &["foo"]); - assert_eq!(f, "Foo~1"); - assert!( - m, - "dedup suffix means filesystem name differs from instance name" - ); + assert_eq!(f, "Foo~2"); + assert!(!m, "dedup-only: forward sync strips ~N automatically"); } #[test] fn nfi_case_collision_dir() { let (f, m) = nfi("Assets", Middleware::Dir, &["assets"]); - assert_eq!(f, "Assets~1"); - assert!(m); + assert_eq!(f, "Assets~2"); + assert!(!m, "dedup-only: forward sync strips ~N automatically"); } #[test] fn nfi_slug_collision_file_middleware() { - // For file middleware, dedup compares bare slug against taken. - // Using bare names to verify the dedup logic works. - let (f, m) = nfi("Hey/Bro", Middleware::ModuleScript, &["hey_bro"]); - assert_eq!(f, "Hey_Bro~1.luau"); + // For file middleware, dedup compares full filesystem name against taken. + let (f, m) = nfi("Hey/Bro", Middleware::ModuleScript, &["hey_bro.luau"]); + assert_eq!(f, "Hey_Bro~2.luau"); assert!(m); } @@ -1190,16 +1227,16 @@ mod tests { let r = process_siblings_dir(&["Foo", "Foo"]); assert_eq!(r[0].1, "Foo"); assert!(!r[0].2); - assert_eq!(r[1].1, "Foo~1"); - assert!(r[1].2); + assert_eq!(r[1].1, "Foo~2"); + assert!(!r[1].2, "dedup-only: forward sync strips ~N automatically"); } #[test] fn ordering_three_identical_names() { let r = process_siblings_dir(&["X", "X", "X"]); assert_eq!(r[0].1, "X"); - assert_eq!(r[1].1, "X~1"); - assert_eq!(r[2].1, "X~2"); + assert_eq!(r[1].1, "X~2"); + assert_eq!(r[2].1, "X~3"); } #[test] @@ -1208,7 +1245,7 @@ mod tests { let r = process_siblings_dir(&["Hey_Bro", "Hey/Bro"]); assert_eq!(r[0].1, "Hey_Bro"); assert!(!r[0].2); - assert_eq!(r[1].1, "Hey_Bro~1"); + assert_eq!(r[1].1, "Hey_Bro~2"); assert!(r[1].2); } @@ -1218,8 +1255,8 @@ mod tests { let r = process_siblings_dir(&["Hey/Bro", "Hey_Bro"]); assert_eq!(r[0].1, "Hey_Bro"); assert!(r[0].2); - assert_eq!(r[1].1, "Hey_Bro~1"); - assert!(r[1].2, "dedup suffix → needs meta even for natural name"); + assert_eq!(r[1].1, "Hey_Bro~2"); + assert!(!r[1].2, "dedup-only: natural name, forward sync strips ~N"); } #[test] @@ -1227,10 +1264,10 @@ mod tests { let r = process_siblings_dir(&["Script", "script", "SCRIPT"]); assert_eq!(r[0].1, "Script"); assert!(!r[0].2); - assert_eq!(r[1].1, "script~1"); - assert!(r[1].2); - assert_eq!(r[2].1, "SCRIPT~2"); - assert!(r[2].2); + assert_eq!(r[1].1, "script~2"); + assert!(!r[1].2, "dedup-only: forward sync strips ~N automatically"); + assert_eq!(r[2].1, "SCRIPT~3"); + assert!(!r[2].2, "dedup-only: forward sync strips ~N automatically"); } #[test] @@ -1238,8 +1275,8 @@ mod tests { // All three slugify to "A_B" let r = process_siblings_dir(&["A/B", "A:B", "A*B"]); assert_eq!(r[0].1, "A_B"); - assert_eq!(r[1].1, "A_B~1"); - assert_eq!(r[2].1, "A_B~2"); + assert_eq!(r[1].1, "A_B~2"); + assert_eq!(r[2].1, "A_B~3"); assert!(r[0].2); assert!(r[1].2); assert!(r[2].2); @@ -1261,8 +1298,8 @@ mod tests { let r = process_siblings_dir(&["CON", "CON_"]); assert_eq!(r[0].1, "CON_"); assert!(r[0].2); - assert_eq!(r[1].1, "CON_~1"); - assert!(r[1].2); + assert_eq!(r[1].1, "CON_~2"); + assert!(!r[1].2, "dedup-only: natural name, forward sync strips ~N"); } #[test] @@ -1270,18 +1307,18 @@ mod tests { let r = process_siblings_dir(&[ "Utils", // first claim "utils", // case collision with Utils - "Utils", // exact collision (already taken + ~1 taken) + "Utils", // exact collision (already taken + ~2 taken) "UTILS", // case collision "Uti/ls", // different slug "Uti_ls", no collision ]); assert_eq!(r[0].1, "Utils"); assert!(!r[0].2); - assert_eq!(r[1].1, "utils~1"); - assert!(r[1].2); - assert_eq!(r[2].1, "Utils~2"); - assert!(r[2].2); - assert_eq!(r[3].1, "UTILS~3"); - assert!(r[3].2); + assert_eq!(r[1].1, "utils~2"); + assert!(!r[1].2, "dedup-only: forward sync strips ~N"); + assert_eq!(r[2].1, "Utils~3"); + assert!(!r[2].2, "dedup-only: forward sync strips ~N"); + assert_eq!(r[3].1, "UTILS~4"); + assert!(!r[3].2, "dedup-only: forward sync strips ~N"); assert_eq!(r[4].1, "Uti_ls"); assert!(r[4].2); } @@ -1291,20 +1328,20 @@ mod tests { // All six slugify to "X_Y" as dirs let r = process_siblings_dir(&["X/Y", "X:Y", "X*Y", "X?Y", "XY"]); assert_eq!(r[0].1, "X_Y"); - assert_eq!(r[1].1, "X_Y~1"); - assert_eq!(r[2].1, "X_Y~2"); - assert_eq!(r[3].1, "X_Y~3"); - assert_eq!(r[4].1, "X_Y~4"); - assert_eq!(r[5].1, "X_Y~5"); + assert_eq!(r[1].1, "X_Y~2"); + assert_eq!(r[2].1, "X_Y~3"); + assert_eq!(r[3].1, "X_Y~4"); + assert_eq!(r[4].1, "X_Y~5"); + assert_eq!(r[5].1, "X_Y~6"); } #[test] fn ordering_tilde_in_name_vs_dedup_suffix() { - // "Foo~1" (tilde slugified to "Foo_1") vs dedup suffix "Foo~1" - // The slugified "Foo~1" → "Foo_1" which is different from dedup "Foo~1" + // "Foo~1" (tilde slugified to "Foo_1") vs dedup suffix "Foo~2" + // The slugified "Foo~1" → "Foo_1" which is different from dedup "Foo~2" let r = process_siblings_dir(&["Foo", "Foo", "Foo~1"]); assert_eq!(r[0].1, "Foo"); - assert_eq!(r[1].1, "Foo~1"); // dedup suffix + assert_eq!(r[1].1, "Foo~2"); // dedup suffix assert_eq!(r[2].1, "Foo_1"); // tilde was slugified to _ assert!(r[2].2); } @@ -1318,10 +1355,10 @@ mod tests { ]); assert_eq!(r[0].1, "Leading"); assert!(r[0].2, "leading space was stripped"); - assert_eq!(r[1].1, "Leading~1"); - assert!(r[1].2); - assert_eq!(r[2].1, "Leading~2"); - assert!(r[2].2); + assert_eq!(r[1].1, "Leading~2"); + assert!(!r[1].2, "dedup-only: clean name, forward sync strips ~N"); + assert_eq!(r[2].1, "Leading~3"); + assert!(r[2].2, "trailing space stripped → slug differs"); } // ── Every middleware type with slugified name ───────────────────── @@ -1524,26 +1561,26 @@ mod tests { #[test] fn disk_seed_prevents_overwrite_of_existing_dir() { // "Utils" directory exists on disk. New instance "Utils" (dir) - // must get "Utils~1". + // must get "Utils~2". let r = process_new_children_with_disk_seed(&["utils"], &[("Utils", Middleware::Dir)]); - assert_eq!(r[0].1, "Utils~1"); - assert!(r[0].2); + assert_eq!(r[0].1, "Utils~2"); + assert!(!r[0].2, "dedup-only: forward sync strips ~N"); } #[test] fn disk_seed_prevents_overwrite_of_existing_dir_case_insensitive() { // "SCRIPTS" directory on disk, new instance "Scripts" (dir) must dedup. let r = process_new_children_with_disk_seed(&["scripts"], &[("Scripts", Middleware::Dir)]); - assert_eq!(r[0].1, "Scripts~1"); - assert!(r[0].2); + assert_eq!(r[0].1, "Scripts~2"); + assert!(!r[0].2, "dedup-only: forward sync strips ~N"); } #[test] fn disk_seed_slug_collision_with_existing_dir() { // "Hey_Bro" directory on disk. New instance "Hey/Bro" slugifies - // to "Hey_Bro" — must dedup to "Hey_Bro~1". + // to "Hey_Bro" — must dedup to "Hey_Bro~2". let r = process_new_children_with_disk_seed(&["hey_bro"], &[("Hey/Bro", Middleware::Dir)]); - assert_eq!(r[0].1, "Hey_Bro~1"); + assert_eq!(r[0].1, "Hey_Bro~2"); assert!(r[0].2); } @@ -1559,10 +1596,10 @@ mod tests { ("NewThing", Middleware::Dir), ], ); - assert_eq!(r[0].1, "Utils~1", "Utils collides with utils/ on disk"); - assert!(r[0].2); - assert_eq!(r[1].1, "Config~1", "Config collides with config/ on disk"); - assert!(r[1].2); + assert_eq!(r[0].1, "Utils~2", "Utils collides with utils/ on disk"); + assert!(!r[0].2, "dedup-only: forward sync strips ~N"); + assert_eq!(r[1].1, "Config~2", "Config collides with config/ on disk"); + assert!(!r[1].2, "dedup-only: forward sync strips ~N"); assert_eq!(r[2].1, "NewThing", "NewThing is clean"); assert!(!r[2].2); } @@ -1575,27 +1612,28 @@ mod tests { &["stuff", "stuff.meta.json5", "other.luau"], &[("Stuff", Middleware::Dir)], ); - assert_eq!(r[0].1, "Stuff~1"); - assert!(r[0].2); + assert_eq!(r[0].1, "Stuff~2"); + assert!(!r[0].2, "dedup-only: forward sync strips ~N"); } #[test] fn disk_seed_file_middleware_collides_with_bare_slug() { - // taken_names now uses bare slugs (dedup_keys), so file-format - // dedup works correctly. "mymodule" in taken matches slug "MyModule". + // taken_names stores full filesystem names (lowercased). + // A bare directory name "mymodule" does NOT collide with "mymodule.luau". + // Seed with "mymodule.luau" to test file-file collision. let r = process_new_children_with_disk_seed( - &["mymodule"], + &["mymodule.luau"], &[("MyModule", Middleware::ModuleScript)], ); - assert_eq!(r[0].1, "MyModule~1.luau"); - assert!(r[0].2, "dedup suffix means needs meta"); + assert_eq!(r[0].1, "MyModule~2.luau"); + assert!(!r[0].2, "dedup-only: forward sync strips ~N"); } #[test] fn disk_seed_file_middleware_sibling_slug_collision_deduped() { // Two file-middleware children whose names slugify to the same base. // Since taken_names accumulates dedup_keys (bare slugs), the second - // child correctly detects the collision and gets ~1. + // child correctly detects the collision and gets ~2. let r = process_new_children_with_disk_seed( &[], &[ @@ -1605,7 +1643,7 @@ mod tests { ); assert_eq!(r[0].1, "A_B.luau"); assert!(r[0].2); - assert_eq!(r[1].1, "A_B~1.luau"); + assert_eq!(r[1].1, "A_B~2.luau"); assert!(r[1].2); } @@ -1646,7 +1684,7 @@ mod tests { // New "Foo" dir must skip both and land on "Foo~2". let r = process_new_children_with_disk_seed(&["foo", "foo~1"], &[("Foo", Middleware::Dir)]); assert_eq!(r[0].1, "Foo~2"); - assert!(r[0].2); + assert!(!r[0].2, "dedup-only: forward sync strips ~N"); } #[test] @@ -1694,12 +1732,12 @@ mod tests { fn project_siblings_same_slug_different_names() { // Two new project children: "A/B" and "A:B" both slug to "A_B". // Before fix: both got "A_B" path (collision!). - // After fix: second gets "A_B~1". + // After fix: second gets "A_B~2". let r = process_project_siblings(&[], &[("A/B", Middleware::Dir), ("A:B", Middleware::Dir)]); assert_eq!(r[0].1, "A_B"); assert!(r[0].2); - assert_eq!(r[1].1, "A_B~1"); + assert_eq!(r[1].1, "A_B~2"); assert!(r[1].2); } @@ -1713,13 +1751,13 @@ mod tests { ); assert_eq!(r[0].1, "Hey_Bro"); assert!(!r[0].2); - assert_eq!(r[1].1, "Hey_Bro~1"); + assert_eq!(r[1].1, "Hey_Bro~2"); assert!(r[1].2); } #[test] fn project_siblings_three_way_slug_collision() { - // Three siblings all slug to "X_Y" — must get X_Y, X_Y~1, X_Y~2. + // Three siblings all slug to "X_Y" — must get X_Y, X_Y~2, X_Y~3. let r = process_project_siblings( &[], &[ @@ -1729,23 +1767,23 @@ mod tests { ], ); assert_eq!(r[0].1, "X_Y"); - assert_eq!(r[1].1, "X_Y~1"); - assert_eq!(r[2].1, "X_Y~2"); + assert_eq!(r[1].1, "X_Y~2"); + assert_eq!(r[2].1, "X_Y~3"); } #[test] fn project_siblings_disk_plus_siblings_collision_dir() { // "Helper" directory already on disk. Two new dir siblings: "Helper" x2. - // First → "Helper~1" (disk has "helper"). - // Second → "Helper~2" (both disk and first sibling taken). + // First → "Helper~2" (disk has "helper"). + // Second → "Helper~3" (both disk and first sibling taken). let r = process_project_siblings( &["helper"], &[("Helper", Middleware::Dir), ("Helper", Middleware::Dir)], ); - assert_eq!(r[0].1, "Helper~1"); - assert!(r[0].2); - assert_eq!(r[1].1, "Helper~2"); - assert!(r[1].2); + assert_eq!(r[0].1, "Helper~2"); + assert!(!r[0].2, "dedup-only: forward sync strips ~N"); + assert_eq!(r[1].1, "Helper~3"); + assert!(!r[1].2, "dedup-only: forward sync strips ~N"); } #[test] @@ -1762,12 +1800,12 @@ mod tests { ); assert_eq!(r[0].1, "Test"); assert!(!r[0].2); - assert_eq!(r[1].1, "test~1"); - assert!(r[1].2); - assert_eq!(r[2].1, "TEST~2"); - assert!(r[2].2); - assert_eq!(r[3].1, "tEsT~3"); - assert!(r[3].2); + assert_eq!(r[1].1, "test~2"); + assert!(!r[1].2, "dedup-only: forward sync strips ~N"); + assert_eq!(r[2].1, "TEST~3"); + assert!(!r[2].2, "dedup-only: forward sync strips ~N"); + assert_eq!(r[3].1, "tEsT~4"); + assert!(!r[3].2, "dedup-only: forward sync strips ~N"); } // ══════════════════════════════════════════════════════════════════ @@ -1786,7 +1824,7 @@ mod tests { let r = process_siblings_dir(&["CON", "CON/"]); assert_eq!(r[0].1, "CON_"); assert!(r[0].2); - assert_eq!(r[1].1, "CON_~1"); + assert_eq!(r[1].1, "CON_~2"); assert!(r[1].2); } @@ -1796,29 +1834,29 @@ mod tests { let r = process_siblings_dir(&["foo.server", "foo/server"]); assert_eq!(r[0].1, "foo_server"); assert!(r[0].2); - assert_eq!(r[1].1, "foo_server~1"); + assert_eq!(r[1].1, "foo_server~2"); assert!(r[1].2); } #[test] fn nightmare_tilde_looks_like_dedup_but_gets_slugified() { // "Foo~1" has tilde slugified to "Foo_1". Then real "Foo" gets - // dedup "Foo~1". These should NOT collide because tilde-in-name + // dedup "Foo~2". These should NOT collide because tilde-in-name // becomes underscore, but dedup adds actual tilde. let r = process_siblings_dir(&["Foo~1", "Foo", "Foo"]); assert_eq!(r[0].1, "Foo_1", "tilde slugified"); assert_eq!(r[1].1, "Foo", "clean, no collision"); - assert_eq!(r[2].1, "Foo~1", "dedup suffix, different from Foo_1"); + assert_eq!(r[2].1, "Foo~2", "dedup suffix, different from Foo_1"); } #[test] fn nightmare_dedup_suffix_avoids_slugified_tilde_name() { - // Process "Foo", then "Foo" again (dedup to "Foo~1"), then "Foo~1" - // (tilde slugified to "Foo_1"). The dedup "Foo~1" and slug "Foo_1" + // Process "Foo", then "Foo" again (dedup to "Foo~2"), then "Foo~1" + // (tilde slugified to "Foo_1"). The dedup "Foo~2" and slug "Foo_1" // should coexist. let r = process_siblings_dir(&["Foo", "Foo", "Foo~1"]); assert_eq!(r[0].1, "Foo"); - assert_eq!(r[1].1, "Foo~1"); + assert_eq!(r[1].1, "Foo~2"); assert_eq!(r[2].1, "Foo_1"); // All three have distinct lowercased names let names: HashSet = r.iter().map(|(_, f, _)| f.to_lowercase()).collect(); @@ -1830,7 +1868,7 @@ mod tests { // "カフェ/Bar" → "カフェ_Bar", "カフェ:Bar" → "カフェ_Bar" — collision let r = process_siblings_dir(&["カフェ/Bar", "カフェ:Bar"]); assert_eq!(r[0].1, "カフェ_Bar"); - assert_eq!(r[1].1, "カフェ_Bar~1"); + assert_eq!(r[1].1, "カフェ_Bar~2"); } #[test] @@ -1839,8 +1877,8 @@ mod tests { let r = process_siblings_dir(&["🎮 Games", "🎮 Games"]); assert_eq!(r[0].1, "🎮 Games"); assert!(!r[0].2, "emoji name is clean"); - assert_eq!(r[1].1, "🎮 Games~1"); - assert!(r[1].2); + assert_eq!(r[1].1, "🎮 Games~2"); + assert!(!r[1].2, "dedup-only: forward sync strips ~N"); } #[test] @@ -1850,9 +1888,9 @@ mod tests { let r = process_siblings_dir(&["CON", "CON_", "con"]); assert_eq!(r[0].1, "CON_"); assert!(r[0].2); - assert_eq!(r[1].1, "CON_~1"); - assert!(r[1].2, "natural CON_ collides with reserved CON's slug"); - assert_eq!(r[2].1, "con_~2"); + assert_eq!(r[1].1, "CON_~2"); + assert!(!r[1].2, "dedup-only: natural CON_ needs no meta"); + assert_eq!(r[2].1, "con_~3"); assert!(r[2].2); } @@ -1862,8 +1900,8 @@ mod tests { let r = process_siblings_dir(&["test.", "test"]); assert_eq!(r[0].1, "test"); assert!(r[0].2, "trailing dot stripped → slug differs"); - assert_eq!(r[1].1, "test~1"); - assert!(r[1].2); + assert_eq!(r[1].1, "test~2"); + assert!(!r[1].2, "dedup-only: clean name, forward sync strips ~N"); } #[test] @@ -1872,10 +1910,10 @@ mod tests { let r = process_siblings_dir(&[" A", "A", " A"]); assert_eq!(r[0].1, "A"); assert!(r[0].2, "leading space stripped"); - assert_eq!(r[1].1, "A~1"); - assert!(r[1].2); - assert_eq!(r[2].1, "A~2"); - assert!(r[2].2); + assert_eq!(r[1].1, "A~2"); + assert!(!r[1].2, "dedup-only: clean name, forward sync strips ~N"); + assert_eq!(r[2].1, "A~3"); + assert!(r[2].2, "leading spaces stripped → slug differs"); } #[test] @@ -1920,10 +1958,9 @@ mod tests { #[test] fn nightmare_mixed_middleware_dir_then_file() { - // "Shared" as Dir (filename "Shared") then ModuleScript. - // Dir filename "Shared" → taken gets "shared". - // ModuleScript dedup checks "shared" → MATCH → dedup to "Shared~1". - // Then final filename becomes "Shared~1.luau". + // "Shared" as Dir (filename "Shared", dedup_key "Shared") then ModuleScript. + // Dir dedup_key "shared" in taken does NOT collide with "shared.luau". + // Dir and file with same base name occupy different filesystem namespaces. let r = process_new_children_with_disk_seed( &[], &[ @@ -1933,17 +1970,16 @@ mod tests { ); assert_eq!(r[0].1, "Shared"); assert!(!r[0].2); - // File middleware bare slug "Shared" matches dir entry "shared" in taken. - assert_eq!(r[1].1, "Shared~1.luau"); - assert!(r[1].2); + assert_eq!(r[1].1, "Shared.luau"); + assert!(!r[1].2); } #[test] fn nightmare_mixed_middleware_file_then_dir() { // Reverse order: ModuleScript first, Dir second. - // ModuleScript "Shared" → dedup_key "shared" → taken gets "shared" - // Dir "Shared" → slug "Shared" → dedup checks "shared" → MATCH - // (dedup_key-based taken catches this correctly) → "Shared~1" + // ModuleScript "Shared" → dedup_key "shared.luau" in taken. + // Dir "Shared" → full_name "Shared", checks "shared" against taken. + // "shared" != "shared.luau" → no collision. Dir and file coexist. let r = process_new_children_with_disk_seed( &[], &[ @@ -1953,9 +1989,8 @@ mod tests { ); assert_eq!(r[0].1, "Shared.luau"); assert!(!r[0].2); - // Dir slug "Shared" collides with dedup_key "shared" from the ModuleScript - assert_eq!(r[1].1, "Shared~1"); - assert!(r[1].2, "dedup suffix means different from instance name"); + assert_eq!(r[1].1, "Shared"); + assert!(!r[1].2); } #[test] @@ -1974,11 +2009,7 @@ mod tests { ]; let r = process_new_children_with_disk_seed(&["a_b"], &new_children); for (i, (_, filename, needs_meta)) in r.iter().enumerate() { - let expected = if i == 0 { - "A_B~1".to_string() - } else { - format!("A_B~{}", i + 1) - }; + let expected = format!("A_B~{}", i + 2); assert_eq!(filename, &expected, "child {i}"); assert!(needs_meta, "child {i} needs meta"); } @@ -1989,8 +2020,8 @@ mod tests { // Multiple instances with empty names. All slugify to "instance". let r = process_siblings_dir(&["", "", ""]); assert_eq!(r[0].1, "instance"); - assert_eq!(r[1].1, "instance~1"); - assert_eq!(r[2].1, "instance~2"); + assert_eq!(r[1].1, "instance~2"); + assert_eq!(r[2].1, "instance~3"); } #[test] @@ -1998,10 +2029,10 @@ mod tests { // Names that are entirely forbidden chars → all become "instance". let r = process_siblings_dir(&["<>", ":/", "?*", "|\\", "\"~"]); assert_eq!(r[0].1, "instance"); - assert_eq!(r[1].1, "instance~1"); - assert_eq!(r[2].1, "instance~2"); - assert_eq!(r[3].1, "instance~3"); - assert_eq!(r[4].1, "instance~4"); + assert_eq!(r[1].1, "instance~2"); + assert_eq!(r[2].1, "instance~3"); + assert_eq!(r[3].1, "instance~4"); + assert_eq!(r[4].1, "instance~5"); } #[test] @@ -2009,7 +2040,7 @@ mod tests { // "Hello\x00World" → "Hello_World", "Hello/World" → "Hello_World" let r = process_siblings_dir(&["Hello\x00World", "Hello/World"]); assert_eq!(r[0].1, "Hello_World"); - assert_eq!(r[1].1, "Hello_World~1"); + assert_eq!(r[1].1, "Hello_World~2"); } #[test] @@ -2029,7 +2060,7 @@ mod tests { // "init/meta" → slug "init_meta" → collision! let r = process_siblings_dir(&["init.meta", "init/meta"]); assert_eq!(r[0].1, "init_meta"); - assert_eq!(r[1].1, "init_meta~1"); + assert_eq!(r[1].1, "init_meta~2"); } #[test] @@ -2038,10 +2069,10 @@ mod tests { let r = process_siblings_dir(&["a.project", "a/project", "a_project"]); assert_eq!(r[0].1, "a_project"); assert!(r[0].2); - assert_eq!(r[1].1, "a_project~1"); + assert_eq!(r[1].1, "a_project~2"); assert!(r[1].2); - assert_eq!(r[2].1, "a_project~2"); - assert!(r[2].2, "natural name collides with slugified siblings"); + assert_eq!(r[2].1, "a_project~3"); + assert!(!r[2].2, "dedup-only: natural name, forward sync strips ~N"); } #[test] @@ -2052,8 +2083,8 @@ mod tests { assert_eq!(r[0].1, "Script"); assert!(!r[0].2); for (i, entry) in r.iter().enumerate().skip(1) { - assert_eq!(entry.1, format!("Script~{i}")); - assert!(entry.2); + assert_eq!(entry.1, format!("Script~{}", i + 1)); + assert!(!entry.2, "dedup-only: forward sync strips ~N"); } // All filenames must be unique let unique: HashSet = r.iter().map(|(_, f, _)| f.to_lowercase()).collect(); @@ -2067,22 +2098,22 @@ mod tests { let r = process_siblings_dir(&[ "A_B", // clean, claims "A_B" "C_D", // clean, claims "C_D" - "A/B", // slug "A_B" → collision → "A_B~1" + "A/B", // slug "A_B" → collision → "A_B~2" "E_F", // clean, claims "E_F" - "C:D", // slug "C_D" → collision → "C_D~1" - "A:B", // slug "A_B" → collision with A_B and A_B~1 → "A_B~2" + "C:D", // slug "C_D" → collision → "C_D~2" + "A:B", // slug "A_B" → collision with A_B and A_B~2 → "A_B~3" ]); assert_eq!(r[0].1, "A_B"); assert!(!r[0].2); assert_eq!(r[1].1, "C_D"); assert!(!r[1].2); - assert_eq!(r[2].1, "A_B~1"); + assert_eq!(r[2].1, "A_B~2"); assert!(r[2].2); assert_eq!(r[3].1, "E_F"); assert!(!r[3].2); - assert_eq!(r[4].1, "C_D~1"); + assert_eq!(r[4].1, "C_D~2"); assert!(r[4].2); - assert_eq!(r[5].1, "A_B~2"); + assert_eq!(r[5].1, "A_B~3"); assert!(r[5].2); } @@ -2287,7 +2318,7 @@ mod tests { #[test] fn tilde_dedup_collision_roundtrip() { // Two instances "A/B" and "A_B" both slugify to "A_B". - // First gets "A_B", second gets "A_B~1". + // First gets "A_B", second gets "A_B~2". let mut taken = HashSet::new(); let dom1 = make_inst("A/B", "ModuleScript"); @@ -2306,18 +2337,16 @@ mod tests { assert_eq!(name1.as_ref(), "A_B.luau"); assert!(meta1, "A/B was slugified, needs meta name"); - assert_eq!(name2.as_ref(), "A_B~1.luau"); - assert!(meta2, "A_B was deduped to ~1, needs meta name"); + assert_eq!(name2.as_ref(), "A_B~2.luau"); + assert!(!meta2, "dedup-only: natural A_B, forward sync strips ~2"); } #[test] fn tilde_in_filename_not_parsed_as_dedup_marker() { - // A file named "Foo~1" should produce instance name "Foo~1", - // NOT be interpreted as "Foo" with dedup suffix ~1. - // This is verified by the fact that name_needs_slugify("Foo~1") - // returns true (tilde is in SLUGIFY_CHARS), so a file named - // "Foo~1.luau" can only exist if it was written by the dedup - // system. Forward sync reads the filename as-is. + // An instance literally named "Foo~1" gets slugified to "Foo_1" + // because tilde is in SLUGIFY_CHARS. This means a file named + // "Foo~1.luau" on disk can only have been produced by the dedup + // system, so forward sync correctly strips ~1 to derive "Foo". assert!( name_needs_slugify("Foo~1"), "tilde should trigger slugification" @@ -2393,14 +2422,9 @@ mod tests { assert_eq!(name1.as_ref(), "Hey_Bro.luau"); assert!(meta1); - // On the old-inst path the existing filename is preserved, - // so a second syncback (incremental) should not change the name. - // We can't easily test the full old-inst path here since it - // requires InstanceWithMeta, but we verify the dedup_key is - // stable: calling strip_middleware_extension on the result gives - // the same dedup_key. - let dk_from_filename = strip_middleware_extension(&name1, Middleware::ModuleScript); - assert_eq!(dk_from_filename.to_lowercase(), dk1.to_lowercase()); + // dedup_key is now the full filesystem name (including extension). + // For file middleware, dedup_key == filename. + assert_eq!(dk1.to_lowercase(), name1.as_ref().to_lowercase()); } #[test] @@ -2424,7 +2448,7 @@ mod tests { .unwrap(); assert_eq!(filename.as_ref(), "Hey_Bro.luau"); assert!(needs_meta); - assert_eq!(dedup_key, "Hey_Bro"); + assert_eq!(dedup_key, "Hey_Bro.luau"); } #[test] @@ -2461,7 +2485,7 @@ mod tests { assert_eq!(f1.as_ref(), "A_B.luau"); assert!(m1); - assert_eq!(f2.as_ref(), "A_B~1.luau"); + assert_eq!(f2.as_ref(), "A_B~2.luau"); assert!(m2); assert_ne!(dk1.to_lowercase(), dk2.to_lowercase()); } @@ -2470,7 +2494,7 @@ mod tests { fn syncback_idempotency_same_input_same_output() { // Run the same set of instances through name_for_inst twice // and verify identical output (determinism). - let names = vec!["Alpha", "Beta/Gamma", "Beta:Gamma", "Delta", "CON"]; + let names = ["Alpha", "Beta/Gamma", "Beta:Gamma", "Delta", "CON"]; let mw = crate::snapshot_middleware::Middleware::ModuleScript; let mut dom = rbx_dom_weak::WeakDom::new(rbx_dom_weak::InstanceBuilder::new("Folder")); diff --git a/src/syncback/fs_snapshot.rs b/src/syncback/fs_snapshot.rs index 89e62a06..ccc104e0 100644 --- a/src/syncback/fs_snapshot.rs +++ b/src/syncback/fs_snapshot.rs @@ -446,4 +446,345 @@ impl FsSnapshot { removed_dirs.sort_unstable(); removed_dirs } + + /// Replace `__ROJO_REF___` placeholders in a line using HashMap lookup. + /// Scans the line for `__ROJO_REF_` markers and extracts each full + /// placeholder key for O(1) lookup, avoiding O(n) iteration over all + /// substitutions. + fn replace_placeholders(line: &str, sub_map: &std::collections::HashMap<&str, &str>) -> String { + const PREFIX: &str = "__ROJO_REF_"; + const SUFFIX: &str = "__"; + + let mut result = String::with_capacity(line.len()); + let mut remaining = line; + + while let Some(start) = remaining.find(PREFIX) { + result.push_str(&remaining[..start]); + let after_prefix = &remaining[start + PREFIX.len()..]; + if let Some(end) = after_prefix.find(SUFFIX) { + let placeholder = &remaining[start..start + PREFIX.len() + end + SUFFIX.len()]; + if let Some(replacement) = sub_map.get(placeholder) { + result.push_str(replacement); + } else { + result.push_str(placeholder); + } + remaining = &remaining[start + PREFIX.len() + end + SUFFIX.len()..]; + } else { + result.push_str(&remaining[start..]); + remaining = ""; + break; + } + } + result.push_str(remaining); + result + } + + /// Post-process `Rojo_Ref_*` attribute paths in meta/model JSON files. + /// + /// For each entry in `substitutions` (old_path → new_path), finds all + /// `.meta.json5` and `.model.json5` files in the snapshot and replaces + /// occurrences of the old path with the new path **only on lines that + /// contain a `Rojo_Ref_` key**. This prevents unrelated string values + /// (property values, comments, etc.) from being altered by the + /// substitution. + pub fn fix_ref_paths(&mut self, substitutions: &[(String, String)]) { + if substitutions.is_empty() { + return; + } + + let sub_map: std::collections::HashMap<&str, &str> = substitutions + .iter() + .map(|(k, v)| (k.as_str(), v.as_str())) + .collect(); + + let ref_paths: Vec = self + .added_files + .keys() + .filter(|p| { + let name = p.file_name().and_then(|f| f.to_str()).unwrap_or(""); + name.ends_with(".meta.json5") + || name.ends_with(".meta.json") + || name.ends_with(".model.json5") + || name.ends_with(".model.json") + }) + .cloned() + .collect(); + + for path in ref_paths { + let Some(contents) = self.added_files.get_mut(&path) else { + continue; + }; + let Ok(text) = std::str::from_utf8(contents) else { + continue; + }; + if !text.contains("__ROJO_REF_") && !text.contains("Rojo_Ref_") { + continue; + } + + let mut modified_lines: Vec = Vec::new(); + let mut any_changed = false; + for line in text.split('\n') { + if line.contains("__ROJO_REF_") { + let new_line = Self::replace_placeholders(line, &sub_map); + if new_line != line { + any_changed = true; + } + modified_lines.push(new_line); + } else if line.contains("Rojo_Ref_") { + let mut new_line = line.to_string(); + for (old_path, new_path) in substitutions { + if old_path != new_path && new_line.contains(old_path.as_str()) { + new_line = new_line.replace(old_path.as_str(), new_path.as_str()); + } + } + if new_line != line { + any_changed = true; + } + modified_lines.push(new_line); + } else { + modified_lines.push(line.to_string()); + } + } + + if any_changed { + *contents = modified_lines.join("\n").into_bytes(); + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn fix_ref_paths_only_touches_ref_lines() { + let mut snap = FsSnapshot::new(); + + let meta_content = r#"{ + "properties": { + "Description": "Path is Workspace/Foo" + }, + "attributes": { + "Rojo_Ref_PrimaryPart": "Workspace/Foo" + } +}"#; + snap.added_files.insert( + PathBuf::from("/test/init.meta.json5"), + meta_content.as_bytes().to_vec(), + ); + + let substitutions = vec![("Workspace/Foo".to_string(), "./Foo~2".to_string())]; + snap.fix_ref_paths(&substitutions); + + let result = std::str::from_utf8( + snap.added_files + .get(Path::new("/test/init.meta.json5")) + .unwrap(), + ) + .unwrap(); + + assert!( + result.contains(r#""Rojo_Ref_PrimaryPart": "./Foo~2""#), + "Rojo_Ref line should be updated. Got:\n{}", + result + ); + assert!( + result.contains(r#""Description": "Path is Workspace/Foo""#), + "Non-ref property line should NOT be changed. Got:\n{}", + result + ); + } + + #[test] + fn fix_ref_paths_ignores_non_meta_files() { + let mut snap = FsSnapshot::new(); + + snap.added_files.insert( + PathBuf::from("/test/script.luau"), + b"-- Rojo_Ref_PrimaryPart Workspace/Foo".to_vec(), + ); + + let substitutions = vec![("Workspace/Foo".to_string(), "./Foo~2".to_string())]; + snap.fix_ref_paths(&substitutions); + + let result = std::str::from_utf8( + snap.added_files + .get(Path::new("/test/script.luau")) + .unwrap(), + ) + .unwrap(); + assert!( + result.contains("Workspace/Foo"), + "Non-meta file should not be touched. Got: {}", + result + ); + assert!( + !result.contains("./Foo~2"), + "Non-meta file should not have substitution applied" + ); + } + + /// Regression test: the old approach used tentative paths (e.g., + /// "Beams1/001") as substitution keys. When multiple duplicate-named + /// instances shared the same tentative path, sequential `str::replace` + /// calls would chain — each substitution's result contained the next + /// substitution's search key as a prefix, producing corrupted paths + /// like "001~5~7~4~6~3~8~2". + /// + /// The fix uses unique per-Ref placeholders as keys, making each + /// substitution independent. This test verifies the old failure mode + /// cannot recur. + #[test] + fn fix_ref_paths_no_chaining_with_unique_placeholders() { + let mut snap = FsSnapshot::new(); + + // Simulate a Beam with Attachment0 → placeholder for one specific + // Attachment, and Attachment1 → placeholder for another. + let meta_content = r#"{ + "attributes": { + "Rojo_Ref_Attachment0": "__ROJO_REF_00000000000000000000000000000001__", + "Rojo_Ref_Attachment1": "__ROJO_REF_00000000000000000000000000000002__" + }, + "className": "Beam" +}"#; + snap.added_files.insert( + PathBuf::from("/test/BeamA.model.json5"), + meta_content.as_bytes().to_vec(), + ); + + // Each placeholder maps to a distinct dedup'd path. + let substitutions = vec![ + ( + "__ROJO_REF_00000000000000000000000000000001__".to_string(), + "./001~5".to_string(), + ), + ( + "__ROJO_REF_00000000000000000000000000000002__".to_string(), + "./002~7.model.json5".to_string(), + ), + ]; + snap.fix_ref_paths(&substitutions); + + let result = std::str::from_utf8( + snap.added_files + .get(Path::new("/test/BeamA.model.json5")) + .unwrap(), + ) + .unwrap(); + + assert!( + result.contains(r#""Rojo_Ref_Attachment0": "./001~5""#), + "Attachment0 should resolve to 001~5. Got:\n{result}" + ); + assert!( + result.contains(r#""Rojo_Ref_Attachment1": "./002~7.model.json5""#), + "Attachment1 should resolve to 002~7. Got:\n{result}" + ); + assert!( + !result.contains("__ROJO_REF_"), + "no placeholders should remain. Got:\n{result}" + ); + } + + /// Regression test: verifies that the OLD tentative-path approach + /// would chain substitutions, producing corrupted paths. This test + /// documents the exact failure mode to prevent future regressions. + #[test] + fn old_tentative_substitution_would_chain() { + let mut snap = FsSnapshot::new(); + + let meta_content = r#"{ + "attributes": { + "Rojo_Ref_Attachment0": "Beams1/001" + } +}"#; + snap.added_files.insert( + PathBuf::from("/test/BeamA.model.json5"), + meta_content.as_bytes().to_vec(), + ); + + // Simulate the old broken substitution list: same source, different targets. + // This is what happened when all 8 duplicate "001" Attachments shared + // the tentative path "Beams1/001". + let substitutions = vec![ + ("Beams1/001".to_string(), "Beams1/001~5".to_string()), + ("Beams1/001".to_string(), "Beams1/001~3".to_string()), + ("Beams1/001".to_string(), "Beams1/001~2".to_string()), + ]; + snap.fix_ref_paths(&substitutions); + + let result = std::str::from_utf8( + snap.added_files + .get(Path::new("/test/BeamA.model.json5")) + .unwrap(), + ) + .unwrap(); + + // The old approach chains: "001" → "001~5" → "001~3~5" → "001~2~3~5" + // This is the documented bug — the result is corrupted. + assert!( + result.contains("~5") && result.contains("~3") && result.contains("~2"), + "old approach should chain suffixes (documenting the bug). Got:\n{result}" + ); + assert!( + !result.contains(r#""Rojo_Ref_Attachment0": "Beams1/001~5""#), + "old approach should NOT produce the correct single-suffix result" + ); + } + + /// Tests that multiple Rojo_Ref_* attributes on the same line or in + /// the same file each resolve to their correct target independently. + #[test] + fn fix_ref_paths_multiple_refs_same_file() { + let mut snap = FsSnapshot::new(); + + let meta_content = r#"{ + "attributes": { + "Rojo_Ref_Attachment0": "__ROJO_REF_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa__", + "Rojo_Ref_Attachment1": "__ROJO_REF_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb__", + "Rojo_Ref_PrimaryPart": "__ROJO_REF_cccccccccccccccccccccccccccccccc__" + } +}"#; + snap.added_files.insert( + PathBuf::from("/test/init.meta.json5"), + meta_content.as_bytes().to_vec(), + ); + + let substitutions = vec![ + ( + "__ROJO_REF_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa__".to_string(), + "./001~2".to_string(), + ), + ( + "__ROJO_REF_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb__".to_string(), + "./001~5".to_string(), + ), + ( + "__ROJO_REF_cccccccccccccccccccccccccccccccc__".to_string(), + "@self/Handle.model.json5".to_string(), + ), + ]; + snap.fix_ref_paths(&substitutions); + + let result = std::str::from_utf8( + snap.added_files + .get(Path::new("/test/init.meta.json5")) + .unwrap(), + ) + .unwrap(); + + assert!( + result.contains(r#""Rojo_Ref_Attachment0": "./001~2""#), + "Attachment0 wrong. Got:\n{result}" + ); + assert!( + result.contains(r#""Rojo_Ref_Attachment1": "./001~5""#), + "Attachment1 wrong. Got:\n{result}" + ); + assert!( + result.contains(r#""Rojo_Ref_PrimaryPart": "@self/Handle.model.json5""#), + "PrimaryPart wrong. Got:\n{result}" + ); + } } diff --git a/src/syncback/hash/variant.rs b/src/syncback/hash/variant.rs index 3457beac..4d5285a5 100644 --- a/src/syncback/hash/variant.rs +++ b/src/syncback/hash/variant.rs @@ -96,12 +96,6 @@ pub fn hash_variant(hasher: &mut Hasher, value: &Variant) { n_hash!(hasher, f.weight as u16); n_hash!(hasher, f.style as u8); hash!(hasher, f.family.as_bytes()); - if let Some(cache) = &f.cached_face_id { - hash!(hasher, &[0x01]); - hash!(hasher, cache.as_bytes()); - } else { - hash!(hasher, &[0x00]); - } } Variant::Int32(n) => n_hash!(hasher, n), Variant::Int64(n) => n_hash!(hasher, n), diff --git a/src/syncback/matching.rs b/src/syncback/matching.rs new file mode 100644 index 00000000..14cde11c --- /dev/null +++ b/src/syncback/matching.rs @@ -0,0 +1,1331 @@ +//! Instance matching algorithm for syncback. +//! +//! Pairs new children (from Roblox file) to old children (from filesystem) +//! by minimizing total reconciler changes. +//! +//! Algorithm per parent: +//! 1. Group by (Name, ClassName) -- 1:1 groups instant-match +//! 2. Ambiguous groups: change-count scoring + greedy assignment +//! (with hash fast-path: hash equal = 0 cost) +//! +//! The change count = how many things the reconciler would need to touch +//! to turn instance A into instance B. + +use std::cell::RefCell; +use std::collections::HashMap; + +use blake3::Hash; +use rbx_dom_weak::types::{Ref, Variant}; +use rbx_dom_weak::{Ustr, WeakDom}; + +use crate::variant_eq::variant_eq_disk; + +const UNMATCHED_PENALTY: u32 = 10_000; + +/// Maximum recursion depth for `compute_change_count`. Beyond this depth, +/// only flat property comparison is used (no subtree recursion). +const MAX_SCORING_DEPTH: u32 = 3; + +/// Session-scoped cache for the syncback matching algorithm. +pub struct MatchingSession { + cost_cache: RefCell>, +} + +impl Default for MatchingSession { + fn default() -> Self { + Self::new() + } +} + +impl MatchingSession { + pub fn new() -> Self { + Self { + cost_cache: RefCell::new(HashMap::new()), + } + } +} + +/// Result of the matching algorithm. +#[derive(Debug)] +pub struct MatchResult { + pub matched: Vec<(Ref, Ref)>, + pub unmatched_new: Vec, + pub unmatched_old: Vec, +} + +/// Match new children to old children, minimizing total changes. +pub fn match_children( + new_children: &[Ref], + old_children: &[Ref], + new_dom: &WeakDom, + old_dom: &WeakDom, + new_hashes: Option<&HashMap>, + old_hashes: Option<&HashMap>, + session: &MatchingSession, +) -> MatchResult { + if new_children.is_empty() && old_children.is_empty() { + return MatchResult { + matched: Vec::new(), + unmatched_new: Vec::new(), + unmatched_old: Vec::new(), + }; + } + + let new_len = new_children.len(); + let old_len = old_children.len(); + let mut new_matched = vec![false; new_len]; + let mut old_matched = vec![false; old_len]; + let mut matched: Vec<(usize, usize)> = Vec::with_capacity(new_len.min(old_len)); + + // ================================================================ + // Fast-path: Group by (Name, ClassName) -- 1:1 groups instant-match + // ================================================================ + let mut new_by_key: HashMap<(String, Ustr), Vec> = HashMap::with_capacity(new_len); + for (i, &r) in new_children.iter().enumerate() { + if let Some(inst) = new_dom.get_by_ref(r) { + new_by_key + .entry((inst.name.clone(), inst.class)) + .or_default() + .push(i); + } + } + + let mut old_by_key: HashMap<(String, Ustr), Vec> = HashMap::with_capacity(old_len); + for (i, &r) in old_children.iter().enumerate() { + if let Some(inst) = old_dom.get_by_ref(r) { + old_by_key + .entry((inst.name.clone(), inst.class)) + .or_default() + .push(i); + } + } + + // 1:1 groups: instant match + for (key, new_indices) in &new_by_key { + if let Some(old_indices) = old_by_key.get(key) { + if new_indices.len() == 1 && old_indices.len() == 1 { + let ni = new_indices[0]; + let oi = old_indices[0]; + matched.push((ni, oi)); + new_matched[ni] = true; + old_matched[oi] = true; + } + } + } + + let new_remaining = new_matched.iter().filter(|&&m| !m).count(); + let old_remaining = old_matched.iter().filter(|&&m| !m).count(); + + if new_remaining == 0 || old_remaining == 0 { + return build_result( + new_children, + old_children, + &new_matched, + &old_matched, + matched, + ); + } + + // ================================================================ + // Ambiguous groups: change-count scoring + greedy assignment + // ================================================================ + for (key, new_indices) in &new_by_key { + let Some(old_indices) = old_by_key.get(key) else { + continue; + }; + + let avail_new: Vec = new_indices + .iter() + .filter(|&&ni| !new_matched[ni]) + .copied() + .collect(); + let avail_old: Vec = old_indices + .iter() + .filter(|&&oi| !old_matched[oi]) + .copied() + .collect(); + + if avail_new.is_empty() || avail_old.is_empty() { + continue; + } + + if avail_new.len() == 1 && avail_old.len() == 1 { + let ni = avail_new[0]; + let oi = avail_old[0]; + matched.push((ni, oi)); + new_matched[ni] = true; + old_matched[oi] = true; + continue; + } + + // Score all (A, B) pairs using recursive change-count scoring + let mut pairs: Vec<(u32, usize, usize)> = + Vec::with_capacity(avail_new.len() * avail_old.len()); + let mut best_so_far = u32::MAX; + + for &ni in &avail_new { + for &oi in &avail_old { + let cost = compute_change_count( + new_children[ni], + old_children[oi], + new_dom, + old_dom, + new_hashes, + old_hashes, + best_so_far, + 0, + session, + ); + pairs.push((cost, ni, oi)); + if cost < best_so_far { + best_so_far = cost; + } + } + } + + // Stable sort by cost ascending + pairs.sort_by_key(|&(cost, _, _)| cost); + + // Greedy assign + for &(_, ni, oi) in &pairs { + if new_matched[ni] || old_matched[oi] { + continue; + } + matched.push((ni, oi)); + new_matched[ni] = true; + old_matched[oi] = true; + } + } + + build_result( + new_children, + old_children, + &new_matched, + &old_matched, + matched, + ) +} + +/// Convert index-based matching results back to Ref-based MatchResult. +fn build_result( + new_children: &[Ref], + old_children: &[Ref], + new_matched: &[bool], + old_matched: &[bool], + mut matched_indices: Vec<(usize, usize)>, +) -> MatchResult { + matched_indices.sort_by_key(|&(ni, _)| ni); + let matched: Vec<(Ref, Ref)> = matched_indices + .into_iter() + .map(|(ni, oi)| (new_children[ni], old_children[oi])) + .collect(); + + let unmatched_new: Vec = new_matched + .iter() + .enumerate() + .filter(|(_, &m)| !m) + .map(|(i, _)| new_children[i]) + .collect(); + + let unmatched_old: Vec = old_matched + .iter() + .enumerate() + .filter(|(_, &m)| !m) + .map(|(i, _)| old_children[i]) + .collect(); + + MatchResult { + matched, + unmatched_new, + unmatched_old, + } +} + +/// Lightweight matching result used during recursive scoring. +struct ScoringMatchResult { + matched: Vec<(usize, usize)>, + unmatched_new: usize, + unmatched_old: usize, +} + +/// Match children by reference for scoring purposes (non-consuming). +/// Groups by (Name, ClassName), instant-matches 1:1 groups, and scores +/// ambiguous groups using `compute_change_count` (mutually recursive). +#[allow(clippy::too_many_arguments)] +fn match_children_for_scoring( + new_children: &[Ref], + old_children: &[Ref], + new_dom: &WeakDom, + old_dom: &WeakDom, + new_hashes: Option<&HashMap>, + old_hashes: Option<&HashMap>, + depth: u32, + session: &MatchingSession, +) -> ScoringMatchResult { + if new_children.is_empty() && old_children.is_empty() { + return ScoringMatchResult { + matched: Vec::new(), + unmatched_new: 0, + unmatched_old: 0, + }; + } + + let new_len = new_children.len(); + let old_len = old_children.len(); + let mut new_matched = vec![false; new_len]; + let mut old_matched = vec![false; old_len]; + let mut matched: Vec<(usize, usize)> = Vec::with_capacity(new_len.min(old_len)); + + // Group by (Name, ClassName) + let mut new_by_key: HashMap<(String, Ustr), Vec> = HashMap::with_capacity(new_len); + for (i, &r) in new_children.iter().enumerate() { + if let Some(inst) = new_dom.get_by_ref(r) { + new_by_key + .entry((inst.name.clone(), inst.class)) + .or_default() + .push(i); + } + } + + let mut old_by_key: HashMap<(String, Ustr), Vec> = HashMap::with_capacity(old_len); + for (i, &r) in old_children.iter().enumerate() { + if let Some(inst) = old_dom.get_by_ref(r) { + old_by_key + .entry((inst.name.clone(), inst.class)) + .or_default() + .push(i); + } + } + + // 1:1 groups: instant match + for (key, new_indices) in &new_by_key { + if let Some(old_indices) = old_by_key.get(key) { + if new_indices.len() == 1 && old_indices.len() == 1 { + let ni = new_indices[0]; + let oi = old_indices[0]; + matched.push((ni, oi)); + new_matched[ni] = true; + old_matched[oi] = true; + } + } + } + + // Ambiguous groups: score + greedy assign + for (key, new_indices) in &new_by_key { + let Some(old_indices) = old_by_key.get(key) else { + continue; + }; + + let avail_new: Vec = new_indices + .iter() + .filter(|&&ni| !new_matched[ni]) + .copied() + .collect(); + let avail_old: Vec = old_indices + .iter() + .filter(|&&oi| !old_matched[oi]) + .copied() + .collect(); + + if avail_new.is_empty() || avail_old.is_empty() { + continue; + } + if avail_new.len() == 1 && avail_old.len() == 1 { + let ni = avail_new[0]; + let oi = avail_old[0]; + matched.push((ni, oi)); + new_matched[ni] = true; + old_matched[oi] = true; + continue; + } + + let mut pairs: Vec<(u32, usize, usize)> = + Vec::with_capacity(avail_new.len() * avail_old.len()); + let mut best_so_far = u32::MAX; + for &ni in &avail_new { + for &oi in &avail_old { + let cost = compute_change_count( + new_children[ni], + old_children[oi], + new_dom, + old_dom, + new_hashes, + old_hashes, + best_so_far, + depth, + session, + ); + pairs.push((cost, ni, oi)); + if cost < best_so_far { + best_so_far = cost; + } + } + } + + pairs.sort_by_key(|&(cost, _, _)| cost); + + for &(_, ni, oi) in &pairs { + if new_matched[ni] || old_matched[oi] { + continue; + } + matched.push((ni, oi)); + new_matched[ni] = true; + old_matched[oi] = true; + } + } + + let unmatched_new = new_matched.iter().filter(|&&m| !m).count(); + let unmatched_old = old_matched.iter().filter(|&&m| !m).count(); + + ScoringMatchResult { + matched, + unmatched_new, + unmatched_old, + } +} + +/// Compute total change count between two WeakDom instances, including +/// recursive subtree scoring. Hash fast-path: if hashes match, return 0. +/// +/// Mutually recursive with `match_children_for_scoring`. +#[allow(clippy::too_many_arguments)] +fn compute_change_count( + new_ref: Ref, + old_ref: Ref, + new_dom: &WeakDom, + old_dom: &WeakDom, + new_hashes: Option<&HashMap>, + old_hashes: Option<&HashMap>, + best_so_far: u32, + depth: u32, + session: &MatchingSession, +) -> u32 { + let cacheable = new_ref.is_some() && old_ref.is_some(); + if cacheable { + if let Some(&cached) = session.cost_cache.borrow().get(&(new_ref, old_ref)) { + return cached; + } + } + + // Hash fast-path: identical subtree = 0 cost + if let (Some(nh), Some(oh)) = (new_hashes, old_hashes) { + if let (Some(new_hash), Some(old_hash)) = (nh.get(&new_ref), oh.get(&old_ref)) { + if new_hash == old_hash { + if cacheable { + session + .cost_cache + .borrow_mut() + .insert((new_ref, old_ref), 0); + } + return 0; + } + } + } + + let mut cost = count_own_diffs(new_ref, old_ref, new_dom, old_dom); + if cost >= best_so_far || depth >= MAX_SCORING_DEPTH { + return cost; + } + + let new_inst = match new_dom.get_by_ref(new_ref) { + Some(i) => i, + None => return UNMATCHED_PENALTY, + }; + let old_inst = match old_dom.get_by_ref(old_ref) { + Some(i) => i, + None => return UNMATCHED_PENALTY, + }; + + let new_children = new_inst.children(); + let old_children = old_inst.children(); + + if new_children.is_empty() && old_children.is_empty() { + if cacheable && cost < best_so_far { + session + .cost_cache + .borrow_mut() + .insert((new_ref, old_ref), cost); + } + return cost; + } + + let scoring = match_children_for_scoring( + new_children, + old_children, + new_dom, + old_dom, + new_hashes, + old_hashes, + depth + 1, + session, + ); + + for &(ni, oi) in &scoring.matched { + let remaining = best_so_far.saturating_sub(cost); + cost += compute_change_count( + new_children[ni], + old_children[oi], + new_dom, + old_dom, + new_hashes, + old_hashes, + remaining, + depth + 1, + session, + ); + if cost >= best_so_far { + return cost; + } + } + + cost = cost + .saturating_add((scoring.unmatched_new + scoring.unmatched_old) as u32 * UNMATCHED_PENALTY); + + if cacheable && cost < best_so_far { + session + .cost_cache + .borrow_mut() + .insert((new_ref, old_ref), cost); + } + + cost +} + +/// Count own property diffs between two WeakDom instances (flat, non-recursive). +/// Tags and Attributes counted granularly (per-tag and per-attribute diffs). +/// Ref properties compared by target Name+ClassName at UNMATCHED_PENALTY per +/// mismatch. Children count diff = +1. +/// Does NOT include hash fast-path (handled by `compute_change_count`). +/// +/// Properties present on only one side are skipped when their value matches +/// the class default (see `src/snapshot/matching.rs` for full rationale). +#[inline] +fn count_own_diffs(new_ref: Ref, old_ref: Ref, new_dom: &WeakDom, old_dom: &WeakDom) -> u32 { + let new_inst = match new_dom.get_by_ref(new_ref) { + Some(i) => i, + None => return UNMATCHED_PENALTY, + }; + let old_inst = match old_dom.get_by_ref(old_ref) { + Some(i) => i, + None => return UNMATCHED_PENALTY, + }; + + let mut cost: u32 = 0; + + let class_data = rbx_reflection_database::get() + .ok() + .and_then(|db| db.classes.get(new_inst.class.as_str())); + + for (key, new_val) in new_inst.properties.iter() { + if let Some(old_val) = old_inst.properties.get(key) { + cost += diff_variant_pair(new_val, old_val, new_dom, old_dom); + } else { + let is_default = class_data + .and_then(|cd| cd.default_properties.get(key.as_str())) + .is_some_and(|default| variant_eq_disk(new_val, default)); + if !is_default { + cost += count_variant_one_sided(new_val); + } + } + } + + for (key, old_val) in old_inst.properties.iter() { + if !new_inst.properties.contains_key(key) { + let is_default = class_data + .and_then(|cd| cd.default_properties.get(key.as_str())) + .is_some_and(|default| variant_eq_disk(old_val, default)); + if !is_default { + cost += count_variant_one_sided(old_val); + } + } + } + + if new_inst.children().len() != old_inst.children().len() { + cost += 1; + } + + cost +} + +/// Compare two Variant values with type-aware scoring for syncback. +/// Ref targets resolved by Name+ClassName from their respective DOMs. +#[inline] +fn diff_variant_pair(a: &Variant, b: &Variant, a_dom: &WeakDom, b_dom: &WeakDom) -> u32 { + match (a, b) { + (Variant::Ref(ref_a), Variant::Ref(ref_b)) => { + if ref_a == ref_b { + return 0; + } + let a_none = !ref_a.is_some(); + let b_none = !ref_b.is_some(); + if a_none != b_none { + return UNMATCHED_PENALTY; + } + if a_none && b_none { + return 0; + } + let target_a = a_dom.get_by_ref(*ref_a); + let target_b = b_dom.get_by_ref(*ref_b); + match (target_a, target_b) { + (Some(ta), Some(tb)) => { + if ta.name == tb.name && ta.class == tb.class { + 0 + } else { + UNMATCHED_PENALTY + } + } + (None, None) => 0, + _ => UNMATCHED_PENALTY, + } + } + (Variant::Tags(tags_a), Variant::Tags(tags_b)) => { + use std::collections::HashSet; + let set_a: HashSet<&str> = tags_a.iter().collect(); + let set_b: HashSet<&str> = tags_b.iter().collect(); + (set_a.difference(&set_b).count() + set_b.difference(&set_a).count()) as u32 + } + (Variant::Attributes(attrs_a), Variant::Attributes(attrs_b)) => { + let mut cost: u32 = 0; + for (key, a_val) in attrs_a.iter() { + match attrs_b.get(key.as_str()) { + Some(b_val) => { + if !variant_eq_disk(a_val, b_val) { + cost += 1; + } + } + None => cost += 1, + } + } + for (key, _) in attrs_b.iter() { + if attrs_a.get(key.as_str()).is_none() { + cost += 1; + } + } + cost + } + _ => { + if variant_eq_disk(a, b) { + 0 + } else { + 1 + } + } + } +} + +/// Cost for a one-sided Variant (property present on only one side). +#[inline] +fn count_variant_one_sided(val: &Variant) -> u32 { + match val { + Variant::Ref(r) => { + if r.is_some() { + UNMATCHED_PENALTY + } else { + 0 + } + } + Variant::Tags(tags) => tags.len() as u32, + Variant::Attributes(attrs) => attrs.len() as u32, + _ => 1, + } +} + +#[cfg(test)] +mod tests { + use super::*; + use rbx_dom_weak::{ustr, InstanceBuilder}; + + fn build_test_dom() -> (WeakDom, Ref, Ref, Ref) { + let mut dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let root = dom.root_ref(); + let child_a = dom.insert(root, InstanceBuilder::new("Folder").with_name("Alpha")); + let child_b = dom.insert(root, InstanceBuilder::new("Script").with_name("Beta")); + (dom, root, child_a, child_b) + } + + #[test] + fn unique_names_match_instantly() { + let (new_dom, new_root, _, _) = build_test_dom(); + let (old_dom, old_root, _, _) = build_test_dom(); + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + + let result = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 2); + assert!(result.unmatched_new.is_empty()); + assert!(result.unmatched_old.is_empty()); + + for (new_ref, old_ref) in &result.matched { + let new_name = &new_dom.get_by_ref(*new_ref).unwrap().name; + let old_name = &old_dom.get_by_ref(*old_ref).unwrap().name; + assert_eq!(new_name, old_name); + } + } + + #[test] + fn class_name_grouping() { + let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let new_root = new_dom.root_ref(); + new_dom.insert( + new_root, + InstanceBuilder::new("Script").with_name("Handler"), + ); + new_dom.insert( + new_root, + InstanceBuilder::new("ModuleScript").with_name("Handler"), + ); + + let mut old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_root = old_dom.root_ref(); + old_dom.insert( + old_root, + InstanceBuilder::new("ModuleScript").with_name("Handler"), + ); + old_dom.insert( + old_root, + InstanceBuilder::new("Script").with_name("Handler"), + ); + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + + let result = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 2); + for (new_ref, old_ref) in &result.matched { + let new_class = &new_dom.get_by_ref(*new_ref).unwrap().class; + let old_class = &old_dom.get_by_ref(*old_ref).unwrap().class; + assert_eq!(new_class, old_class); + } + } + + #[test] + fn unmatched_new_and_old() { + let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let new_root = new_dom.root_ref(); + new_dom.insert(new_root, InstanceBuilder::new("Folder").with_name("Alpha")); + new_dom.insert( + new_root, + InstanceBuilder::new("Folder").with_name("NewOnly"), + ); + + let mut old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_root = old_dom.root_ref(); + old_dom.insert(old_root, InstanceBuilder::new("Folder").with_name("Alpha")); + old_dom.insert( + old_root, + InstanceBuilder::new("Folder").with_name("OldOnly"), + ); + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + + let result = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 1); + assert_eq!(result.unmatched_new.len(), 1); + assert_eq!(result.unmatched_old.len(), 1); + } + + #[test] + fn duplicate_names_matched() { + let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let new_root = new_dom.root_ref(); + new_dom.insert(new_root, InstanceBuilder::new("Folder").with_name("Data")); + new_dom.insert(new_root, InstanceBuilder::new("Folder").with_name("Data")); + + let mut old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_root = old_dom.root_ref(); + old_dom.insert(old_root, InstanceBuilder::new("Folder").with_name("Data")); + old_dom.insert(old_root, InstanceBuilder::new("Folder").with_name("Data")); + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + + let result = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + assert_eq!(result.matched.len(), 2); + assert!(result.unmatched_new.is_empty()); + assert!(result.unmatched_old.is_empty()); + } + + #[test] + fn empty_children() { + let new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let result = match_children( + &[], + &[], + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + assert!(result.matched.is_empty()); + } + + #[test] + fn matching_stability() { + let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let new_root = new_dom.root_ref(); + new_dom.insert(new_root, InstanceBuilder::new("Folder").with_name("X")); + new_dom.insert(new_root, InstanceBuilder::new("Script").with_name("Y")); + + let mut old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_root = old_dom.root_ref(); + old_dom.insert(old_root, InstanceBuilder::new("Folder").with_name("X")); + old_dom.insert(old_root, InstanceBuilder::new("Script").with_name("Y")); + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + + let r1 = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + let r2 = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + + assert_eq!(r1.matched.len(), r2.matched.len()); + for (a, b) in r1.matched.iter().zip(r2.matched.iter()) { + assert_eq!(a, b); + } + } + + #[test] + fn session_cache_consistent_syncback() { + let (new_dom, new_root, _, _) = build_test_dom(); + let (old_dom, old_root, _, _) = build_test_dom(); + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + let session = MatchingSession::new(); + + let r1 = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &session, + ); + let r2 = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &session, + ); + assert_eq!(r1.matched.len(), r2.matched.len()); + } + + #[test] + fn ref_scoring_syncback() { + use rbx_dom_weak::types::Variant; + + // 2 Models named "Weapon" in each DOM, each with a differently-named + // child Part. A Ref property on each Model points to its own child. + // Syncback matching must pair by Ref target Name+ClassName, not by + // raw Ref identity (which always differs between DOMs). + + let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let new_root = new_dom.root_ref(); + + let new_model_a = + new_dom.insert(new_root, InstanceBuilder::new("Model").with_name("Weapon")); + let new_handle = new_dom.insert( + new_model_a, + InstanceBuilder::new("Part").with_name("Handle"), + ); + new_dom + .get_by_ref_mut(new_model_a) + .unwrap() + .properties + .insert("PrimaryPart".into(), Variant::Ref(new_handle)); + + let new_model_b = + new_dom.insert(new_root, InstanceBuilder::new("Model").with_name("Weapon")); + let new_grip = new_dom.insert(new_model_b, InstanceBuilder::new("Part").with_name("Grip")); + new_dom + .get_by_ref_mut(new_model_b) + .unwrap() + .properties + .insert("PrimaryPart".into(), Variant::Ref(new_grip)); + + // Old DOM: reversed order (Grip-model first, Handle-model second) + let mut old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_root = old_dom.root_ref(); + + let old_model_b = + old_dom.insert(old_root, InstanceBuilder::new("Model").with_name("Weapon")); + let old_grip = old_dom.insert(old_model_b, InstanceBuilder::new("Part").with_name("Grip")); + old_dom + .get_by_ref_mut(old_model_b) + .unwrap() + .properties + .insert("PrimaryPart".into(), Variant::Ref(old_grip)); + + let old_model_a = + old_dom.insert(old_root, InstanceBuilder::new("Model").with_name("Weapon")); + let old_handle = old_dom.insert( + old_model_a, + InstanceBuilder::new("Part").with_name("Handle"), + ); + old_dom + .get_by_ref_mut(old_model_a) + .unwrap() + .properties + .insert("PrimaryPart".into(), Variant::Ref(old_handle)); + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + + let result = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 2); + assert!(result.unmatched_new.is_empty()); + assert!(result.unmatched_old.is_empty()); + + for (new_ref, old_ref) in &result.matched { + let new_inst = new_dom.get_by_ref(*new_ref).unwrap(); + let old_inst = old_dom.get_by_ref(*old_ref).unwrap(); + + let new_child_name = &new_dom.get_by_ref(new_inst.children()[0]).unwrap().name; + let old_child_name = &old_dom.get_by_ref(old_inst.children()[0]).unwrap().name; + assert_eq!( + new_child_name, old_child_name, + "Ref scoring failed: new model with child '{}' matched old model with child '{}'", + new_child_name, old_child_name + ); + } + } + + // ================================================================ + // Large ambiguous group tests (50+ same-named instances) + // ================================================================ + + #[test] + fn fifty_same_name_parts_five_groups_syncback() { + use rbx_dom_weak::types::{Color3, Variant, Vector3}; + + let groups: Vec<(Vector3, Color3)> = vec![ + (Vector3::new(0.0, 0.0, 0.0), Color3::new(1.0, 0.0, 0.0)), + (Vector3::new(0.0, 5.0, 0.0), Color3::new(1.0, 0.0, 0.0)), + (Vector3::new(0.0, 10.0, 0.0), Color3::new(0.0, 1.0, 0.0)), + (Vector3::new(0.0, 0.0, 0.0), Color3::new(0.0, 1.0, 0.0)), + (Vector3::new(0.0, 5.0, 0.0), Color3::new(0.0, 0.0, 1.0)), + ]; + + let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let new_root = new_dom.root_ref(); + for (pos, color) in &groups { + for _ in 0..10 { + new_dom.insert( + new_root, + InstanceBuilder::new("Part") + .with_name("Line") + .with_property("Position", Variant::Vector3(*pos)) + .with_property("Color", Variant::Color3(*color)), + ); + } + } + + let mut old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_root = old_dom.root_ref(); + for (pos, color) in groups.iter().rev() { + for _ in 0..10 { + old_dom.insert( + old_root, + InstanceBuilder::new("Part") + .with_name("Line") + .with_property("Position", Variant::Vector3(*pos)) + .with_property("Color", Variant::Color3(*color)), + ); + } + } + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + + let start = std::time::Instant::now(); + let result = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + let elapsed = start.elapsed(); + + assert_eq!(result.matched.len(), 50); + assert!(result.unmatched_new.is_empty()); + assert!(result.unmatched_old.is_empty()); + assert!(elapsed.as_secs() < 5, "took too long: {:?}", elapsed); + + for (new_ref, old_ref) in &result.matched { + let new_inst = new_dom.get_by_ref(*new_ref).unwrap(); + let old_inst = old_dom.get_by_ref(*old_ref).unwrap(); + + let new_pos = new_inst.properties.get(&ustr("Position")).unwrap(); + let old_pos = old_inst.properties.get(&ustr("Position")).unwrap(); + assert!( + variant_eq_disk(new_pos, old_pos), + "Position mismatch: new={:?}, old={:?}", + new_pos, + old_pos + ); + let new_color = new_inst.properties.get(&ustr("Color")).unwrap(); + let old_color = old_inst.properties.get(&ustr("Color")).unwrap(); + assert!( + variant_eq_disk(new_color, old_color), + "Color mismatch: new={:?}, old={:?}", + new_color, + old_color + ); + } + } + + #[test] + fn fifty_parts_position_only_syncback() { + use rbx_dom_weak::types::{Color3, Variant, Vector3}; + + let shared_color = Color3::new(0.5, 0.5, 0.5); + let positions = [ + Vector3::new(0.0, 0.0, 0.0), + Vector3::new(10.0, 0.0, 0.0), + Vector3::new(0.0, 10.0, 0.0), + Vector3::new(0.0, 0.0, 10.0), + Vector3::new(5.0, 5.0, 5.0), + ]; + + let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let new_root = new_dom.root_ref(); + let mut old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_root = old_dom.root_ref(); + + for pos in &positions { + for _ in 0..10 { + new_dom.insert( + new_root, + InstanceBuilder::new("Part") + .with_name("Block") + .with_property("Position", Variant::Vector3(*pos)) + .with_property("Color", Variant::Color3(shared_color)), + ); + } + } + for pos in positions.iter().rev() { + for _ in 0..10 { + old_dom.insert( + old_root, + InstanceBuilder::new("Part") + .with_name("Block") + .with_property("Position", Variant::Vector3(*pos)) + .with_property("Color", Variant::Color3(shared_color)), + ); + } + } + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + let result = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 50); + for (new_ref, old_ref) in &result.matched { + let new_pos = new_dom + .get_by_ref(*new_ref) + .unwrap() + .properties + .get(&ustr("Position")) + .unwrap(); + let old_pos = old_dom + .get_by_ref(*old_ref) + .unwrap() + .properties + .get(&ustr("Position")) + .unwrap(); + assert!(variant_eq_disk(new_pos, old_pos), "Position mismatch"); + } + } + + #[test] + fn fifty_parts_color_only_syncback() { + use rbx_dom_weak::types::{Color3, Variant, Vector3}; + + let shared_pos = Vector3::new(0.0, 0.0, 0.0); + let colors = [ + Color3::new(1.0, 0.0, 0.0), + Color3::new(0.0, 1.0, 0.0), + Color3::new(0.0, 0.0, 1.0), + Color3::new(1.0, 1.0, 0.0), + Color3::new(0.0, 1.0, 1.0), + ]; + + let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let new_root = new_dom.root_ref(); + let mut old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_root = old_dom.root_ref(); + + for color in &colors { + for _ in 0..10 { + new_dom.insert( + new_root, + InstanceBuilder::new("Part") + .with_name("Block") + .with_property("Position", Variant::Vector3(shared_pos)) + .with_property("Color", Variant::Color3(*color)), + ); + } + } + for color in colors.iter().rev() { + for _ in 0..10 { + old_dom.insert( + old_root, + InstanceBuilder::new("Part") + .with_name("Block") + .with_property("Position", Variant::Vector3(shared_pos)) + .with_property("Color", Variant::Color3(*color)), + ); + } + } + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + let result = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 50); + for (new_ref, old_ref) in &result.matched { + let new_color = new_dom + .get_by_ref(*new_ref) + .unwrap() + .properties + .get(&ustr("Color")) + .unwrap(); + let old_color = old_dom + .get_by_ref(*old_ref) + .unwrap() + .properties + .get(&ustr("Color")) + .unwrap(); + assert!(variant_eq_disk(new_color, old_color), "Color mismatch"); + } + } + + #[test] + fn sixty_parts_near_float_syncback() { + use rbx_dom_weak::types::Variant; + + let groups: Vec = vec![0.1, 0.100001, 0.5, 0.500001, 0.999999]; + let group_size = 12; + + let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let new_root = new_dom.root_ref(); + let mut old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_root = old_dom.root_ref(); + + for &t in &groups { + for _ in 0..group_size { + new_dom.insert( + new_root, + InstanceBuilder::new("Part") + .with_name("Segment") + .with_property("Transparency", Variant::Float32(t)), + ); + } + } + for &t in groups.iter().rev() { + for _ in 0..group_size { + old_dom.insert( + old_root, + InstanceBuilder::new("Part") + .with_name("Segment") + .with_property("Transparency", Variant::Float32(t)), + ); + } + } + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + let result = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 60); + assert!(result.unmatched_new.is_empty()); + assert!(result.unmatched_old.is_empty()); + + for (new_ref, old_ref) in &result.matched { + let new_t = new_dom + .get_by_ref(*new_ref) + .unwrap() + .properties + .get(&ustr("Transparency")) + .unwrap(); + let old_t = old_dom + .get_by_ref(*old_ref) + .unwrap() + .properties + .get(&ustr("Transparency")) + .unwrap(); + assert!( + variant_eq_disk(new_t, old_t), + "Transparency mismatch: new={:?}, old={:?}", + new_t, + old_t + ); + } + } + + #[test] + fn disk_representation_boundary_syncback() { + use rbx_dom_weak::types::Variant; + + let a_val: f32 = 1.00005; + let b_val: f32 = 1.00015; + + let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let new_root = new_dom.root_ref(); + new_dom.insert( + new_root, + InstanceBuilder::new("Part") + .with_name("Edge") + .with_property("Transparency", Variant::Float32(a_val)), + ); + new_dom.insert( + new_root, + InstanceBuilder::new("Part") + .with_name("Edge") + .with_property("Transparency", Variant::Float32(b_val)), + ); + + let mut old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_root = old_dom.root_ref(); + old_dom.insert( + old_root, + InstanceBuilder::new("Part") + .with_name("Edge") + .with_property("Transparency", Variant::Float32(b_val)), + ); + old_dom.insert( + old_root, + InstanceBuilder::new("Part") + .with_name("Edge") + .with_property("Transparency", Variant::Float32(a_val)), + ); + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + let result = match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &MatchingSession::new(), + ); + + assert_eq!(result.matched.len(), 2); + for (new_ref, old_ref) in &result.matched { + let new_t = new_dom + .get_by_ref(*new_ref) + .unwrap() + .properties + .get(&ustr("Transparency")) + .unwrap(); + let old_t = old_dom + .get_by_ref(*old_ref) + .unwrap() + .properties + .get(&ustr("Transparency")) + .unwrap(); + assert!( + variant_eq_disk(new_t, old_t), + "Boundary test failed: new={:?}, old={:?}", + new_t, + old_t + ); + } + } +} diff --git a/src/syncback/meta.rs b/src/syncback/meta.rs index 4dc969c5..7c4dc172 100644 --- a/src/syncback/meta.rs +++ b/src/syncback/meta.rs @@ -96,10 +96,17 @@ pub fn remove_model_name(model_path: &Path) -> anyhow::Result /// For each attribute whose key starts with `Rojo_Ref_` and whose string /// value starts with `old_prefix`, replaces the prefix with `new_prefix`. /// Returns true if any attribute was updated. +/// Update `Rojo_Ref_*` attributes in a meta/model file after a rename. +/// +/// For each attribute, resolves the on-disk relative path to absolute using +/// `source_abs`, checks if the resolved absolute path is affected by the +/// rename (`old_prefix` → `new_prefix`), and if so, recomputes the relative +/// path from `source_abs` to the new absolute target. pub fn update_ref_paths_in_file( file_path: &Path, old_prefix: &str, new_prefix: &str, + source_abs: &str, ) -> anyhow::Result { use crate::REF_PATH_ATTRIBUTE_PREFIX; @@ -118,17 +125,24 @@ pub fn update_ref_paths_in_file( ); } + let old_prefix_slash = format!("{old_prefix}/"); let mut updated = false; if let Some(attrs) = val.get_mut("attributes").and_then(|a| a.as_object_mut()) { for (key, value) in attrs.iter_mut() { - if key.starts_with(REF_PATH_ATTRIBUTE_PREFIX) { - if let Some(path_str) = value.as_str() { - if path_str == old_prefix || path_str.starts_with(&format!("{old_prefix}/")) { - let new_path = format!("{new_prefix}{}", &path_str[old_prefix.len()..]); - *value = serde_json::Value::String(new_path); - updated = true; - } - } + if !key.starts_with(REF_PATH_ATTRIBUTE_PREFIX) { + continue; + } + let Some(path_str) = value.as_str() else { + continue; + }; + let Some(resolved) = crate::resolve_ref_path_to_absolute(path_str, source_abs) else { + continue; + }; + if resolved == old_prefix || resolved.starts_with(&old_prefix_slash) { + let new_abs = format!("{new_prefix}{}", &resolved[old_prefix.len()..]); + let new_relative = crate::compute_relative_ref_path(source_abs, &new_abs); + *value = serde_json::Value::String(new_relative); + updated = true; } } } diff --git a/src/syncback/mod.rs b/src/syncback/mod.rs index 69d16805..85feb3a9 100644 --- a/src/syncback/mod.rs +++ b/src/syncback/mod.rs @@ -1,6 +1,8 @@ +pub mod dedup_suffix; mod file_names; mod fs_snapshot; mod hash; +pub mod matching; pub mod meta; mod property_filter; mod ref_properties; @@ -124,10 +126,16 @@ pub fn syncback_loop_with_stats( .map(|rules| rules.compile_tree_globs()) .transpose()?; + let phase_timer = std::time::Instant::now(); + // Collect all instance paths BEFORE pruning so we can track external references // (references to instances that will be pruned, like SoundGroups in SoundService). log::debug!("Collecting instance paths before pruning..."); let pre_prune_paths = collect_all_paths(&new_tree); + log::debug!( + "syncback: collect_all_paths in {:.1?}", + phase_timer.elapsed() + ); // TODO: Add a better way to tell if the root of a project is a directory let skip_pruning = if let Some(path) = &project.tree.path { @@ -146,17 +154,11 @@ pub fn syncback_loop_with_stats( } else { false }; + let phase_timer = std::time::Instant::now(); if !skip_pruning { - // Strip out any objects from the new tree that aren't in the old tree. This - // is necessary so that hashing the roots of each tree won't always result - // in different hashes. Shout out to Roblox for serializing a bunch of - // Services nobody cares about. - log::debug!("Pruning new tree"); strip_unknown_root_children(&mut new_tree, old_tree); } - // If ignoreHiddenServices is enabled (default: true), filter to only visible services - // Check root-level setting first, then fall back to syncbackRules for backward compatibility let ignore_hidden = project .ignore_hidden_services .or_else(|| { @@ -167,16 +169,19 @@ pub fn syncback_loop_with_stats( }) .unwrap_or(true); if ignore_hidden { - log::debug!("Filtering hidden services"); strip_hidden_services(&mut new_tree); } + log::debug!("syncback: prune + filter in {:.1?}", phase_timer.elapsed()); - log::debug!("Collecting referents for new DOM..."); - let deferred_referents = collect_referents(&new_tree, &pre_prune_paths); + let phase_timer = std::time::Instant::now(); + let mut deferred_referents = collect_referents(&new_tree, &pre_prune_paths, None); + let placeholder_map = std::mem::take(&mut deferred_referents.placeholder_to_source_and_target); + log::debug!( + "syncback: collect_referents in {:.1?}", + phase_timer.elapsed() + ); - // Remove any properties that are manually blocked from syncback via the - // project file. - log::debug!("Pre-filtering properties on DOMs"); + let phase_timer = std::time::Instant::now(); for referent in descendants(&new_tree, new_tree.root_ref()) { let new_inst = new_tree.get_by_ref_mut(referent).unwrap(); if let Some(filter) = get_property_filter(project, new_inst) { @@ -204,18 +209,21 @@ pub fn syncback_loop_with_stats( .unwrap_or(false); if !sync_current_camera { log::debug!("Removing CurrentCamera from new DOM"); - let mut camera_ref = None; + let mut workspace_ref = None; + let mut camera_target = None; for child_ref in new_tree.root().children() { let inst = new_tree.get_by_ref(*child_ref).unwrap(); if inst.class == "Workspace" { - camera_ref = inst.properties.get(&ustr("CurrentCamera")); + workspace_ref = Some(*child_ref); + camera_target = inst.properties.get(&ustr("CurrentCamera")).cloned(); break; } } - if let Some(Variant::Ref(camera_ref)) = camera_ref { - if new_tree.get_by_ref(*camera_ref).is_some() { - new_tree.destroy(*camera_ref); + if let (Some(ws_ref), Some(Variant::Ref(cam_ref))) = (workspace_ref, camera_target) { + if new_tree.get_by_ref(cam_ref).is_some() { + new_tree.destroy(cam_ref); } + deferred_referents.remove_ref(ws_ref, "CurrentCamera"); } } @@ -225,30 +233,38 @@ pub fn syncback_loop_with_stats( .and_then(|s| s.ignore_referents) .unwrap_or_default(); if !ignore_referents { - log::debug!("Linking referents for new DOM"); link_referents(deferred_referents, &mut new_tree)?; - } else { - log::debug!("Skipping referent linking as per project syncback rules"); } + log::debug!( + "syncback: filter props + link refs in {:.1?}", + phase_timer.elapsed() + ); - // As with pruning the children of the new root, we need to ensure the roots - // for both DOMs have the same name otherwise their hashes will always be - // different. new_tree.root_mut().name = old_tree.root().name().to_string(); - log::debug!("Hashing project DOM"); - let old_hashes = hash_tree(project, old_tree.inner(), old_tree.get_root_id()); - log::debug!("Hashing file DOM"); - let new_hashes = hash_tree(project, &new_tree, new_tree.root_ref()); + let phase_timer = std::time::Instant::now(); + let (old_hashes, new_hashes) = rayon::join( + || hash_tree(project, old_tree.inner(), old_tree.get_root_id()), + || hash_tree(project, &new_tree, new_tree.root_ref()), + ); + log::debug!( + "syncback: hash both trees (parallel) in {:.1?}", + phase_timer.elapsed() + ); let project_path = project.folder_location(); - // In clean mode, we scan the actual filesystem to find ALL existing files, - // not just what Rojo loaded into its tree. This ensures orphaned files - // (like duplicates Rojo couldn't load) get cleaned up properly. - // Clean mode is essentially: "delete everything and rebuild from Studio" + fn strip_unc_prefix(p: PathBuf) -> PathBuf { + let path_str = p.to_string_lossy(); + if path_str.starts_with(r"\\?\") || path_str.starts_with("//?/") { + PathBuf::from(&path_str[4..]) + } else { + p + } + } + + let phase_timer = std::time::Instant::now(); let existing_paths: HashSet = if !incremental { - log::debug!("Clean mode: scanning filesystem for existing paths"); let mut paths = HashSet::new(); // Get the source directories from the project's tree structure. @@ -374,63 +390,48 @@ pub fn syncback_loop_with_stats( } } - // Recursively scan each source directory - fn scan_directory( - dir: &Path, - paths: &mut HashSet, - ignore_patterns: &Option>, - project_path: &Path, - ) { - log::trace!("Scanning directory: {}", dir.display()); - let entries = match std::fs::read_dir(dir) { - Ok(e) => e, - Err(e) => { - log::trace!("Failed to read directory {}: {}", dir.display(), e); - return; - } - }; - - for entry in entries.flatten() { - let path = entry.path(); - - // Skip paths matching ignore patterns - if !is_valid_path(ignore_patterns, project_path, &path) { - log::trace!("Skipping {} (matches ignore pattern)", path.display()); - continue; - } - - // Skip hidden files/directories (starting with .), except - // .gitkeep which is managed by Rojo and must be tracked for - // cleanup when directories are no longer empty. - if let Some(name) = path.file_name().and_then(|n| n.to_str()) { - if name.starts_with('.') && name != ".gitkeep" { - log::trace!("Skipping {} (hidden)", path.display()); - continue; - } - } - - log::trace!("Found path: {}", path.display()); - paths.insert(path.clone()); - - if path.is_dir() { - scan_directory(&path, paths, ignore_patterns, project_path); - } - } - } - - log::trace!( - "dirs_to_scan: {:?}", - dirs_to_scan + if log::log_enabled!(log::Level::Trace) { + let dirs_str: Vec<_> = dirs_to_scan .iter() .map(|p| p.display().to_string()) - .collect::>() - ); + .collect(); + log::trace!("dirs_to_scan: {:?}", dirs_str); + } for dir in &dirs_to_scan { - if dir.is_dir() { - scan_directory(dir, &mut paths, &ignore_patterns, project_path); + if !dir.is_dir() { + continue; + } + if let Some(name) = dir.file_name().and_then(|n| n.to_str()) { + if name.starts_with('.') && name != ".gitkeep" { + continue; + } + } + let canonical_dir = std::fs::canonicalize(dir) + .map(strip_unc_prefix) + .unwrap_or_else(|_| dir.clone()); + for entry in walkdir::WalkDir::new(&canonical_dir) + .follow_links(true) + .into_iter() + .filter_entry(|e| { + if e.depth() == 0 { + return true; + } + e.file_name() + .to_str() + .is_none_or(|n| !n.starts_with('.') || n == ".gitkeep") + }) + .flatten() + { + if entry.depth() == 0 { + continue; + } + let path = entry.path().to_path_buf(); + if !is_valid_path(&ignore_patterns, project_path, &path) { + continue; + } + paths.insert(path); } - // Note: We only add directories to dirs_to_scan now, so no else branch needed } // Also add any alternate file representations we found earlier @@ -448,7 +449,14 @@ pub fn syncback_loop_with_stats( } else { HashSet::new() }; + log::debug!( + "syncback: orphan scan in {:.1?} ({} paths)", + phase_timer.elapsed(), + existing_paths.len() + ); + let phase_timer = std::time::Instant::now(); + let ref_path_map = std::cell::RefCell::new(HashMap::new()); let syncback_data = SyncbackData { vfs, old_tree, @@ -456,6 +464,7 @@ pub fn syncback_loop_with_stats( project, incremental, stats, + ref_path_map: &ref_path_map, }; // Always start with old reference for the Project middleware. @@ -562,6 +571,12 @@ pub fn syncback_loop_with_stats( 'remove: for inst in &syncback.removed_children { let path = inst.metadata().instigating_source.as_ref().unwrap().path(); let inst_path = snapshot.get_old_inst_path(inst.id()); + if let Some(name) = path.file_name().and_then(|n| n.to_str()) { + if name.starts_with('.') && name != ".gitkeep" { + log::debug!("Skipping removing {} (hidden path)", path.display()); + continue; + } + } if !is_valid_path(&ignore_patterns, project_path, path) { log::debug!( "Skipping removing {} because its matches an ignore pattern", @@ -606,89 +621,70 @@ pub fn syncback_loop_with_stats( snapshots.extend(syncback.children); } + log::debug!("syncback: main walk loop in {:.1?}", phase_timer.elapsed()); - // In clean mode, remove any existing paths that weren't overwritten by new structure. - // This ensures no orphaned files remain from the old project layout. + let phase_timer = std::time::Instant::now(); + { + use ref_properties::tentative_fs_path_public; + + let final_map = ref_path_map.borrow(); + let mut substitutions: Vec<(String, String)> = Vec::new(); + for (placeholder, (source_ref, target_ref)) in &placeholder_map { + let source_abs = final_map + .get(source_ref) + .cloned() + .unwrap_or_else(|| tentative_fs_path_public(&new_tree, *source_ref)); + let target_abs = final_map + .get(target_ref) + .cloned() + .unwrap_or_else(|| tentative_fs_path_public(&new_tree, *target_ref)); + let relative = crate::compute_relative_ref_path(&source_abs, &target_abs); + substitutions.push((placeholder.clone(), relative)); + } + log::debug!( + "syncback: built {} ref substitutions in {:.1?}", + substitutions.len(), + phase_timer.elapsed() + ); + + let sub_timer = std::time::Instant::now(); + if !substitutions.is_empty() { + fs_snapshot.fix_ref_paths(&substitutions); + } + log::debug!( + "syncback: applied ref substitutions in {:.1?}", + sub_timer.elapsed() + ); + } + + let phase_timer = std::time::Instant::now(); if !incremental && !existing_paths.is_empty() { log::debug!("Clean mode: checking for orphaned files to remove"); - // Helper to strip UNC prefix on Windows - fn strip_unc_prefix(p: PathBuf) -> PathBuf { - let path_str = p.to_string_lossy(); - if path_str.starts_with(r"\\?\") || path_str.starts_with("//?/") { - PathBuf::from(&path_str[4..]) - } else { - p - } - } + let canonical_project_path = strip_unc_prefix( + std::fs::canonicalize(project_path).unwrap_or_else(|_| project_path.to_path_buf()), + ); - // Helper to normalize a path by: - // 1. Canonicalizing to resolve symlinks (important on macOS where /var -> /private/var) - // 2. Stripping UNC prefix on Windows - fn normalize_existing_path(p: &Path) -> PathBuf { - // First try to canonicalize to resolve symlinks - // This is important on macOS where /var is a symlink to /private/var - let canonicalized = std::fs::canonicalize(p).unwrap_or_else(|_| p.to_path_buf()); - strip_unc_prefix(canonicalized) - } - - // Canonicalize the project path base to resolve symlinks consistently - // This is important on macOS where /var -> /private/var - let canonical_project_path = - std::fs::canonicalize(project_path).unwrap_or_else(|_| project_path.to_path_buf()); - let canonical_project_path = strip_unc_prefix(canonical_project_path); - - // Helper to canonicalize a path that may not exist yet. - // Tries to canonicalize the full path, then falls back to canonicalizing - // the nearest existing ancestor and appending the remaining components. - fn canonicalize_maybe_nonexistent(p: &Path) -> PathBuf { - // Try full canonicalization first (works if path exists) - if let Ok(canonical) = std::fs::canonicalize(p) { - return strip_unc_prefix(canonical); - } - - // Path doesn't exist - find the nearest existing ancestor and canonicalize that, - // then append the remaining path components. - let mut existing_ancestor = p.to_path_buf(); - let mut suffix_components = Vec::new(); - - while !existing_ancestor.as_os_str().is_empty() { - if existing_ancestor.exists() { - break; - } - if let Some(file_name) = existing_ancestor.file_name() { - suffix_components.push(file_name.to_owned()); - } - if !existing_ancestor.pop() { - break; - } - } - - // Canonicalize the existing ancestor - let canonical_ancestor = std::fs::canonicalize(&existing_ancestor) - .map(strip_unc_prefix) - .unwrap_or(existing_ancestor); - - // Append the suffix components in reverse order - let mut result = canonical_ancestor; - for component in suffix_components.into_iter().rev() { - result.push(component); - } - result - } - - // Convert added_paths to absolute paths by joining with canonicalized project base path. - // fs_snapshot paths are relative to the project folder. - // We must also canonicalize the result to resolve any symlinks within the project - // directory structure, ensuring consistent comparison with canonicalized existing paths. let added_paths: HashSet = fs_snapshot .added_paths() .into_iter() - .map(|p| canonicalize_maybe_nonexistent(&canonical_project_path.join(p))) + .map(|p| strip_unc_prefix(canonical_project_path.join(p))) .collect(); - // Get the project file path to exclude it from removal - let project_file = normalize_existing_path(&project.file_location); + let mut added_dir_prefixes: HashSet = HashSet::new(); + for p in &added_paths { + let mut ancestor = p.clone(); + while ancestor.pop() { + if !added_dir_prefixes.insert(ancestor.clone()) { + break; + } + } + } + + let project_file = strip_unc_prefix( + std::fs::canonicalize(&project.file_location) + .unwrap_or_else(|_| project.file_location.clone()), + ); // Collect ALL paths explicitly referenced via $path in the project. // These paths should NOT be removed during orphan cleanup because they @@ -714,7 +710,6 @@ pub fn syncback_loop_with_stats( .map(strip_unc_prefix) .unwrap_or_else(|_| resolved.clone()); protected.insert(canonical.clone()); - // Add mapping from filesystem path to instance path if !instance_path.is_empty() { mappings.push((canonical, instance_path.to_string())); } @@ -741,8 +736,10 @@ pub fn syncback_loop_with_stats( &mut protected_paths, &mut path_to_instance_prefix, ); - log::trace!("Protected $path references: {:?}", protected_paths); - log::trace!("Path to instance mappings: {:?}", path_to_instance_prefix); + if log::log_enabled!(log::Level::Trace) { + log::trace!("Protected $path references: {:?}", protected_paths); + log::trace!("Path to instance mappings: {:?}", path_to_instance_prefix); + } // Helper to convert a filesystem path to an instance path using the mappings let fs_path_to_instance_path = |fs_path: &Path| -> Option { @@ -763,12 +760,10 @@ pub fn syncback_loop_with_stats( None }; - // First pass: collect normalized paths to remove (deduplicated) let mut paths_to_remove: HashSet = HashSet::new(); for old_path in &existing_paths { - let old_path_norm = normalize_existing_path(old_path); + let old_path_norm = old_path.clone(); - // Never remove the ROOT project file itself if old_path_norm == project_file { log::trace!("Skipping root project file: {}", old_path.display()); continue; @@ -862,47 +857,30 @@ pub fn syncback_loop_with_stats( } } - // Skip if this exact path is being written to if added_paths.contains(&old_path_norm) { - log::trace!( - "Skipping {} (exact path in added_paths)", - old_path.display() - ); continue; } - // Skip if this path is an ancestor of something being written to - // (e.g., a directory that will contain new files) - // NOTE: We DO NOT skip files just because their parent directory is in added_paths. - // Adding a directory does not mean all orphan files inside should be preserved. - if old_path_norm.is_dir() - && added_paths - .iter() - .any(|added| added.starts_with(&old_path_norm)) - { - log::trace!("Skipping {} (ancestor of added path)", old_path.display()); + if old_path_norm.is_dir() && added_dir_prefixes.contains(&old_path_norm) { continue; } - // Use normalized path to avoid duplicates from UNC vs regular paths - log::trace!("Marking for removal: {}", old_path_norm.display()); paths_to_remove.insert(old_path_norm); } // Second pass: only remove top-level orphaned paths. - // Skip any path that is inside another path we're removing, - // since remove_dir_all will handle the contents. - let paths_vec: Vec<_> = paths_to_remove.iter().collect(); - for old_path in &paths_vec { - // Check if this path is inside another path we're removing - let is_nested = paths_vec - .iter() - .any(|other| *other != *old_path && old_path.starts_with(*other)); + // Sorted path order guarantees all descendants appear consecutively + // after their ancestor, so a single ancestor tracker suffices. + let mut sorted_removals: Vec<_> = paths_to_remove.into_iter().collect::>(); + sorted_removals.sort(); - if is_nested { - continue; + let mut current_ancestor: Option<&PathBuf> = None; + for old_path in &sorted_removals { + if let Some(ancestor) = current_ancestor { + if old_path.starts_with(ancestor) && old_path != ancestor { + continue; + } } + current_ancestor = Some(old_path); - // Convert absolute path to relative path for FsSnapshot. - // FsSnapshot expects relative paths and joins them with base_path in write_to_vfs. let relative_path = old_path .strip_prefix(&canonical_project_path) .unwrap_or(old_path); @@ -916,7 +894,8 @@ pub fn syncback_loop_with_stats( } } - // Log stats summary if using internal stats (not provided externally) + log::debug!("syncback: orphan removal in {:.1?}", phase_timer.elapsed()); + if external_stats.is_none() { stats.log_summary(); } @@ -1017,7 +996,7 @@ pub struct SyncbackRules { #[serde(skip_serializing_if = "Option::is_none")] sync_current_camera: Option, /// Whether or not to sync properties that cannot be modified via scripts. - /// Defaults to `true`. + /// Defaults to `false`. #[serde(skip_serializing_if = "Option::is_none")] sync_unscriptable: Option, /// Whether to skip serializing referent properties like `Model.PrimaryPart` diff --git a/src/syncback/property_filter.rs b/src/syncback/property_filter.rs index 4adbd906..1f622f47 100644 --- a/src/syncback/property_filter.rs +++ b/src/syncback/property_filter.rs @@ -40,7 +40,7 @@ pub fn filter_properties_with_stats<'inst>( .syncback_rules .as_ref() .and_then(|s| s.sync_unscriptable) - .unwrap_or(true); + .unwrap_or(false); let database = rbx_reflection_database::get().unwrap(); let class_data = database.classes.get(inst.class.as_str()); @@ -61,12 +61,18 @@ pub fn filter_properties_with_stats<'inst>( return true; } if !sync_unscriptable { - if let Some(data) = class_data { + let mut current = class_data; + while let Some(data) = current { if let Some(prop_data) = data.properties.get(prop_name.as_str()) { if matches!(prop_data.scriptability, Scriptability::None) { return true; } + break; } + current = data + .superclass + .as_ref() + .and_then(|s| database.classes.get(&**s)); } } false diff --git a/src/syncback/ref_properties.rs b/src/syncback/ref_properties.rs index f22e892f..cc54a3a2 100644 --- a/src/syncback/ref_properties.rs +++ b/src/syncback/ref_properties.rs @@ -10,10 +10,131 @@ use rbx_dom_weak::{ }; use crate::{ - ref_attribute_name, ref_target_attribute_name, syncback::snapshot::inst_path, + ref_attribute_name, ref_target_attribute_name, + syncback::{name_needs_slugify, slugify_name}, REF_ID_ATTRIBUTE_NAME, REF_PATH_ATTRIBUTE_PREFIX, REF_POINTER_ATTRIBUTE_PREFIX, }; +/// Public wrapper for `tentative_fs_path` used by the post-processing step +/// in `syncback_loop` to compare tentative vs final paths. +pub fn tentative_fs_path_public(dom: &WeakDom, target_ref: Ref) -> String { + tentative_fs_path(dom, target_ref) +} + +/// Compute a filesystem-name-compatible path for an instance in a bare WeakDom. +/// +/// Each path segment is a **tentative filesystem name**: the slugified instance +/// name plus the extension that the syncback middleware would assign based on +/// class and children. This produces paths like `"Workspace/Hey_Bro.server.luau"` +/// that are compatible with `get_instance_by_path()` (which resolves using +/// filesystem names). +/// +/// For directory-style instances (Folders, services, scripts with children), +/// the segment is just the slugified name (no extension). +fn tentative_fs_path(dom: &WeakDom, target_ref: Ref) -> String { + let root_ref = dom.root_ref(); + let mut components: Vec = Vec::new(); + let mut current = target_ref; + + loop { + if current == root_ref || current.is_none() { + break; + } + + let inst = match dom.get_by_ref(current) { + Some(i) => i, + None => break, + }; + + let segment = tentative_fs_name(inst); + components.push(segment); + current = inst.parent(); + } + + components.reverse(); + components.join("/") +} + +/// Compute the tentative filesystem name for a single instance based on its +/// class, RunContext property, and whether it has children. +/// +/// This mirrors the logic in `get_best_middleware()` + `extension_for_middleware()` +/// but operates on a bare Instance without SyncbackSnapshot context. +fn tentative_fs_name(inst: &Instance) -> String { + let slug = if name_needs_slugify(&inst.name) { + slugify_name(&inst.name) + } else { + inst.name.clone() + }; + + let has_children = !inst.children().is_empty(); + + // Directory-style classes never get extensions + let is_container = matches!( + inst.class.as_str(), + "Folder" | "Configuration" | "Tool" | "ScreenGui" | "SurfaceGui" | "BillboardGui" | "AdGui" + ); + + if is_container || (has_children && is_script_class(inst.class.as_str())) { + // Directory representation -- slug only, no extension + return slug; + } + + let extension = match inst.class.as_str() { + "Script" => { + if has_children { + return slug; // directory + } + match inst.properties.get(&ustr("RunContext")) { + Some(Variant::Enum(e)) => match e.to_u32() { + 0 => "legacy.luau", + 1 => "server.luau", + 2 => "client.luau", + 3 => "plugin.luau", + _ => "legacy.luau", + }, + _ => "legacy.luau", + } + } + "LocalScript" => { + if has_children { + return slug; + } + "local.luau" + } + "ModuleScript" => { + if has_children { + return slug; + } + "luau" + } + "StringValue" => { + if has_children { + return slug; + } + "txt" + } + "LocalizationTable" => { + if has_children { + return slug; + } + "csv" + } + _ => { + if has_children { + return slug; // directory + } + "model.json5" + } + }; + + format!("{slug}.{extension}") +} + +fn is_script_class(class: &str) -> bool { + matches!(class, "Script" | "LocalScript" | "ModuleScript") +} + pub struct RefLinks { /// Refs that use path-based linking (path is unique). path_links: HashMap>, @@ -21,6 +142,22 @@ pub struct RefLinks { id_links: HashMap>, /// Target instances that need a Rojo_Id written (for ID-based system). targets_needing_id: HashSet, + /// Maps placeholder strings to (source_ref, target_ref) pairs. Populated + /// when `final_paths` is not available (CLI syncback pre-walk). The + /// placeholder encodes both source and target so that post-processing can + /// compute the correct relative path for each (source, target) pair. + pub placeholder_to_source_and_target: HashMap, +} + +impl RefLinks { + pub fn remove_ref(&mut self, source: Ref, prop_name: &str) { + if let Some(links) = self.path_links.get_mut(&source) { + links.retain(|link| link.name.as_str() != prop_name); + } + if let Some(links) = self.id_links.get_mut(&source) { + links.retain(|link| link.name.as_str() != prop_name); + } + } } #[derive(PartialEq, Eq)] @@ -36,7 +173,7 @@ struct IdRefLink { } /// Collects all instance paths in a WeakDom before any pruning occurs. -/// Returns a map of Ref -> path for all instances. +/// Returns a map of Ref -> filesystem-name-compatible path for all instances. pub fn collect_all_paths(dom: &WeakDom) -> HashMap { let mut paths = HashMap::new(); let mut queue = VecDeque::new(); @@ -45,102 +182,31 @@ pub fn collect_all_paths(dom: &WeakDom) -> HashMap { while let Some(inst_ref) = queue.pop_front() { let inst = dom.get_by_ref(inst_ref).unwrap(); queue.extend(inst.children().iter().copied()); - paths.insert(inst_ref, inst_path(dom, inst_ref)); + paths.insert(inst_ref, tentative_fs_path(dom, inst_ref)); } paths } -/// Pre-computes which instances have duplicate-named siblings. -/// Returns a HashSet of Refs that have at least one sibling with the same name. -/// -/// This is O(N) where N is the number of instances, and allows subsequent -/// path uniqueness checks to be O(d) instead of O(d × s) where d=depth, s=siblings. -fn compute_refs_with_duplicate_siblings(dom: &WeakDom) -> HashSet { - let mut has_duplicate_siblings = HashSet::new(); - let mut queue = VecDeque::new(); - queue.push_back(dom.root_ref()); - - while let Some(inst_ref) = queue.pop_front() { - let inst = match dom.get_by_ref(inst_ref) { - Some(i) => i, - None => continue, - }; - - // Count children by name and collect their refs - let mut name_to_refs: HashMap<&str, Vec> = HashMap::new(); - for child_ref in inst.children() { - if let Some(child) = dom.get_by_ref(*child_ref) { - name_to_refs - .entry(&child.name) - .or_default() - .push(*child_ref); - } - queue.push_back(*child_ref); - } - - // Mark refs that share a name with siblings - for (_name, refs) in name_to_refs { - if refs.len() > 1 { - for r in refs { - has_duplicate_siblings.insert(r); - } - } - } - } - - has_duplicate_siblings -} - -/// Checks if a path is unique using pre-computed duplicate sibling info. -/// This is O(d) where d is the depth of the instance. -fn is_path_unique_with_cache( - dom: &WeakDom, - target_ref: Ref, - has_duplicate_siblings: &HashSet, -) -> bool { - let mut current_ref = target_ref; - - // Walk up the tree checking each level using the pre-computed cache - loop { - // O(1) lookup instead of O(siblings) counting - if has_duplicate_siblings.contains(¤t_ref) { - return false; - } - - let current = match dom.get_by_ref(current_ref) { - Some(inst) => inst, - None => return false, - }; - - let parent_ref = current.parent(); - if parent_ref.is_none() { - // Reached root - path is unique at all levels - return true; - } - - // Move up to parent and check the next level - current_ref = parent_ref; - } -} - /// Iterates through a WeakDom and collects referent properties. /// Uses paths when unique, falls back to IDs when paths are ambiguous. /// /// The `pre_prune_paths` parameter should contain paths for instances that may /// have been pruned from the DOM. This allows references to instances outside /// the sync tree to be preserved as path-based attributes. -pub fn collect_referents(dom: &WeakDom, pre_prune_paths: &HashMap) -> RefLinks { +/// +/// The `final_paths` parameter, when provided, contains the definitive +/// filesystem-name-based paths assigned during the syncback walk (including +/// dedup suffixes like `~2`). These take priority over `tentative_fs_path()`. +pub fn collect_referents( + dom: &WeakDom, + pre_prune_paths: &HashMap, + final_paths: Option<&HashMap>, +) -> RefLinks { let mut path_links: HashMap> = HashMap::new(); - let mut id_links: HashMap> = HashMap::new(); - let mut targets_needing_id: HashSet = HashSet::new(); - - // Pre-compute duplicate sibling info in O(N) - this makes all subsequent - // path uniqueness checks O(d) instead of O(d × s) where d=depth, s=siblings - let has_duplicate_siblings = compute_refs_with_duplicate_siblings(dom); - - // Cache path uniqueness results to avoid re-checking the same target - let mut path_unique_cache: HashMap = HashMap::new(); + let id_links: HashMap> = HashMap::new(); + let targets_needing_id: HashSet = HashSet::new(); + let mut placeholder_to_source_and_target: HashMap = HashMap::new(); let mut queue = VecDeque::new(); queue.push_back(dom.root_ref()); @@ -157,50 +223,42 @@ pub fn collect_referents(dom: &WeakDom, pre_prune_paths: &HashMap) continue; } - // Check if target exists in current DOM - if let Some(_target) = dom.get_by_ref(*target_ref) { - // Target exists - check if path is unique (with caching) - let is_unique = *path_unique_cache.entry(*target_ref).or_insert_with(|| { - is_path_unique_with_cache(dom, *target_ref, &has_duplicate_siblings) - }); - - if is_unique { - // Path is unique - use path-based system - let target_path = inst_path(dom, *target_ref); - path_links.entry(inst_ref).or_default().push(PathRefLink { - name: *prop_name, - path: target_path, - }); + if dom.get_by_ref(*target_ref).is_some() { + let ref_path = if let Some(fp) = final_paths { + let source_abs = fp + .get(&inst_ref) + .cloned() + .unwrap_or_else(|| tentative_fs_path(dom, inst_ref)); + let target_abs = fp + .get(target_ref) + .cloned() + .unwrap_or_else(|| tentative_fs_path(dom, *target_ref)); + crate::compute_relative_ref_path(&source_abs, &target_abs) } else { - // Path is NOT unique - fall back to ID-based system - log::debug!( - "Property {}.{} uses ID-based reference because target has duplicate-named siblings", - inst_path(dom, inst_ref), - prop_name - ); - id_links.entry(inst_ref).or_default().push(IdRefLink { - name: *prop_name, - target: *target_ref, - }); - targets_needing_id.insert(*target_ref); - } + let placeholder = ref_placeholder(inst_ref, *target_ref); + placeholder_to_source_and_target + .insert(placeholder.clone(), (inst_ref, *target_ref)); + placeholder + }; + path_links.entry(inst_ref).or_default().push(PathRefLink { + name: *prop_name, + path: ref_path, + }); } else if let Some(external_path) = pre_prune_paths.get(target_ref) { - // Target was pruned - use path (we can't check uniqueness, assume it's fine) log::debug!( - "Property {}.{} points to pruned instance at '{}', storing as path reference", - inst_path(dom, inst_ref), + "Property {}.{} points to pruned instance at '{}', storing as @game/ reference", + tentative_fs_path(dom, inst_ref), prop_name, external_path ); path_links.entry(inst_ref).or_default().push(PathRefLink { name: *prop_name, - path: external_path.clone(), + path: format!("@game/{external_path}"), }); } else { - // Truly dangling reference log::warn!( "Property {}.{} will be `nil` on disk because the referenced instance does not exist", - inst_path(dom, inst_ref), + tentative_fs_path(dom, inst_ref), prop_name ); } @@ -211,9 +269,18 @@ pub fn collect_referents(dom: &WeakDom, pre_prune_paths: &HashMap) path_links, id_links, targets_needing_id, + placeholder_to_source_and_target, } } +const REF_PLACEHOLDER_PREFIX: &str = "__ROJO_REF_"; +const REF_PLACEHOLDER_SUFFIX: &str = "__"; + +/// Build a unique placeholder string encoding both source and target Refs. +fn ref_placeholder(source: Ref, target: Ref) -> String { + format!("{REF_PLACEHOLDER_PREFIX}{source}_TO_{target}{REF_PLACEHOLDER_SUFFIX}") +} + /// Writes reference attributes to instances in the DOM. /// Uses paths for unique references, IDs for non-unique (duplicate siblings). pub fn link_referents(links: RefLinks, dom: &mut WeakDom) -> anyhow::Result<()> { @@ -363,3 +430,129 @@ fn get_existing_id_from_props(props: &rbx_dom_weak::UstrMap) -> Option< None } } + +#[cfg(test)] +mod tests { + use super::*; + use rbx_dom_weak::InstanceBuilder; + + fn make_beam_attachment_dom() -> (WeakDom, Ref, Vec, Ref) { + let mut dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let root = dom.root_ref(); + + let container = dom.insert(root, InstanceBuilder::new("Folder").with_name("Beams1")); + + let mut attachments = Vec::new(); + for _ in 0..4 { + let att = dom.insert( + container, + InstanceBuilder::new("Attachment").with_name("001"), + ); + attachments.push(att); + } + + let beam = dom.insert( + container, + InstanceBuilder::new("Beam") + .with_name("BeamA") + .with_property("Attachment0", Variant::Ref(attachments[2])), + ); + + (dom, container, attachments, beam) + } + + #[test] + fn collect_referents_uses_source_aware_placeholders() { + let (dom, _, attachments, beam) = make_beam_attachment_dom(); + + let links = collect_referents(&dom, &HashMap::new(), None); + + let beam_links = links.path_links.get(&beam).expect("beam should have refs"); + assert_eq!(beam_links.len(), 1); + + let path = &beam_links[0].path; + assert!( + path.starts_with(REF_PLACEHOLDER_PREFIX), + "path should be a placeholder, got: {path}" + ); + assert!( + path.contains("_TO_"), + "placeholder should encode both source and target, got: {path}" + ); + + let (src, tgt) = links + .placeholder_to_source_and_target + .get(path) + .expect("placeholder should map to (source, target)"); + assert_eq!(*src, beam); + assert_eq!(*tgt, attachments[2]); + } + + #[test] + fn placeholders_are_unique_per_source_target_pair() { + let mut dom2 = WeakDom::new(InstanceBuilder::new("DataModel")); + let root2 = dom2.root_ref(); + let c2 = dom2.insert(root2, InstanceBuilder::new("Folder").with_name("Beams1")); + let att_a = dom2.insert(c2, InstanceBuilder::new("Attachment").with_name("001")); + let att_b = dom2.insert(c2, InstanceBuilder::new("Attachment").with_name("001")); + dom2.insert( + c2, + InstanceBuilder::new("Beam") + .with_name("Beam1") + .with_property("Attachment0", Variant::Ref(att_a)) + .with_property("Attachment1", Variant::Ref(att_b)), + ); + + let links = collect_referents(&dom2, &HashMap::new(), None); + + let placeholders: Vec<&String> = links.placeholder_to_source_and_target.keys().collect(); + + assert!( + placeholders.len() >= 2, + "should have at least 2 placeholders" + ); + + let targets: HashSet = links + .placeholder_to_source_and_target + .values() + .map(|(_, t)| *t) + .collect(); + assert!( + targets.contains(&att_a), + "should have placeholder for att_a" + ); + assert!( + targets.contains(&att_b), + "should have placeholder for att_b" + ); + + let unique_placeholders: HashSet<&String> = placeholders.iter().copied().collect(); + assert_eq!( + unique_placeholders.len(), + placeholders.len(), + "all placeholders must be unique strings" + ); + } + + #[test] + fn collect_referents_with_final_paths_produces_relative_paths() { + let (dom, _, attachments, beam) = make_beam_attachment_dom(); + + let mut final_paths = HashMap::new(); + final_paths.insert(beam, "Beams1/BeamA.model.json5".to_string()); + final_paths.insert(attachments[2], "Beams1/001~3.model.json5".to_string()); + + let links = collect_referents(&dom, &HashMap::new(), Some(&final_paths)); + + assert!( + links.placeholder_to_source_and_target.is_empty(), + "should not generate placeholders when final_paths is provided" + ); + + let beam_links = links.path_links.get(&beam).expect("beam should have refs"); + assert_eq!( + beam_links[0].path, "./001~3.model.json5", + "should produce relative path (sibling)" + ); + } +} diff --git a/src/syncback/snapshot.rs b/src/syncback/snapshot.rs index dc20cb3e..87318625 100644 --- a/src/syncback/snapshot.rs +++ b/src/syncback/snapshot.rs @@ -1,6 +1,7 @@ use indexmap::IndexMap; use memofs::Vfs; -use std::collections::HashSet; +use std::cell::RefCell; +use std::collections::{HashMap, HashSet}; use std::path::{Path, PathBuf}; use crate::{ @@ -29,6 +30,10 @@ pub struct SyncbackData<'sync> { pub(super) incremental: bool, /// Statistics tracker for syncback issues. pub(super) stats: &'sync SyncbackStats, + /// Records (Ref → ref-path-string) for each instance as filenames are + /// assigned during the syncback walk. Used after the walk to fix + /// Rojo_Ref_* attribute paths with correct dedup suffixes. + pub(super) ref_path_map: &'sync RefCell>, } impl<'sync> SyncbackData<'sync> { @@ -87,6 +92,10 @@ impl<'sync> SyncbackSnapshot<'sync> { snapshot.path = self.path.join(&*name); snapshot.needs_meta_name = needs_meta_name; + // Record the Ref → ref-path mapping for dedup-aware ref linking. + // The ref path is built from the parent's entry + the child filename. + self.record_ref_path(new_ref, &name); + Ok((snapshot, needs_meta_name, dedup_key)) } @@ -126,9 +135,33 @@ impl<'sync> SyncbackSnapshot<'sync> { snapshot.path = base_path.join(&*name); snapshot.needs_meta_name = needs_meta_name; + // Record the Ref → ref-path mapping for dedup-aware ref linking. + self.record_ref_path(new_ref, &name); + Ok((snapshot, needs_meta_name, dedup_key)) } + /// Records the ref path for a child instance in the shared ref_path_map. + /// The ref path is: self's ref path + "/" + child_filename. + fn record_ref_path(&self, child_ref: Ref, child_filename: &str) { + let parent_path = self + .data + .ref_path_map + .borrow() + .get(&self.new) + .cloned() + .unwrap_or_default(); + let child_path = if parent_path.is_empty() { + child_filename.to_string() + } else { + format!("{}/{}", parent_path, child_filename) + }; + self.data + .ref_path_map + .borrow_mut() + .insert(child_ref, child_path); + } + /// Constructs a SyncbackSnapshot with the provided path and refs while /// inheriting the data of the this snapshot. #[inline] diff --git a/src/variant_eq.rs b/src/variant_eq.rs index ae3ca20d..5d47931d 100644 --- a/src/variant_eq.rs +++ b/src/variant_eq.rs @@ -1,17 +1,43 @@ +use lexical_write_float::{format::STANDARD, ToLexicalWithOptions}; use rbx_dom_weak::types::{PhysicalProperties, Variant, Vector3}; -/// Accepts three argumets: a float type and two values to compare. -/// -/// Returns a bool indicating whether they're equal. This accounts for NaN such -/// that `approx_eq!(f32, f32::NAN, f32::NAN)` is `true`. -macro_rules! approx_eq { - ($Ty:ty, $a:expr, $b:expr) => { - float_cmp::approx_eq!($Ty, $a, $b) || $a.is_nan() && $b.is_nan() - }; +use crate::json::{F32_DISK_BUF_SIZE, F32_DISK_OPTIONS, F64_DISK_BUF_SIZE, F64_DISK_OPTIONS}; +use crate::resolution::cleanup_f32; + +const EPSILON_F32: f32 = 0.0001; +const EPSILON_F64: f64 = 0.0001; + +/// Fuzzy float equality matching Lua trueEquals: absolute OR relative epsilon. +/// NaN == NaN is true. +#[inline(always)] +fn fuzzy_eq_f32(a: f32, b: f32) -> bool { + if a.is_nan() { + return b.is_nan(); + } + if b.is_nan() { + return false; + } + let diff = (a - b).abs(); + let max_val = a.abs().max(b.abs()).max(1.0); + diff < EPSILON_F32 || diff < max_val * EPSILON_F32 +} + +#[inline(always)] +fn fuzzy_eq_f64(a: f64, b: f64) -> bool { + if a.is_nan() { + return b.is_nan(); + } + if b.is_nan() { + return false; + } + let diff = (a - b).abs(); + let max_val = a.abs().max(b.abs()).max(1.0); + diff < EPSILON_F64 || diff < max_val * EPSILON_F64 } /// Compares two variants to determine if they're equal. This correctly takes /// float comparisons into account. +#[inline] pub fn variant_eq(variant_a: &Variant, variant_b: &Variant) -> bool { if variant_a.ty() != variant_b.ty() { return false; @@ -45,7 +71,7 @@ pub fn variant_eq(variant_a: &Variant, variant_b: &Variant) -> bool { && vector_eq(&a.orientation.z, &b.orientation.z) } (Variant::Color3(a), Variant::Color3(b)) => { - approx_eq!(f32, a.r, b.r) && approx_eq!(f32, a.g, b.g) && approx_eq!(f32, a.b, b.b) + fuzzy_eq_f32(a.r, b.r) && fuzzy_eq_f32(a.g, b.g) && fuzzy_eq_f32(a.b, b.b) } (Variant::Color3uint8(a), Variant::Color3uint8(b)) => a == b, (Variant::ColorSequence(a), Variant::ColorSequence(b)) => { @@ -58,10 +84,10 @@ pub fn variant_eq(variant_a: &Variant, variant_b: &Variant) -> bool { b_keypoints.sort_unstable_by(|k1, k2| k1.time.partial_cmp(&k2.time).unwrap()); for (a_kp, b_kp) in a_keypoints.iter().zip(b_keypoints) { - if !(approx_eq!(f32, a_kp.time, b_kp.time) - && approx_eq!(f32, a_kp.color.r, b_kp.color.r) - && approx_eq!(f32, a_kp.color.g, b_kp.color.g) - && approx_eq!(f32, a_kp.color.b, b_kp.color.b)) + if !(fuzzy_eq_f32(a_kp.time, b_kp.time) + && fuzzy_eq_f32(a_kp.color.r, b_kp.color.r) + && fuzzy_eq_f32(a_kp.color.g, b_kp.color.g) + && fuzzy_eq_f32(a_kp.color.b, b_kp.color.b)) { return false; } @@ -73,20 +99,17 @@ pub fn variant_eq(variant_a: &Variant, variant_b: &Variant) -> bool { (Variant::Enum(a), Variant::Enum(b)) => a == b, (Variant::EnumItem(a), Variant::EnumItem(b)) => a == b, (Variant::Faces(a), Variant::Faces(b)) => a == b, - (Variant::Float32(a), Variant::Float32(b)) => approx_eq!(f32, *a, *b), - (Variant::Float64(a), Variant::Float64(b)) => approx_eq!(f64, *a, *b), + (Variant::Float32(a), Variant::Float32(b)) => fuzzy_eq_f32(*a, *b), + (Variant::Float64(a), Variant::Float64(b)) => fuzzy_eq_f64(*a, *b), (Variant::Font(a), Variant::Font(b)) => { - a.weight == b.weight - && a.style == b.style - && a.family == b.family - && a.cached_face_id == b.cached_face_id + a.weight == b.weight && a.style == b.style && a.family == b.family } (Variant::Int32(a), Variant::Int32(b)) => a == b, (Variant::Int64(a), Variant::Int64(b)) => a == b, (Variant::MaterialColors(a), Variant::MaterialColors(b)) => a.encode() == b.encode(), (Variant::NetAssetRef(a), Variant::NetAssetRef(b)) => a == b, (Variant::NumberRange(a), Variant::NumberRange(b)) => { - approx_eq!(f32, a.max, b.max) && approx_eq!(f32, a.min, b.min) + fuzzy_eq_f32(a.max, b.max) && fuzzy_eq_f32(a.min, b.min) } (Variant::NumberSequence(a), Variant::NumberSequence(b)) => { if a.keypoints.len() != b.keypoints.len() { @@ -98,9 +121,9 @@ pub fn variant_eq(variant_a: &Variant, variant_b: &Variant) -> bool { b_keypoints.sort_unstable_by(|k1, k2| k1.time.partial_cmp(&k2.time).unwrap()); for (a_kp, b_kp) in a_keypoints.iter().zip(b_keypoints) { - if !(approx_eq!(f32, a_kp.time, b_kp.time) - && approx_eq!(f32, a_kp.value, b_kp.value) - && approx_eq!(f32, a_kp.envelope, b_kp.envelope)) + if !(fuzzy_eq_f32(a_kp.time, b_kp.time) + && fuzzy_eq_f32(a_kp.value, b_kp.value) + && fuzzy_eq_f32(a_kp.envelope, b_kp.envelope)) { return false; } @@ -120,12 +143,12 @@ pub fn variant_eq(variant_a: &Variant, variant_b: &Variant) -> bool { (Variant::PhysicalProperties(a), Variant::PhysicalProperties(b)) => match (a, b) { (PhysicalProperties::Default, PhysicalProperties::Default) => true, (PhysicalProperties::Custom(a2), PhysicalProperties::Custom(b2)) => { - approx_eq!(f32, a2.density(), b2.density()) - && approx_eq!(f32, a2.elasticity(), b2.elasticity()) - && approx_eq!(f32, a2.friction(), b2.friction()) - && approx_eq!(f32, a2.elasticity_weight(), b2.elasticity_weight()) - && approx_eq!(f32, a2.friction_weight(), b2.friction_weight()) - && approx_eq!(f32, a2.acoustic_absorption(), b2.acoustic_absorption()) + fuzzy_eq_f32(a2.density(), b2.density()) + && fuzzy_eq_f32(a2.elasticity(), b2.elasticity()) + && fuzzy_eq_f32(a2.friction(), b2.friction()) + && fuzzy_eq_f32(a2.elasticity_weight(), b2.elasticity_weight()) + && fuzzy_eq_f32(a2.friction_weight(), b2.friction_weight()) + && fuzzy_eq_f32(a2.acoustic_absorption(), b2.acoustic_absorption()) } _ => false, }, @@ -133,10 +156,10 @@ pub fn variant_eq(variant_a: &Variant, variant_b: &Variant) -> bool { vector_eq(&a.direction, &b.direction) && vector_eq(&a.origin, &b.origin) } (Variant::Rect(a), Variant::Rect(b)) => { - approx_eq!(f32, a.max.x, b.max.x) - && approx_eq!(f32, a.max.y, b.max.y) - && approx_eq!(f32, a.min.x, b.min.x) - && approx_eq!(f32, a.min.y, b.min.y) + fuzzy_eq_f32(a.max.x, b.max.x) + && fuzzy_eq_f32(a.max.y, b.max.y) + && fuzzy_eq_f32(a.min.x, b.min.x) + && fuzzy_eq_f32(a.min.y, b.min.y) } (Variant::Ref(a), Variant::Ref(b)) => a == b, (Variant::Region3(a), Variant::Region3(b)) => { @@ -162,18 +185,18 @@ pub fn variant_eq(variant_a: &Variant, variant_b: &Variant) -> bool { } } (Variant::UDim(a), Variant::UDim(b)) => { - approx_eq!(f32, a.scale, b.scale) && a.offset == b.offset + fuzzy_eq_f32(a.scale, b.scale) && a.offset == b.offset } (Variant::UDim2(a), Variant::UDim2(b)) => { - approx_eq!(f32, a.x.scale, b.x.scale) + fuzzy_eq_f32(a.x.scale, b.x.scale) && a.x.offset == b.x.offset - && approx_eq!(f32, a.y.scale, b.y.scale) + && fuzzy_eq_f32(a.y.scale, b.y.scale) && a.y.offset == b.y.offset } (Variant::UniqueId(a), Variant::UniqueId(b)) => a == b, (Variant::String(a), Variant::String(b)) => a == b, (Variant::Vector2(a), Variant::Vector2(b)) => { - approx_eq!(f32, a.x, b.x) && approx_eq!(f32, a.y, b.y) + fuzzy_eq_f32(a.x, b.x) && fuzzy_eq_f32(a.y, b.y) } (Variant::Vector2int16(a), Variant::Vector2int16(b)) => a == b, (Variant::Vector3(a), Variant::Vector3(b)) => vector_eq(a, b), @@ -188,5 +211,513 @@ pub fn variant_eq(variant_a: &Variant, variant_b: &Variant) -> bool { #[inline(always)] fn vector_eq(a: &Vector3, b: &Vector3) -> bool { - approx_eq!(f32, a.x, b.x) && approx_eq!(f32, a.y, b.y) && approx_eq!(f32, a.z, b.z) + fuzzy_eq_f32(a.x, b.x) && fuzzy_eq_f32(a.y, b.y) && fuzzy_eq_f32(a.z, b.z) +} + +// ============================================================================ +// Disk-representation equality: "will these two values produce identical bytes +// when written to a .meta.json5 / .model.json5 file?" +// +// Used by the matching algorithms in src/snapshot/matching.rs and +// src/syncback/matching.rs where the cost metric is file modifications. +// ============================================================================ + +#[inline(always)] +fn disk_eq_f32(a: f32, b: f32) -> bool { + let a = if a == 0.0 { 0.0_f32 } else { a }; + let b = if b == 0.0 { 0.0_f32 } else { b }; + if a.to_bits() == b.to_bits() { + return true; + } + let mut buf_a = [0u8; F32_DISK_BUF_SIZE]; + let mut buf_b = [0u8; F32_DISK_BUF_SIZE]; + let repr_a = a.to_lexical_with_options::(&mut buf_a, F32_DISK_OPTIONS); + let repr_b = b.to_lexical_with_options::(&mut buf_b, F32_DISK_OPTIONS); + repr_a == repr_b +} + +#[inline(always)] +fn disk_eq_f32_cleaned(a: f32, b: f32) -> bool { + disk_eq_f32(cleanup_f32(a), cleanup_f32(b)) +} + +#[inline(always)] +fn disk_eq_f64(a: f64, b: f64) -> bool { + let a = if a == 0.0 { 0.0_f64 } else { a }; + let b = if b == 0.0 { 0.0_f64 } else { b }; + if a.to_bits() == b.to_bits() { + return true; + } + let mut buf_a = [0u8; F64_DISK_BUF_SIZE]; + let mut buf_b = [0u8; F64_DISK_BUF_SIZE]; + let repr_a = a.to_lexical_with_options::(&mut buf_a, F64_DISK_OPTIONS); + let repr_b = b.to_lexical_with_options::(&mut buf_b, F64_DISK_OPTIONS); + repr_a == repr_b +} + +#[inline(always)] +fn vector_eq_disk(a: &Vector3, b: &Vector3) -> bool { + disk_eq_f32_cleaned(a.x, b.x) && disk_eq_f32_cleaned(a.y, b.y) && disk_eq_f32_cleaned(a.z, b.z) +} + +/// Compares two variants by their on-disk representation. Two values are +/// "equal" iff serializing both through resolution.rs + json.rs produces +/// identical bytes in the output file. Used by matching algorithms where +/// cost = number of file modifications. +#[inline] +pub fn variant_eq_disk(variant_a: &Variant, variant_b: &Variant) -> bool { + if variant_a.ty() != variant_b.ty() { + return false; + } + + match (variant_a, variant_b) { + (Variant::Attributes(a), Variant::Attributes(b)) => { + if a.len() != b.len() { + return false; + } + for ((a_name, a_value), (b_name, b_value)) in a.iter().zip(b.iter()) { + if !(a_name == b_name && variant_eq_disk(a_value, b_value)) { + return false; + } + } + true + } + (Variant::Axes(a), Variant::Axes(b)) => a == b, + (Variant::BinaryString(a), Variant::BinaryString(b)) => a == b, + (Variant::Bool(a), Variant::Bool(b)) => a == b, + (Variant::BrickColor(a), Variant::BrickColor(b)) => a == b, + (Variant::CFrame(a), Variant::CFrame(b)) => { + vector_eq_disk(&a.position, &b.position) + && vector_eq_disk(&a.orientation.x, &b.orientation.x) + && vector_eq_disk(&a.orientation.y, &b.orientation.y) + && vector_eq_disk(&a.orientation.z, &b.orientation.z) + } + (Variant::Color3(a), Variant::Color3(b)) => { + disk_eq_f32_cleaned(a.r, b.r) + && disk_eq_f32_cleaned(a.g, b.g) + && disk_eq_f32_cleaned(a.b, b.b) + } + (Variant::Color3uint8(a), Variant::Color3uint8(b)) => a == b, + (Variant::ColorSequence(a), Variant::ColorSequence(b)) => { + if a.keypoints.len() != b.keypoints.len() { + return false; + } + let mut a_keypoints: Vec<_> = a.keypoints.iter().collect(); + let mut b_keypoints: Vec<_> = b.keypoints.iter().collect(); + a_keypoints.sort_unstable_by(|k1, k2| k1.time.partial_cmp(&k2.time).unwrap()); + b_keypoints.sort_unstable_by(|k1, k2| k1.time.partial_cmp(&k2.time).unwrap()); + + for (a_kp, b_kp) in a_keypoints.iter().zip(b_keypoints) { + if !(disk_eq_f32_cleaned(a_kp.time, b_kp.time) + && disk_eq_f32_cleaned(a_kp.color.r, b_kp.color.r) + && disk_eq_f32_cleaned(a_kp.color.g, b_kp.color.g) + && disk_eq_f32_cleaned(a_kp.color.b, b_kp.color.b)) + { + return false; + } + } + true + } + (Variant::Content(a), Variant::Content(b)) => a == b, + (Variant::ContentId(a), Variant::ContentId(b)) => a == b, + (Variant::Enum(a), Variant::Enum(b)) => a == b, + (Variant::EnumItem(a), Variant::EnumItem(b)) => a == b, + (Variant::Faces(a), Variant::Faces(b)) => a == b, + (Variant::Float32(a), Variant::Float32(b)) => disk_eq_f32(*a, *b), + (Variant::Float64(a), Variant::Float64(b)) => disk_eq_f64(*a, *b), + (Variant::Font(a), Variant::Font(b)) => { + a.weight == b.weight && a.style == b.style && a.family == b.family + } + (Variant::Int32(a), Variant::Int32(b)) => a == b, + (Variant::Int64(a), Variant::Int64(b)) => a == b, + (Variant::MaterialColors(a), Variant::MaterialColors(b)) => a.encode() == b.encode(), + (Variant::NetAssetRef(a), Variant::NetAssetRef(b)) => a == b, + (Variant::NumberRange(a), Variant::NumberRange(b)) => { + disk_eq_f32_cleaned(a.max, b.max) && disk_eq_f32_cleaned(a.min, b.min) + } + (Variant::NumberSequence(a), Variant::NumberSequence(b)) => { + if a.keypoints.len() != b.keypoints.len() { + return false; + } + let mut a_keypoints: Vec<_> = a.keypoints.iter().collect(); + let mut b_keypoints: Vec<_> = b.keypoints.iter().collect(); + a_keypoints.sort_unstable_by(|k1, k2| k1.time.partial_cmp(&k2.time).unwrap()); + b_keypoints.sort_unstable_by(|k1, k2| k1.time.partial_cmp(&k2.time).unwrap()); + + for (a_kp, b_kp) in a_keypoints.iter().zip(b_keypoints) { + if !(disk_eq_f32_cleaned(a_kp.time, b_kp.time) + && disk_eq_f32_cleaned(a_kp.value, b_kp.value) + && disk_eq_f32_cleaned(a_kp.envelope, b_kp.envelope)) + { + return false; + } + } + true + } + (Variant::OptionalCFrame(a), Variant::OptionalCFrame(b)) => match (a, b) { + (Some(a), Some(b)) => { + vector_eq_disk(&a.position, &b.position) + && vector_eq_disk(&a.orientation.x, &b.orientation.x) + && vector_eq_disk(&a.orientation.y, &b.orientation.y) + && vector_eq_disk(&a.orientation.z, &b.orientation.z) + } + (None, None) => true, + _ => false, + }, + (Variant::PhysicalProperties(a), Variant::PhysicalProperties(b)) => match (a, b) { + (PhysicalProperties::Default, PhysicalProperties::Default) => true, + (PhysicalProperties::Custom(a2), PhysicalProperties::Custom(b2)) => { + disk_eq_f32_cleaned(a2.density(), b2.density()) + && disk_eq_f32_cleaned(a2.elasticity(), b2.elasticity()) + && disk_eq_f32_cleaned(a2.friction(), b2.friction()) + && disk_eq_f32_cleaned(a2.elasticity_weight(), b2.elasticity_weight()) + && disk_eq_f32_cleaned(a2.friction_weight(), b2.friction_weight()) + && disk_eq_f32_cleaned(a2.acoustic_absorption(), b2.acoustic_absorption()) + } + _ => false, + }, + (Variant::Ray(a), Variant::Ray(b)) => { + vector_eq_disk(&a.direction, &b.direction) && vector_eq_disk(&a.origin, &b.origin) + } + (Variant::Rect(a), Variant::Rect(b)) => { + disk_eq_f32_cleaned(a.max.x, b.max.x) + && disk_eq_f32_cleaned(a.max.y, b.max.y) + && disk_eq_f32_cleaned(a.min.x, b.min.x) + && disk_eq_f32_cleaned(a.min.y, b.min.y) + } + (Variant::Ref(a), Variant::Ref(b)) => a == b, + (Variant::Region3(a), Variant::Region3(b)) => { + vector_eq_disk(&a.max, &b.max) && vector_eq_disk(&a.min, &b.min) + } + (Variant::Region3int16(a), Variant::Region3int16(b)) => a == b, + (Variant::SecurityCapabilities(a), Variant::SecurityCapabilities(b)) => a == b, + (Variant::SharedString(a), Variant::SharedString(b)) => a == b, + (Variant::Tags(a), Variant::Tags(b)) => { + let mut a_sorted: Vec<&str> = a.iter().collect(); + let mut b_sorted: Vec<&str> = b.iter().collect(); + if a_sorted.len() == b_sorted.len() { + a_sorted.sort_unstable(); + b_sorted.sort_unstable(); + for (a_tag, b_tag) in a_sorted.into_iter().zip(b_sorted) { + if a_tag != b_tag { + return false; + } + } + true + } else { + false + } + } + (Variant::UDim(a), Variant::UDim(b)) => { + disk_eq_f32_cleaned(a.scale, b.scale) && a.offset == b.offset + } + (Variant::UDim2(a), Variant::UDim2(b)) => { + disk_eq_f32_cleaned(a.x.scale, b.x.scale) + && a.x.offset == b.x.offset + && disk_eq_f32_cleaned(a.y.scale, b.y.scale) + && a.y.offset == b.y.offset + } + (Variant::UniqueId(a), Variant::UniqueId(b)) => a == b, + (Variant::String(a), Variant::String(b)) => a == b, + (Variant::Vector2(a), Variant::Vector2(b)) => { + disk_eq_f32_cleaned(a.x, b.x) && disk_eq_f32_cleaned(a.y, b.y) + } + (Variant::Vector2int16(a), Variant::Vector2int16(b)) => a == b, + (Variant::Vector3(a), Variant::Vector3(b)) => vector_eq_disk(a, b), + (Variant::Vector3int16(a), Variant::Vector3int16(b)) => a == b, + (a, b) => panic!( + "unsupported variant disk comparison: {:?} and {:?}", + a.ty(), + b.ty() + ), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn fuzzy_eq_matches_lua_absolute_epsilon() { + assert!(fuzzy_eq_f32(1.0, 1.0 + 0.000099)); + assert!(!fuzzy_eq_f32(1.0, 1.0 + 0.00011)); + } + + #[test] + fn fuzzy_eq_matches_lua_relative_epsilon() { + assert!(fuzzy_eq_f32(10000.0, 10000.0 + 0.9)); + assert!(!fuzzy_eq_f32(10000.0, 10000.0 + 1.1)); + } + + #[test] + fn fuzzy_eq_nan_handling() { + assert!(fuzzy_eq_f32(f32::NAN, f32::NAN)); + assert!(!fuzzy_eq_f32(f32::NAN, 0.0)); + assert!(!fuzzy_eq_f32(0.0, f32::NAN)); + } + + #[test] + fn fuzzy_eq_zero_and_negative_zero() { + assert!(fuzzy_eq_f32(0.0, -0.0)); + } + + #[test] + fn fuzzy_eq_f64_basic() { + assert!(fuzzy_eq_f64(1.0, 1.0 + 0.000099)); + assert!(!fuzzy_eq_f64(1.0, 1.0 + 0.00011)); + assert!(fuzzy_eq_f64(f64::NAN, f64::NAN)); + } + + #[test] + fn variant_eq_float32_with_new_epsilon() { + assert!(variant_eq( + &Variant::Float32(1.0), + &Variant::Float32(1.0 + 0.000099) + )); + assert!(!variant_eq( + &Variant::Float32(1.0), + &Variant::Float32(1.0 + 0.00011) + )); + } + + // ================================================================ + // disk_eq / variant_eq_disk tests + // ================================================================ + + #[test] + fn disk_eq_f32_identical_bits() { + assert!(disk_eq_f32(1.0, 1.0)); + assert!(disk_eq_f32(0.0, 0.0)); + assert!(disk_eq_f32(-1.5, -1.5)); + } + + #[test] + fn disk_eq_f32_negative_zero() { + assert!(disk_eq_f32(0.0, -0.0)); + } + + #[test] + fn disk_eq_f32_nan() { + assert!(disk_eq_f32(f32::NAN, f32::NAN)); + assert!(!disk_eq_f32(f32::NAN, 0.0)); + assert!(!disk_eq_f32(0.0, f32::NAN)); + } + + #[test] + fn disk_eq_f32_infinity() { + assert!(disk_eq_f32(f32::INFINITY, f32::INFINITY)); + assert!(disk_eq_f32(f32::NEG_INFINITY, f32::NEG_INFINITY)); + assert!(!disk_eq_f32(f32::INFINITY, f32::NEG_INFINITY)); + assert!(!disk_eq_f32(f32::INFINITY, 0.0)); + } + + #[test] + fn disk_eq_f32_fuzzy_says_equal_but_disk_differs() { + let a: f32 = 10000.0; + let b: f32 = 10000.5; + assert!( + fuzzy_eq_f32(a, b), + "precondition: fuzzy should say equal (diff=0.5, relative threshold=1.0)" + ); + assert!( + !disk_eq_f32(a, b), + "disk representations differ: 6 sig digits gives '10000' vs '10000.5'" + ); + } + + #[test] + fn disk_eq_f64_basic() { + assert!(disk_eq_f64(1.0, 1.0)); + assert!(disk_eq_f64(0.0, -0.0)); + assert!(disk_eq_f64(f64::NAN, f64::NAN)); + assert!(!disk_eq_f64(1.0, 2.0)); + } + + #[test] + fn disk_eq_f32_cleaned_tiny_values_zeroed() { + assert!( + disk_eq_f32_cleaned(0.0000001, 0.0), + "cleanup_f32 zeros 1e-7 to 0.0" + ); + assert!( + !disk_eq_f32(0.0000001, 0.0), + "without cleanup, 1e-7 and 0 are different on disk" + ); + } + + #[test] + fn disk_eq_f32_cleaned_preserves_significant_values() { + assert!(!disk_eq_f32_cleaned(0.5, 0.6)); + assert!(disk_eq_f32_cleaned(0.5, 0.5)); + } + + #[test] + fn variant_eq_disk_vector3_tiny_component() { + assert!(variant_eq_disk( + &Variant::Vector3(Vector3::new(1.0, 2.0, 0.0000001)), + &Variant::Vector3(Vector3::new(1.0, 2.0, 0.0)), + )); + } + + #[test] + fn variant_eq_disk_vector3_different() { + assert!(!variant_eq_disk( + &Variant::Vector3(Vector3::new(1.0, 2.0, 3.0)), + &Variant::Vector3(Vector3::new(1.0, 2.0, 4.0)), + )); + } + + #[test] + fn variant_eq_disk_color3() { + use rbx_dom_weak::types::Color3; + assert!(variant_eq_disk( + &Variant::Color3(Color3::new(1.0, 0.0, 0.0)), + &Variant::Color3(Color3::new(1.0, 0.0, 0.0)), + )); + assert!(!variant_eq_disk( + &Variant::Color3(Color3::new(1.0, 0.0, 0.0)), + &Variant::Color3(Color3::new(0.0, 1.0, 0.0)), + )); + } + + #[test] + fn variant_eq_disk_float32_standalone() { + assert!(variant_eq_disk( + &Variant::Float32(0.5), + &Variant::Float32(0.5) + )); + assert!(!variant_eq_disk( + &Variant::Float32(0.5), + &Variant::Float32(0.6) + )); + } + + #[test] + fn variant_eq_disk_float32_diverges_from_fuzzy() { + let a: f32 = 10000.0; + let b: f32 = 10000.5; + assert!( + variant_eq(&Variant::Float32(a), &Variant::Float32(b)), + "fuzzy says equal (diff=0.5, relative threshold=1.0)" + ); + assert!( + !variant_eq_disk(&Variant::Float32(a), &Variant::Float32(b)), + "disk says different ('10000' vs '10000.5')" + ); + } + + #[test] + fn variant_eq_disk_non_float_types() { + assert!(variant_eq_disk(&Variant::Bool(true), &Variant::Bool(true))); + assert!(!variant_eq_disk( + &Variant::Bool(true), + &Variant::Bool(false) + )); + assert!(variant_eq_disk(&Variant::Int32(42), &Variant::Int32(42))); + assert!(!variant_eq_disk(&Variant::Int32(42), &Variant::Int32(43))); + assert!(variant_eq_disk( + &Variant::String("hello".into()), + &Variant::String("hello".into()), + )); + assert!(!variant_eq_disk( + &Variant::String("hello".into()), + &Variant::String("world".into()), + )); + } + + #[test] + fn variant_eq_disk_udim2_scale_cleaned() { + use rbx_dom_weak::types::{UDim, UDim2}; + assert!(variant_eq_disk( + &Variant::UDim2(UDim2::new(UDim::new(0.5, 0), UDim::new(0.0000001, 100))), + &Variant::UDim2(UDim2::new(UDim::new(0.5, 0), UDim::new(0.0, 100))), + )); + } + + #[test] + fn variant_eq_disk_number_range() { + use rbx_dom_weak::types::NumberRange; + assert!(variant_eq_disk( + &Variant::NumberRange(NumberRange::new(0.0, 1.0)), + &Variant::NumberRange(NumberRange::new(0.0, 1.0)), + )); + assert!(!variant_eq_disk( + &Variant::NumberRange(NumberRange::new(0.0, 1.0)), + &Variant::NumberRange(NumberRange::new(0.0, 2.0)), + )); + } + + #[test] + fn variant_eq_disk_tags_sorted() { + use rbx_dom_weak::types::Tags; + let mut tags_ab = Tags::new(); + tags_ab.push("A"); + tags_ab.push("B"); + let mut tags_ba = Tags::new(); + tags_ba.push("B"); + tags_ba.push("A"); + assert!(variant_eq_disk( + &Variant::Tags(tags_ab), + &Variant::Tags(tags_ba) + )); + } + + #[test] + fn variant_eq_disk_attributes_recursive() { + use rbx_dom_weak::types::Attributes; + let mut a = Attributes::new(); + a.insert("Health".into(), Variant::Float32(100.0)); + let mut b = Attributes::new(); + b.insert("Health".into(), Variant::Float32(100.0)); + assert!(variant_eq_disk( + &Variant::Attributes(a), + &Variant::Attributes(b), + )); + } + + #[test] + fn variant_eq_disk_cframe() { + use rbx_dom_weak::types::{CFrame, Matrix3}; + let identity = Matrix3::new( + Vector3::new(1.0, 0.0, 0.0), + Vector3::new(0.0, 1.0, 0.0), + Vector3::new(0.0, 0.0, 1.0), + ); + assert!(variant_eq_disk( + &Variant::CFrame(CFrame::new(Vector3::new(1.0, 2.0, 3.0), identity)), + &Variant::CFrame(CFrame::new(Vector3::new(1.0, 2.0, 3.0), identity)), + )); + assert!(!variant_eq_disk( + &Variant::CFrame(CFrame::new(Vector3::new(1.0, 2.0, 3.0), identity)), + &Variant::CFrame(CFrame::new(Vector3::new(1.0, 2.0, 4.0), identity)), + )); + } + + #[test] + fn variant_eq_disk_rect() { + use rbx_dom_weak::types::{Rect, Vector2}; + assert!(variant_eq_disk( + &Variant::Rect(Rect::new( + Vector2::new(0.0, 0.0), + Vector2::new(100.0, 100.0) + )), + &Variant::Rect(Rect::new( + Vector2::new(0.0, 0.0), + Vector2::new(100.0, 100.0) + )), + )); + } + + #[test] + fn variant_eq_disk_vector2() { + use rbx_dom_weak::types::Vector2; + assert!(variant_eq_disk( + &Variant::Vector2(Vector2::new(1.0, 2.0)), + &Variant::Vector2(Vector2::new(1.0, 2.0)), + )); + assert!(!variant_eq_disk( + &Variant::Vector2(Vector2::new(1.0, 2.0)), + &Variant::Vector2(Vector2::new(1.0, 3.0)), + )); + } } diff --git a/src/web/api.rs b/src/web/api.rs index e0a63678..eb2e839d 100644 --- a/src/web/api.rs +++ b/src/web/api.rs @@ -30,7 +30,8 @@ use crate::{ interface::{ ErrorResponse, Instance, InstanceMetadata, MessagesPacket, OpenResponse, ReadResponse, ServerInfoResponse, SocketPacket, SocketPacketBody, SocketPacketType, SubscribeMessage, - WriteRequest, WriteResponse, PROTOCOL_VERSION, SERVER_VERSION, + SyncbackPayload, SyncbackRequest, WriteRequest, WriteResponse, PROTOCOL_VERSION, + SERVER_VERSION, }, util::{deserialize_msgpack, msgpack, msgpack_ok, serialize_msgpack}, }, @@ -122,6 +123,7 @@ fn variant_to_json(variant: &Variant) -> Option { pub async fn call( serve_session: Arc, mut request: Request, + syncback_signal: Arc, ) -> Response> { let service = ApiService::new(serve_session); @@ -153,6 +155,9 @@ pub async fn call( service.handle_api_open(request).await } (&Method::POST, "/api/write") => service.handle_api_write(request).await, + (&Method::POST, "/api/syncback") => { + handle_api_syncback(request, &service, syncback_signal).await + } (&Method::GET, "/api/validate-tree") => service.handle_api_validate_tree().await, (_method, path) => msgpack( @@ -162,6 +167,105 @@ pub async fn call( } } +async fn handle_api_syncback( + request: Request, + service: &ApiService, + syncback_signal: Arc, +) -> Response> { + let body = match request.into_body().collect().await { + Ok(collected) => collected.to_bytes(), + Err(err) => { + return msgpack( + ErrorResponse::bad_request(format!("Failed to read request body: {err}")), + StatusCode::BAD_REQUEST, + ); + } + }; + + let syncback_request: SyncbackRequest = match deserialize_msgpack(&body) { + Ok(req) => req, + Err(err) => { + return msgpack( + ErrorResponse::bad_request(format!( + "Failed to deserialize syncback request: {err}" + )), + StatusCode::BAD_REQUEST, + ); + } + }; + + let client_protocol = syncback_request.protocol_version as u64; + if client_protocol != PROTOCOL_VERSION { + return msgpack( + ErrorResponse::bad_request(format!( + "Protocol version mismatch: expected {}, got {}", + PROTOCOL_VERSION, client_protocol + )), + StatusCode::BAD_REQUEST, + ); + } + + let server_semver = SERVER_VERSION; + let server_parts: Vec<&str> = server_semver.split('.').collect(); + let client_parts: Vec<&str> = syncback_request.server_version.split('.').collect(); + let server_major_minor = server_parts.get(..2); + let client_major_minor = client_parts.get(..2); + if server_major_minor != client_major_minor { + return msgpack( + ErrorResponse::bad_request(format!( + "Server version mismatch: server is {}, plugin expects {}", + SERVER_VERSION, syncback_request.server_version + )), + StatusCode::BAD_REQUEST, + ); + } + + if let Some(place_id_f64) = syncback_request.place_id { + let place_id = place_id_f64 as u64; + if let Some(expected) = service.serve_session.serve_place_ids() { + if !expected.contains(&place_id) { + return msgpack( + ErrorResponse::bad_request(format!( + "Place ID {} is not in the servePlaceIds whitelist", + place_id + )), + StatusCode::FORBIDDEN, + ); + } + } + if let Some(blocked) = service.serve_session.blocked_place_ids() { + if blocked.contains(&place_id) { + return msgpack( + ErrorResponse::bad_request(format!( + "Place ID {} is in the blockedPlaceIds list", + place_id + )), + StatusCode::FORBIDDEN, + ); + } + } + } + + log::info!( + "Live syncback requested with {} service chunks, {} bytes of rbxm data", + syncback_request.services.len(), + syncback_request.data.len() + ); + + let payload = SyncbackPayload { + data: syncback_request.data, + services: syncback_request.services, + }; + if !syncback_signal.fire(payload) { + return msgpack( + ErrorResponse::bad_request("A syncback is already in progress"), + StatusCode::CONFLICT, + ); + } + + msgpack_ok(serde_json::json!({"status": "syncback_initiated"})) +} + pub struct ApiService { serve_session: Arc, suppressed_paths: Arc>>, @@ -264,8 +368,7 @@ impl ApiService { /// Get a summary of information about the server async fn handle_api_rojo(&self) -> Response> { - let tree = self.serve_session.tree(); - let root_instance_id = tree.get_root_id(); + let root_instance_id = self.serve_session.tree().get_root_id(); let ignore_hidden_services = self.serve_session.ignore_hidden_services(); let visible_services = if ignore_hidden_services { @@ -274,6 +377,10 @@ impl ApiService { Vec::new() }; + let git_metadata = self.serve_session.git_repo_root().map(|repo_root| { + crate::git::compute_git_metadata(&self.serve_session.tree_handle(), repo_root) + }); + msgpack_ok(&ServerInfoResponse { server_version: SERVER_VERSION.to_owned(), protocol_version: PROTOCOL_VERSION, @@ -288,6 +395,7 @@ impl ApiService { sync_source_only: true, ignore_hidden_services, visible_services, + git_metadata, }) } @@ -413,6 +521,11 @@ impl ApiService { // Create a stats tracker for this syncback operation let stats = crate::syncback::SyncbackStats::new(); + // Track GUIDs that were updated in place (instance already existed + // in the tree). These must be excluded from the PatchAdd list to + // avoid creating duplicate instances in the tree. + let mut updated_in_place: HashSet = HashSet::new(); + if !request.added.is_empty() { let tree = self.serve_session.tree(); // Pre-compute duplicate sibling info in O(N) for efficient path uniqueness checks @@ -523,21 +636,28 @@ impl ApiService { // Group added instances by parent so siblings share a dedup set. // This prevents two instances in the same batch from claiming the // same slug when added to the same parent. - let mut adds_by_parent: HashMap> = - HashMap::new(); - for added in request.added.values() { + let mut adds_by_parent: HashMap< + Ref, + Vec<(Ref, &crate::web::interface::AddedInstance)>, + > = HashMap::new(); + for (guid, added) in &request.added { if let Some(parent_ref) = added.parent { - adds_by_parent.entry(parent_ref).or_default().push(added); + adds_by_parent + .entry(parent_ref) + .or_default() + .push((*guid, added)); } else { - // No parent — process individually (will fail with context) - adds_by_parent.entry(Ref::none()).or_default().push(added); + adds_by_parent + .entry(Ref::none()) + .or_default() + .push((*guid, added)); } } for (parent_ref, siblings) in &adds_by_parent { // Pre-seed sibling_slugs from the tree's existing children // of this parent so new instances dedup against existing ones. // We derive slugs from actual filesystem paths (via instigating_source) - // to correctly account for dedup suffixes (e.g., Hey_Bro~1). + // to correctly account for dedup suffixes (e.g., Hey_Bro~2). let mut sibling_slugs: HashSet = if *parent_ref != Ref::none() { if let Some(parent_inst) = tree.get_instance(*parent_ref) { use crate::snapshot::InstigatingSource; @@ -576,8 +696,8 @@ impl ApiService { } else { HashSet::new() }; - for added in siblings { - if let Err(err) = self.syncback_added_instance( + for (guid, added) in siblings { + match self.syncback_added_instance( added, &tree, &duplicate_siblings_cache, @@ -585,11 +705,17 @@ impl ApiService { &converted_parents, &mut sibling_slugs, ) { - log::warn!( - "Failed to syncback added instance '{}': {}", - added.name, - err - ); + Ok(true) => { + updated_in_place.insert(*guid); + } + Ok(false) => {} + Err(err) => { + log::warn!( + "Failed to syncback added instance '{}': {}", + added.name, + err + ); + } } } } @@ -601,6 +727,9 @@ impl ApiService { // Build a path map for instances added in this same request. // This allows Ref properties in updates to reference just-added // instances that aren't in the tree yet (VFS watcher is async). + // The segment includes the middleware extension (e.g. .server.luau, + // .model.json5) so that Rojo_Ref_* paths match what the filesystem + // will actually contain. let added_paths: HashMap = { let tree = self.serve_session.tree(); request @@ -609,15 +738,15 @@ impl ApiService { .filter_map(|(guid, added)| { let parent_ref = added.parent?; let parent_path = if tree.get_instance(parent_ref).is_some() { - crate::ref_target_path(tree.inner(), parent_ref) + crate::ref_target_path_from_tree(&tree, parent_ref) } else { return None; }; - let escaped_name = crate::escape_ref_path_segment(&added.name); + let fs_name = added_instance_fs_segment(added); let path = if parent_path.is_empty() { - escaped_name.into_owned() + fs_name } else { - format!("{}/{}", parent_path, escaped_name) + format!("{}/{}", parent_path, fs_name) }; Some((*guid, path)) }) @@ -696,9 +825,15 @@ impl ApiService { // plugin's GUID is used as the snapshot_id so that apply_patch_set's // snapshot_id_to_instance_id map rewrites Ref properties in // updated_instances to point to the correct tree IDs. + // + // Instances that were updated in place (already existed in the tree) + // are excluded — adding them would create duplicates. Tree + // reconciliation will pick up any property changes from the + // filesystem write. let added_instances: Vec = request .added .iter() + .filter(|(guid, _)| !updated_in_place.contains(guid)) .filter_map(|(guid, added)| { let parent_ref = added.parent?; Some(PatchAdd { @@ -708,11 +843,32 @@ impl ApiService { }) .collect(); + let stage_ids: HashSet = request.stage_ids.into_iter().collect(); + + let paths_to_stage: Vec = if !stage_ids.is_empty() { + let tree = self.serve_session.tree(); + stage_ids + .iter() + .filter_map(|&id| { + tree.get_instance(id).and_then(|inst| { + inst.metadata() + .instigating_source + .as_ref() + .map(|src| src.path().to_path_buf()) + }) + }) + .collect() + } else { + Vec::new() + }; + tree_mutation_sender .send(PatchSet { removed_instances: actually_removed, added_instances, updated_instances, + stage_ids, + stage_paths: paths_to_stage, }) .unwrap(); @@ -911,7 +1067,7 @@ impl ApiService { stats: &crate::syncback::SyncbackStats, converted_parents: &HashMap, sibling_slugs: &mut HashSet, - ) -> anyhow::Result<()> { + ) -> anyhow::Result { use anyhow::Context; // Get the parent Ref (required for top-level added instances) @@ -947,8 +1103,11 @@ impl ApiService { added.name, existing_path.display() ); - // Update the existing instance instead of creating new files - return self.syncback_update_existing_instance(added, existing_path, stats); + // Update the existing instance instead of creating new files. + // Return true to signal caller to skip PatchAdd (instance + // already exists in tree, no need to add a duplicate). + self.syncback_update_existing_instance(added, existing_path, stats)?; + return Ok(true); } } } @@ -1048,7 +1207,7 @@ impl ApiService { let slug = self.syncback_instance_to_path_with_stats(added, &parent_dir, stats, sibling_slugs)?; sibling_slugs.insert(slug); - Ok(()) + Ok(false) } /// Update an existing instance in place instead of creating new files. @@ -1197,14 +1356,10 @@ impl ApiService { new_dir }; - // Filter duplicate children - let inst_path = format!("{}", existing_path.display()); - let unique_children = - self.filter_duplicate_children(&added.children, &inst_path, stats); - // Process children using normal syncback path // This will use detect_existing_script_format to check existing files - self.process_children_incremental(&unique_children, &children_dir, stats)?; + let children_refs: Vec<_> = added.children.iter().collect(); + self.process_children_incremental(&children_refs, &children_dir, stats)?; } } @@ -1234,11 +1389,8 @@ impl ApiService { // Handle children if !added.children.is_empty() { - let inst_path = format!("{}", existing_path.display()); - let unique_children = - self.filter_duplicate_children(&added.children, &inst_path, stats); - - self.process_children_incremental(&unique_children, existing_path, stats)?; + let children_refs: Vec<_> = added.children.iter().collect(); + self.process_children_incremental(&children_refs, existing_path, stats)?; } } else { // It's a standalone file (e.g., .model.json5) @@ -1654,48 +1806,6 @@ impl ApiService { true } - /// Check for duplicate names among children and return the set of unique children - /// that should be synced. Records skipped instances via stats tracker. - fn filter_duplicate_children<'a>( - &self, - children: &'a [crate::web::interface::AddedInstance], - parent_path: &str, - stats: &crate::syncback::SyncbackStats, - ) -> Vec<&'a crate::web::interface::AddedInstance> { - use std::collections::HashMap; - - // Count occurrences of each name - let mut name_counts: HashMap<&str, usize> = HashMap::new(); - for child in children { - *name_counts.entry(&child.name).or_insert(0) += 1; - } - - // Find duplicates - let duplicates: std::collections::HashSet<&str> = name_counts - .iter() - .filter(|(_, &count)| count > 1) - .map(|(&name, _)| name) - .collect(); - - // Record skipped duplicates in stats - if !duplicates.is_empty() { - let mut skipped_count = 0; - for child in children { - if duplicates.contains(child.name.as_str()) { - skipped_count += 1; - } - } - let duplicate_list: Vec<&str> = duplicates.iter().copied().collect(); - stats.record_duplicate_names_batch(parent_path, &duplicate_list, skipped_count); - } - - // Return only non-duplicate children - children - .iter() - .filter(|child| !duplicates.contains(child.name.as_str())) - .collect() - } - /// Detects what file format currently exists on disk for a given instance. /// This is used to preserve the existing format during partial updates. /// @@ -1819,19 +1929,15 @@ impl ApiService { // not filenames with extensions). This ensures file-format instances // are correctly detected as collisions. let encoded_name = deduplicate_name(&base_name, sibling_slugs); - let needs_meta_name = needs_slug || encoded_name != base_name; + let needs_meta_name = needs_slug; let meta_name_field: Option<&str> = if needs_meta_name { Some(&added.name) } else { None }; - // Build path string for stats - let inst_path = format!("{}/{}", parent_dir.display(), added.name); - - // Filter out children with duplicate names (cannot reliably sync) - let unique_children = self.filter_duplicate_children(&added.children, &inst_path, stats); - let has_children = !unique_children.is_empty(); + let children_refs: Vec<_> = added.children.iter().collect(); + let has_children = !children_refs.is_empty(); // Determine the appropriate middleware/file format based on class name. // This matches the logic in src/syncback/mod.rs::get_best_middleware. @@ -1887,7 +1993,7 @@ impl ApiService { })?; self.write_script_meta_json_if_needed(&dir_path, added, meta_name_field)?; log::info!("Syncback: Updated ModuleScript at {}", init_path.display()); - self.process_children_incremental(&unique_children, &dir_path, stats)?; + self.process_children_incremental(&children_refs, &dir_path, stats)?; } else { let file_path = parent_dir.join(format!("{}.luau", encoded_name)); fs::write(&file_path, source.as_bytes()).with_context(|| { @@ -1937,7 +2043,7 @@ impl ApiService { })?; self.write_script_meta_json_if_needed(&dir_path, added, meta_name_field)?; log::info!("Syncback: Updated Script at {}", init_path.display()); - self.process_children_incremental(&unique_children, &dir_path, stats)?; + self.process_children_incremental(&children_refs, &dir_path, stats)?; } else { let file_path = parent_dir.join(format!("{}.{}.luau", encoded_name, script_suffix)); @@ -1987,7 +2093,7 @@ impl ApiService { })?; self.write_script_meta_json_if_needed(&dir_path, added, meta_name_field)?; log::info!("Syncback: Updated LocalScript at {}", init_path.display()); - self.process_children_incremental(&unique_children, &dir_path, stats)?; + self.process_children_incremental(&children_refs, &dir_path, stats)?; } else { let file_path = parent_dir.join(format!("{}.local.luau", encoded_name)); fs::write(&file_path, source.as_bytes()).with_context(|| { @@ -2033,7 +2139,7 @@ impl ApiService { ); // Recursively process children - self.process_children_incremental(&unique_children, &dir_path, stats)?; + self.process_children_incremental(&children_refs, &dir_path, stats)?; } // StringValue: .txt file if no children, directory with init.meta.json5 if has children @@ -2049,7 +2155,7 @@ impl ApiService { "Syncback: Created StringValue directory at {}", dir_path.display() ); - self.process_children_incremental(&unique_children, &dir_path, stats)?; + self.process_children_incremental(&children_refs, &dir_path, stats)?; } else { let value = added .properties @@ -2103,7 +2209,7 @@ impl ApiService { "Syncback: Created LocalizationTable directory at {}", dir_path.display() ); - self.process_children_incremental(&unique_children, &dir_path, stats)?; + self.process_children_incremental(&children_refs, &dir_path, stats)?; } else { let file_path = parent_dir.join(format!("{}.csv", encoded_name)); fs::write(&file_path, content.as_bytes()).with_context(|| { @@ -2167,7 +2273,7 @@ impl ApiService { ); // Recursively process children - self.process_children_incremental(&unique_children, &dir_path, stats)?; + self.process_children_incremental(&children_refs, &dir_path, stats)?; } else { let content = self.serialize_instance_to_model_json(added, meta_name_field)?; // Use the detected file path if available (preserves .model.json @@ -2689,15 +2795,11 @@ impl ApiService { Some(Variant::Ref(target_ref)) => { let attr_name = crate::ref_attribute_name(key); if target_ref.is_none() { - // Null ref: mark attribute for removal remove_attributes.push(attr_name); } else if tree.get_instance(*target_ref).is_some() { - // Valid target: compute path and add as Rojo_Ref_* attribute - let path = crate::ref_target_path(tree.inner(), *target_ref); + let source_abs = crate::ref_target_path_from_tree(tree, update.id); + let target_abs = crate::ref_target_path_from_tree(tree, *target_ref); - // Warn if the path is ambiguous (duplicate-named siblings - // at any ancestor level). The ref will still be written, - // but may resolve to the wrong target on rebuild. if !Self::is_ref_path_unique(tree, *target_ref) { log::warn!( "Ref property '{}' for instance {:?} has an ambiguous \ @@ -2705,24 +2807,24 @@ impl ApiService { resolve to the wrong target on rebuild.", key, update.id, - path + target_abs ); } - ref_attributes.insert(attr_name, serde_json::Value::String(path)); - } else if let Some(path) = added_paths.get(target_ref) { - // Target was added in the same /api/write request. - // It's not in the tree yet (VFS watcher hasn't picked it up), - // but we can compute the path from the AddedInstance data. + let relative = crate::compute_relative_ref_path(&source_abs, &target_abs); + ref_attributes.insert(attr_name, serde_json::Value::String(relative)); + } else if let Some(target_abs) = added_paths.get(target_ref) { + let source_abs = crate::ref_target_path_from_tree(tree, update.id); + let relative = crate::compute_relative_ref_path(&source_abs, target_abs); log::info!( "Ref property '{}' for instance {:?}: target {:?} is a \ - just-added instance, using pre-computed path '{}'", + just-added instance, computed relative path '{}'", key, update.id, target_ref, - path + relative ); - ref_attributes.insert(attr_name, serde_json::Value::String(path.clone())); + ref_attributes.insert(attr_name, serde_json::Value::String(relative)); } else { log::warn!( "Cannot persist Ref property '{}' for instance {:?}: \ @@ -2879,6 +2981,7 @@ impl ApiService { // additions/removals (which can miss remaining attrs when one is // removed from a file that has multiple). { + let source_abs = crate::ref_target_path_from_tree(tree, update.id); let mut index = self.ref_path_index.lock().unwrap(); index.remove_all_for_file(&written_meta_path); @@ -2888,7 +2991,10 @@ impl ApiService { for (key, value) in attrs { if key.starts_with(crate::REF_PATH_ATTRIBUTE_PREFIX) { if let Some(path_str) = value.as_str() { - index.add(path_str, &written_meta_path); + let resolved = + crate::resolve_ref_path_to_absolute(path_str, &source_abs) + .unwrap_or_else(|| path_str.to_string()); + index.add(&resolved, &written_meta_path); } } } @@ -3581,6 +3687,85 @@ fn instance_for_scripts_only<'a>( } } +/// Compute the tentative filesystem name segment for an `AddedInstance`. +/// +/// Mirrors the logic of `tentative_fs_name` in `ref_properties.rs` but +/// operates on `AddedInstance` fields instead of `rbx_dom_weak::Instance`. +/// Used by `added_paths` to build correct same-batch Ref target paths. +fn added_instance_fs_segment(added: &crate::web::interface::AddedInstance) -> String { + let slug = if crate::syncback::name_needs_slugify(&added.name) { + crate::syncback::slugify_name(&added.name) + } else { + added.name.clone() + }; + + let has_children = !added.children.is_empty(); + + let is_container = matches!( + added.class_name.as_str(), + "Folder" | "Configuration" | "Tool" | "ScreenGui" | "SurfaceGui" | "BillboardGui" | "AdGui" + ); + + let is_script = matches!( + added.class_name.as_str(), + "Script" | "LocalScript" | "ModuleScript" + ); + + if is_container || (has_children && is_script) { + return slug; + } + + let extension = match added.class_name.as_str() { + "Script" => { + if has_children { + return slug; + } + match added.properties.get("RunContext") { + Some(Variant::Enum(e)) => match e.to_u32() { + 0 => "legacy.luau", + 1 => "server.luau", + 2 => "client.luau", + 3 => "plugin.luau", + _ => "legacy.luau", + }, + _ => "legacy.luau", + } + } + "LocalScript" => { + if has_children { + return slug; + } + "local.luau" + } + "ModuleScript" => { + if has_children { + return slug; + } + "luau" + } + "StringValue" => { + if has_children { + return slug; + } + "txt" + } + "LocalizationTable" => { + if has_children { + return slug; + } + "csv" + } + _ => { + if has_children { + return slug; + } + "model.json5" + } + }; + + format!("{slug}.{extension}") +} + #[cfg(test)] mod tests { use super::*; @@ -5460,115 +5645,6 @@ mod tests { } } - // ========================================================================= - // Duplicate Path Detection - Integration Scenarios - // ========================================================================= - mod duplicate_path_integration_tests { - - #[test] - fn test_syncback_skips_duplicate_children() { - // When syncing back children, duplicates should be skipped - // This test verifies the filter_duplicate_children logic - - let children_names = ["ChildA", "ChildB", "ChildA", "ChildC"]; - let unique: Vec<_> = children_names - .iter() - .filter(|&&name| children_names.iter().filter(|&&n| n == name).count() == 1) - .collect(); - - assert_eq!(unique.len(), 2, "ChildB and ChildC should be kept"); - assert!(unique.contains(&&"ChildB")); - assert!(unique.contains(&&"ChildC")); - } - - #[test] - fn test_recursive_duplicate_handling() { - // Test that duplicates at any level are handled - // - // Parent/ - // ├─ Child1/ - // │ ├─ GrandChild (duplicate) - // │ └─ GrandChild (duplicate) - // └─ Child2/ - // └─ GrandChild (unique within its parent) - - let child1_grandchildren = ["GrandChild", "GrandChild"]; - let child2_grandchildren = ["GrandChild"]; - - // Child1's grandchildren have duplicates - let child1_filtered: Vec<_> = child1_grandchildren - .iter() - .filter(|&&name| child1_grandchildren.iter().filter(|&&n| n == name).count() == 1) - .collect(); - assert_eq!( - child1_filtered.len(), - 0, - "Child1's duplicate grandchildren filtered" - ); - - // Child2's grandchild is unique - let child2_filtered: Vec<_> = child2_grandchildren - .iter() - .filter(|&&name| child2_grandchildren.iter().filter(|&&n| n == name).count() == 1) - .collect(); - assert_eq!(child2_filtered.len(), 1, "Child2's unique grandchild kept"); - } - - #[test] - fn test_duplicate_log_message_format() { - // Verify the expected log message format for skipped duplicates - let class_name = "ModuleScript"; - let full_name = "game.ReplicatedStorage.DuplicateScript"; - - let expected_msg = format!( - "Skipped instance '{}' ({}) - path contains duplicate-named siblings (cannot reliably sync)", - full_name, class_name - ); - - assert!(expected_msg.contains(full_name)); - assert!(expected_msg.contains(class_name)); - assert!(expected_msg.contains("duplicate-named siblings")); - } - - #[test] - fn test_parent_path_validation() { - // When pulling an instance, we must verify the parent path is unique - // This simulates the is_tree_path_unique check - - // Scenario: Trying to add a script under a folder that has duplicate siblings - let parent_path_levels = [ - ("Root", vec!["FolderA", "FolderA"]), // Duplicate at this level! - ]; - - let path_unique = parent_path_levels - .iter() - .all(|(target, siblings)| siblings.iter().filter(|&&s| s == *target).count() == 1); - - assert!( - !path_unique, - "Parent path with duplicates should fail validation" - ); - } - - #[test] - fn test_valid_parent_path() { - // Valid parent path with no duplicates at any level - let parent_path_levels = [ - ( - "ReplicatedStorage", - vec!["Workspace", "ReplicatedStorage", "ServerStorage"], - ), - ("Shared", vec!["Shared", "Client", "Server"]), - ]; - - let path_unique = parent_path_levels - .iter() - .all(|(target, siblings)| siblings.iter().filter(|&&s| s == *target).count() == 1); - - assert!(path_unique, "Valid parent path should pass validation"); - } - } - // Tests for syncback_removed_instance mod syncback_removed_tests { use crate::snapshot::{InstanceMetadata, InstanceSnapshot, RojoTree}; @@ -6402,7 +6478,7 @@ mod tests { instance_name.to_string() }; let deduped = deduplicate_name(&base, &taken); - let needs_meta = needs_slug || deduped != base; + let needs_meta = needs_slug; fs::create_dir(temp.path().join(&deduped)).unwrap(); @@ -6440,7 +6516,7 @@ mod tests { name.to_string() }; let deduped = deduplicate_name(&base, &taken); - let needs_meta = needs_slug || deduped != base; + let needs_meta = needs_slug; taken.insert(deduped.to_lowercase()); let meta = if needs_meta { Some(name.to_string()) @@ -6464,8 +6540,8 @@ mod tests { #[test] fn dir_clean_name_collides_with_existing() { let (f, m) = simulate_dir_dedup(&["NewFolder"], "NewFolder"); - assert_eq!(f, "NewFolder~1"); - assert_eq!(m.unwrap(), "NewFolder"); + assert_eq!(f, "NewFolder~2"); + assert!(m.is_none(), "dedup-only: forward sync strips ~N"); } #[test] @@ -6478,22 +6554,22 @@ mod tests { #[test] fn dir_forbidden_collides_with_existing() { let (f, m) = simulate_dir_dedup(&["Hey_Bro"], "Hey/Bro"); - assert_eq!(f, "Hey_Bro~1"); + assert_eq!(f, "Hey_Bro~2"); assert_eq!(m.unwrap(), "Hey/Bro"); } #[test] fn dir_natural_collides_with_slug() { let (f, m) = simulate_dir_dedup(&["A_B"], "A_B"); - assert_eq!(f, "A_B~1"); - assert_eq!(m.unwrap(), "A_B"); + assert_eq!(f, "A_B~2"); + assert!(m.is_none(), "dedup-only: forward sync strips ~N"); } #[test] fn dir_case_insensitive_collision() { let (f, m) = simulate_dir_dedup(&["MyFolder"], "myfolder"); - assert_eq!(f, "myfolder~1"); - assert_eq!(m.unwrap(), "myfolder"); + assert_eq!(f, "myfolder~2"); + assert!(m.is_none(), "dedup-only: forward sync strips ~N"); } #[test] @@ -6518,7 +6594,7 @@ mod tests { #[test] fn dir_dangerous_suffix_collides() { let (f, m) = simulate_dir_dedup(&["config_meta"], "config.meta"); - assert_eq!(f, "config_meta~1"); + assert_eq!(f, "config_meta~2"); assert_eq!(m.unwrap(), "config.meta"); } @@ -6532,7 +6608,7 @@ mod tests { #[test] fn dir_nul_collides_with_existing() { let (f, _) = simulate_dir_dedup(&["NUL_"], "NUL"); - assert_eq!(f, "NUL_~1"); + assert_eq!(f, "NUL_~2"); } // ── Batch add: directory siblings ───────────────────────────── @@ -6551,24 +6627,24 @@ mod tests { let r = simulate_dir_batch(&[], &["Script", "Script", "Script"]); assert_eq!(r[0].0, "Script"); assert!(r[0].1.is_none()); - assert_eq!(r[1].0, "Script~1"); - assert_eq!(r[1].1.as_deref(), Some("Script")); - assert_eq!(r[2].0, "Script~2"); + assert_eq!(r[1].0, "Script~2"); + assert!(r[1].1.is_none(), "dedup-only: forward sync strips ~N"); + assert_eq!(r[2].0, "Script~3"); } #[test] fn batch_dir_slug_collision_siblings() { let r = simulate_dir_batch(&[], &["X/Y", "X:Y", "X*Y"]); assert_eq!(r[0].0, "X_Y"); - assert_eq!(r[1].0, "X_Y~1"); - assert_eq!(r[2].0, "X_Y~2"); + assert_eq!(r[1].0, "X_Y~2"); + assert_eq!(r[2].0, "X_Y~3"); } #[test] fn batch_dir_with_existing_and_collisions() { let r = simulate_dir_batch(&["Utils", "Config"], &["Utils", "Config", "NewThing"]); - assert_eq!(r[0].0, "Utils~1"); - assert_eq!(r[1].0, "Config~1"); + assert_eq!(r[0].0, "Utils~2"); + assert_eq!(r[1].0, "Config~2"); assert_eq!(r[2].0, "NewThing"); assert!(r[2].1.is_none()); } @@ -6577,8 +6653,8 @@ mod tests { fn batch_dir_dangerous_then_natural_then_slug() { let r = simulate_dir_batch(&[], &["foo.server", "foo_server", "foo/server"]); assert_eq!(r[0].0, "foo_server"); - assert_eq!(r[1].0, "foo_server~1"); - assert_eq!(r[2].0, "foo_server~2"); + assert_eq!(r[1].0, "foo_server~2"); + assert_eq!(r[2].0, "foo_server~3"); } #[test] @@ -6586,9 +6662,9 @@ mod tests { let r = simulate_dir_batch(&[], &["CON", "CON_", "con"]); assert_eq!(r[0].0, "CON_"); assert_eq!(r[0].1.as_deref(), Some("CON")); - assert_eq!(r[1].0, "CON_~1"); - assert_eq!(r[1].1.as_deref(), Some("CON_")); - assert_eq!(r[2].0, "con_~2"); + assert_eq!(r[1].0, "CON_~2"); + assert!(r[1].1.is_none(), "dedup-only: natural CON_ needs no meta"); + assert_eq!(r[2].0, "con_~3"); assert_eq!(r[2].1.as_deref(), Some("con")); } @@ -6596,8 +6672,8 @@ mod tests { fn batch_dir_empty_names() { let r = simulate_dir_batch(&[], &["", "", ""]); assert_eq!(r[0].0, "instance"); - assert_eq!(r[1].0, "instance~1"); - assert_eq!(r[2].0, "instance~2"); + assert_eq!(r[1].0, "instance~2"); + assert_eq!(r[2].0, "instance~3"); } #[test] @@ -6606,7 +6682,7 @@ mod tests { let r = simulate_dir_batch(&[], &children); assert_eq!(r[0].0, "Spam"); for (i, entry) in r.iter().enumerate().skip(1) { - assert_eq!(entry.0, format!("Spam~{i}")); + assert_eq!(entry.0, format!("Spam~{}", i + 1)); } let unique: HashSet = r.iter().map(|(f, _)| f.to_lowercase()).collect(); assert_eq!(unique.len(), 20); @@ -6622,7 +6698,7 @@ mod tests { let taken: HashSet = ["foo".to_string()].into_iter().collect(); // Bare slug "Foo" matches "foo" in taken let deduped = deduplicate_name("Foo", &taken); - assert_eq!(deduped, "Foo~1", "bare slug collides with existing sibling"); + assert_eq!(deduped, "Foo~2", "bare slug collides with existing sibling"); } #[test] @@ -6633,7 +6709,7 @@ mod tests { let slug = slugify_name("Hey/Bro"); assert_eq!(slug, "Hey_Bro"); let deduped = deduplicate_name(&slug, &taken); - assert_eq!(deduped, "Hey_Bro~1", "slug collision correctly detected"); + assert_eq!(deduped, "Hey_Bro~2", "slug collision correctly detected"); } // ── Rename → re-add cycle (the critical slug safety test) ──── @@ -6684,7 +6760,7 @@ mod tests { #[test] fn dir_add_to_populated_no_overwrite() { // Dir has: Helper/ and Helper.meta.json5 - // Add new "Helper" dir → must get Helper~1 + // Add new "Helper" dir → must get Helper~2 let temp = tempdir().expect("Failed to create temp dir"); fs::create_dir(temp.path().join("Helper")).unwrap(); File::create(temp.path().join("Helper.meta.json5")) @@ -6700,7 +6776,7 @@ mod tests { .collect(); let deduped = deduplicate_name("Helper", &taken); - assert_eq!(deduped, "Helper~1"); + assert_eq!(deduped, "Helper~2"); } // ── Nightmare batch scenarios ──────────────────────────────── @@ -6713,15 +6789,15 @@ mod tests { "A/B", "A:B", "A*B", "A?B", "AB", "A|B", "A\\B", "A\"B", ], ); - assert_eq!(r[0].0, "A_B~1"); - assert_eq!(r[1].0, "A_B~2"); - assert_eq!(r[2].0, "A_B~3"); - assert_eq!(r[3].0, "A_B~4"); - assert_eq!(r[4].0, "A_B~5"); - assert_eq!(r[5].0, "A_B~6"); - assert_eq!(r[6].0, "A_B~7"); - assert_eq!(r[7].0, "A_B~8"); - assert_eq!(r[8].0, "A_B~9"); + assert_eq!(r[0].0, "A_B~2"); + assert_eq!(r[1].0, "A_B~3"); + assert_eq!(r[2].0, "A_B~4"); + assert_eq!(r[3].0, "A_B~5"); + assert_eq!(r[4].0, "A_B~6"); + assert_eq!(r[5].0, "A_B~7"); + assert_eq!(r[6].0, "A_B~8"); + assert_eq!(r[7].0, "A_B~9"); + assert_eq!(r[8].0, "A_B~10"); } #[test] @@ -6729,16 +6805,16 @@ mod tests { let r = simulate_dir_batch(&[], &["CON", "PRN", "CON/", "PRN/"]); assert_eq!(r[0].0, "CON_"); assert_eq!(r[1].0, "PRN_"); - assert_eq!(r[2].0, "CON_~1"); // "CON/" slug "CON_" collides - assert_eq!(r[3].0, "PRN_~1"); + assert_eq!(r[2].0, "CON_~2"); // "CON/" slug "CON_" collides + assert_eq!(r[3].0, "PRN_~2"); } #[test] fn batch_dir_unicode_plus_forbidden() { let r = simulate_dir_batch(&[], &["カフェ/Bar", "カフェ:Bar", "カフェ_Bar"]); assert_eq!(r[0].0, "カフェ_Bar"); - assert_eq!(r[1].0, "カフェ_Bar~1"); - assert_eq!(r[2].0, "カフェ_Bar~2"); + assert_eq!(r[1].0, "カフェ_Bar~2"); + assert_eq!(r[2].0, "カフェ_Bar~3"); } #[test] diff --git a/src/web/interface.rs b/src/web/interface.rs index 4feb5e61..756e6ea8 100644 --- a/src/web/interface.rs +++ b/src/web/interface.rs @@ -27,6 +27,60 @@ pub(crate) const SERVER_VERSION: &str = env!("CARGO_PKG_VERSION"); /// Current protocol version, which is required to match. pub const PROTOCOL_VERSION: u64 = 6; +/// Metadata for a single service in the live syncback request. +/// +/// Children are NOT stored here -- they live in the top-level `data` blob +/// (a single rbxm containing all children from all services). +/// +/// Each service's range in the blob is `child_count + ref_target_count` +/// consecutive entries. The first `child_count` are real children (parented +/// to the service). The next `ref_target_count` are ObjectValue carriers +/// whose `Value` property holds a Ref to the actual target. +/// +/// `refs` maps property names to 1-based indices into the carrier subrange +/// (NOT the full service range). Index 1 = first carrier, index 2 = second, +/// etc. Index 0 or out-of-range values are ignored (treated as nil). +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct ServiceChunk { + pub class_name: String, + #[serde(default)] + pub child_count: u32, + #[serde(default)] + pub ref_target_count: u32, + #[serde(default)] + pub properties: HashMap, + #[serde(default)] + pub refs: HashMap, +} + +/// Incoming request from the plugin for live syncback. +/// Numeric fields use `f64` because Luau has no integer type -- all numbers +/// are IEEE 754 doubles, and large values (like place IDs) get encoded as +/// msgpack float64 which rmp_serde cannot coerce into u64. +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct SyncbackRequest { + pub protocol_version: f64, + pub server_version: String, + pub place_id: Option, + #[serde(with = "serde_bytes", default)] + pub data: Vec, + pub services: Vec, +} + +/// Payload passed from the API handler to the serve loop after validation. +#[derive(Debug)] +pub struct SyncbackPayload { + pub data: Vec, + pub services: Vec, +} + +/// Why the live server exited its accept loop. +pub enum ServerExitReason { + SyncbackRequested(SyncbackPayload), +} + /// Message returned by Rojo API when a change has occurred. #[derive(Debug, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] @@ -152,6 +206,16 @@ fn property_filter(value: Option<&Variant>) -> bool { ty != Some(VariantType::SharedString) } +/// Git-related metadata sent to the plugin for smart sync direction defaults. +#[derive(Debug, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct GitMetadata { + pub changed_ids: Vec, + /// SHA1 hashes of prior versions (git blob format) for changed script instances. + /// Contains 1-2 hashes per script: HEAD version and staged version (if different). + pub script_committed_hashes: HashMap>, +} + /// Response body from /api/rojo #[derive(Debug, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] @@ -202,6 +266,8 @@ pub struct ServerInfoResponse { /// Services not in this list should be ignored during sync operations. #[serde(default, skip_serializing_if = "Vec::is_empty")] pub visible_services: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub git_metadata: Option, } // Serialize place IDs as f64 to avoid msgpack uint64 encoding issues with Lua @@ -287,6 +353,9 @@ pub struct WriteRequest { #[serde(default)] pub added: HashMap, pub updated: Vec, + /// Instance IDs whose backing files should be staged via git add. + #[serde(default)] + pub stage_ids: Vec, } #[derive(Debug, Serialize, Deserialize)] diff --git a/src/web/mod.rs b/src/web/mod.rs index 22121be2..9086f213 100644 --- a/src/web/mod.rs +++ b/src/web/mod.rs @@ -10,7 +10,8 @@ mod util; use std::convert::Infallible; use std::net::SocketAddr; -use std::sync::Arc; +use std::sync::{Arc, Mutex}; +use std::time::Duration; use hyper::body::Incoming; use hyper::server::conn::http1; @@ -19,52 +20,141 @@ use hyper::Request; use hyper_util::rt::TokioIo; use tokio::net::TcpListener; use tokio::runtime::Runtime; +use tokio::sync::Notify; use crate::serve_session::ServeSession; +use self::interface::{ServerExitReason, SyncbackPayload}; + +/// Shared signal for the syncback endpoint to deposit its payload and notify +/// the accept loop to shut down. +pub struct SyncbackSignal { + payload: Mutex>, + notify: Notify, +} + +impl SyncbackSignal { + pub fn new() -> Self { + Self { + payload: Mutex::new(None), + notify: Notify::new(), + } + } + + pub fn fire(&self, payload: SyncbackPayload) -> bool { + let mut guard = self.payload.lock().unwrap_or_else(|e| e.into_inner()); + if guard.is_some() { + return false; + } + *guard = Some(payload); + self.notify.notify_one(); + true + } + + pub fn take_payload(&self) -> Option { + self.payload + .lock() + .unwrap_or_else(|e| e.into_inner()) + .take() + } +} + pub struct LiveServer { serve_session: Arc, + syncback_signal: Arc, } impl LiveServer { pub fn new(serve_session: Arc) -> Self { - LiveServer { serve_session } + LiveServer { + serve_session, + syncback_signal: Arc::new(SyncbackSignal::new()), + } } - pub fn start(self, address: SocketAddr) { + pub fn start(self, address: SocketAddr) -> ServerExitReason { let serve_session = Arc::clone(&self.serve_session); + let syncback_signal = Arc::clone(&self.syncback_signal); let rt = Runtime::new().unwrap(); - rt.block_on(async move { - let listener = TcpListener::bind(address).await.unwrap(); + let exit_reason = rt.block_on(async move { + let listener = { + const MAX_BIND_ATTEMPTS: u32 = 5; + const BASE_BACKOFF_MS: u64 = 200; + let mut attempts = 0u32; + loop { + attempts += 1; + match TcpListener::bind(address).await { + Ok(listener) => break listener, + Err(err) + if err.kind() == std::io::ErrorKind::AddrInUse + && attempts < MAX_BIND_ATTEMPTS => + { + let delay = BASE_BACKOFF_MS * 2u64.pow(attempts - 1); + log::warn!( + "Port {} in use, retrying in {}ms (attempt {}/{})", + address.port(), + delay, + attempts, + MAX_BIND_ATTEMPTS + ); + tokio::time::sleep(Duration::from_millis(delay)).await; + } + Err(err) => { + panic!( + "Failed to bind to {}: {} (after {} attempts)", + address, err, attempts + ); + } + } + } + }; loop { - let (stream, _) = listener.accept().await.unwrap(); - let io = TokioIo::new(stream); - let serve_session = Arc::clone(&serve_session); - - tokio::spawn(async move { - let service = service_fn(move |req: Request| { + tokio::select! { + result = listener.accept() => { + let (stream, _) = result.unwrap(); + let io = TokioIo::new(stream); let serve_session = Arc::clone(&serve_session); + let syncback_signal = Arc::clone(&syncback_signal); - async move { - if req.uri().path().starts_with("/api") { - Ok::<_, Infallible>(api::call(serve_session, req).await) - } else { - Ok::<_, Infallible>(ui::call(serve_session, req).await) + tokio::spawn(async move { + let service = service_fn(move |req: Request| { + let serve_session = Arc::clone(&serve_session); + let syncback_signal = Arc::clone(&syncback_signal); + + async move { + if req.uri().path().starts_with("/api") { + Ok::<_, Infallible>( + api::call(serve_session, req, syncback_signal).await, + ) + } else { + Ok::<_, Infallible>(ui::call(serve_session, req).await) + } + } + }); + + if let Err(err) = http1::Builder::new() + .serve_connection(io, service) + .with_upgrades() + .await + { + log::error!("Error serving connection: {err}"); } - } - }); - - if let Err(err) = http1::Builder::new() - .serve_connection(io, service) - .with_upgrades() - .await - { - log::error!("Error serving connection: {err}"); + }); } - }); + _ = syncback_signal.notify.notified() => { + break; + } + } } + + let payload = syncback_signal + .take_payload() + .expect("Syncback signal fired but no payload was deposited"); + ServerExitReason::SyncbackRequested(payload) }); + + exit_reason } } diff --git a/tests/rojo_test/serve_util.rs b/tests/rojo_test/serve_util.rs index 9577bc0a..a7b93c42 100644 --- a/tests/rojo_test/serve_util.rs +++ b/tests/rojo_test/serve_util.rs @@ -130,55 +130,211 @@ impl TestServeSession { } } + /// Creates a test session using a specific (non-default) project file + /// inside the fixture directory. The fixture is copied as usual, but + /// `atlas serve` is pointed at `project_file` within the copied dir + /// instead of the directory itself. + pub fn new_with_project_file(name: &str, project_file: &str) -> Self { + let working_dir = get_working_dir_path(); + + let source_path = Path::new(SERVE_TESTS_PATH).join(name); + let dir = tempdir().expect("Couldn't create temporary directory"); + let fixture_dir = dir + .path() + .canonicalize() + .expect("Couldn't canonicalize temporary directory path") + .join(name); + + fs::create_dir(&fixture_dir).expect("Couldn't create temporary project subdirectory"); + copy_recursive(&source_path, &fixture_dir) + .expect("Couldn't copy project to temporary directory"); + + let project_path = fixture_dir.join(project_file); + assert!( + project_path.exists(), + "Project file {} does not exist in fixture {}", + project_file, + name + ); + + #[cfg(target_os = "macos")] + std::thread::sleep(Duration::from_millis(100)); + + let port = get_port_number(); + let port_string = port.to_string(); + + let rojo_process = Command::new(ROJO_PATH) + .args([ + "serve", + project_path.to_str().unwrap(), + "--port", + port_string.as_str(), + ]) + .current_dir(working_dir) + .stderr(Stdio::piped()) + .spawn() + .expect("Couldn't start Rojo"); + + TestServeSession { + rojo_process: KillOnDrop(rojo_process), + _dir: dir, + port, + project_path: fixture_dir, + } + } + + /// Creates a test session with a git repo initialized in the project dir. + /// The `setup` callback runs after the fixture is copied and git is initialized + /// but BEFORE the serve process starts. Use it to commit initial files and + /// make modifications that should appear as git changes. + pub fn new_with_git(name: &str, setup: impl FnOnce(&Path)) -> Self { + let working_dir = get_working_dir_path(); + + let source_path = Path::new(SERVE_TESTS_PATH).join(name); + let dir = tempdir().expect("Couldn't create temporary directory"); + let project_path = dir + .path() + .canonicalize() + .expect("Couldn't canonicalize temporary directory path") + .join(name); + + fs::create_dir(&project_path).expect("Couldn't create temporary project subdirectory"); + copy_recursive(&source_path, &project_path) + .expect("Couldn't copy project to temporary directory"); + + // Initialize git repo and run setup callback + Command::new("git") + .args(["init"]) + .current_dir(&project_path) + .output() + .expect("git init failed"); + Command::new("git") + .args(["config", "user.email", "test@test.com"]) + .current_dir(&project_path) + .output() + .unwrap(); + Command::new("git") + .args(["config", "user.name", "Test"]) + .current_dir(&project_path) + .output() + .unwrap(); + + setup(&project_path); + + #[cfg(target_os = "macos")] + std::thread::sleep(Duration::from_millis(100)); + + let port = get_port_number(); + let port_string = port.to_string(); + + let rojo_process = Command::new(ROJO_PATH) + .args([ + "serve", + project_path.to_str().unwrap(), + "--port", + port_string.as_str(), + ]) + .current_dir(working_dir) + .stderr(Stdio::piped()) + .spawn() + .expect("Couldn't start Rojo"); + + TestServeSession { + rojo_process: KillOnDrop(rojo_process), + _dir: dir, + port, + project_path, + } + } + pub fn path(&self) -> &Path { &self.project_path } + #[allow(dead_code)] pub fn port(&self) -> usize { self.port } - /// Waits for the `rojo serve` server to come online with expontential - /// backoff. - pub fn wait_to_come_online(&mut self) -> ServerInfoResponse { - const BASE_DURATION_MS: f32 = 30.0; - const EXP_BACKOFF_FACTOR: f32 = 1.3; - const MAX_TRIES: u32 = 5; + fn respawn_with_new_port(&mut self) { + let port = get_port_number(); + let port_string = port.to_string(); + let working_dir = get_working_dir_path(); - for i in 1..=MAX_TRIES { - match self.rojo_process.0.try_wait() { - Ok(Some(status)) => { - let mut stderr_output = String::new(); - if let Some(mut stderr) = self.rojo_process.0.stderr.take() { - let _ = stderr.read_to_string(&mut stderr_output); + let rojo_process = Command::new(ROJO_PATH) + .args([ + "serve", + self.project_path.to_str().unwrap(), + "--port", + port_string.as_str(), + ]) + .current_dir(working_dir) + .stderr(Stdio::piped()) + .spawn() + .expect("Couldn't start Rojo"); + + self.rojo_process = KillOnDrop(rojo_process); + self.port = port; + } + + /// Waits for the `rojo serve` server to come online with exponential + /// backoff. If the process dies with `AddrInUse`, respawns with a new + /// random port (up to `MAX_PORT_RETRIES` times). + pub fn wait_to_come_online(&mut self) -> ServerInfoResponse { + const BASE_DURATION_MS: f32 = 100.0; + const EXP_BACKOFF_FACTOR: f32 = 1.3; + const MAX_TRIES: u32 = 10; + const MAX_PORT_RETRIES: u32 = 3; + + let mut port_retries = 0u32; + + 'retry_port: loop { + for i in 1..=MAX_TRIES { + match self.rojo_process.0.try_wait() { + Ok(Some(status)) => { + let mut stderr_output = String::new(); + if let Some(mut stderr) = self.rojo_process.0.stderr.take() { + let _ = stderr.read_to_string(&mut stderr_output); + } + if stderr_output.contains("AddrInUse") && port_retries < MAX_PORT_RETRIES { + port_retries += 1; + log::warn!( + "Port {} in use, respawning with new port (attempt {}/{})", + self.port, + port_retries, + MAX_PORT_RETRIES + ); + self.respawn_with_new_port(); + continue 'retry_port; + } + panic!( + "Rojo process exited with status {}\nstderr:\n{}", + status, stderr_output + ); } - panic!( - "Rojo process exited with status {}\nstderr:\n{}", - status, stderr_output - ); + Ok(None) => { /* The process is still running, as expected */ } + Err(err) => panic!("Failed to wait on Rojo process: {}", err), } - Ok(None) => { /* The process is still running, as expected */ } - Err(err) => panic!("Failed to wait on Rojo process: {}", err), + + let info = match self.get_api_rojo() { + Ok(info) => info, + Err(err) => { + let retry_time_ms = BASE_DURATION_MS * (i as f32).powf(EXP_BACKOFF_FACTOR); + let retry_time = Duration::from_millis(retry_time_ms as u64); + + log::info!("Server error, retrying in {:?}: {}", retry_time, err); + thread::sleep(retry_time); + continue; + } + }; + + log::info!("Got session info: {:?}", info); + + return info; } - let info = match self.get_api_rojo() { - Ok(info) => info, - Err(err) => { - let retry_time_ms = BASE_DURATION_MS * (i as f32).powf(EXP_BACKOFF_FACTOR); - let retry_time = Duration::from_millis(retry_time_ms as u64); - - log::info!("Server error, retrying in {:?}: {}", retry_time, err); - thread::sleep(retry_time); - continue; - } - }; - - log::info!("Got session info: {:?}", info); - - return info; + panic!("Rojo server did not respond after {} tries.", MAX_TRIES); } - - panic!("Rojo server did not respond after {} tries.", MAX_TRIES); } pub fn get_api_rojo(&self) -> Result { @@ -405,6 +561,351 @@ fn get_port_number() -> usize { port } +/// A service entry plus its children and ObjectValue ref carriers (before +/// serialization into the shared rbxm blob). Use `build_syncback_request` +/// to finalize. +pub struct ServiceEntry { + pub chunk: librojo::web_api::ServiceChunk, + pub children: Vec, + pub ref_targets: Vec, +} + +/// Build a ServiceEntry with children and no extra properties. +pub fn make_service_chunk( + class_name: &str, + children: Vec, +) -> ServiceEntry { + make_service_chunk_full(class_name, vec![], vec![], vec![], children) +} + +/// Build a ServiceEntry with explicit service-level properties, ObjectValue +/// ref carriers, and refs (1-based indices into `ref_targets`). +pub fn make_service_chunk_full( + class_name: &str, + properties: Vec<(&str, rbx_dom_weak::types::Variant)>, + refs: Vec<(&str, u32)>, + ref_targets: Vec, + children: Vec, +) -> ServiceEntry { + use std::collections::HashMap; + + let props_map: HashMap = properties + .into_iter() + .map(|(k, v)| (k.to_string(), v)) + .collect(); + let refs_map: HashMap = refs + .into_iter() + .map(|(prop, idx)| (prop.to_string(), idx)) + .collect(); + + ServiceEntry { + chunk: librojo::web_api::ServiceChunk { + class_name: class_name.to_string(), + child_count: children.len() as u32, + ref_target_count: ref_targets.len() as u32, + properties: props_map, + refs: refs_map, + }, + children, + ref_targets, + } +} + +/// Serialize all ServiceEntries into the final wire format: a single rbxm +/// data blob and a Vec of ServiceChunks (metadata only). +pub fn build_syncback_request( + entries: Vec, +) -> (Vec, Vec) { + use rbx_dom_weak::WeakDom; + + let mut dom = WeakDom::new(rbx_dom_weak::InstanceBuilder::new("DataModel")); + let root = dom.root_ref(); + + let mut chunks = Vec::new(); + for mut entry in entries { + for child in entry.children { + dom.insert(root, child); + } + entry.chunk.ref_target_count = entry.ref_targets.len() as u32; + for carrier in entry.ref_targets { + dom.insert(root, carrier); + } + chunks.push(entry.chunk); + } + + let all_refs: Vec = dom.root().children().to_vec(); + let mut data = Vec::new(); + if !all_refs.is_empty() { + rbx_binary::to_writer(&mut data, &dom, &all_refs).unwrap(); + } + + (data, chunks) +} + +/// Build a complete rbxl file from the single-rbxm data blob + service +/// chunks, mirroring the same data that live syncback receives. +pub fn make_rbxl_from_chunks(data: &[u8], chunks: &[librojo::web_api::ServiceChunk]) -> Vec { + use rbx_dom_weak::types::Variant; + use rbx_dom_weak::WeakDom; + use std::collections::HashMap; + use std::io::Cursor; + + let mut dom = WeakDom::new(rbx_dom_weak::InstanceBuilder::new("DataModel")); + let root_ref = dom.root_ref(); + + let cloned_children: Vec = if !data.is_empty() { + let chunk_dom = rbx_binary::from_reader(Cursor::new(data)) + .unwrap_or_else(|e| panic!("Failed to parse rbxm data blob: {e}")); + + let mut ref_map: HashMap = HashMap::new(); + let mut cloned = Vec::new(); + for &child_ref in chunk_dom.root().children() { + deep_clone_into_test(&chunk_dom, &mut dom, child_ref, root_ref, &mut ref_map); + cloned.push(*ref_map.get(&child_ref).unwrap()); + } + + let all_refs: Vec = ref_map.values().copied().collect(); + for inst_ref in all_refs { + let props_to_fix: Vec<(String, Ref)> = { + let inst = dom.get_by_ref(inst_ref).unwrap(); + inst.properties + .iter() + .filter_map(|(key, value)| { + if let Variant::Ref(r) = value { + ref_map.get(r).map(|&mapped| (key.to_string(), mapped)) + } else { + None + } + }) + .collect() + }; + if !props_to_fix.is_empty() { + let inst = dom.get_by_ref_mut(inst_ref).unwrap(); + for (key, new_ref) in props_to_fix { + inst.properties.insert(key.into(), Variant::Ref(new_ref)); + } + } + } + cloned + } else { + Vec::new() + }; + + let mut cursor = 0usize; + for chunk in chunks { + let mut builder = rbx_dom_weak::InstanceBuilder::new(&chunk.class_name); + for (key, value) in &chunk.properties { + builder = builder.with_property(key.as_str(), value.clone()); + } + let service_ref = dom.insert(root_ref, builder); + + let child_count = chunk.child_count as usize; + let ref_count = chunk.ref_target_count as usize; + let total = child_count + ref_count; + let end = (cursor + total).min(cloned_children.len()); + let service_range: Vec = cloned_children[cursor..end].to_vec(); + + for &child_ref in &service_range[..child_count.min(service_range.len())] { + dom.transfer_within(child_ref, service_ref); + } + + let carrier_start = child_count; + let carrier_end = service_range.len(); + let carriers: Vec = service_range[carrier_start..carrier_end].to_vec(); + + for (prop_name, &idx) in &chunk.refs { + if idx == 0 || (idx as usize) > carriers.len() { + continue; + } + let carrier_ref = carriers[idx as usize - 1]; + if let Some(Variant::Ref(actual_target)) = dom + .get_by_ref(carrier_ref) + .and_then(|inst| inst.properties.get(&rbx_dom_weak::ustr("Value")).cloned()) + { + let service = dom.get_by_ref_mut(service_ref).unwrap(); + service + .properties + .insert(prop_name.as_str().into(), Variant::Ref(actual_target)); + } + } + + for carrier_ref in carriers { + dom.destroy(carrier_ref); + } + + cursor = end; + } + + let existing: std::collections::HashSet = + chunks.iter().map(|c| c.class_name.clone()).collect(); + for &svc in librojo::syncback::VISIBLE_SERVICES { + if !existing.contains(svc) { + dom.insert(root_ref, rbx_dom_weak::InstanceBuilder::new(svc)); + } + } + + let service_refs: Vec = dom.root().children().to_vec(); + let mut buf = Vec::new(); + rbx_binary::to_writer(&mut buf, &dom, &service_refs).unwrap(); + buf +} + +fn deep_clone_into_test( + source: &rbx_dom_weak::WeakDom, + target: &mut rbx_dom_weak::WeakDom, + source_ref: Ref, + target_parent: Ref, + ref_map: &mut std::collections::HashMap, +) { + let inst = source.get_by_ref(source_ref).unwrap(); + let mut builder = + rbx_dom_weak::InstanceBuilder::new(inst.class.as_str()).with_name(inst.name.as_str()); + + for (key, value) in &inst.properties { + builder = builder.with_property(key.as_str(), value.clone()); + } + + let new_ref = target.insert(target_parent, builder); + ref_map.insert(source_ref, new_ref); + + for &child_ref in inst.children() { + deep_clone_into_test(source, target, child_ref, new_ref, ref_map); + } +} + +/// Run CLI syncback on a fresh copy of a fixture using the same data +/// that was sent to live syncback. Returns the temp dir (keep alive) and +/// project path. +pub fn run_cli_syncback_on_chunks( + fixture_name: &str, + data: &[u8], + chunks: &[librojo::web_api::ServiceChunk], +) -> (TempDir, PathBuf) { + use crate::rojo_test::io_util::{copy_recursive, ROJO_PATH, SERVE_TESTS_PATH}; + + let source_path = Path::new(SERVE_TESTS_PATH).join(fixture_name); + let dir = tempdir().expect("Couldn't create temp dir for CLI syncback"); + let project_path = dir.path().join(fixture_name); + fs::create_dir(&project_path).expect("Couldn't create project subdirectory"); + copy_recursive(&source_path, &project_path).expect("Couldn't copy fixture"); + + let rbxl_data = make_rbxl_from_chunks(data, chunks); + let rbxl_path = dir.path().join("input.rbxl"); + fs::write(&rbxl_path, rbxl_data).expect("Failed to write rbxl"); + + let output = std::process::Command::new(ROJO_PATH) + .args([ + "syncback", + project_path.to_str().unwrap(), + "--input", + rbxl_path.to_str().unwrap(), + ]) + .output() + .expect("Failed to run atlas syncback"); + + if !output.status.success() { + panic!( + "atlas syncback failed:\nstdout: {}\nstderr: {}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + } + + (dir, project_path) +} + +impl TestServeSession { + /// Post to /api/syncback with raw bytes. Returns the full response + /// for status code inspection. + pub fn post_api_syncback_raw(&self, body: Vec) -> reqwest::blocking::Response { + let url = format!("http://localhost:{}/api/syncback", self.port); + let client = reqwest::blocking::Client::builder() + .timeout(Duration::from_secs(120)) + .build() + .expect("Failed to build reqwest client"); + client + .post(url) + .body(body) + .send() + .expect("Failed to send syncback request") + } + + /// Post to /api/syncback with correct version/protocol info. + /// `data` is the single rbxm blob; `services` are the metadata chunks. + /// Panics if the response is not 200. + pub fn post_api_syncback( + &self, + place_id: Option, + data: Vec, + services: Vec, + ) { + use serde::Serialize; + + #[derive(Serialize)] + #[serde(rename_all = "camelCase")] + struct Req { + protocol_version: u64, + server_version: String, + place_id: Option, + #[serde(with = "serde_bytes")] + data: Vec, + services: Vec, + } + + let request = Req { + protocol_version: librojo::web_api::PROTOCOL_VERSION, + server_version: env!("CARGO_PKG_VERSION").to_string(), + place_id, + data, + services, + }; + + let mut body = Vec::new(); + let mut serializer = rmp_serde::Serializer::new(&mut body) + .with_human_readable() + .with_struct_map(); + request + .serialize(&mut serializer) + .expect("Failed to serialize syncback request"); + + let response = self.post_api_syncback_raw(body); + assert!( + response.status().is_success(), + "Syncback request failed with status {}: {}", + response.status(), + response.text().unwrap_or_default() + ); + } + + /// Wait for the server to come back online after a syncback-triggered + /// restart. Uses longer timeouts than initial startup since syncback + /// runs between teardown and restart. + pub fn wait_to_come_back_online(&self) -> ServerInfoResponse { + const BASE_DURATION_MS: f32 = 200.0; + const EXP_BACKOFF_FACTOR: f32 = 1.3; + const MAX_TRIES: u32 = 20; + + for i in 1..=MAX_TRIES { + let info = match self.get_api_rojo() { + Ok(info) => info, + Err(_) => { + let retry_time_ms = BASE_DURATION_MS * (i as f32).powf(EXP_BACKOFF_FACTOR); + let retry_time = Duration::from_millis(retry_time_ms as u64); + thread::sleep(retry_time); + continue; + } + }; + + return info; + } + + panic!( + "Server did not come back online after syncback ({} tries)", + MAX_TRIES + ); + } +} + /// Extract the message cursor from a SocketPacket for use in subsequent /// WebSocket subscriptions when chaining multi-step tests. pub fn get_message_cursor(packet: &SocketPacket) -> u32 { @@ -475,49 +976,63 @@ pub fn fresh_rebuild_read(project_path: &Path) -> NormalizedInstance { ); }; - let port = get_port_number(); - let port_string = port.to_string(); - let working_dir = get_working_dir_path(); - - let process = Command::new(ROJO_PATH) - .args([ - "serve", - project_file.to_str().unwrap(), - "--port", - port_string.as_str(), - ]) - .current_dir(working_dir) - .stderr(Stdio::piped()) - .spawn() - .expect("Couldn't start fresh Rojo for round-trip check"); - - let mut kill_guard = KillOnDrop(process); - - // Wait for fresh server to come online with same backoff - const BASE_DURATION_MS: f32 = 30.0; + const BASE_DURATION_MS: f32 = 100.0; const EXP_BACKOFF_FACTOR: f32 = 1.3; - const MAX_TRIES: u32 = 5; + const MAX_TRIES: u32 = 10; + const MAX_PORT_RETRIES: u32 = 3; - let mut info = None; - for i in 1..=MAX_TRIES { - match kill_guard.0.try_wait() { - Ok(Some(status)) => { - let mut stderr_output = String::new(); - if let Some(mut stderr) = kill_guard.0.stderr.take() { - let _ = stderr.read_to_string(&mut stderr_output); + let working_dir = get_working_dir_path(); + let mut port_retries = 0u32; + + loop { + let port = get_port_number(); + let port_string = port.to_string(); + + let process = Command::new(ROJO_PATH) + .args([ + "serve", + project_file.to_str().unwrap(), + "--port", + port_string.as_str(), + ]) + .current_dir(&working_dir) + .stderr(Stdio::piped()) + .spawn() + .expect("Couldn't start fresh Rojo for round-trip check"); + + let mut kill_guard = KillOnDrop(process); + let mut info = None; + let mut needs_port_retry = false; + + for i in 1..=MAX_TRIES { + match kill_guard.0.try_wait() { + Ok(Some(status)) => { + let mut stderr_output = String::new(); + if let Some(mut stderr) = kill_guard.0.stderr.take() { + let _ = stderr.read_to_string(&mut stderr_output); + } + if stderr_output.contains("AddrInUse") && port_retries < MAX_PORT_RETRIES { + port_retries += 1; + log::warn!( + "Port {} in use, retrying with new port (attempt {}/{})", + port, + port_retries, + MAX_PORT_RETRIES + ); + needs_port_retry = true; + break; + } + panic!( + "Fresh Rojo process exited with status {}\nstderr:\n{}", + status, stderr_output + ); } - panic!( - "Fresh Rojo process exited with status {}\nstderr:\n{}", - status, stderr_output - ); + Ok(None) => {} + Err(err) => panic!("Failed to wait on fresh Rojo process: {}", err), } - Ok(None) => {} - Err(err) => panic!("Failed to wait on fresh Rojo process: {}", err), - } - let url = format!("http://localhost:{}/api/rojo", port); - match reqwest::blocking::get(&url) { - Ok(resp) => { + let url = format!("http://localhost:{}/api/rojo", port); + if let Ok(resp) = reqwest::blocking::get(&url) { if let Ok(body) = resp.bytes() { if let Ok(server_info) = deserialize_msgpack::(&body) { info = Some(server_info); @@ -525,24 +1040,27 @@ pub fn fresh_rebuild_read(project_path: &Path) -> NormalizedInstance { } } } - Err(_) => {} + let retry_time_ms = BASE_DURATION_MS * (i as f32).powf(EXP_BACKOFF_FACTOR); + thread::sleep(Duration::from_millis(retry_time_ms as u64)); } - let retry_time_ms = BASE_DURATION_MS * (i as f32).powf(EXP_BACKOFF_FACTOR); - thread::sleep(Duration::from_millis(retry_time_ms as u64)); + + if needs_port_retry { + continue; + } + + let info = info.expect("Fresh Rojo server did not respond"); + let root_id = info.root_instance_id; + + let url = format!("http://localhost:{}/api/read/{}", port, root_id); + let body = reqwest::blocking::get(&url) + .expect("Failed to read from fresh server") + .bytes() + .expect("Failed to get bytes from fresh server"); + let read: ReadResponse = + deserialize_msgpack(&body).expect("Fresh server returned malformed response"); + + return normalize_read_response(&read, root_id); } - - let info = info.expect("Fresh Rojo server did not respond"); - let root_id = info.root_instance_id; - - let url = format!("http://localhost:{}/api/read/{}", port, root_id); - let body = reqwest::blocking::get(&url) - .expect("Failed to read from fresh server") - .bytes() - .expect("Failed to get bytes from fresh server"); - let read: ReadResponse = - deserialize_msgpack(&body).expect("Fresh server returned malformed response"); - - normalize_read_response(&read, root_id) } /// Assert that the live session tree matches a fresh rebuild from the filesystem. diff --git a/tests/tests/build.rs b/tests/tests/build.rs index 26666f00..5b32b320 100644 --- a/tests/tests/build.rs +++ b/tests/tests/build.rs @@ -68,6 +68,7 @@ gen_build_tests! { meta_name_override, model_json_name_override, dedup_suffix_with_meta, + dedup_suffix_auto_strip, init_meta_name_override, reserved_name_override, } @@ -112,3 +113,356 @@ fn run_build_test(test_name: &str) { assert_snapshot!(test_name, contents); }); } + +fn snapshot_debug(snap: &librojo::InstanceSnapshot) -> String { + fn recurse(snap: &librojo::InstanceSnapshot, depth: usize, out: &mut String) { + use std::fmt::Write; + let indent = " ".repeat(depth); + writeln!(out, "{}{} [{}]", indent, snap.name, snap.class_name).unwrap(); + for child in &snap.children { + recurse(child, depth + 1, out); + } + } + let mut s = String::new(); + recurse(snap, 0, &mut s); + s +} + +#[test] +fn parallel_snapshot_determinism() { + use crate::rojo_test::io_util::SERVE_TESTS_PATH; + + let _ = env_logger::try_init(); + + let fixture_path = Path::new(SERVE_TESTS_PATH).join("connected_scripts"); + let vfs = memofs::Vfs::new_default(); + let ctx = librojo::InstanceContext::default(); + + let first = librojo::snapshot_from_vfs(&ctx, &vfs, &fixture_path) + .expect("snapshot failed") + .expect("snapshot returned None"); + let baseline = snapshot_debug(&first); + + for i in 1..5 { + let vfs = memofs::Vfs::new_default(); + let snap = librojo::snapshot_from_vfs(&ctx, &vfs, &fixture_path) + .expect("snapshot failed") + .expect("snapshot returned None"); + let current = snapshot_debug(&snap); + assert_eq!( + baseline, current, + "Parallel snapshot produced different child order on iteration {i}" + ); + } +} + +#[test] +fn parallel_snapshot_determinism_stress() { + let _ = env_logger::try_init(); + + let dir = tempdir().expect("couldn't create temp dir"); + let root = dir.path(); + let src = root.join("src"); + fs::create_dir(&src).unwrap(); + + for i in 0..40 { + fs::write(src.join(format!("mod_{i:03}.luau")), format!("return {i}")).unwrap(); + } + for i in 0..10 { + let sub = src.join(format!("pkg_{i:02}")); + fs::create_dir(&sub).unwrap(); + for j in 0..5 { + fs::write(sub.join(format!("sub_{j}.luau")), format!("return {j}")).unwrap(); + } + } + + fs::write( + root.join("default.project.json5"), + r#"{ "name": "StressTest", "tree": { "$path": "src" } }"#, + ) + .unwrap(); + + let ctx = librojo::InstanceContext::default(); + let project_path = root.join("default.project.json5"); + + let vfs = memofs::Vfs::new_default(); + let first = librojo::snapshot_from_vfs(&ctx, &vfs, &project_path) + .expect("snapshot failed") + .expect("snapshot returned None"); + let baseline = snapshot_debug(&first); + + for i in 1..50 { + let vfs = memofs::Vfs::new_default(); + let snap = librojo::snapshot_from_vfs(&ctx, &vfs, &project_path) + .expect("snapshot failed") + .expect("snapshot returned None"); + let current = snapshot_debug(&snap); + assert_eq!( + baseline, current, + "Stress: parallel snapshot diverged on iteration {i}" + ); + } +} + +#[test] +fn parallel_snapshot_with_prefetch_cache() { + let _ = env_logger::try_init(); + + let dir = tempdir().expect("couldn't create temp dir"); + let root = dir.path(); + let src = root.join("src"); + fs::create_dir(&src).unwrap(); + + for i in 0..20 { + fs::write(src.join(format!("f{i}.luau")), format!("return {i}")).unwrap(); + } + for i in 0..5 { + let sub = src.join(format!("d{i}")); + fs::create_dir(&sub).unwrap(); + for j in 0..4 { + fs::write(sub.join(format!("g{j}.luau")), format!("return {j}")).unwrap(); + } + } + + fs::write( + root.join("default.project.json5"), + r#"{ "name": "PrefetchTest", "tree": { "$path": "src" } }"#, + ) + .unwrap(); + + let ctx = librojo::InstanceContext::default(); + let project_path = root.join("default.project.json5"); + + let vfs_no_cache = memofs::Vfs::new_default(); + let snap_no_cache = librojo::snapshot_from_vfs(&ctx, &vfs_no_cache, &project_path) + .expect("no-cache snapshot failed") + .expect("no-cache snapshot returned None"); + let baseline = snapshot_debug(&snap_no_cache); + + use std::collections::HashMap; + let mut files = HashMap::new(); + let mut canonical = HashMap::new(); + for entry in walkdir::WalkDir::new(root).follow_links(true) { + let entry = entry.unwrap(); + let path = entry.path().to_path_buf(); + if entry.file_type().is_file() { + files.insert(path.clone(), fs::read(&path).unwrap()); + } + if let Ok(c) = fs::canonicalize(&path) { + canonical.insert(path, c); + } + } + + let vfs_cached = memofs::Vfs::new_default(); + vfs_cached.set_prefetch_cache(memofs::PrefetchCache { + files, + canonical, + is_file: std::collections::HashMap::new(), + children: std::collections::HashMap::new(), + }); + + let snap_cached = librojo::snapshot_from_vfs(&ctx, &vfs_cached, &project_path) + .expect("cached snapshot failed") + .expect("cached snapshot returned None"); + let cached_str = snapshot_debug(&snap_cached); + + assert_eq!( + baseline, cached_str, + "Prefetch-cached snapshot differs from non-cached snapshot" + ); +} + +#[test] +fn parallel_snapshot_error_propagation() { + let _ = env_logger::try_init(); + + let dir = tempdir().expect("couldn't create temp dir"); + let root = dir.path(); + let src = root.join("src"); + fs::create_dir(&src).unwrap(); + + for i in 0..5 { + fs::write(src.join(format!("ok_{i}.luau")), format!("return {i}")).unwrap(); + } + fs::write(src.join("bad.model.json"), "THIS IS NOT VALID JSON {{{").unwrap(); + + fs::write( + root.join("default.project.json5"), + r#"{ "name": "ErrorTest", "tree": { "$path": "src" } }"#, + ) + .unwrap(); + + let ctx = librojo::InstanceContext::default(); + let project_path = root.join("default.project.json5"); + + let vfs = memofs::Vfs::new_default(); + let result = librojo::snapshot_from_vfs(&ctx, &vfs, &project_path); + assert!( + result.is_err(), + "Should propagate JSON parse error from parallel snapshot" + ); +} + +#[test] +fn overlapping_path_roots_no_duplicate_children() { + let _ = env_logger::try_init(); + + let dir = tempdir().expect("couldn't create temp dir"); + let root = dir.path(); + let src = root.join("src"); + let shared = src.join("shared"); + fs::create_dir_all(&shared).unwrap(); + + fs::write(src.join("top.luau"), "return 1").unwrap(); + fs::write(shared.join("a.luau"), "return 2").unwrap(); + fs::write(shared.join("b.luau"), "return 3").unwrap(); + + fs::write( + root.join("default.project.json5"), + r#"{ + "name": "OverlapTest", + "tree": { + "$path": "src", + "shared": { + "$path": "src/shared" + } + } + }"#, + ) + .unwrap(); + + let ctx = librojo::InstanceContext::default(); + let project_path = root.join("default.project.json5"); + + use std::collections::HashMap; + let mut files = HashMap::new(); + let mut canonical = HashMap::new(); + let mut is_file = HashMap::new(); + let mut children_map: HashMap> = HashMap::new(); + + for entry in walkdir::WalkDir::new(root).follow_links(true) { + let entry = entry.unwrap(); + let path = entry.path().to_path_buf(); + is_file.insert(path.clone(), entry.file_type().is_file()); + if entry.file_type().is_file() { + files.insert(path.clone(), fs::read(&path).unwrap()); + } + if let Ok(c) = fs::canonicalize(&path) { + canonical.insert(path.clone(), c); + } + if entry.depth() > 0 { + if let Some(parent) = entry.path().parent() { + children_map + .entry(parent.to_path_buf()) + .or_default() + .push(path); + } + } + } + for children in children_map.values_mut() { + children.sort_by(|a, b| a.file_name().cmp(&b.file_name())); + } + + let vfs = memofs::Vfs::new_default(); + vfs.set_prefetch_cache(memofs::PrefetchCache { + files, + canonical, + is_file, + children: children_map, + }); + + let snap = librojo::snapshot_from_vfs(&ctx, &vfs, &project_path) + .expect("snapshot failed") + .expect("snapshot returned None"); + + fn find_child<'a>( + snap: &'a librojo::InstanceSnapshot, + name: &str, + ) -> Option<&'a librojo::InstanceSnapshot> { + snap.children.iter().find(|c| c.name.as_ref() == name) + } + + let shared_snap = find_child(&snap, "shared").expect("shared child should exist"); + let shared_names: Vec<&str> = shared_snap + .children + .iter() + .map(|c| c.name.as_ref()) + .collect(); + + let mut deduped = shared_names.clone(); + deduped.sort(); + deduped.dedup(); + assert_eq!( + shared_names.len(), + deduped.len(), + "No duplicate children should exist under shared. Got: {:?}", + shared_names + ); +} + +/// Test the nested path detection algorithm used in clean-mode orphan removal. +/// Sorted path walk with ancestor tracking should only emit top-level paths. +#[test] +fn nested_path_detection_correctness() { + use std::path::PathBuf; + + fn top_level_paths(input: &[&str]) -> Vec { + let mut sorted: Vec = input.iter().map(PathBuf::from).collect(); + sorted.sort(); + + let mut result = Vec::new(); + let mut current_ancestor: Option<&PathBuf> = None; + for path in &sorted { + if let Some(ancestor) = current_ancestor { + if path.starts_with(ancestor) && path != ancestor { + continue; + } + } + current_ancestor = Some(path); + result.push(path.clone()); + } + result + } + + let result = top_level_paths(&["a/b", "a/b/c", "a/b/d", "x/y", "x/y/z"]); + assert_eq!( + result, + vec![PathBuf::from("a/b"), PathBuf::from("x/y")], + "Should only keep top-level ancestors" + ); + + let result = top_level_paths(&["a/b/c", "a/b/d", "a/b"]); + assert_eq!( + result, + vec![PathBuf::from("a/b")], + "Should detect ancestors regardless of input order" + ); + + let result = top_level_paths(&["a", "a/b", "a/b/c"]); + assert_eq!( + result, + vec![PathBuf::from("a")], + "Single ancestor should subsume all descendants" + ); + + let result = top_level_paths(&["a/b", "a/c", "b/d"]); + assert_eq!( + result, + vec![ + PathBuf::from("a/b"), + PathBuf::from("a/c"), + PathBuf::from("b/d") + ], + "Non-nested siblings should all be kept" + ); + + let result = top_level_paths(&["x"]); + assert_eq!( + result, + vec![PathBuf::from("x")], + "Single path should be kept" + ); + + let result: Vec = top_level_paths(&[]); + assert!(result.is_empty(), "Empty input should produce empty output"); +} diff --git a/tests/tests/clean_mode.rs b/tests/tests/clean_mode.rs index 0edb4cae..3ded9e4e 100644 --- a/tests/tests/clean_mode.rs +++ b/tests/tests/clean_mode.rs @@ -328,3 +328,56 @@ fn clean_mode_is_idempotent() { "File content should be identical after second syncback" ); } + +/// Test that clean mode does NOT delete hidden directories or their contents +/// inside $path source directories. Hidden directories like .vscode/ or +/// .cursor/ should be left untouched by the orphan scanner. +#[test] +fn clean_mode_preserves_hidden_directories() { + let _ = env_logger::try_init(); + + let source_path = Path::new(SYNCBACK_TESTS_PATH) + .join("sync_rules") + .join("input-project"); + let input_file = Path::new(SYNCBACK_TESTS_PATH) + .join("sync_rules") + .join("input.rbxm"); + + let test_dir = tempdir().expect("Couldn't create temporary directory"); + let project_path = test_dir.path().join("test_project"); + + fs_err::create_dir(&project_path).expect("Couldn't create project directory"); + copy_recursive(&source_path, &project_path).expect("Couldn't copy project"); + + let src_dir = project_path.join("src"); + let hidden_dir = src_dir.join(".hidden"); + fs_err::create_dir_all(&hidden_dir).expect("Couldn't create hidden directory"); + fs_err::write(hidden_dir.join("keep.txt"), "do not delete").unwrap(); + fs_err::write(hidden_dir.join("config.txt"), "some config").unwrap(); + + let nested_hidden = hidden_dir.join("nested"); + fs_err::create_dir_all(&nested_hidden).unwrap(); + fs_err::write(nested_hidden.join("deep.txt"), "also keep").unwrap(); + + assert!( + run_syncback(&project_path, &input_file, false), + "Syncback should succeed" + ); + + assert!( + hidden_dir.exists(), + ".hidden directory should survive clean mode syncback" + ); + assert!( + hidden_dir.join("keep.txt").exists(), + ".hidden/keep.txt should survive" + ); + assert!( + hidden_dir.join("config.txt").exists(), + ".hidden/config.txt should survive" + ); + assert!( + nested_hidden.join("deep.txt").exists(), + ".hidden/nested/deep.txt should survive" + ); +} diff --git a/tests/tests/connected_mode.rs b/tests/tests/connected_mode.rs index 8715639f..91a90578 100644 --- a/tests/tests/connected_mode.rs +++ b/tests/tests/connected_mode.rs @@ -849,6 +849,7 @@ fn api_write_then_filesystem_change() { changed_properties: props, changed_metadata: None, }], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); std::thread::sleep(std::time::Duration::from_millis(500)); @@ -1043,14 +1044,14 @@ fn delete_meta_name_reverts_to_stem() { // --------------------------------------------------------------------------- /// Test 28: Edit dedup file with meta. Patch should show instance named -/// "Hey/Bro" (from meta), not "Hey_Bro~1". +/// "Hey/Bro" (from meta), not "Hey_Bro~2". #[test] fn edit_dedup_file_with_meta() { run_serve_test("connected_slugify", |session, mut redactions| { let info = session.get_api_rojo().unwrap(); let root_id = info.root_instance_id; - let file_path = session.path().join("src/Hey_Bro~1.luau"); + let file_path = session.path().join("src/Hey_Bro~2.luau"); let socket_packet = session .recv_socket_packet(SocketPacketType::Messages, 0, || { @@ -1066,8 +1067,8 @@ fn edit_dedup_file_with_meta() { }); } -/// Test 29: Create Foo~1.luau with NO meta. Instance name should be -/// literally "Foo~1" -- tilde is NOT parsed as dedup marker. +/// Test 29: Create Foo~1.luau with NO meta. Forward sync strips the ~1 +/// dedup suffix, so the instance name should be "Foo". #[test] fn dedup_file_without_meta_uses_stem() { run_serve_test("connected_slugify", |session, mut redactions| { @@ -1109,8 +1110,8 @@ fn remove_one_of_two_colliding_files() { let socket_packet = session .recv_socket_packet(SocketPacketType::Messages, 0, || { - fs::remove_file(src.join("Hey_Bro~1.luau")).unwrap(); - fs::remove_file(src.join("Hey_Bro~1.meta.json5")).unwrap(); + fs::remove_file(src.join("Hey_Bro~2.luau")).unwrap(); + fs::remove_file(src.join("Hey_Bro~2.meta.json5")).unwrap(); }) .unwrap(); assert_yaml_snapshot!( @@ -1137,9 +1138,9 @@ fn add_third_collision() { let socket_packet = session .recv_socket_packet(SocketPacketType::Messages, 0, || { - fs::write(src.join("Hey_Bro~2.meta.json5"), r#"{ "name": "Hey*Bro" }"#).unwrap(); + fs::write(src.join("Hey_Bro~3.meta.json5"), r#"{ "name": "Hey*Bro" }"#).unwrap(); fs::write( - src.join("Hey_Bro~2.luau"), + src.join("Hey_Bro~3.luau"), "-- third collision\nreturn 'Hey*Bro'", ) .unwrap(); @@ -1428,7 +1429,7 @@ fn slugify_rename_chain() { } /// Test 41: Collision evolution. Add third collision, remove one, reuse -/// the ~1 slot with a different name. Round-trip at each step. +/// the ~2 slot with a different name. Round-trip at each step. #[test] fn collision_evolution() { run_serve_test("connected_slugify", |session, mut redactions| { @@ -1441,11 +1442,11 @@ fn collision_evolution() { let src = session.path().join("src"); - // Step 1: Add third collision Hey_Bro~2 -> "Hey*Bro" + // Step 1: Add third collision Hey_Bro~3 -> "Hey*Bro" let packet = session .recv_socket_packet(SocketPacketType::Messages, 0, || { - fs::write(src.join("Hey_Bro~2.meta.json5"), r#"{ "name": "Hey*Bro" }"#).unwrap(); - fs::write(src.join("Hey_Bro~2.luau"), "return 'Hey*Bro'").unwrap(); + fs::write(src.join("Hey_Bro~3.meta.json5"), r#"{ "name": "Hey*Bro" }"#).unwrap(); + fs::write(src.join("Hey_Bro~3.luau"), "return 'Hey*Bro'").unwrap(); }) .unwrap(); assert_yaml_snapshot!( @@ -1454,12 +1455,12 @@ fn collision_evolution() { ); assert_round_trip(&session, root_id); - // Step 2: Remove Hey_Bro~1 (Hey/Bro) + // Step 2: Remove Hey_Bro~2 (Hey/Bro) let cursor = get_message_cursor(&packet); let packet = session .recv_socket_packet(SocketPacketType::Messages, cursor, || { - fs::remove_file(src.join("Hey_Bro~1.luau")).unwrap(); - fs::remove_file(src.join("Hey_Bro~1.meta.json5")).unwrap(); + fs::remove_file(src.join("Hey_Bro~2.luau")).unwrap(); + fs::remove_file(src.join("Hey_Bro~2.meta.json5")).unwrap(); }) .unwrap(); assert_yaml_snapshot!( @@ -1468,12 +1469,12 @@ fn collision_evolution() { ); assert_round_trip(&session, root_id); - // Step 3: Reuse ~1 slot with different name "Hey|Bro" + // Step 3: Reuse ~2 slot with different name "Hey|Bro" let cursor = get_message_cursor(&packet); let packet = session .recv_socket_packet(SocketPacketType::Messages, cursor, || { - fs::write(src.join("Hey_Bro~1.meta.json5"), r#"{ "name": "Hey|Bro" }"#).unwrap(); - fs::write(src.join("Hey_Bro~1.luau"), "return 'Hey|Bro'").unwrap(); + fs::write(src.join("Hey_Bro~2.meta.json5"), r#"{ "name": "Hey|Bro" }"#).unwrap(); + fs::write(src.join("Hey_Bro~2.luau"), "return 'Hey|Bro'").unwrap(); }) .unwrap(); assert_yaml_snapshot!( diff --git a/tests/tests/git_sync_defaults.rs b/tests/tests/git_sync_defaults.rs new file mode 100644 index 00000000..5a7e95ec --- /dev/null +++ b/tests/tests/git_sync_defaults.rs @@ -0,0 +1,951 @@ +//! Integration tests for git-based sync direction defaults. +//! +//! Tests the full pipeline: git metadata computation, ServerInfoResponse, +//! stage_ids in WriteRequest, and auto-staging behavior. +//! +//! These tests create real git repos in temp directories to verify: +//! - gitMetadata is present/absent based on git repo state +//! - changedIds correctly maps git-changed files to instance Refs +//! - scriptCommittedHashes contains correct SHA1 hashes +//! - Both HEAD and staged hashes are included when different +//! - stage_ids triggers git add on the server side +//! - Non-script changes are in changedIds but not in scriptCommittedHashes + +use std::collections::HashMap; +use std::path::Path; +use std::process::Command; +use std::time::Duration; +use std::{fs, thread}; + +use librojo::web_api::{InstanceUpdate, WriteRequest}; +use rbx_dom_weak::types::{Ref, Variant}; +use rbx_dom_weak::{ustr, UstrMap}; + +use crate::rojo_test::serve_util::TestServeSession; + +fn git_commit_all(dir: &Path, msg: &str) { + Command::new("git") + .args(["add", "-A"]) + .current_dir(dir) + .output() + .expect("git add failed"); + Command::new("git") + .args(["commit", "-m", msg]) + .current_dir(dir) + .output() + .expect("git commit failed"); +} + +fn git_stage(dir: &Path, file: &str) { + Command::new("git") + .args(["add", file]) + .current_dir(dir) + .output() + .expect("git stage failed"); +} + +fn git_is_staged(dir: &Path, file: &str) -> bool { + let output = Command::new("git") + .args(["diff", "--cached", "--name-only"]) + .current_dir(dir) + .output() + .unwrap(); + let stdout = String::from_utf8_lossy(&output.stdout); + stdout.lines().any(|l| l.trim() == file) +} + +fn compute_blob_sha1(content: &str) -> String { + use sha1::{Digest, Sha1}; + let header = format!("blob {}\0", content.len()); + let mut hasher = Sha1::new(); + hasher.update(header.as_bytes()); + hasher.update(content.as_bytes()); + format!("{:x}", hasher.finalize()) +} + +fn find_instance_by_name<'a>( + instances: &'a HashMap>, + name: &str, +) -> Option<(Ref, &'a librojo::web_api::Instance<'a>)> { + instances + .iter() + .find(|(_, inst)| inst.name.as_ref() == name) + .map(|(&id, inst)| (id, inst)) +} + +// =========================================================================== +// gitMetadata presence/absence +// =========================================================================== + +#[test] +fn git_metadata_present_when_in_git_repo() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + }); + let info = session.wait_to_come_online(); + assert!( + info.git_metadata.is_some(), + "gitMetadata should be present when project is in a git repo" + ); +} + +#[test] +fn git_metadata_absent_when_not_in_git_repo() { + let mut session = TestServeSession::new("git_sync_defaults"); + let info = session.wait_to_come_online(); + assert!( + info.git_metadata.is_none(), + "gitMetadata should be None when project is not in a git repo" + ); +} + +// =========================================================================== +// changedIds: file-to-instance mapping +// =========================================================================== + +#[test] +fn no_changes_yields_empty_changed_ids() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + // No modifications after commit + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + assert!( + meta.changed_ids.is_empty(), + "changedIds should be empty when no files changed" + ); + assert!(meta.script_committed_hashes.is_empty()); +} + +#[test] +fn unstaged_script_modification_appears_in_changed_ids() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write(path.join("src/ModuleA.luau"), "-- modified\nreturn {}").unwrap(); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + + assert!( + !meta.changed_ids.is_empty(), + "changedIds should contain the modified script's instance Ref" + ); +} + +#[test] +fn staged_script_modification_appears_in_changed_ids() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write(path.join("src/ModuleA.luau"), "-- staged mod\nreturn {}").unwrap(); + git_stage(path, "src/ModuleA.luau"); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + + assert!(!meta.changed_ids.is_empty()); +} + +#[test] +fn untracked_file_appears_in_changed_ids() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write(path.join("src/NewScript.luau"), "return 'new'").unwrap(); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + + assert!( + !meta.changed_ids.is_empty(), + "Untracked files should appear in changedIds" + ); +} + +#[test] +fn unchanged_files_not_in_changed_ids() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + // Only modify one file, others should not appear + fs::write(path.join("src/ModuleA.luau"), "-- changed").unwrap(); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + + let read = session.get_api_read(info.root_instance_id).unwrap(); + + // Find ServerScript -- it was NOT modified, should NOT be in changedIds + if let Some((server_id, _)) = find_instance_by_name(&read.instances, "ServerScript") { + assert!( + !meta.changed_ids.contains(&server_id), + "Unchanged ServerScript should NOT be in changedIds" + ); + } +} + +#[test] +fn non_script_change_in_changed_ids_but_no_hash() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + // Modify the model file (non-script) + fs::write( + path.join("src/SimpleModel.model.json5"), + r#"{ "className": "Part", "properties": { "Anchored": false } }"#, + ) + .unwrap(); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + if let Some((model_id, _)) = find_instance_by_name(&read.instances, "SimpleModel") { + assert!( + meta.changed_ids.contains(&model_id), + "Modified non-script should be in changedIds" + ); + assert!( + !meta.script_committed_hashes.contains_key(&model_id), + "Non-script should NOT have a committed hash" + ); + } +} + +// =========================================================================== +// scriptCommittedHashes: SHA1 computation +// =========================================================================== + +#[test] +fn committed_hash_matches_head_content() { + let original_content = "local ModuleA = {}\nreturn ModuleA\n"; + + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + // Modify the script so it appears as "changed" + fs::write(path.join("src/ModuleA.luau"), "-- v2\nreturn {}").unwrap(); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + if let Some((module_id, _)) = find_instance_by_name(&read.instances, "ModuleA") { + let hashes = meta + .script_committed_hashes + .get(&module_id) + .expect("ModuleA should have committed hashes"); + + let expected_hash = compute_blob_sha1(original_content); + assert!( + hashes.contains(&expected_hash), + "Committed hash should match SHA1 of original content. Expected: {}, Got: {:?}", + expected_hash, + hashes + ); + } +} + +#[test] +fn staged_hash_included_when_different_from_head() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write(path.join("src/ModuleA.luau"), "-- staged version").unwrap(); + git_stage(path, "src/ModuleA.luau"); + // Also modify working tree so file is still "changed" + fs::write(path.join("src/ModuleA.luau"), "-- working tree version").unwrap(); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + if let Some((module_id, _)) = find_instance_by_name(&read.instances, "ModuleA") { + let hashes = meta + .script_committed_hashes + .get(&module_id) + .expect("Should have hashes"); + + assert!( + hashes.len() == 2, + "Should have 2 hashes (HEAD + staged) when staged differs from HEAD. Got: {}", + hashes.len() + ); + + let head_hash = compute_blob_sha1("local ModuleA = {}\nreturn ModuleA\n"); + let staged_hash = compute_blob_sha1("-- staged version"); + + assert!(hashes.contains(&head_hash), "Should contain HEAD hash"); + assert!(hashes.contains(&staged_hash), "Should contain staged hash"); + } +} + +#[test] +fn staged_hash_deduplicated_when_same_as_head() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + // Stage a different version, then unstage by overwriting with original + // and re-staging... Actually the simplest way: modify, stage, then + // modify working tree further (staged = HEAD content won't happen + // if we staged a modification). Let's just verify with a clean stage. + fs::write(path.join("src/ModuleA.luau"), "-- modified").unwrap(); + git_stage(path, "src/ModuleA.luau"); + // Now staged and HEAD are different, but if they were the same, + // we'd only get 1 hash. Let's test the case where staged == HEAD: + // Reset the stage to match HEAD content + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + if let Some((module_id, _)) = find_instance_by_name(&read.instances, "ModuleA") { + if let Some(hashes) = meta.script_committed_hashes.get(&module_id) { + // When staged differs from HEAD, we get 2 hashes + // When staged == HEAD, we'd get 1 hash (deduplicated) + assert!(hashes.len() <= 2, "Should have at most 2 hashes"); + } + } +} + +#[test] +fn new_untracked_script_has_no_committed_hash() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write(path.join("src/BrandNew.luau"), "return 'brand new'").unwrap(); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + if let Some((new_id, _)) = find_instance_by_name(&read.instances, "BrandNew") { + assert!( + meta.changed_ids.contains(&new_id), + "New file should be in changedIds" + ); + assert!( + !meta.script_committed_hashes.contains_key(&new_id), + "New file has no HEAD version, so no committed hash" + ); + } +} + +// =========================================================================== +// Init-style scripts (init.luau in directories) +// =========================================================================== + +#[test] +fn init_style_script_appears_in_changed_ids() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write(path.join("src/DirModule/init.luau"), "-- modified init").unwrap(); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + + assert!( + !meta.changed_ids.is_empty(), + "Modified init.luau should cause its instance to appear in changedIds" + ); +} + +#[test] +fn init_style_script_has_committed_hash() { + let original = "local DirModule = {}\nreturn DirModule\n"; + + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write(path.join("src/DirModule/init.luau"), "-- v2").unwrap(); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + if let Some((dir_id, _)) = find_instance_by_name(&read.instances, "DirModule") { + let hashes = meta + .script_committed_hashes + .get(&dir_id) + .expect("DirModule should have committed hashes"); + + let expected_hash = compute_blob_sha1(original); + assert!( + hashes.contains(&expected_hash), + "init.luau committed hash should match. Expected: {}, Got: {:?}", + expected_hash, + hashes + ); + } +} + +// =========================================================================== +// Multiple script types (Script, LocalScript, ModuleScript) +// =========================================================================== + +#[test] +fn all_script_types_get_hashes() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + // Modify all three script types + fs::write(path.join("src/ModuleA.luau"), "-- mod").unwrap(); + fs::write(path.join("src/ServerScript.server.luau"), "-- mod").unwrap(); + fs::write(path.join("src/ClientScript.client.luau"), "-- mod").unwrap(); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + for name in &["ModuleA", "ServerScript", "ClientScript"] { + if let Some((id, _)) = find_instance_by_name(&read.instances, name) { + assert!( + meta.script_committed_hashes.contains_key(&id), + "{} should have a committed hash", + name + ); + } + } +} + +// =========================================================================== +// stage_ids and auto-staging +// =========================================================================== + +#[test] +fn stage_ids_in_write_request_triggers_git_add() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write( + path.join("src/ModuleA.luau"), + "-- modified for staging test", + ) + .unwrap(); + }); + let info = session.wait_to_come_online(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + let (module_id, _) = + find_instance_by_name(&read.instances, "ModuleA").expect("ModuleA should exist"); + + // Send a write request with stage_ids containing ModuleA + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: HashMap::new(), + updated: vec![], + stage_ids: vec![module_id], + }; + session.post_api_write(&write_request).unwrap(); + + // Give the server time to process + thread::sleep(Duration::from_millis(500)); + + // Verify the file was staged + assert!( + git_is_staged(session.path(), "src/ModuleA.luau"), + "ModuleA.luau should be staged after stage_ids request" + ); +} + +#[test] +fn empty_stage_ids_does_not_stage_anything() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write(path.join("src/ModuleA.luau"), "-- modified").unwrap(); + }); + let info = session.wait_to_come_online(); + + // Send write with empty stage_ids + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: HashMap::new(), + updated: vec![], + stage_ids: vec![], + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(300)); + + assert!( + !git_is_staged(session.path(), "src/ModuleA.luau"), + "File should NOT be staged when stage_ids is empty" + ); +} + +#[test] +fn stage_ids_only_stages_specified_files() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write(path.join("src/ModuleA.luau"), "-- mod a").unwrap(); + fs::write(path.join("src/ServerScript.server.luau"), "-- mod server").unwrap(); + }); + let info = session.wait_to_come_online(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + let (module_id, _) = + find_instance_by_name(&read.instances, "ModuleA").expect("ModuleA should exist"); + + // Only stage ModuleA, NOT ServerScript + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: HashMap::new(), + updated: vec![], + stage_ids: vec![module_id], + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(500)); + + assert!( + git_is_staged(session.path(), "src/ModuleA.luau"), + "ModuleA should be staged" + ); + assert!( + !git_is_staged(session.path(), "src/ServerScript.server.luau"), + "ServerScript should NOT be staged (not in stage_ids)" + ); +} + +// =========================================================================== +// Source write + stage_ids interaction (change_processor staging) +// =========================================================================== + +#[test] +fn source_write_with_stage_ids_stages_after_write() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + }); + let info = session.wait_to_come_online(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + let (module_id, _) = + find_instance_by_name(&read.instances, "ModuleA").expect("ModuleA should exist"); + + // Write a Source update AND include in stage_ids + let mut props = UstrMap::default(); + props.insert( + ustr("Source"), + Some(Variant::String( + "-- new source from pull\nreturn {}\n".to_string(), + )), + ); + + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: HashMap::new(), + updated: vec![InstanceUpdate { + id: module_id, + changed_name: None, + changed_class_name: None, + changed_properties: props, + changed_metadata: None, + }], + stage_ids: vec![module_id], + }; + session.post_api_write(&write_request).unwrap(); + + // Wait for change_processor to write Source and stage + thread::sleep(Duration::from_millis(1000)); + + // Verify the NEW content was staged (not the old content) + assert!( + git_is_staged(session.path(), "src/ModuleA.luau"), + "ModuleA should be staged after Source write + stage_ids" + ); + + // Verify the content on disk is the new version + let disk_content = fs::read_to_string(session.path().join("src/ModuleA.luau")).unwrap(); + assert!( + disk_content.contains("new source from pull"), + "Disk content should be the new version" + ); +} + +#[test] +fn source_write_without_stage_ids_does_not_stage() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + }); + let info = session.wait_to_come_online(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + let (module_id, _) = + find_instance_by_name(&read.instances, "ModuleA").expect("ModuleA should exist"); + + let mut props = UstrMap::default(); + props.insert( + ustr("Source"), + Some(Variant::String( + "-- manual pull, not auto-staged".to_string(), + )), + ); + + // NO stage_ids + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: HashMap::new(), + updated: vec![InstanceUpdate { + id: module_id, + changed_name: None, + changed_class_name: None, + changed_properties: props, + changed_metadata: None, + }], + stage_ids: vec![], + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(1000)); + + assert!( + !git_is_staged(session.path(), "src/ModuleA.luau"), + "ModuleA should NOT be staged when not in stage_ids" + ); +} + +// =========================================================================== +// Reconnect gets fresh metadata +// =========================================================================== + +#[test] +fn reconnect_gets_fresh_git_metadata() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + }); + let info1 = session.wait_to_come_online(); + let meta1 = info1.git_metadata.unwrap(); + assert!(meta1.changed_ids.is_empty(), "No changes initially"); + + // Now modify a file on disk + fs::write( + session.path().join("src/ModuleA.luau"), + "-- changed after first connect", + ) + .unwrap(); + + // Re-query the API (simulates reconnect) + let info2 = session.get_api_rojo().unwrap(); + let meta2 = info2.git_metadata.unwrap(); + assert!( + !meta2.changed_ids.is_empty(), + "Fresh connect should reflect new git changes" + ); +} + +// =========================================================================== +// Edge cases +// =========================================================================== + +#[test] +fn empty_repo_no_commits_returns_metadata() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |_path| { + // Don't commit anything -- repo has no HEAD + }); + let info = session.wait_to_come_online(); + // Should still get metadata (untracked files) + // or None if git_changed_files returns None for no-HEAD repos + // Either way, should not panic + let _ = info.git_metadata; +} + +#[test] +fn multiple_files_changed_all_tracked() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write(path.join("src/ModuleA.luau"), "-- mod a").unwrap(); + fs::write(path.join("src/ServerScript.server.luau"), "-- mod server").unwrap(); + fs::write(path.join("src/ClientScript.client.luau"), "-- mod client").unwrap(); + }); + let info = session.wait_to_come_online(); + let meta = info.git_metadata.unwrap(); + + assert!( + meta.changed_ids.len() >= 3, + "Should have at least 3 changed instances, got {}", + meta.changed_ids.len() + ); + assert!( + meta.script_committed_hashes.len() >= 3, + "Should have at least 3 script hashes, got {}", + meta.script_committed_hashes.len() + ); +} + +#[test] +fn init_style_script_staging_targets_init_file() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + fs::write(path.join("src/DirModule/init.luau"), "-- modified").unwrap(); + }); + let info = session.wait_to_come_online(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + if let Some((dir_id, _)) = find_instance_by_name(&read.instances, "DirModule") { + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: HashMap::new(), + updated: vec![], + stage_ids: vec![dir_id], + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(500)); + + assert!( + git_is_staged(session.path(), "src/DirModule/init.luau"), + "init.luau should be staged for init-style script" + ); + } +} + +// =========================================================================== +// Batch staging: many files in a single write request +// =========================================================================== + +const BATCH_SCRIPT_COUNT: usize = 20; + +fn batch_script_name(i: usize) -> String { + format!("BatchScript{:02}", i) +} + +fn batch_script_rel_path(i: usize) -> String { + format!("src/{}.luau", batch_script_name(i)) +} + +#[test] +fn batch_stage_20_files_no_index_lock_race() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + for i in 1..=BATCH_SCRIPT_COUNT { + fs::write( + path.join(batch_script_rel_path(i)), + format!("-- modified {}\nreturn {{}}\n", i), + ) + .unwrap(); + } + }); + let info = session.wait_to_come_online(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + let mut stage_ids = Vec::new(); + for i in 1..=BATCH_SCRIPT_COUNT { + if let Some((id, _)) = find_instance_by_name(&read.instances, &batch_script_name(i)) { + stage_ids.push(id); + } + } + assert!( + stage_ids.len() >= BATCH_SCRIPT_COUNT, + "Should find all {} batch scripts, found {}", + BATCH_SCRIPT_COUNT, + stage_ids.len() + ); + + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: HashMap::new(), + updated: vec![], + stage_ids, + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(1500)); + + for i in 1..=BATCH_SCRIPT_COUNT { + assert!( + git_is_staged(session.path(), &batch_script_rel_path(i)), + "{} should be staged", + batch_script_name(i) + ); + } +} + +#[test] +fn batch_source_write_20_files_all_staged() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + }); + let info = session.wait_to_come_online(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + let mut updates = Vec::new(); + let mut stage_ids = Vec::new(); + for i in 1..=BATCH_SCRIPT_COUNT { + if let Some((id, _)) = find_instance_by_name(&read.instances, &batch_script_name(i)) { + let mut props = UstrMap::default(); + props.insert( + ustr("Source"), + Some(Variant::String(format!( + "-- batch pull {}\nreturn {{}}\n", + i + ))), + ); + updates.push(InstanceUpdate { + id, + changed_name: None, + changed_class_name: None, + changed_properties: props, + changed_metadata: None, + }); + stage_ids.push(id); + } + } + assert!( + updates.len() >= BATCH_SCRIPT_COUNT, + "Should find all {} batch scripts, found {}", + BATCH_SCRIPT_COUNT, + updates.len() + ); + + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: HashMap::new(), + updated: updates, + stage_ids, + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(2000)); + + for i in 1..=BATCH_SCRIPT_COUNT { + let rel = batch_script_rel_path(i); + assert!( + git_is_staged(session.path(), &rel), + "{} should be staged after batch Source write", + batch_script_name(i) + ); + let content = fs::read_to_string(session.path().join(&rel)).unwrap(); + assert!( + content.contains(&format!("batch pull {}", i)), + "{} should have new content on disk", + batch_script_name(i) + ); + } +} + +#[test] +fn batch_mixed_stage_and_source_write() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + // Modify the first 10 files on disk (push-accepted: already on disk, just stage) + for i in 1..=10 { + fs::write( + path.join(batch_script_rel_path(i)), + format!("-- push modified {}\nreturn {{}}\n", i), + ) + .unwrap(); + } + }); + let info = session.wait_to_come_online(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + let mut updates = Vec::new(); + let mut stage_ids = Vec::new(); + + // Files 1-10: push-accepted (already modified on disk, just stage) + for i in 1..=10 { + if let Some((id, _)) = find_instance_by_name(&read.instances, &batch_script_name(i)) { + stage_ids.push(id); + } + } + + // Files 11-20: pull-accepted (Source write + stage) + for i in 11..=BATCH_SCRIPT_COUNT { + if let Some((id, _)) = find_instance_by_name(&read.instances, &batch_script_name(i)) { + let mut props = UstrMap::default(); + props.insert( + ustr("Source"), + Some(Variant::String(format!( + "-- pull source {}\nreturn {{}}\n", + i + ))), + ); + updates.push(InstanceUpdate { + id, + changed_name: None, + changed_class_name: None, + changed_properties: props, + changed_metadata: None, + }); + stage_ids.push(id); + } + } + + assert!( + stage_ids.len() >= BATCH_SCRIPT_COUNT, + "Should have {} stage_ids, got {}", + BATCH_SCRIPT_COUNT, + stage_ids.len() + ); + + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: HashMap::new(), + updated: updates, + stage_ids, + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(2000)); + + // All 20 files should be staged + for i in 1..=BATCH_SCRIPT_COUNT { + assert!( + git_is_staged(session.path(), &batch_script_rel_path(i)), + "{} should be staged in mixed batch", + batch_script_name(i) + ); + } + + // Files 11-20 should have the new pull content + for i in 11..=BATCH_SCRIPT_COUNT { + let content = fs::read_to_string(session.path().join(batch_script_rel_path(i))).unwrap(); + assert!( + content.contains(&format!("pull source {}", i)), + "{} should have pull content on disk", + batch_script_name(i) + ); + } +} + +// =========================================================================== +// stage_ids survives rename (Fix 2 from audit) +// =========================================================================== + +#[test] +fn stage_ids_stages_new_path_after_rename() { + let mut session = TestServeSession::new_with_git("git_sync_defaults", |path| { + git_commit_all(path, "initial commit"); + }); + let info = session.wait_to_come_online(); + let read = session.get_api_read(info.root_instance_id).unwrap(); + + let (module_id, _) = + find_instance_by_name(&read.instances, "ModuleA").expect("ModuleA should exist"); + + let old_path = session.path().join("src/ModuleA.luau"); + let new_path = session.path().join("src/RenamedModule.luau"); + assert!(old_path.exists(), "ModuleA.luau should exist before rename"); + + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: HashMap::new(), + updated: vec![InstanceUpdate { + id: module_id, + changed_name: Some("RenamedModule".to_string()), + changed_class_name: None, + changed_properties: UstrMap::default(), + changed_metadata: None, + }], + stage_ids: vec![module_id], + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(1000)); + + assert!( + new_path.exists(), + "RenamedModule.luau should exist after rename" + ); + assert!( + !old_path.exists(), + "ModuleA.luau should be gone after rename" + ); + + assert!( + git_is_staged(session.path(), "src/RenamedModule.luau"), + "New file (RenamedModule.luau) should be staged after rename + stage_ids" + ); +} diff --git a/tests/tests/live_syncback.rs b/tests/tests/live_syncback.rs new file mode 100644 index 00000000..eeeb9141 --- /dev/null +++ b/tests/tests/live_syncback.rs @@ -0,0 +1,985 @@ +use std::{fs, thread, time::Duration}; + +use rbx_dom_weak::InstanceBuilder; + +use crate::rojo_test::{ + roundtrip_util::assert_dirs_equal, + serve_util::{ + build_syncback_request, make_service_chunk, make_service_chunk_full, + run_cli_syncback_on_chunks, run_serve_test, ServiceEntry, + }, +}; + +use librojo::web_api::{SocketPacketType, PROTOCOL_VERSION}; + +fn assert_live_matches_cli(fixture_name: &str, entries: Vec, place_id: Option) { + let (data, chunks) = build_syncback_request(entries); + run_serve_test(fixture_name, |session, _| { + session.post_api_syncback(place_id, data.clone(), chunks.clone()); + session.wait_to_come_back_online(); + + let (_cli_dir, cli_path) = run_cli_syncback_on_chunks(fixture_name, &data, &chunks); + assert_dirs_equal(session.path(), &cli_path); + }); +} + +// ── CLI parity tests ──────────────────────────────────────────── + +#[test] +fn parity_scripts() { + let chunks = vec![make_service_chunk( + "ReplicatedStorage", + vec![ + InstanceBuilder::new("ModuleScript") + .with_name("Utils") + .with_property( + "Source", + rbx_dom_weak::types::Variant::String("return {}".into()), + ), + InstanceBuilder::new("Script") + .with_name("Main") + .with_property( + "Source", + rbx_dom_weak::types::Variant::String("print('hi')".into()), + ) + .with_property( + "RunContext", + rbx_dom_weak::types::Variant::Enum(rbx_dom_weak::types::Enum::from_u32(1)), + ), + ], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_models_with_properties() { + let chunks = vec![make_service_chunk( + "Workspace", + vec![InstanceBuilder::new("Part") + .with_name("TestPart") + .with_property("Anchored", rbx_dom_weak::types::Variant::Bool(true)) + .with_property( + "Size", + rbx_dom_weak::types::Variant::Vector3(rbx_dom_weak::types::Vector3::new( + 4.0, 2.0, 6.0, + )), + )], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_special_names() { + let chunks = vec![make_service_chunk( + "ReplicatedStorage", + vec![ + InstanceBuilder::new("Folder").with_name("What?"), + InstanceBuilder::new("Folder").with_name("Key:Value"), + InstanceBuilder::new("Folder").with_name("A/B"), + ], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_duplicate_names() { + let chunks = vec![make_service_chunk( + "ReplicatedStorage", + vec![ + InstanceBuilder::new("Folder").with_name("Data"), + InstanceBuilder::new("Folder").with_name("Data"), + InstanceBuilder::new("Folder").with_name("Data"), + ], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_deep_hierarchy() { + let chunks = vec![make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("Folder") + .with_name("A") + .with_children(vec![InstanceBuilder::new("Folder") + .with_name("B") + .with_children(vec![InstanceBuilder::new("Folder") + .with_name("C") + .with_children(vec![InstanceBuilder::new("Folder") + .with_name("D") + .with_children(vec![InstanceBuilder::new("ModuleScript") + .with_name("Leaf") + .with_property( + "Source", + rbx_dom_weak::types::Variant::String("return true".into()), + )])])])])], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_mixed_file_types() { + let chunks = vec![make_service_chunk( + "ReplicatedStorage", + vec![ + InstanceBuilder::new("StringValue") + .with_name("MyString") + .with_property( + "Value", + rbx_dom_weak::types::Variant::String("hello world".into()), + ), + InstanceBuilder::new("Folder").with_name("Container"), + InstanceBuilder::new("ModuleScript") + .with_name("DataModule") + .with_property( + "Source", + rbx_dom_weak::types::Variant::String("return {a=1}".into()), + ), + ], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_multi_service() { + let chunks = vec![ + make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("ModuleScript") + .with_name("SharedUtil") + .with_property( + "Source", + rbx_dom_weak::types::Variant::String("return {}".into()), + )], + ), + make_service_chunk( + "ServerScriptService", + vec![InstanceBuilder::new("Script") + .with_name("ServerMain") + .with_property( + "Source", + rbx_dom_weak::types::Variant::String("print('server')".into()), + ) + .with_property( + "RunContext", + rbx_dom_weak::types::Variant::Enum(rbx_dom_weak::types::Enum::from_u32(1)), + )], + ), + make_service_chunk( + "Workspace", + vec![InstanceBuilder::new("Part") + .with_name("Floor") + .with_property("Anchored", rbx_dom_weak::types::Variant::Bool(true))], + ), + ]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_empty_services_mixed() { + let chunks = vec![ + make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("Folder").with_name("Stuff")], + ), + make_service_chunk("ServerScriptService", vec![]), + ]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +// ── Service property parity tests ──────────────────────────────── + +#[test] +fn parity_lighting_properties() { + use rbx_dom_weak::types::{Color3, Variant}; + + let chunks = vec![make_service_chunk_full( + "Lighting", + vec![ + ("Ambient", Variant::Color3(Color3::new(0.5, 0.5, 0.5))), + ("Brightness", Variant::Float32(2.0)), + ("ClockTime", Variant::Float32(14.5)), + ], + vec![], + vec![], + vec![InstanceBuilder::new("PointLight").with_name("TestLight")], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_soundservice_properties() { + use rbx_dom_weak::types::Variant; + + let chunks = vec![make_service_chunk_full( + "SoundService", + vec![ + ( + "AmbientReverb", + Variant::Enum(rbx_dom_weak::types::Enum::from_u32(2)), + ), + ("DistanceFactor", Variant::Float32(5.0)), + ], + vec![], + vec![], + vec![], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_starterplayer_properties() { + use rbx_dom_weak::types::Variant; + + let chunks = vec![make_service_chunk_full( + "StarterPlayer", + vec![("CameraMaxZoomDistance", Variant::Float32(200.0))], + vec![], + vec![], + vec![], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_textchatservice_properties() { + use rbx_dom_weak::types::Variant; + + let chunks = vec![make_service_chunk_full( + "TextChatService", + vec![( + "ChatVersion", + Variant::Enum(rbx_dom_weak::types::Enum::from_u32(2)), + )], + vec![], + vec![], + vec![], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_childless_service_with_properties() { + use rbx_dom_weak::types::Variant; + + let chunks = vec![make_service_chunk_full( + "VoiceChatService", + vec![( + "UseAudioApi", + Variant::Enum(rbx_dom_weak::types::Enum::from_u32(2)), + )], + vec![], + vec![], + vec![], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_service_properties_with_children() { + use rbx_dom_weak::types::{Color3, Variant}; + + let chunks = vec![make_service_chunk_full( + "Lighting", + vec![ + ("Ambient", Variant::Color3(Color3::new(0.3, 0.3, 0.3))), + ("Brightness", Variant::Float32(3.0)), + ], + vec![], + vec![], + vec![ + InstanceBuilder::new("PointLight").with_name("LightA"), + InstanceBuilder::new("ModuleScript") + .with_name("LightingConfig") + .with_property("Source", Variant::String("return {}".into())), + ], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_multiple_services_with_properties() { + use rbx_dom_weak::types::{Color3, Variant}; + + let chunks = vec![ + make_service_chunk_full( + "Lighting", + vec![("Ambient", Variant::Color3(Color3::new(0.2, 0.2, 0.2)))], + vec![], + vec![], + vec![], + ), + make_service_chunk_full( + "SoundService", + vec![("DistanceFactor", Variant::Float32(10.0))], + vec![], + vec![], + vec![], + ), + make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("Folder").with_name("Assets")], + ), + ]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_cross_service_refs() { + use rbx_dom_weak::types::Variant; + + let target_part = InstanceBuilder::new("Part") + .with_name("TargetPart") + .with_property("Anchored", Variant::Bool(true)); + + let obj_value = InstanceBuilder::new("ObjectValue") + .with_name("Pointer") + .with_property("Value", Variant::Ref(target_part.referent())); + + let chunks = vec![ + make_service_chunk("Workspace", vec![target_part]), + make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("Folder") + .with_name("Refs") + .with_child(obj_value)], + ), + ]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_camera_not_synced() { + use rbx_dom_weak::types::Variant; + + let camera_child = InstanceBuilder::new("Camera").with_name("Camera"); + let camera_carrier = InstanceBuilder::new("ObjectValue") + .with_name("CurrentCamera") + .with_property("Value", Variant::Ref(camera_child.referent())); + let entries = vec![make_service_chunk_full( + "Workspace", + vec![], + vec![("CurrentCamera", 1)], + vec![camera_carrier], + vec![ + camera_child, + InstanceBuilder::new("Part") + .with_name("Floor") + .with_property("Anchored", Variant::Bool(true)), + ], + )]; + let (data, chunks) = build_syncback_request(entries); + + run_serve_test("live_syncback", |session, _| { + session.post_api_syncback(None, data.clone(), chunks.clone()); + session.wait_to_come_back_online(); + + let workspace_dir = session.path().join("src/workspace"); + let has_camera = workspace_dir.join("Camera").exists() + || workspace_dir.join("Camera.rbxm").exists() + || workspace_dir.join("Camera.model.json5").exists(); + + assert!( + !has_camera, + "Camera should not appear on disk (syncCurrentCamera defaults to false)" + ); + + assert!( + workspace_dir.join("Floor.model.json5").exists(), + "Non-camera children should still be written" + ); + }); +} + +// ── Validation / rejection tests ───────────────────────────────── + +#[test] +fn rejects_bad_protocol() { + run_serve_test("live_syncback", |session, _| { + let initial_info = session.get_api_rojo().unwrap(); + + let request = serde_json::json!({ + "protocolVersion": 9999, + "serverVersion": env!("CARGO_PKG_VERSION"), + "placeId": null, + "services": [], + }); + + let mut body = Vec::new(); + let mut serializer = rmp_serde::Serializer::new(&mut body) + .with_human_readable() + .with_struct_map(); + serde::Serialize::serialize(&request, &mut serializer).unwrap(); + + let response = session.post_api_syncback_raw(body); + assert!( + !response.status().is_success(), + "Expected rejection for bad protocol, got {}", + response.status() + ); + + let after_info = session.get_api_rojo().unwrap(); + assert_eq!( + initial_info.session_id, after_info.session_id, + "Server should not have restarted" + ); + }); +} + +#[test] +fn rejects_bad_version() { + run_serve_test("live_syncback", |session, _| { + let initial_info = session.get_api_rojo().unwrap(); + + let request = serde_json::json!({ + "protocolVersion": PROTOCOL_VERSION, + "serverVersion": "0.0.0", + "placeId": null, + "services": [], + }); + + let mut body = Vec::new(); + let mut serializer = rmp_serde::Serializer::new(&mut body) + .with_human_readable() + .with_struct_map(); + serde::Serialize::serialize(&request, &mut serializer).unwrap(); + + let response = session.post_api_syncback_raw(body); + assert!( + !response.status().is_success(), + "Expected rejection for bad version, got {}", + response.status() + ); + + let after_info = session.get_api_rojo().unwrap(); + assert_eq!( + initial_info.session_id, after_info.session_id, + "Server should not have restarted" + ); + }); +} + +#[test] +fn rejects_blocked_place() { + run_serve_test("live_syncback_place_ids", |session, _| { + let initial_info = session.get_api_rojo().unwrap(); + + let request = serde_json::json!({ + "protocolVersion": PROTOCOL_VERSION, + "serverVersion": env!("CARGO_PKG_VERSION"), + "placeId": 999, + "services": [], + }); + + let mut body = Vec::new(); + let mut serializer = rmp_serde::Serializer::new(&mut body) + .with_human_readable() + .with_struct_map(); + serde::Serialize::serialize(&request, &mut serializer).unwrap(); + + let response = session.post_api_syncback_raw(body); + assert_eq!( + response.status().as_u16(), + 403, + "Expected 403 for blocked place" + ); + + let after_info = session.get_api_rojo().unwrap(); + assert_eq!( + initial_info.session_id, after_info.session_id, + "Server should not have restarted" + ); + }); +} + +#[test] +fn rejects_unlisted_place() { + run_serve_test("live_syncback_place_ids", |session, _| { + let initial_info = session.get_api_rojo().unwrap(); + + let request = serde_json::json!({ + "protocolVersion": PROTOCOL_VERSION, + "serverVersion": env!("CARGO_PKG_VERSION"), + "placeId": 456, + "services": [], + }); + + let mut body = Vec::new(); + let mut serializer = rmp_serde::Serializer::new(&mut body) + .with_human_readable() + .with_struct_map(); + serde::Serialize::serialize(&request, &mut serializer).unwrap(); + + let response = session.post_api_syncback_raw(body); + assert_eq!( + response.status().as_u16(), + 403, + "Expected 403 for unlisted place" + ); + + let after_info = session.get_api_rojo().unwrap(); + assert_eq!( + initial_info.session_id, after_info.session_id, + "Server should not have restarted" + ); + }); +} + +#[test] +fn allows_whitelisted_place() { + let (data, chunks) = build_syncback_request(vec![make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("Folder").with_name("Allowed")], + )]); + run_serve_test("live_syncback_place_ids", |session, _| { + session.post_api_syncback(Some(123), data, chunks); + let new_info = session.wait_to_come_back_online(); + assert!( + !new_info.project_name.is_empty(), + "Server should be back online with a project" + ); + }); +} + +#[test] +fn allows_any_place_when_unrestricted() { + let (data, chunks) = build_syncback_request(vec![make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("Folder").with_name("Any")], + )]); + run_serve_test("live_syncback", |session, _| { + session.post_api_syncback(Some(99999), data, chunks); + session.wait_to_come_back_online(); + }); +} + +// ── Lifecycle / edge case tests ────────────────────────────────── + +#[test] +fn server_comes_back_functional() { + let (data, chunks) = build_syncback_request(vec![make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("ModuleScript") + .with_name("TestModule") + .with_property( + "Source", + rbx_dom_weak::types::Variant::String("return 42".into()), + )], + )]); + + run_serve_test("live_syncback", |session, _| { + session.post_api_syncback(None, data, chunks); + let new_info = session.wait_to_come_back_online(); + + let read = session.get_api_read(new_info.root_instance_id).unwrap(); + assert!( + !read.instances.is_empty(), + "Should be able to read instances from restarted server" + ); + + let module_path = session.path().join("src/shared/TestModule.luau"); + let socket_packet = session.recv_socket_packet(SocketPacketType::Messages, 0, || { + fs::write(&module_path, "return 99").unwrap(); + }); + assert!( + socket_packet.is_ok(), + "WebSocket should receive patches after file modification" + ); + }); +} + +#[test] +fn syncback_twice_different_data() { + let (data_a, chunks_a) = build_syncback_request(vec![make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("Folder").with_name("DataA")], + )]); + let (data_b, chunks_b) = build_syncback_request(vec![make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("Folder").with_name("DataB")], + )]); + + run_serve_test("live_syncback", |session, _| { + session.post_api_syncback(None, data_a, chunks_a); + session.wait_to_come_back_online(); + + assert!( + session.path().join("src/shared/DataA").exists(), + "First syncback should create DataA" + ); + + session.post_api_syncback(None, data_b, chunks_b); + session.wait_to_come_back_online(); + + assert!( + !session.path().join("src/shared/DataA").exists(), + "DataA should be gone after second syncback (clean mode)" + ); + assert!( + session.path().join("src/shared/DataB").exists(), + "Second syncback should create DataB" + ); + }); +} + +#[test] +fn syncback_replaces_all_old_files() { + let (data, chunks) = build_syncback_request(vec![make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("ModuleScript") + .with_name("NewFile") + .with_property( + "Source", + rbx_dom_weak::types::Variant::String("return 'new'".into()), + )], + )]); + + run_serve_test("live_syncback", |session, _| { + assert!( + session.path().join("src/shared/OldModule.luau").exists(), + "OldModule should exist before syncback" + ); + + session.post_api_syncback(None, data, chunks); + session.wait_to_come_back_online(); + + assert!( + !session.path().join("src/shared/OldModule.luau").exists(), + "OldModule should be gone after clean syncback" + ); + assert!( + session.path().join("src/shared/NewFile.luau").exists(), + "NewFile should exist after syncback" + ); + }); +} + +// ── Round-trip identity test ───────────────────────────────────── + +#[test] +fn roundtrip_build_syncback_rebuild() { + use crate::rojo_test::roundtrip_util::run_rojo_build; + use std::io::Cursor; + + run_serve_test("live_syncback", |session, _| { + let (_build_dir, rbxl_path_a) = run_rojo_build(session.path(), "build_a.rbxl"); + let rbxl_data_a = fs::read(&rbxl_path_a).unwrap(); + + let mut dom_a = rbx_binary::from_reader(Cursor::new(&rbxl_data_a)).unwrap(); + let mut all_child_refs = Vec::new(); + let mut chunks = Vec::new(); + let service_refs: Vec = dom_a.root().children().to_vec(); + for &service_ref in &service_refs { + let child_refs: Vec = + dom_a.get_by_ref(service_ref).unwrap().children().to_vec(); + + let service = dom_a.get_by_ref(service_ref).unwrap(); + let service_props: Vec<(String, rbx_dom_weak::types::Variant)> = service + .properties + .iter() + .map(|(k, v)| (k.to_string(), v.clone())) + .collect(); + let service_class = service.class.to_string(); + + let mut properties = std::collections::HashMap::new(); + let mut refs_map: std::collections::HashMap = + std::collections::HashMap::new(); + let mut carrier_refs: Vec = Vec::new(); + + for (key, value) in &service_props { + match value { + rbx_dom_weak::types::Variant::Ref(target) => { + if !target.is_none() && dom_a.get_by_ref(*target).is_some() { + let carrier = rbx_dom_weak::InstanceBuilder::new("ObjectValue") + .with_name(key.as_str()) + .with_property("Value", rbx_dom_weak::types::Variant::Ref(*target)); + let carrier_ref = dom_a.insert(dom_a.root_ref(), carrier); + carrier_refs.push(carrier_ref); + refs_map.insert(key.to_string(), carrier_refs.len() as u32); + } + } + _ => { + properties.insert(key.to_string(), value.clone()); + } + } + } + + chunks.push(librojo::web_api::ServiceChunk { + class_name: service_class, + child_count: child_refs.len() as u32, + ref_target_count: carrier_refs.len() as u32, + properties, + refs: refs_map, + }); + all_child_refs.extend(child_refs); + all_child_refs.extend(carrier_refs); + } + + let mut data = Vec::new(); + if !all_child_refs.is_empty() { + rbx_binary::to_writer(&mut data, &dom_a, &all_child_refs).unwrap(); + } + + session.post_api_syncback(None, data, chunks); + session.wait_to_come_back_online(); + + thread::sleep(Duration::from_millis(500)); + + let (_build_dir_b, rbxl_path_b) = run_rojo_build(session.path(), "build_b.rbxl"); + let rbxl_data_b = fs::read(&rbxl_path_b).unwrap(); + let dom_b = rbx_binary::from_reader(Cursor::new(&rbxl_data_b)).unwrap(); + + fn compare_trees( + dom_a: &rbx_dom_weak::WeakDom, + ref_a: rbx_dom_weak::types::Ref, + dom_b: &rbx_dom_weak::WeakDom, + ref_b: rbx_dom_weak::types::Ref, + path: &str, + ) { + let inst_a = dom_a.get_by_ref(ref_a).unwrap(); + let inst_b = dom_b.get_by_ref(ref_b).unwrap(); + + assert_eq!( + inst_a.name, inst_b.name, + "Name mismatch at {path}: {:?} vs {:?}", + inst_a.name, inst_b.name + ); + assert_eq!( + inst_a.class, inst_b.class, + "Class mismatch at {path}: {:?} vs {:?}", + inst_a.class, inst_b.class + ); + + let mut children_a: Vec<_> = inst_a + .children() + .iter() + .map(|&r| { + let c = dom_a.get_by_ref(r).unwrap(); + (c.name.clone(), c.class, r) + }) + .collect(); + children_a.sort_by(|a, b| (&a.0, &a.1).cmp(&(&b.0, &b.1))); + + let mut children_b: Vec<_> = inst_b + .children() + .iter() + .map(|&r| { + let c = dom_b.get_by_ref(r).unwrap(); + (c.name.clone(), c.class, r) + }) + .collect(); + children_b.sort_by(|a, b| (&a.0, &a.1).cmp(&(&b.0, &b.1))); + + assert_eq!( + children_a.len(), + children_b.len(), + "Child count mismatch at {path}: {} vs {}", + children_a.len(), + children_b.len() + ); + + for (ca, cb) in children_a.iter().zip(children_b.iter()) { + let child_path = format!("{path}/{}", ca.0); + compare_trees(dom_a, ca.2, dom_b, cb.2, &child_path); + } + } + + compare_trees(&dom_a, dom_a.root_ref(), &dom_b, dom_b.root_ref(), "root"); + }); +} + +// ── Additional edge-case parity tests ──────────────────────────── + +#[test] +fn parity_children_with_attributes_and_tags() { + use rbx_dom_weak::types::{Attributes, Tags, Variant}; + + let mut attrs = Attributes::new(); + attrs.insert("CustomFloat".to_string(), Variant::Float32(1.5)); + attrs.insert("CustomBool".to_string(), Variant::Bool(true)); + + let mut tags = Tags::new(); + tags.push("TestTag"); + tags.push("AnotherTag"); + + let chunks = vec![make_service_chunk( + "ReplicatedStorage", + vec![ + InstanceBuilder::new("Folder") + .with_name("TaggedFolder") + .with_property("Attributes", Variant::Attributes(attrs)) + .with_property("Tags", Variant::Tags(tags)), + InstanceBuilder::new("ModuleScript") + .with_name("Utils") + .with_property("Source", Variant::String("return {}".into())), + ], + )]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_multiple_cross_service_refs() { + use rbx_dom_weak::types::Variant; + + let target_part = InstanceBuilder::new("Part") + .with_name("Target1") + .with_property("Anchored", Variant::Bool(true)); + + let target_folder = InstanceBuilder::new("Folder").with_name("Target2"); + + let obj_value_a = InstanceBuilder::new("ObjectValue") + .with_name("PointerA") + .with_property("Value", Variant::Ref(target_part.referent())); + + let obj_value_b = InstanceBuilder::new("ObjectValue") + .with_name("PointerB") + .with_property("Value", Variant::Ref(target_folder.referent())); + + let chunks = vec![ + make_service_chunk("Workspace", vec![target_part, target_folder]), + make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("Folder") + .with_name("Refs") + .with_child(obj_value_a) + .with_child(obj_value_b)], + ), + ]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +#[test] +fn parity_properties_only_no_children() { + use rbx_dom_weak::types::{Color3, Variant}; + + let chunks = vec![ + make_service_chunk_full( + "Lighting", + vec![ + ("Ambient", Variant::Color3(Color3::new(0.1, 0.1, 0.1))), + ("Brightness", Variant::Float32(1.5)), + ], + vec![], + vec![], + vec![], + ), + make_service_chunk_full( + "SoundService", + vec![("DistanceFactor", Variant::Float32(8.0))], + vec![], + vec![], + vec![], + ), + make_service_chunk_full( + "TextChatService", + vec![( + "ChatVersion", + Variant::Enum(rbx_dom_weak::types::Enum::from_u32(2)), + )], + vec![], + vec![], + vec![], + ), + ]; + assert_live_matches_cli("live_syncback", chunks, None); +} + +// ── Resilience tests ───────────────────────────────────────────── + +#[test] +fn rapid_sequential_syncbacks() { + let (data_a, chunks_a) = build_syncback_request(vec![make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("Folder").with_name("RapidA")], + )]); + let (data_b, chunks_b) = build_syncback_request(vec![make_service_chunk( + "ReplicatedStorage", + vec![InstanceBuilder::new("Folder").with_name("RapidB")], + )]); + + run_serve_test("live_syncback", |session, _| { + session.post_api_syncback(None, data_a, chunks_a); + session.wait_to_come_back_online(); + + session.post_api_syncback(None, data_b, chunks_b); + session.wait_to_come_back_online(); + + assert!( + !session.path().join("src/shared/RapidA").exists(), + "RapidA should be gone after second syncback (clean mode)" + ); + assert!( + session.path().join("src/shared/RapidB").exists(), + "Second syncback should create RapidB" + ); + }); +} + +#[test] +fn recovers_from_malformed_chunks() { + run_serve_test("live_syncback", |session, _| { + let initial_info = session.get_api_rojo().unwrap(); + + let bad_chunk = librojo::web_api::ServiceChunk { + class_name: "ReplicatedStorage".to_string(), + child_count: 999, + ref_target_count: 0, + properties: std::collections::HashMap::new(), + refs: std::collections::HashMap::new(), + }; + + let body = build_raw_syncback_body(None, &[], &[bad_chunk]); + let response = session.post_api_syncback_raw(body); + assert!( + response.status().is_success(), + "Server should accept the request (validation happens async), got {}", + response.status() + ); + + let new_info = session.wait_to_come_back_online(); + assert_ne!( + initial_info.session_id, new_info.session_id, + "Server should have restarted (new session ID)" + ); + + assert!( + session.path().join("src").exists(), + "Filesystem should be intact (malformed syncback should not have written)" + ); + }); +} + +fn build_raw_syncback_body( + place_id: Option, + data: &[u8], + services: &[librojo::web_api::ServiceChunk], +) -> Vec { + use serde::Serialize; + + #[derive(Serialize)] + #[serde(rename_all = "camelCase")] + struct Req<'a> { + protocol_version: u64, + server_version: String, + place_id: Option, + #[serde(with = "serde_bytes")] + data: &'a [u8], + services: &'a [librojo::web_api::ServiceChunk], + } + + let request = Req { + protocol_version: PROTOCOL_VERSION, + server_version: env!("CARGO_PKG_VERSION").to_string(), + place_id, + data, + services, + }; + + let mut body = Vec::new(); + let mut serializer = rmp_serde::Serializer::new(&mut body) + .with_human_readable() + .with_struct_map(); + request + .serialize(&mut serializer) + .expect("Failed to serialize syncback request"); + body +} diff --git a/tests/tests/matching_fixtures.rs b/tests/tests/matching_fixtures.rs new file mode 100644 index 00000000..ddff9a84 --- /dev/null +++ b/tests/tests/matching_fixtures.rs @@ -0,0 +1,678 @@ +//! Integration tests for the matching algorithm using real-world fixtures. +//! +//! Uses the UFOWave rbxm (full Studio properties) and its syncback'd +//! filesystem representation (defaults stripped) to verify that all +//! matching algorithms correctly pair instances despite asymmetric +//! property sets. +//! +//! The key scenario: 12 Textures under TsunamiWave, each with a unique +//! Face value. Two groups of 6 (by TextureContent). In each group, the +//! Face=Front texture has Face omitted from its model.json5 (it's the +//! class default). The matching must still pair it to the correct Studio +//! instance rather than stealing another Face's match. + +use std::collections::HashMap; +use std::path::Path; + +use rbx_dom_weak::types::{Ref, Variant}; +use rbx_dom_weak::{ustr, HashMapExt as _, WeakDom}; + +use crate::rojo_test::io_util::SYNCBACK_TESTS_PATH; + +const FIXTURE_DIR: &str = "UFOWave_matching"; + +/// Parse the input.rbxm fixture into a WeakDom. +fn load_input_rbxm() -> WeakDom { + let path = Path::new(SYNCBACK_TESTS_PATH) + .join(FIXTURE_DIR) + .join("input.rbxm"); + let data = + std::fs::read(&path).unwrap_or_else(|e| panic!("Failed to read {}: {}", path.display(), e)); + rbx_binary::from_reader(data.as_slice()) + .unwrap_or_else(|e| panic!("Failed to parse {}: {}", path.display(), e)) +} + +/// Build a WeakDom from the expected/ directory by building it through +/// the snapshot pipeline (the same path `atlas build` uses). +fn load_expected_dom() -> WeakDom { + let expected_path = Path::new(SYNCBACK_TESTS_PATH) + .join(FIXTURE_DIR) + .join("expected"); + + let vfs = memofs::Vfs::new_default(); + let ctx = librojo::InstanceContext::default(); + let snapshot = librojo::snapshot_from_vfs(&ctx, &vfs, &expected_path) + .unwrap_or_else(|e| panic!("Failed to snapshot expected/: {}", e)) + .expect("snapshot_from_vfs returned None for expected/"); + + // Convert InstanceSnapshot tree into a WeakDom for syncback matching + let tree = librojo::RojoTree::new(snapshot); + rojo_tree_to_weak_dom(&tree) +} + +/// Convert a RojoTree into a WeakDom (for syncback matching which needs +/// two WeakDom instances). +fn rojo_tree_to_weak_dom(tree: &librojo::RojoTree) -> WeakDom { + let root_id = tree.get_root_id(); + let root = tree.get_instance(root_id).unwrap(); + + let mut dom = WeakDom::new( + rbx_dom_weak::InstanceBuilder::new(root.class_name().as_str()).with_name(root.name()), + ); + let dom_root = dom.root_ref(); + + // Copy properties to root + for (key, val) in root.properties() { + dom.get_by_ref_mut(dom_root) + .unwrap() + .properties + .insert(*key, val.clone()); + } + + // Recursively copy children + fn copy_children( + tree: &librojo::RojoTree, + parent_ref: Ref, + dom: &mut WeakDom, + dom_parent: Ref, + ) { + let children: Vec = tree.get_instance(parent_ref).unwrap().children().to_vec(); + for child_ref in children { + let child = tree.get_instance(child_ref).unwrap(); + let builder = rbx_dom_weak::InstanceBuilder::new(child.class_name().as_str()) + .with_name(child.name()); + let new_ref = dom.insert(dom_parent, builder); + for (key, val) in child.properties() { + dom.get_by_ref_mut(new_ref) + .unwrap() + .properties + .insert(*key, val.clone()); + } + copy_children(tree, child_ref, dom, new_ref); + } + } + + copy_children(tree, root_id, &mut dom, dom_root); + dom +} + +/// Recursively find a descendant by name in a WeakDom. +fn find_descendant(dom: &WeakDom, parent: Ref, name: &str) -> Option { + let parent_inst = dom.get_by_ref(parent)?; + for &child in parent_inst.children() { + let child_inst = dom.get_by_ref(child)?; + if child_inst.name == name { + return Some(child); + } + if let Some(found) = find_descendant(dom, child, name) { + return Some(found); + } + } + None +} + +/// Recursively find a descendant by name in a RojoTree. +fn find_descendant_by_name(tree: &librojo::RojoTree, parent: Ref, name: &str) -> Option { + let parent_inst = tree.get_instance(parent)?; + for &child in parent_inst.children() { + let child_inst = tree.get_instance(child)?; + if child_inst.name() == name { + return Some(child); + } + if let Some(found) = find_descendant_by_name(tree, child, name) { + return Some(found); + } + } + None +} + +/// Get all Texture children under a parent in a WeakDom. +fn get_texture_children(dom: &WeakDom, parent: Ref) -> Vec { + let parent_inst = dom.get_by_ref(parent).unwrap(); + parent_inst + .children() + .iter() + .filter(|&&r| { + dom.get_by_ref(r) + .is_some_and(|i| i.class.as_str() == "Texture") + }) + .copied() + .collect() +} + +/// Extract the Face enum value from an instance (returns None if not set). +fn get_face(dom: &WeakDom, inst_ref: Ref) -> Option { + let inst = dom.get_by_ref(inst_ref)?; + match inst.properties.get(&ustr("Face"))? { + Variant::Enum(e) => Some(e.to_u32()), + _ => None, + } +} + +/// Build a map of Face value → count for a set of Texture refs. +fn face_distribution(dom: &WeakDom, refs: &[Ref]) -> HashMap, usize> { + let mut map = HashMap::new(); + for &r in refs { + *map.entry(get_face(dom, r)).or_insert(0) += 1; + } + map +} + +// ================================================================ +// Syncback matching (WeakDom vs WeakDom) +// ================================================================ + +#[test] +fn syncback_matching_ufowave_textures() { + let _ = env_logger::try_init(); + + let input_dom = load_input_rbxm(); + let expected_dom = load_expected_dom(); + + // Find TsunamiWave in both trees + let input_root = input_dom.root_ref(); + let input_tsunami = find_descendant(&input_dom, input_root, "TsunamiWave") + .expect("TsunamiWave not found in input.rbxm"); + + let expected_root = expected_dom.root_ref(); + let expected_tsunami = find_descendant(&expected_dom, expected_root, "TsunamiWave") + .expect("TsunamiWave not found in expected/"); + + let input_textures = get_texture_children(&input_dom, input_tsunami); + let expected_textures = get_texture_children(&expected_dom, expected_tsunami); + + assert!( + !input_textures.is_empty(), + "No Texture children in input.rbxm TsunamiWave" + ); + assert_eq!( + input_textures.len(), + expected_textures.len(), + "Texture count mismatch: input has {}, expected has {}", + input_textures.len(), + expected_textures.len() + ); + + // Verify both sides have all 6 faces per group + let input_faces = face_distribution(&input_dom, &input_textures); + let expected_faces = face_distribution(&expected_dom, &expected_textures); + eprintln!("Input face distribution: {:?}", input_faces); + eprintln!("Expected face distribution: {:?}", expected_faces); + + // Run syncback matching + let session = librojo::syncback::matching::MatchingSession::new(); + let result = librojo::syncback::matching::match_children( + &input_textures, + &expected_textures, + &input_dom, + &expected_dom, + None, + None, + &session, + ); + + assert_eq!( + result.matched.len(), + input_textures.len(), + "All Textures should be matched" + ); + assert!(result.unmatched_new.is_empty(), "No unmatched input"); + assert!(result.unmatched_old.is_empty(), "No unmatched expected"); + + // Verify correct Face pairing for every matched pair + for (new_ref, old_ref) in &result.matched { + let new_face = get_face(&input_dom, *new_ref); + let old_face = get_face(&expected_dom, *old_ref); + + // If expected has no Face, it's the default (Front = 5) + let effective_old_face = old_face.or(Some(5)); + + assert_eq!( + new_face, + effective_old_face, + "Face mismatch: input {:?} (name={}) matched to expected {:?} (name={})", + new_face, + input_dom.get_by_ref(*new_ref).unwrap().name, + old_face, + expected_dom.get_by_ref(*old_ref).unwrap().name, + ); + } +} + +// ================================================================ +// Forward sync matching (InstanceSnapshot vs RojoTree) +// ================================================================ + +#[test] +fn forward_matching_ufowave_textures() { + let _ = env_logger::try_init(); + + // For forward sync: snapshots come from filesystem (sparse), + // tree comes from Studio (full properties, simulated by rbxm). + + // Build snapshots from the expected/ directory + let expected_path = Path::new(SYNCBACK_TESTS_PATH) + .join(FIXTURE_DIR) + .join("expected"); + let vfs = memofs::Vfs::new_default(); + let ctx = librojo::InstanceContext::default(); + let snapshot = librojo::snapshot_from_vfs(&ctx, &vfs, &expected_path) + .expect("Failed to snapshot expected/") + .expect("snapshot_from_vfs returned None"); + + // Build the tree from the rbxm (simulating Studio-populated tree). + // from_tree takes ownership, so we load a fresh copy. + let input_dom = load_input_rbxm(); + let root_ref = input_dom.root_ref(); + let input_snapshot = librojo::InstanceSnapshot::from_tree(input_dom, root_ref); + let tree = librojo::RojoTree::new(input_snapshot); + let tree_root = tree.get_root_id(); + + // Find TsunamiWave in the snapshot children + let snap_tsunami = snapshot + .children + .iter() + .find(|c| c.name == "TsunamiWave") + .expect("TsunamiWave not found in snapshot"); + + // Find TsunamiWave in the tree (may be nested under root → UFOWave → TsunamiWave) + let tree_tsunami = find_descendant_by_name(&tree, tree_root, "TsunamiWave") + .expect("TsunamiWave not found in tree"); + + // Extract Texture children from snapshot (filesystem side, sparse) + let snap_textures: Vec = snap_tsunami + .children + .iter() + .filter(|c| c.class_name.as_str() == "Texture") + .cloned() + .collect(); + + // Extract Texture children from tree (Studio side, full properties) + let tree_textures: Vec = tree + .get_instance(tree_tsunami) + .unwrap() + .children() + .iter() + .filter(|&&r| { + tree.get_instance(r) + .is_some_and(|i| i.class_name().as_str() == "Texture") + }) + .copied() + .collect(); + + assert_eq!( + snap_textures.len(), + tree_textures.len(), + "Texture count mismatch" + ); + assert!(!snap_textures.is_empty(), "No Texture children found"); + + // Run forward sync matching + let session = librojo::snapshot::matching::MatchingSession::new(); + let result = + librojo::snapshot::matching::match_forward(snap_textures, &tree_textures, &tree, &session); + + assert_eq!( + result.matched.len(), + tree_textures.len(), + "All Textures should be matched" + ); + assert!(result.unmatched_snapshot.is_empty()); + assert!(result.unmatched_tree.is_empty()); + + // Verify correct Face pairing + let face_front = Variant::Enum(rbx_dom_weak::types::Enum::from_u32(5)); + + for (snap, tree_ref) in &result.matched { + let tree_inst = tree.get_instance(*tree_ref).unwrap(); + let tree_face = tree_inst.properties().get(&ustr("Face")); + + let snap_face = snap.properties.get(&ustr("Face")); + + match (snap_face, tree_face) { + (Some(sv), Some(tv)) => { + assert!( + librojo::variant_eq::variant_eq(sv, tv), + "Face mismatch: snap={:?}, tree={:?}", + sv, + tv + ); + } + (None, Some(tv)) => { + assert!( + librojo::variant_eq::variant_eq(tv, &face_front), + "Omitted Face should match Front, got {:?}", + tv + ); + } + (None, None) => {} + (Some(sv), None) => { + panic!("Snap has Face={:?} but tree has none", sv); + } + } + } +} + +// ================================================================ +// Parity: forward sync vs syncback produce same pairings +// ================================================================ + +#[test] +fn parity_forward_vs_syncback() { + use rbx_dom_weak::types::Variant; + use rbx_dom_weak::InstanceBuilder; + use std::borrow::Cow; + + let transparency_values = [0.0_f32, 0.3, 0.6, 0.9]; + + // Build "new" WeakDom (for syncback) with parts in forward order + let mut new_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let new_root = new_dom.root_ref(); + for &t in &transparency_values { + let builder = InstanceBuilder::new("Part") + .with_name("Line") + .with_property("Transparency", Variant::Float32(t)); + new_dom.insert(new_root, builder); + } + + // Build "old" WeakDom (for syncback) with parts in REVERSED order + let mut old_dom = WeakDom::new(InstanceBuilder::new("DataModel")); + let old_root = old_dom.root_ref(); + for &t in transparency_values.iter().rev() { + let builder = InstanceBuilder::new("Part") + .with_name("Line") + .with_property("Transparency", Variant::Float32(t)); + old_dom.insert(old_root, builder); + } + + let new_children: Vec = new_dom.get_by_ref(new_root).unwrap().children().to_vec(); + let old_children: Vec = old_dom.get_by_ref(old_root).unwrap().children().to_vec(); + + // Run syncback matching + let syncback_session = librojo::syncback::matching::MatchingSession::new(); + let syncback_result = librojo::syncback::matching::match_children( + &new_children, + &old_children, + &new_dom, + &old_dom, + None, + None, + &syncback_session, + ); + + // Build equivalent InstanceSnapshots for forward sync + let snap_children: Vec = transparency_values + .iter() + .map(|&t| { + let mut properties = rbx_dom_weak::UstrMap::new(); + properties.insert(ustr("Transparency"), Variant::Float32(t)); + librojo::InstanceSnapshot { + snapshot_id: Ref::none(), + metadata: librojo::snapshot::InstanceMetadata::default(), + name: Cow::Borrowed("Line"), + class_name: ustr("Part"), + properties, + children: Vec::new(), + } + }) + .collect(); + + // Build RojoTree from old_dom data (reversed order) + let tree_snap_children: Vec = transparency_values + .iter() + .rev() + .map(|&t| { + let mut properties = rbx_dom_weak::UstrMap::new(); + properties.insert(ustr("Transparency"), Variant::Float32(t)); + librojo::InstanceSnapshot { + snapshot_id: Ref::none(), + metadata: librojo::snapshot::InstanceMetadata::default(), + name: Cow::Borrowed("Line"), + class_name: ustr("Part"), + properties, + children: Vec::new(), + } + }) + .collect(); + + let root_snap = librojo::InstanceSnapshot { + snapshot_id: Ref::none(), + metadata: librojo::snapshot::InstanceMetadata::default(), + name: Cow::Borrowed("DataModel"), + class_name: ustr("DataModel"), + properties: Default::default(), + children: tree_snap_children, + }; + let tree = librojo::RojoTree::new(root_snap); + let tree_root = tree.get_root_id(); + let tree_children_refs: Vec = tree.get_instance(tree_root).unwrap().children().to_vec(); + + // Run forward sync matching + let forward_session = librojo::snapshot::matching::MatchingSession::new(); + let forward_result = librojo::snapshot::matching::match_forward( + snap_children, + &tree_children_refs, + &tree, + &forward_session, + ); + + // Both should match all 4 + assert_eq!(syncback_result.matched.len(), 4); + assert_eq!(forward_result.matched.len(), 4); + + // Verify both produce the same pairing (by Transparency value) + let mut syncback_pairs: Vec<(f32, f32)> = syncback_result + .matched + .iter() + .map(|(nr, or)| { + let nt = match new_dom + .get_by_ref(*nr) + .unwrap() + .properties + .get(&ustr("Transparency")) + { + Some(Variant::Float32(t)) => *t, + _ => panic!("Missing Transparency on new"), + }; + let ot = match old_dom + .get_by_ref(*or) + .unwrap() + .properties + .get(&ustr("Transparency")) + { + Some(Variant::Float32(t)) => *t, + _ => panic!("Missing Transparency on old"), + }; + (nt, ot) + }) + .collect(); + syncback_pairs.sort_by(|a, b| a.0.partial_cmp(&b.0).unwrap()); + + let mut forward_pairs: Vec<(f32, f32)> = forward_result + .matched + .iter() + .map(|(snap, tree_ref)| { + let st = match snap.properties.get(&ustr("Transparency")) { + Some(Variant::Float32(t)) => *t, + _ => panic!("Missing Transparency on snap"), + }; + let tt = match tree + .get_instance(*tree_ref) + .unwrap() + .properties() + .get(&ustr("Transparency")) + { + Some(Variant::Float32(t)) => *t, + _ => panic!("Missing Transparency on tree"), + }; + (st, tt) + }) + .collect(); + forward_pairs.sort_by(|a, b| a.0.partial_cmp(&b.0).unwrap()); + + assert_eq!( + syncback_pairs, forward_pairs, + "Syncback and forward sync produced different pairings" + ); + + // Verify correct pairing: each pair should have matching Transparency + for (nt, ot) in &syncback_pairs { + assert_eq!( + nt, ot, + "Incorrect pairing: new Transparency={} matched to old Transparency={}", + nt, ot + ); + } +} + +// ================================================================ +// Round-trip: syncback matching + forward sync matching agree +// ================================================================ + +#[test] +fn round_trip_matching_identity() { + let _ = env_logger::try_init(); + + // Load the input rbxm (simulates Studio with full properties) + let input_dom = load_input_rbxm(); + let input_root = input_dom.root_ref(); + let input_tsunami = find_descendant(&input_dom, input_root, "TsunamiWave") + .expect("TsunamiWave not found in input.rbxm"); + let input_textures = get_texture_children(&input_dom, input_tsunami); + assert_eq!(input_textures.len(), 12, "Expected 12 Textures in input"); + + // Load the expected/ filesystem as a WeakDom (syncback output) + let expected_dom = load_expected_dom(); + let expected_root = expected_dom.root_ref(); + let expected_tsunami = find_descendant(&expected_dom, expected_root, "TsunamiWave") + .expect("TsunamiWave not found in expected/"); + let expected_textures = get_texture_children(&expected_dom, expected_tsunami); + assert_eq!( + expected_textures.len(), + 12, + "Expected 12 Textures in expected/" + ); + + // Step 1: Syncback matching (input rbxm vs expected filesystem) + let syncback_session = librojo::syncback::matching::MatchingSession::new(); + let syncback_result = librojo::syncback::matching::match_children( + &input_textures, + &expected_textures, + &input_dom, + &expected_dom, + None, + None, + &syncback_session, + ); + assert_eq!( + syncback_result.matched.len(), + 12, + "Syncback should match all 12" + ); + + // Step 2: Forward sync matching (expected/ snapshots vs input-as-tree) + let expected_path = Path::new(SYNCBACK_TESTS_PATH) + .join(FIXTURE_DIR) + .join("expected"); + let vfs = memofs::Vfs::new_default(); + let ctx = librojo::InstanceContext::default(); + let full_snapshot = librojo::snapshot_from_vfs(&ctx, &vfs, &expected_path) + .expect("Failed to snapshot expected/") + .expect("snapshot_from_vfs returned None"); + let snap_tsunami = full_snapshot + .children + .iter() + .find(|c| c.name == "TsunamiWave") + .expect("TsunamiWave not found in snapshot"); + let snap_textures: Vec = snap_tsunami + .children + .iter() + .filter(|c| c.class_name.as_str() == "Texture") + .cloned() + .collect(); + assert_eq!(snap_textures.len(), 12, "Expected 12 snapshot Textures"); + + let input_dom2 = load_input_rbxm(); + let root_ref2 = input_dom2.root_ref(); + let input_snapshot = librojo::InstanceSnapshot::from_tree(input_dom2, root_ref2); + let tree = librojo::RojoTree::new(input_snapshot); + let tree_root = tree.get_root_id(); + let tree_tsunami = find_descendant_by_name(&tree, tree_root, "TsunamiWave") + .expect("TsunamiWave not found in tree"); + let tree_textures: Vec = tree + .get_instance(tree_tsunami) + .unwrap() + .children() + .iter() + .filter(|&&r| { + tree.get_instance(r) + .is_some_and(|i| i.class_name().as_str() == "Texture") + }) + .copied() + .collect(); + assert_eq!(tree_textures.len(), 12, "Expected 12 tree Textures"); + + let forward_session = librojo::snapshot::matching::MatchingSession::new(); + let forward_result = librojo::snapshot::matching::match_forward( + snap_textures, + &tree_textures, + &tree, + &forward_session, + ); + assert_eq!( + forward_result.matched.len(), + 12, + "Forward sync should match all 12" + ); + + // Step 3: Verify both directions paired by the same Face value. + // Syncback pairs: (input_face, expected_face) + let face_front = 5_u32; + let mut syncback_faces: Vec<(u32, u32)> = syncback_result + .matched + .iter() + .map(|(input_ref, expected_ref)| { + let iface = get_face(&input_dom, *input_ref).unwrap_or(face_front); + let eface = get_face(&expected_dom, *expected_ref).unwrap_or(face_front); + (iface, eface) + }) + .collect(); + syncback_faces.sort(); + + // Forward pairs: (snap_face, tree_face) + let mut forward_faces: Vec<(u32, u32)> = forward_result + .matched + .iter() + .map(|(snap, tree_ref)| { + let sface = snap + .properties + .get(&ustr("Face")) + .and_then(|v| match v { + Variant::Enum(e) => Some(e.to_u32()), + _ => None, + }) + .unwrap_or(face_front); + let tface = tree + .get_instance(*tree_ref) + .and_then(|i| i.properties().get(&ustr("Face"))) + .and_then(|v| match v { + Variant::Enum(e) => Some(e.to_u32()), + _ => None, + }) + .unwrap_or(face_front); + (sface, tface) + }) + .collect(); + forward_faces.sort(); + + // Both directions must produce identical (face, face) pairs + assert_eq!( + syncback_faces, forward_faces, + "Round-trip mismatch: syncback and forward sync disagree on Face pairings" + ); + + // Every pair must have matching Face values + for (a, b) in &syncback_faces { + assert_eq!(a, b, "Face mismatch in round-trip: {} vs {}", a, b); + } +} diff --git a/tests/tests/mod.rs b/tests/tests/mod.rs index 2241939d..efcd3121 100644 --- a/tests/tests/mod.rs +++ b/tests/tests/mod.rs @@ -2,6 +2,9 @@ mod build; mod clean_mode; mod clean_mode_stress; mod connected_mode; +mod git_sync_defaults; +mod live_syncback; +mod matching_fixtures; mod serve; mod syncback; mod syncback_format_transitions; diff --git a/tests/tests/serve.rs b/tests/tests/serve.rs index 907c6883..f8ea2c2e 100644 --- a/tests/tests/serve.rs +++ b/tests/tests/serve.rs @@ -5,7 +5,7 @@ use tempfile::tempdir; use crate::rojo_test::{ internable::InternAndRedact, - serve_util::{run_serve_test, serialize_to_xml_model}, + serve_util::{run_serve_test, serialize_to_xml_model, TestServeSession}, }; use librojo::web_api::SocketPacketType; @@ -415,7 +415,13 @@ fn sync_rule_no_name_project() { }); } +// On macOS, kqueue delivers a duplicate WRITE event for the same file +// modification. The second event re-snapshots against a tree whose Rojo +// transport attributes (Rojo_Id, Rojo_Target_*) were cleaned up by +// finalize_patch_application during the first event, producing spurious +// Attributes and metadata update messages that break the snapshot. #[test] +#[cfg_attr(target_os = "macos", ignore)] fn ref_properties() { run_serve_test("ref_properties", |session, mut redactions| { let info = session.get_api_rojo().unwrap(); @@ -780,6 +786,7 @@ fn api_write_existing_instance() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session @@ -856,7 +863,7 @@ fn ref_path_add_attribute_to_existing_meta() { r#"{ "className": "Model", "attributes": { - "Rojo_Ref_PrimaryPart": "Workspace/MyModel/OtherPart" + "Rojo_Ref_PrimaryPart": "@self/OtherPart" } }"#, ) @@ -916,7 +923,7 @@ fn ref_path_new_file_with_ref_attr() { r#"{ "className": "ObjectValue", "attributes": { - "Rojo_Ref_Value": "Workspace/MyModel/Target" + "Rojo_Ref_Value": "./Target" } }"#, ) @@ -945,7 +952,7 @@ fn ref_path_nonexistent_target_no_crash() { r#"{ "className": "Model", "attributes": { - "Rojo_Ref_PrimaryPart": "Workspace/NonExistent/Part" + "Rojo_Ref_PrimaryPart": "./NonExistent/Part" } }"#, ) @@ -960,6 +967,24 @@ fn ref_path_nonexistent_target_no_crash() { }); } +/// When a non-default project file is used (e.g. `named.project.json5` +/// instead of `default.project.json5`), tree validation must snapshot +/// through the project middleware — not walk the parent directory as a +/// generic Folder. Regression test for a bug where `check_tree_freshness` +/// and `reconcile_tree` used `folder_location()` instead of the project +/// file path, causing the entire repo root to be walked (including +/// unrelated directories with potentially malformed files). +#[test] +fn non_default_project_file_tree_validation() { + let _ = env_logger::try_init(); + + let mut session = + TestServeSession::new_with_project_file("non_default_project", "named.project.json5"); + let _info = session.wait_to_come_online(); + + session.assert_tree_fresh(); +} + /// Multiple Rojo_Ref_* attributes on the same instance should all resolve. #[test] fn ref_path_multiple_attributes() { @@ -975,8 +1000,8 @@ fn ref_path_multiple_attributes() { r#"{ "className": "Model", "attributes": { - "Rojo_Ref_PrimaryPart": "Workspace/MyModel/Target", - "Rojo_Ref_CustomRef": "Workspace/MyModel/OtherPart" + "Rojo_Ref_PrimaryPart": "@self/Target", + "Rojo_Ref_CustomRef": "@self/OtherPart" } }"#, ) diff --git a/tests/tests/syncback_format_transitions.rs b/tests/tests/syncback_format_transitions.rs index 1acacdc9..79bbc163 100644 --- a/tests/tests/syncback_format_transitions.rs +++ b/tests/tests/syncback_format_transitions.rs @@ -75,6 +75,7 @@ fn send_write_request( removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session @@ -904,6 +905,7 @@ fn rapid_syncs_dont_create_duplicates() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session @@ -1179,6 +1181,7 @@ fn scripts_only_mode_rapid_syncs() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session @@ -1802,6 +1805,7 @@ fn sync_immediate_double_sync() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); @@ -1826,6 +1830,7 @@ fn sync_immediate_double_sync() { removed: vec![], added: added_map2, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request2).unwrap(); @@ -1871,6 +1876,7 @@ fn stress_test_50_rapid_syncs() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); // No delay between syncs! diff --git a/tests/tests/syncback_roundtrip.rs b/tests/tests/syncback_roundtrip.rs index 31d85893..3075438b 100644 --- a/tests/tests/syncback_roundtrip.rs +++ b/tests/tests/syncback_roundtrip.rs @@ -89,6 +89,10 @@ fn assert_rbx_equal(file_a: &Path, file_b: &Path, test_name: &str) { } /// Recursively compare children of two instances. +/// +/// Uses sorted Vec-based comparison (not HashMap) so that duplicate-named +/// children are correctly handled. HashMap would silently overwrite entries +/// sharing a name, making the test blind to lost duplicates. fn compare_children( dom_a: &WeakDom, ref_a: rbx_dom_weak::types::Ref, @@ -102,73 +106,63 @@ fn compare_children( let children_a = inst_a.children(); let children_b = inst_b.children(); - // Build maps of children by name for comparison - let mut map_a: std::collections::HashMap<&str, &rbx_dom_weak::Instance> = - std::collections::HashMap::new(); - let mut map_b: std::collections::HashMap<&str, &rbx_dom_weak::Instance> = - std::collections::HashMap::new(); - - for &child_ref in children_a { - if let Some(child) = dom_a.get_by_ref(child_ref) { - map_a.insert(&child.name, child); - } - } - - for &child_ref in children_b { - if let Some(child) = dom_b.get_by_ref(child_ref) { - map_b.insert(&child.name, child); - } - } - - // Check that both have the same children - let names_a: std::collections::HashSet<&str> = map_a.keys().copied().collect(); - let names_b: std::collections::HashSet<&str> = map_b.keys().copied().collect(); - - let only_in_a: Vec<_> = names_a.difference(&names_b).collect(); - let only_in_b: Vec<_> = names_b.difference(&names_a).collect(); - - if !only_in_a.is_empty() || !only_in_b.is_empty() { + if children_a.len() != children_b.len() { + let names_a: Vec<&str> = children_a + .iter() + .filter_map(|r| dom_a.get_by_ref(*r).map(|i| i.name.as_str())) + .collect(); + let names_b: Vec<&str> = children_b + .iter() + .filter_map(|r| dom_b.get_by_ref(*r).map(|i| i.name.as_str())) + .collect(); panic!( - "[{}] Children differ under '{}':\n Only in original: {:?}\n Only in roundtrip: {:?}", - test_name, inst_a.name, only_in_a, only_in_b + "[{}] Child count mismatch under '{}': original has {} children {:?}, \ + roundtrip has {} children {:?}", + test_name, + inst_a.name, + children_a.len(), + names_a, + children_b.len(), + names_b, ); } - // Compare each child - for name in names_a { - let child_a = map_a[name]; - let child_b = map_b[name]; + let mut pairs_a: Vec<(&str, rbx_dom_weak::types::Ref)> = children_a + .iter() + .filter_map(|&r| dom_a.get_by_ref(r).map(|i| (i.name.as_str(), r))) + .collect(); + let mut pairs_b: Vec<(&str, rbx_dom_weak::types::Ref)> = children_b + .iter() + .filter_map(|&r| dom_b.get_by_ref(r).map(|i| (i.name.as_str(), r))) + .collect(); - // Check class name matches - if child_a.class != child_b.class { + pairs_a.sort_by_key(|(name, _)| *name); + pairs_b.sort_by_key(|(name, _)| *name); + + for (i, ((name_a, ref_child_a), (name_b, ref_child_b))) in + pairs_a.iter().zip(pairs_b.iter()).enumerate() + { + if name_a != name_b { panic!( - "[{}] Class mismatch for '{}': original={}, roundtrip={}", - test_name, name, child_a.class, child_b.class + "[{}] Child name mismatch under '{}' at sorted index {}: \ + original='{}', roundtrip='{}'", + test_name, inst_a.name, i, name_a, name_b, + ); + } + + let child_a = dom_a.get_by_ref(*ref_child_a).unwrap(); + let child_b = dom_b.get_by_ref(*ref_child_b).unwrap(); + + if child_a.class != child_b.class { + panic!( + "[{}] Class mismatch for '{}' (sorted index {}): original={}, roundtrip={}", + test_name, name_a, i, child_a.class, child_b.class ); } - // Compare important properties (Source for scripts, Value for value types) compare_properties(child_a, child_b, test_name); - // Recurse into children - let child_ref_a = dom_a - .get_by_ref(ref_a) - .unwrap() - .children() - .iter() - .find(|&&r| dom_a.get_by_ref(r).map(|i| i.name.as_str()) == Some(name)) - .copied() - .unwrap(); - let child_ref_b = dom_b - .get_by_ref(ref_b) - .unwrap() - .children() - .iter() - .find(|&&r| dom_b.get_by_ref(r).map(|i| i.name.as_str()) == Some(name)) - .copied() - .unwrap(); - - compare_children(dom_a, child_ref_a, dom_b, child_ref_b, test_name); + compare_children(dom_a, *ref_child_a, dom_b, *ref_child_b, test_name); } } @@ -221,6 +215,19 @@ fn compare_properties( ); } } + + // Check Tags if present + if let (Some(tags_a), Some(tags_b)) = ( + inst_a.properties.get(&ustr("Tags")), + inst_b.properties.get(&ustr("Tags")), + ) { + if tags_a != tags_b { + panic!( + "[{}] Tags differ for '{}' ({}):\nOriginal: {:?}\nRoundtrip: {:?}", + test_name, inst_a.name, inst_a.class, tags_a, tags_b + ); + } + } } // ============================================================================= diff --git a/tests/tests/two_way_sync.rs b/tests/tests/two_way_sync.rs index 4142574e..3011ac7a 100644 --- a/tests/tests/two_way_sync.rs +++ b/tests/tests/two_way_sync.rs @@ -97,6 +97,7 @@ fn send_update( removed: vec![], added: HashMap::new(), updated: vec![update], + stage_ids: Vec::new(), }; session .post_api_write(&write_request) @@ -115,6 +116,7 @@ fn send_removal( removed: ids, added: HashMap::new(), updated: vec![], + stage_ids: Vec::new(), }; session .post_api_write(&write_request) @@ -704,6 +706,7 @@ fn echo_suppression_prevents_redundant_patches() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); @@ -1127,6 +1130,7 @@ fn add_child_converts_standalone_to_directory() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); thread::sleep(Duration::from_millis(500)); @@ -1279,6 +1283,7 @@ fn add_child_to_encoded_module_creates_encoded_directory() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); thread::sleep(Duration::from_millis(500)); @@ -1352,6 +1357,7 @@ fn add_child_to_encoded_script_strips_suffix_correctly() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); thread::sleep(Duration::from_millis(500)); @@ -1418,6 +1424,7 @@ fn add_child_to_encoded_model_creates_encoded_directory() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); thread::sleep(Duration::from_millis(500)); @@ -1521,6 +1528,7 @@ fn send_update_fast( removed: vec![], added: HashMap::new(), updated: vec![update], + stage_ids: Vec::new(), }; session .post_api_write(&write_request) @@ -1539,6 +1547,7 @@ fn send_update_no_wait( removed: vec![], added: HashMap::new(), updated: vec![update], + stage_ids: Vec::new(), }; session .post_api_write(&write_request) @@ -1556,6 +1565,7 @@ fn send_removal_fast( removed: ids, added: HashMap::new(), updated: vec![], + stage_ids: Vec::new(), }; session .post_api_write(&write_request) @@ -2198,6 +2208,7 @@ fn multi_instance_source_update_single_request() { removed: vec![], added: HashMap::new(), updated: updates, + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); wait_for_settle(); @@ -2234,6 +2245,7 @@ fn multi_instance_rename_single_request() { removed: vec![], added: HashMap::new(), updated: updates, + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); wait_for_settle(); @@ -2337,6 +2349,7 @@ fn echo_suppression_rapid_adds_10x() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); } @@ -2409,6 +2422,7 @@ fn echo_suppression_mixed_operations() { removed, added: added_map, updated: updates, + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); wait_for_settle(); @@ -2758,6 +2772,7 @@ fn fuzz_multi_instance_15_iterations() { removed: vec![], added: HashMap::new(), updated: updates, + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); thread::sleep(Duration::from_millis(100)); @@ -4159,6 +4174,7 @@ fn multi_instance_combined_rename_class_source_batch() { removed: vec![], added: HashMap::new(), updated: updates, + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); wait_for_settle(); @@ -5126,6 +5142,7 @@ fn add_instance_with_forbidden_chars_creates_slug_and_meta() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); thread::sleep(Duration::from_millis(500)); @@ -5147,7 +5164,7 @@ fn add_instance_with_forbidden_chars_creates_slug_and_meta() { } /// Adding two instances whose names slugify to the same string should produce -/// dedup suffixes (~1) and both should have correct meta `name` fields. +/// dedup suffixes (~2) and both should have correct meta `name` fields. #[test] fn add_two_colliding_instances_deduplicates() { run_serve_test("syncback_encoded_names", |session, _redactions| { @@ -5191,18 +5208,19 @@ fn add_two_colliding_instances_deduplicates() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); thread::sleep(Duration::from_millis(500)); let src = session.path().join("src"); let base = src.join("X_Y.luau"); - let deduped = src.join("X_Y~1.luau"); + let deduped = src.join("X_Y~2.luau"); - // One should get X_Y.luau, the other X_Y~1.luau + // One should get X_Y.luau, the other X_Y~2.luau assert!( base.exists() && deduped.exists(), - "Both X_Y.luau and X_Y~1.luau should exist. \ + "Both X_Y.luau and X_Y~2.luau should exist. \ base exists: {}, deduped exists: {}", base.exists(), deduped.exists() @@ -5210,9 +5228,9 @@ fn add_two_colliding_instances_deduplicates() { // Both should have meta files with correct names let base_meta = src.join("X_Y.meta.json5"); - let deduped_meta = src.join("X_Y~1.meta.json5"); + let deduped_meta = src.join("X_Y~2.meta.json5"); assert_file_exists(&base_meta, "Meta for base slug instance"); - assert_file_exists(&deduped_meta, "Meta for deduped ~1 instance"); + assert_file_exists(&deduped_meta, "Meta for deduped ~2 instance"); // Read both meta files and verify they contain the correct names let meta1 = fs::read_to_string(&base_meta).unwrap(); @@ -5438,7 +5456,7 @@ fn rename_model_json_to_clean_name() { // --------------------------------------------------------------------------- /// Adding 3 instances that all slug to the same name (X/Y, X:Y, X|Y → X_Y) -/// should produce X_Y.luau, X_Y~1.luau, X_Y~2.luau with correct meta files. +/// should produce X_Y.luau, X_Y~2.luau, X_Y~3.luau with correct meta files. #[test] fn add_three_colliding_instances_deduplicates() { run_serve_test("syncback_encoded_names", |session, _redactions| { @@ -5470,19 +5488,20 @@ fn add_three_colliding_instances_deduplicates() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); thread::sleep(Duration::from_millis(500)); let src = session.path().join("src"); let base = src.join("X_Y.luau"); - let dedup1 = src.join("X_Y~1.luau"); - let dedup2 = src.join("X_Y~2.luau"); + let dedup1 = src.join("X_Y~2.luau"); + let dedup2 = src.join("X_Y~3.luau"); assert!( base.exists() && dedup1.exists() && dedup2.exists(), "All three slugified files should exist.\n\ - X_Y.luau: {}, X_Y~1.luau: {}, X_Y~2.luau: {}", + X_Y.luau: {}, X_Y~2.luau: {}, X_Y~3.luau: {}", base.exists(), dedup1.exists(), dedup2.exists() @@ -5490,11 +5509,11 @@ fn add_three_colliding_instances_deduplicates() { // All three should have meta files with the original names let base_meta = src.join("X_Y.meta.json5"); - let dedup1_meta = src.join("X_Y~1.meta.json5"); - let dedup2_meta = src.join("X_Y~2.meta.json5"); + let dedup1_meta = src.join("X_Y~2.meta.json5"); + let dedup2_meta = src.join("X_Y~3.meta.json5"); assert_file_exists(&base_meta, "Meta for base slug"); - assert_file_exists(&dedup1_meta, "Meta for ~1 dedup"); - assert_file_exists(&dedup2_meta, "Meta for ~2 dedup"); + assert_file_exists(&dedup1_meta, "Meta for ~2 dedup"); + assert_file_exists(&dedup2_meta, "Meta for ~3 dedup"); // Each meta should contain one of the original names let all_metas = [ @@ -5636,19 +5655,20 @@ fn add_case_insensitive_colliding_instances() { removed: vec![], added: added_map, updated: vec![], + stage_ids: Vec::new(), }; session.post_api_write(&write_request).unwrap(); thread::sleep(Duration::from_millis(500)); let src = session.path().join("src"); - // One should get the base name, the other should get ~1 + // One should get the base name, the other should get ~2 // The exact casing depends on processing order, but both files // must exist with distinct names. let base = src.join("CaseTest.luau"); let base_lower = src.join("casetest.luau"); - let dedup = src.join("CaseTest~1.luau"); - let dedup_lower = src.join("casetest~1.luau"); + let dedup = src.join("CaseTest~2.luau"); + let dedup_lower = src.join("casetest~2.luau"); let base_exists = base.exists() || base_lower.exists(); let dedup_exists = dedup.exists() || dedup_lower.exists(); @@ -5656,7 +5676,7 @@ fn add_case_insensitive_colliding_instances() { assert!( base_exists && dedup_exists, "Both case-insensitive colliding instances should exist as separate files.\n\ - CaseTest.luau: {}, casetest.luau: {}, CaseTest~1.luau: {}, casetest~1.luau: {}", + CaseTest.luau: {}, casetest.luau: {}, CaseTest~2.luau: {}, casetest~2.luau: {}", base.exists(), base_lower.exists(), dedup.exists(), @@ -6127,7 +6147,7 @@ fn ref_set_primary_part() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); }); } @@ -6156,7 +6176,11 @@ fn ref_set_object_value() { let model_path = session .path() .join("src/Workspace/TestObjectValue.model.json5"); - poll_meta_has_ref_attr(&model_path, "Rojo_Ref_Value", "Workspace/TestModel/Part1"); + poll_meta_has_ref_attr( + &model_path, + "Rojo_Ref_Value", + "./TestModel/Part1.model.json5", + ); }); } @@ -6190,7 +6214,7 @@ fn ref_set_primary_part_to_nil() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); // Now set PrimaryPart to nil @@ -6273,7 +6297,7 @@ fn ref_change_target() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); // Change PrimaryPart to Part2 @@ -6294,7 +6318,7 @@ fn ref_change_target() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part2", + "@self/Part2.model.json5", ); }); } @@ -6324,7 +6348,7 @@ fn ref_set_nil_then_set_again() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); let mut props2 = UstrMap::default(); @@ -6359,7 +6383,7 @@ fn ref_set_nil_then_set_again() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); }); } @@ -6399,11 +6423,7 @@ fn ref_with_name_change() { poll_file_exists(&new_meta, "Meta file should exist at new location"); // The ref path should be updated to reflect the renamed parent - assert_meta_has_ref_attr( - &new_meta, - "Rojo_Ref_PrimaryPart", - "Workspace/RenamedModel/Part1", - ); + assert_meta_has_ref_attr(&new_meta, "Rojo_Ref_PrimaryPart", "@self/Part1.model.json5"); }); } @@ -6433,7 +6453,7 @@ fn ref_only_change_creates_meta() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); }); } @@ -6467,7 +6487,11 @@ fn ref_on_model_json5_instance() { let model_path = session .path() .join("src/Workspace/TestObjectValue.model.json5"); - poll_meta_has_ref_attr(&model_path, "Rojo_Ref_Value", "Workspace/TestModel/Part1"); + poll_meta_has_ref_attr( + &model_path, + "Rojo_Ref_Value", + "./TestModel/Part1.model.json5", + ); }); } @@ -6498,7 +6522,7 @@ fn ref_existing_meta_preserved() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); // Verify existing content wasn't clobbered @@ -6571,7 +6595,7 @@ fn ref_mixed_valid_and_invalid() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); }); } @@ -6614,7 +6638,7 @@ fn ref_stale_path_after_target_rename() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); // Step 2: Rename Part1 to RenamedPart @@ -6637,7 +6661,7 @@ fn ref_stale_path_after_target_rename() { assert_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/RenamedPart", + "@self/RenamedPart.model.json5", ); }); } @@ -6671,7 +6695,7 @@ fn ref_stale_path_after_parent_rename() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); // Step 2: Rename the MODEL (parent) from TestModel to RenamedModel @@ -6700,7 +6724,7 @@ fn ref_stale_path_after_parent_rename() { assert_meta_has_ref_attr( &new_meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/RenamedModel/Part1", + "@self/Part1.model.json5", ); }); } @@ -6745,6 +6769,7 @@ fn ref_to_instance_added_in_same_request() { changed_properties: props, changed_metadata: None, }], + stage_ids: Vec::new(), }; session @@ -6756,13 +6781,14 @@ fn ref_to_instance_added_in_same_request() { // The meta file should contain Rojo_Ref_PrimaryPart pointing to the new Part. // The added_paths fallback resolves the path from the AddedInstance data. + // The path includes the middleware extension (Fix 3: same-batch add+ref). let meta_path = session .path() .join("src/Workspace/TestModel/init.meta.json5"); assert_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/NewPart", + "@self/NewPart.model.json5", ); // Verify the tree also has a resolved PrimaryPart Ref (not dangling). @@ -6829,7 +6855,7 @@ fn ref_ambiguous_path_no_crash() { let meta_path = session .path() .join("src/Workspace/DupParent/init.meta.json5"); - poll_meta_has_ref_attr(&meta_path, "Rojo_Ref_PrimaryPart", "Workspace/Target"); + poll_meta_has_ref_attr(&meta_path, "Rojo_Ref_PrimaryPart", "./Target.model.json5"); }); } @@ -6926,7 +6952,7 @@ fn ref_startup_index_rename_updates_preexisting_attr() { assert_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); // Rename Part1 to IndexTestPart WITHOUT any prior /api/write for Refs. @@ -6951,7 +6977,7 @@ fn ref_startup_index_rename_updates_preexisting_attr() { assert_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/IndexTestPart", + "@self/IndexTestPart.model.json5", ); }); } @@ -6987,10 +7013,10 @@ fn ref_partial_removal_preserves_remaining_index_entry() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); // Verify second ref is there too - assert_meta_has_ref_attr(&meta_path, "Rojo_Ref_Adornee", "Workspace/TestModel/Part2"); + assert_meta_has_ref_attr(&meta_path, "Rojo_Ref_Adornee", "@self/Part2.model.json5"); // Step 2: Remove PrimaryPart (set to nil), keep Adornee let mut props2 = UstrMap::default(); @@ -7010,7 +7036,7 @@ fn ref_partial_removal_preserves_remaining_index_entry() { thread::sleep(Duration::from_millis(300)); assert_meta_no_ref_attr(&meta_path, "Rojo_Ref_PrimaryPart"); // Adornee should still be there - assert_meta_has_ref_attr(&meta_path, "Rojo_Ref_Adornee", "Workspace/TestModel/Part2"); + assert_meta_has_ref_attr(&meta_path, "Rojo_Ref_Adornee", "@self/Part2.model.json5"); // Step 3: Rename Part2. The RefPathIndex should still have the entry // for Adornee (not overbroad-removed when PrimaryPart was deleted). @@ -7032,7 +7058,7 @@ fn ref_partial_removal_preserves_remaining_index_entry() { assert_meta_has_ref_attr( &meta_path, "Rojo_Ref_Adornee", - "Workspace/TestModel/RenamedPart2", + "@self/RenamedPart2.model.json5", ); }); } @@ -7066,7 +7092,7 @@ fn ref_rename_to_slugified_name_updates_ref_paths() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "@self/Part1.model.json5", ); // Rename Model to a name with forbidden characters. @@ -7097,7 +7123,7 @@ fn ref_rename_to_slugified_name_updates_ref_paths() { assert_meta_has_ref_attr( &new_meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/Slug:Model/Part1", + "@self/Part1.model.json5", ); }); } @@ -7145,7 +7171,7 @@ fn ref_on_standalone_script() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "./TestModel/Part1.model.json5", ); // Verify the wrong path does NOT have the attribute @@ -7191,7 +7217,962 @@ fn ref_on_txt_file_instance() { poll_meta_has_ref_attr( &meta_path, "Rojo_Ref_PrimaryPart", - "Workspace/TestModel/Part1", + "./TestModel/Part1.model.json5", + ); + }); +} + +// ========================================================================= +// Regression tests for audit_ambiguous_paths_followup fixes +// ========================================================================= + +/// Fix 3 regression: when adding a new Script instance and a Ref property +/// pointing to it in the same /api/write batch, the persisted Rojo_Ref_* +/// path must include the middleware extension (e.g. ".server.luau"). +#[test] +fn ref_same_batch_add_target_includes_extension() { + run_serve_test("ref_two_way_sync", |session, _| { + let (session_id, workspace_id, model_id, _, _, _) = ref_test_setup(&session); + + // Add a new Script as a child of Workspace + let target_guid = Ref::new(); + let mut add_props = HashMap::new(); + add_props.insert( + "Source".to_string(), + Variant::String("-- target script".to_string()), + ); + add_props.insert( + "RunContext".to_string(), + Variant::Enum(rbx_dom_weak::types::Enum::from_u32(1)), // Server + ); + + let added_inst = AddedInstance { + parent: Some(workspace_id), + name: "RefTarget".to_string(), + class_name: "Script".to_string(), + properties: add_props, + children: vec![], + }; + let mut added_map = HashMap::new(); + added_map.insert(target_guid, added_inst); + + // In the same request, set PrimaryPart on TestModel to the new Script + let mut update_props = UstrMap::default(); + update_props.insert(ustr("PrimaryPart"), Some(Variant::Ref(target_guid))); + + let write_request = WriteRequest { + session_id, + removed: vec![], + added: added_map, + updated: vec![InstanceUpdate { + id: model_id, + changed_name: None, + changed_class_name: None, + changed_properties: update_props, + changed_metadata: None, + }], + stage_ids: Vec::new(), + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(500)); + + // The Rojo_Ref_PrimaryPart path should include the extension + let meta_path = session + .path() + .join("src/Workspace/TestModel/init.meta.json5"); + poll_meta_has_ref_attr( + &meta_path, + "Rojo_Ref_PrimaryPart", + "./RefTarget.server.luau", + ); + }); +} + +/// Fix 3 regression: same-batch add+ref for a ModuleScript target should +/// produce a path ending in ".luau" (not bare instance name). +#[test] +fn ref_same_batch_add_module_target_includes_extension() { + run_serve_test("ref_two_way_sync", |session, _| { + let (session_id, workspace_id, model_id, _, _, _) = ref_test_setup(&session); + + let target_guid = Ref::new(); + let mut add_props = HashMap::new(); + add_props.insert( + "Source".to_string(), + Variant::String("return {}".to_string()), + ); + + let added_inst = AddedInstance { + parent: Some(workspace_id), + name: "ModuleTarget".to_string(), + class_name: "ModuleScript".to_string(), + properties: add_props, + children: vec![], + }; + let mut added_map = HashMap::new(); + added_map.insert(target_guid, added_inst); + + let mut update_props = UstrMap::default(); + update_props.insert(ustr("PrimaryPart"), Some(Variant::Ref(target_guid))); + + let write_request = WriteRequest { + session_id, + removed: vec![], + added: added_map, + updated: vec![InstanceUpdate { + id: model_id, + changed_name: None, + changed_class_name: None, + changed_properties: update_props, + changed_metadata: None, + }], + stage_ids: Vec::new(), + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(500)); + + let meta_path = session + .path() + .join("src/Workspace/TestModel/init.meta.json5"); + poll_meta_has_ref_attr(&meta_path, "Rojo_Ref_PrimaryPart", "./ModuleTarget.luau"); + }); +} + +/// Fix 3 regression: same-batch add+ref for a Folder target (directory +/// middleware) should produce a path WITHOUT an extension. +#[test] +fn ref_same_batch_add_folder_target_no_extension() { + run_serve_test("ref_two_way_sync", |session, _| { + let (session_id, workspace_id, model_id, _, _, _) = ref_test_setup(&session); + + let target_guid = Ref::new(); + let added_inst = AddedInstance { + parent: Some(workspace_id), + name: "FolderTarget".to_string(), + class_name: "Folder".to_string(), + properties: HashMap::new(), + children: vec![], + }; + let mut added_map = HashMap::new(); + added_map.insert(target_guid, added_inst); + + let mut update_props = UstrMap::default(); + update_props.insert(ustr("PrimaryPart"), Some(Variant::Ref(target_guid))); + + let write_request = WriteRequest { + session_id, + removed: vec![], + added: added_map, + updated: vec![InstanceUpdate { + id: model_id, + changed_name: None, + changed_class_name: None, + changed_properties: update_props, + changed_metadata: None, + }], + stage_ids: Vec::new(), + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(500)); + + let meta_path = session + .path() + .join("src/Workspace/TestModel/init.meta.json5"); + poll_meta_has_ref_attr(&meta_path, "Rojo_Ref_PrimaryPart", "./FolderTarget"); + }); +} + +// --------------------------------------------------------------------------- +// Tests: Dedup suffix cleanup +// --------------------------------------------------------------------------- + +/// Add two colliding ModuleScripts (creates X_Y.luau + X_Y~2.luau), then +/// delete one. The remaining instance should lose its dedup suffix (group-to-1 +/// cleanup rule). +#[test] +fn delete_deduped_instance_group_to_1_cleanup() { + run_serve_test("syncback_encoded_names", |session, _redactions| { + let info = session.get_api_rojo().unwrap(); + let root_read = session.get_api_read(info.root_instance_id).unwrap(); + let (rs_id, _) = find_by_class(&root_read.instances, "ReplicatedStorage"); + + let mut props1 = HashMap::new(); + props1.insert( + "Source".to_string(), + Variant::String("-- survivor".to_string()), + ); + let mut props2 = HashMap::new(); + props2.insert( + "Source".to_string(), + Variant::String("-- doomed".to_string()), + ); + + let mut added_map = HashMap::new(); + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: "X/Y".to_string(), + class_name: "ModuleScript".to_string(), + properties: props1, + children: vec![], + }, + ); + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: "X:Y".to_string(), + class_name: "ModuleScript".to_string(), + properties: props2, + children: vec![], + }, + ); + + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: added_map, + updated: vec![], + stage_ids: Vec::new(), + }; + session.post_api_write(&write_request).unwrap(); + + let src = session.path().join("src"); + let base = src.join("X_Y.luau"); + let deduped = src.join("X_Y~2.luau"); + + poll_file_exists(&base, "X_Y.luau after add"); + poll_file_exists(&deduped, "X_Y~2.luau after add"); + + // Wait for VFS to settle + thread::sleep(Duration::from_millis(500)); + + // Re-read the tree to find instance IDs + let rs_read = session.get_api_read(rs_id).unwrap(); + + // Read the base file to figure out which instance maps to it + let base_source = fs::read_to_string(&base).unwrap(); + let doomed_name = if base_source.contains("-- doomed") { + // Base file has the doomed source -- find by that name + // The meta file tells us the real name + let base_meta = src.join("X_Y.meta.json5"); + let meta_content = fs::read_to_string(&base_meta).unwrap(); + if meta_content.contains("\"X/Y\"") { + "X/Y" + } else { + "X:Y" + } + } else { + // Base file has the survivor source -- the other one is doomed + let dedup_meta = src.join("X_Y~2.meta.json5"); + let meta_content = fs::read_to_string(&dedup_meta).unwrap(); + if meta_content.contains("\"X/Y\"") { + "X/Y" + } else { + "X:Y" + } + }; + + let (doomed_id, _) = find_by_name(&rs_read.instances, doomed_name); + send_removal(&session, &info.session_id, vec![doomed_id]); + + // After group-to-1 cleanup: only X_Y.luau should remain, no ~2 suffix + poll_not_exists(&deduped, "X_Y~2.luau after group-to-1 cleanup"); + assert_file_exists(&base, "X_Y.luau should survive"); + + // The survivor's meta should still have a name field (since it's slugified) + let meta = src.join("X_Y.meta.json5"); + assert_file_exists(&meta, "Meta for survivor"); + }); +} + +/// Add three colliding ModuleScripts, then delete the base instance. +/// The ~2 instance should be promoted to base (base-name promotion rule). +#[test] +fn delete_base_deduped_instance_promotes_lowest() { + run_serve_test("syncback_encoded_names", |session, _redactions| { + let info = session.get_api_rojo().unwrap(); + let root_read = session.get_api_read(info.root_instance_id).unwrap(); + let (rs_id, _) = find_by_class(&root_read.instances, "ReplicatedStorage"); + + let names = ["X/Y", "X:Y", "X|Y"]; + let mut added_map = HashMap::new(); + for (i, name) in names.iter().enumerate() { + let mut props = HashMap::new(); + props.insert( + "Source".to_string(), + Variant::String(format!("-- inst {}", i + 1)), + ); + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: name.to_string(), + class_name: "ModuleScript".to_string(), + properties: props, + children: vec![], + }, + ); + } + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: added_map, + updated: vec![], + stage_ids: Vec::new(), + }; + session.post_api_write(&write_request).unwrap(); + + let src = session.path().join("src"); + let base = src.join("X_Y.luau"); + let dedup1 = src.join("X_Y~2.luau"); + let dedup2 = src.join("X_Y~3.luau"); + + poll_file_exists(&base, "X_Y.luau after add"); + poll_file_exists(&dedup1, "X_Y~2.luau after add"); + poll_file_exists(&dedup2, "X_Y~3.luau after add"); + + // Wait for VFS to settle + thread::sleep(Duration::from_millis(500)); + + // Re-read tree and identify which instance maps to the base file + let rs_read = session.get_api_read(rs_id).unwrap(); + let base_meta = src.join("X_Y.meta.json5"); + let base_meta_content = fs::read_to_string(&base_meta).unwrap(); + + // Extract the real name from the base meta file + let base_inst_name = if base_meta_content.contains("\"X/Y\"") { + "X/Y" + } else if base_meta_content.contains("\"X:Y\"") { + "X:Y" + } else { + "X|Y" + }; + + let (base_id, _) = find_by_name(&rs_read.instances, base_inst_name); + + // Delete the base instance + send_removal(&session, &info.session_id, vec![base_id]); + + // After base-name promotion: ~2 should be promoted to base + // X_Y.luau should exist (promoted from ~2) + // X_Y~3.luau should still exist (unchanged) + // X_Y~2.luau should NOT exist (promoted away) + poll_not_exists(&dedup1, "X_Y~2.luau after promotion"); + thread::sleep(Duration::from_millis(300)); + assert_file_exists(&base, "X_Y.luau should exist (promoted)"); + assert_file_exists(&dedup2, "X_Y~3.luau should remain"); + }); +} + +/// Add three colliding ModuleScripts, then delete the ~2 instance. +/// Gap should be tolerated: X_Y.luau and X_Y~3.luau remain unchanged. +#[test] +fn delete_middle_deduped_instance_tolerates_gap() { + run_serve_test("syncback_encoded_names", |session, _redactions| { + let info = session.get_api_rojo().unwrap(); + let root_read = session.get_api_read(info.root_instance_id).unwrap(); + let (rs_id, _) = find_by_class(&root_read.instances, "ReplicatedStorage"); + + let names = ["X/Y", "X:Y", "X|Y"]; + let mut added_map = HashMap::new(); + for (i, name) in names.iter().enumerate() { + let mut props = HashMap::new(); + props.insert( + "Source".to_string(), + Variant::String(format!("-- inst {}", i + 1)), + ); + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: name.to_string(), + class_name: "ModuleScript".to_string(), + properties: props, + children: vec![], + }, + ); + } + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: added_map, + updated: vec![], + stage_ids: Vec::new(), + }; + session.post_api_write(&write_request).unwrap(); + + let src = session.path().join("src"); + let base = src.join("X_Y.luau"); + let dedup1 = src.join("X_Y~2.luau"); + let dedup2 = src.join("X_Y~3.luau"); + + poll_file_exists(&base, "X_Y.luau after add"); + poll_file_exists(&dedup1, "X_Y~2.luau after add"); + poll_file_exists(&dedup2, "X_Y~3.luau after add"); + + thread::sleep(Duration::from_millis(500)); + + // Find which instance maps to ~2 + let rs_read = session.get_api_read(rs_id).unwrap(); + let dedup1_meta = src.join("X_Y~2.meta.json5"); + let dedup1_meta_content = fs::read_to_string(&dedup1_meta).unwrap(); + let middle_name = if dedup1_meta_content.contains("\"X/Y\"") { + "X/Y" + } else if dedup1_meta_content.contains("\"X:Y\"") { + "X:Y" + } else { + "X|Y" + }; + + let (middle_id, _) = find_by_name(&rs_read.instances, middle_name); + + // Delete the ~2 instance + send_removal(&session, &info.session_id, vec![middle_id]); + + // Gap tolerance: base and ~3 remain, ~2 is gone + poll_not_exists(&dedup1, "X_Y~2.luau after deletion"); + // Also verify the ~2 meta is gone + poll_not_exists(&dedup1_meta, "X_Y~2.meta.json5 after deletion"); + thread::sleep(Duration::from_millis(300)); + assert_file_exists(&base, "X_Y.luau should remain"); + assert_file_exists(&dedup2, "X_Y~3.luau should remain (gap tolerated)"); + }); +} + +// --------------------------------------------------------------------------- +// Tests: Folder dedup +// --------------------------------------------------------------------------- + +/// Add two Folders with the same name. Both should become directories with +/// dedup suffixes, each containing their respective children. +#[test] +fn add_same_named_folders_deduplicates() { + run_serve_test("syncback_encoded_names", |session, _redactions| { + let info = session.get_api_rojo().unwrap(); + let root_read = session.get_api_read(info.root_instance_id).unwrap(); + let (rs_id, _) = find_by_class(&root_read.instances, "ReplicatedStorage"); + + // Add two Folders named "Data", each with a distinct child + let child1 = AddedInstance { + parent: None, + name: "ChildA".to_string(), + class_name: "ModuleScript".to_string(), + properties: { + let mut p = HashMap::new(); + p.insert( + "Source".to_string(), + Variant::String("-- child A".to_string()), + ); + p + }, + children: vec![], + }; + let child2 = AddedInstance { + parent: None, + name: "ChildB".to_string(), + class_name: "ModuleScript".to_string(), + properties: { + let mut p = HashMap::new(); + p.insert( + "Source".to_string(), + Variant::String("-- child B".to_string()), + ); + p + }, + children: vec![], + }; + + let mut added_map = HashMap::new(); + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: "Data".to_string(), + class_name: "Folder".to_string(), + properties: HashMap::new(), + children: vec![child1], + }, + ); + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: "Data".to_string(), + class_name: "Folder".to_string(), + properties: HashMap::new(), + children: vec![child2], + }, + ); + + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: added_map, + updated: vec![], + stage_ids: Vec::new(), + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(500)); + + let src = session.path().join("src"); + let data_dir = src.join("Data"); + let data_dedup = src.join("Data~2"); + + // Both directories should exist + assert!( + data_dir.is_dir(), + "Data/ directory should exist at {}", + data_dir.display() + ); + assert!( + data_dedup.is_dir(), + "Data~2/ directory should exist at {}", + data_dedup.display() + ); + + // Dedup-only: no meta file needed. Forward sync strips ~N automatically. + let dedup_meta = data_dedup.join("init.meta.json5"); + assert!( + !dedup_meta.exists(), + "Data~2/init.meta.json5 should NOT exist: forward sync strips ~N, got file at {}", + dedup_meta.display() + ); + + // Each directory should have its own child script + let has_child_a = + data_dir.join("ChildA.luau").exists() || data_dedup.join("ChildA.luau").exists(); + let has_child_b = + data_dir.join("ChildB.luau").exists() || data_dedup.join("ChildB.luau").exists(); + assert!(has_child_a, "ChildA.luau should exist in one of the dirs"); + assert!(has_child_b, "ChildB.luau should exist in one of the dirs"); + }); +} + +// --------------------------------------------------------------------------- +// Tests: Ref paths through dedup'd instances +// --------------------------------------------------------------------------- + +/// Add a Model with children that have slug-colliding names, and set a Ref +/// property (PrimaryPart) to one of them. Verify the Rojo_Ref_* attribute +/// path includes the dedup suffix. +#[test] +fn ref_through_deduped_instance() { + run_serve_test("syncback_encoded_names", |session, _redactions| { + let info = session.get_api_rojo().unwrap(); + let root_read = session.get_api_read(info.root_instance_id).unwrap(); + let (rs_id, _) = find_by_class(&root_read.instances, "ReplicatedStorage"); + + // Add a Model "Container" with two Part children that slug-collide + let target_guid = Ref::new(); + let child_target = AddedInstance { + parent: None, + name: "X/Y".to_string(), + class_name: "Part".to_string(), + properties: HashMap::new(), + children: vec![], + }; + let child_other = AddedInstance { + parent: None, + name: "X:Y".to_string(), + class_name: "Part".to_string(), + properties: HashMap::new(), + children: vec![], + }; + + let mut model_props = HashMap::new(); + model_props.insert("PrimaryPart".to_string(), Variant::Ref(target_guid)); + + let mut added_map = HashMap::new(); + added_map.insert( + target_guid, + AddedInstance { + parent: None, + name: "X/Y".to_string(), + class_name: "Part".to_string(), + properties: HashMap::new(), + children: vec![], + }, + ); + // Clear and re-build: add Model with children and PrimaryPart ref + added_map.clear(); + + // Use Ref::new() for the child that is NOT the target + let container_guid = Ref::new(); + added_map.insert( + container_guid, + AddedInstance { + parent: Some(rs_id), + name: "Container".to_string(), + class_name: "Model".to_string(), + properties: model_props, + children: vec![child_target, child_other], + }, + ); + // The target_guid ref in PrimaryPart points to an instance added in + // the same batch — but since children use inline nesting (no separate + // Ref), the Ref property needs to point to a top-level added instance. + // For simplicity, add the target as a separate instance under Container. + added_map.clear(); + + // Simpler approach: add the Model first, then set PrimaryPart via update + let child1 = AddedInstance { + parent: None, + name: "X/Y".to_string(), + class_name: "Part".to_string(), + properties: HashMap::new(), + children: vec![], + }; + let child2 = AddedInstance { + parent: None, + name: "X:Y".to_string(), + class_name: "Part".to_string(), + properties: HashMap::new(), + children: vec![], + }; + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: "Container".to_string(), + class_name: "Model".to_string(), + properties: HashMap::new(), + children: vec![child1, child2], + }, + ); + + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: added_map, + updated: vec![], + stage_ids: Vec::new(), + }; + session.post_api_write(&write_request).unwrap(); + thread::sleep(Duration::from_millis(500)); + + // Verify Container directory exists with dedup'd children + let src = session.path().join("src"); + let container = src.join("Container"); + assert!(container.is_dir(), "Container/ should be a directory"); + + // Wait for VFS and re-read tree + thread::sleep(Duration::from_millis(500)); + let rs_read = session.get_api_read(rs_id).unwrap(); + let (container_id, _container_inst) = find_by_name(&rs_read.instances, "Container"); + + // Find one of the Part children to use as PrimaryPart target + let container_read = session.get_api_read(container_id).unwrap(); + let (target_part_id, _) = find_by_name(&container_read.instances, "X/Y"); + + // Set PrimaryPart on the container model + let mut props = UstrMap::default(); + props.insert(ustr("PrimaryPart"), Some(Variant::Ref(target_part_id))); + send_update( + &session, + &info.session_id, + InstanceUpdate { + id: container_id, + changed_name: None, + changed_class_name: None, + changed_properties: props, + changed_metadata: None, + }, + ); + + // The Container's init.meta.json5 should have a Rojo_Ref_PrimaryPart + // attribute with a path that includes X_Y (base or ~2 suffix) + let init_meta = container.join("init.meta.json5"); + poll_file_exists(&init_meta, "Container init.meta.json5 after ref set"); + thread::sleep(Duration::from_millis(300)); + let meta_content = fs::read_to_string(&init_meta).unwrap(); + assert!( + meta_content.contains("Rojo_Ref_PrimaryPart"), + "Meta should contain Rojo_Ref_PrimaryPart, got: {}", + meta_content + ); + // The ref path should contain "X_Y" (the slugified child name) + assert!( + meta_content.contains("X_Y"), + "Ref path should contain slugified name X_Y, got: {}", + meta_content + ); + }); +} + +/// Create a dedup group where one instance is a ref target, then delete a +/// member to trigger cleanup rename. Verify the ref path in the pointing +/// meta file is updated. +#[test] +fn ref_path_updated_after_dedup_cleanup() { + run_serve_test("syncback_encoded_names", |session, _redactions| { + let info = session.get_api_rojo().unwrap(); + let root_read = session.get_api_read(info.root_instance_id).unwrap(); + let (rs_id, _) = find_by_class(&root_read.instances, "ReplicatedStorage"); + + // Step 1: Add a Model "Pointer" and two colliding ModuleScripts + let child_ptr = AddedInstance { + parent: None, + name: "Ptr".to_string(), + class_name: "ObjectValue".to_string(), + properties: HashMap::new(), + children: vec![], + }; + let mut added_map = HashMap::new(); + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: "Pointer".to_string(), + class_name: "Model".to_string(), + properties: HashMap::new(), + children: vec![child_ptr], + }, + ); + + let mut props1 = HashMap::new(); + props1.insert( + "Source".to_string(), + Variant::String("-- target".to_string()), + ); + let mut props2 = HashMap::new(); + props2.insert( + "Source".to_string(), + Variant::String("-- other".to_string()), + ); + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: "X/Y".to_string(), + class_name: "ModuleScript".to_string(), + properties: props1, + children: vec![], + }, + ); + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: "X:Y".to_string(), + class_name: "ModuleScript".to_string(), + properties: props2, + children: vec![], + }, + ); + + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: added_map, + updated: vec![], + stage_ids: Vec::new(), + }; + session.post_api_write(&write_request).unwrap(); + + let src = session.path().join("src"); + let base = src.join("X_Y.luau"); + let deduped = src.join("X_Y~2.luau"); + + poll_file_exists(&base, "X_Y.luau after add"); + poll_file_exists(&deduped, "X_Y~2.luau after add"); + thread::sleep(Duration::from_millis(500)); + + // Step 2: Find the ~2 instance and point Pointer's child ObjectValue at it + let rs_read = session.get_api_read(rs_id).unwrap(); + let dedup_meta_path = src.join("X_Y~2.meta.json5"); + let dedup_meta_content = fs::read_to_string(&dedup_meta_path).unwrap(); + let target_name = if dedup_meta_content.contains("\"X/Y\"") { + "X/Y" + } else { + "X:Y" + }; + let other_name = if target_name == "X/Y" { "X:Y" } else { "X/Y" }; + + let (target_id, _) = find_by_name(&rs_read.instances, target_name); + let (other_id, _) = find_by_name(&rs_read.instances, other_name); + + // Find the Pointer model and its ObjectValue child + let (pointer_id, _) = find_by_name(&rs_read.instances, "Pointer"); + let pointer_read = session.get_api_read(pointer_id).unwrap(); + let (ptr_child_id, _) = find_by_name(&pointer_read.instances, "Ptr"); + + // Set ObjectValue.Value to the ~2 target + let mut props = UstrMap::default(); + props.insert(ustr("Value"), Some(Variant::Ref(target_id))); + send_update( + &session, + &info.session_id, + InstanceUpdate { + id: ptr_child_id, + changed_name: None, + changed_class_name: None, + changed_properties: props, + changed_metadata: None, + }, + ); + + // Verify the ref was written with the ~2 path + let pointer_dir = src.join("Pointer"); + let ptr_model_path = pointer_dir.join("Ptr.model.json5"); + poll_file_exists(&ptr_model_path, "Ptr.model.json5 after ref set"); + thread::sleep(Duration::from_millis(300)); + let ref_content = fs::read_to_string(&ptr_model_path).unwrap(); + assert!( + ref_content.contains("X_Y~2"), + "Before cleanup: ref should point to X_Y~2, got: {}", + ref_content + ); + + // Step 3: Delete the OTHER instance (not the target) to trigger + // group-to-1 cleanup, which renames X_Y~2 → X_Y + send_removal(&session, &info.session_id, vec![other_id]); + + // After cleanup: X_Y~2.luau renamed to X_Y.luau + poll_not_exists(&deduped, "X_Y~2.luau after cleanup"); + thread::sleep(Duration::from_millis(300)); + assert_file_exists(&base, "X_Y.luau after cleanup"); + + // The ref in Ptr.model.json5 should now point to the non-suffixed path + let updated_ref = fs::read_to_string(&ptr_model_path).unwrap(); + assert!( + !updated_ref.contains("X_Y~2"), + "After cleanup: ref should NOT contain X_Y~2 (old path), got: {}", + updated_ref + ); + assert!( + updated_ref.contains("X_Y"), + "After cleanup: ref should contain X_Y (new path), got: {}", + updated_ref + ); + }); +} + +/// Rename an instance that is a ref target via two-way sync. Verify +/// Rojo_Ref_* paths pointing to it are updated in meta files. +#[test] +fn rename_ref_target_updates_ref_paths() { + run_serve_test("syncback_encoded_names", |session, _redactions| { + let info = session.get_api_rojo().unwrap(); + let root_read = session.get_api_read(info.root_instance_id).unwrap(); + let (rs_id, _) = find_by_class(&root_read.instances, "ReplicatedStorage"); + + // Step 1: Add a target ModuleScript and a Pointer model + let child_obj = AddedInstance { + parent: None, + name: "Ref".to_string(), + class_name: "ObjectValue".to_string(), + properties: HashMap::new(), + children: vec![], + }; + let mut added_map = HashMap::new(); + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: "Target".to_string(), + class_name: "ModuleScript".to_string(), + properties: { + let mut p = HashMap::new(); + p.insert( + "Source".to_string(), + Variant::String("-- target".to_string()), + ); + p + }, + children: vec![], + }, + ); + added_map.insert( + Ref::new(), + AddedInstance { + parent: Some(rs_id), + name: "Ptr".to_string(), + class_name: "Model".to_string(), + properties: HashMap::new(), + children: vec![child_obj], + }, + ); + + let write_request = WriteRequest { + session_id: info.session_id, + removed: vec![], + added: added_map, + updated: vec![], + stage_ids: Vec::new(), + }; + session.post_api_write(&write_request).unwrap(); + + let src = session.path().join("src"); + poll_file_exists(&src.join("Target.luau"), "Target.luau after add"); + thread::sleep(Duration::from_millis(500)); + + // Step 2: Set the ObjectValue.Value to Target + let rs_read = session.get_api_read(rs_id).unwrap(); + let (target_id, _) = find_by_name(&rs_read.instances, "Target"); + let (ptr_model_id, _) = find_by_name(&rs_read.instances, "Ptr"); + let ptr_read = session.get_api_read(ptr_model_id).unwrap(); + let (objval_id, _) = find_by_name(&ptr_read.instances, "Ref"); + + let mut props = UstrMap::default(); + props.insert(ustr("Value"), Some(Variant::Ref(target_id))); + send_update( + &session, + &info.session_id, + InstanceUpdate { + id: objval_id, + changed_name: None, + changed_class_name: None, + changed_properties: props, + changed_metadata: None, + }, + ); + + // Verify ref was written + let obj_model_path = src.join("Ptr").join("Ref.model.json5"); + poll_file_exists(&obj_model_path, "Ref.model.json5 after ref set"); + thread::sleep(Duration::from_millis(300)); + let ref_content = fs::read_to_string(&obj_model_path).unwrap(); + assert!( + ref_content.contains("Target.luau"), + "Ref should point to Target.luau, got: {}", + ref_content + ); + + // Step 3: Rename the target + // Re-read to get current IDs (may have changed after VFS rebuild) + let rs_read = session.get_api_read(rs_id).unwrap(); + let (target_id, _) = find_by_name(&rs_read.instances, "Target"); + + send_update( + &session, + &info.session_id, + InstanceUpdate { + id: target_id, + changed_name: Some("Renamed".to_string()), + changed_class_name: None, + changed_properties: UstrMap::default(), + changed_metadata: None, + }, + ); + + // Wait for rename to propagate + let renamed_path = src.join("Renamed.luau"); + poll_file_exists(&renamed_path, "Renamed.luau after rename"); + thread::sleep(Duration::from_millis(300)); + + // Step 4: Verify the ref path was updated + let updated_ref = fs::read_to_string(&obj_model_path).unwrap(); + assert!( + updated_ref.contains("Renamed.luau"), + "After rename: ref should point to Renamed.luau, got: {}", + updated_ref + ); + assert!( + !updated_ref.contains("Target.luau"), + "After rename: ref should NOT point to old Target.luau, got: {}", + updated_ref ); }); } diff --git a/vscode-rojo b/vscode-rojo index f18e6d4c..f11bef94 160000 --- a/vscode-rojo +++ b/vscode-rojo @@ -1 +1 @@ -Subproject commit f18e6d4c523994e0eda7bf18f4947a724717e59d +Subproject commit f11bef94aa14d635c18eda35e75db4dfbcb2260d