Commit graph

180 commits

Author SHA1 Message Date
Joe
b39c01e5e9 Add matching session caching and native codegen improvements
- Introduced a new matching session caching mechanism in the Lua matching module to enhance performance with native code generation.
- Rewrote the Lua matching algorithm to utilize session-based caching, reducing redundant computations and improving efficiency.
- Updated Rust implementations to include session caching, ensuring consistency across both Lua and Rust.
- Enhanced the `hydrate` function to support session management during instance hydration.
- Added comprehensive tests to validate the new caching mechanism and its integration with existing functionality.

These changes significantly improve the performance and maintainability of the matching process in the application.
2026-02-18 04:55:31 -06:00
Joe
d836ebddc8 Update deduplication logic and documentation for suffix handling
- Removed the `luau` dependency from `rokit.toml`.
- Updated deduplication logic to append `~2`, `~3`, etc. instead of `~1`, `~2` for naming collisions in various files, ensuring consistent suffix management.
- Adjusted comments and documentation across multiple files to reflect the new suffix handling rules.
- Added new test cases to validate the updated deduplication behavior, ensuring accurate naming for instances with collisions.

These changes enhance the clarity and functionality of the deduplication process, improving overall consistency in naming conventions.
2026-02-18 01:57:17 -06:00
Joe
bb9d241253 Refactor Reconciler tests and update path handling logic
- Updated test descriptions in `diff.spec.lua` and `diff.stress.spec.lua` to reflect changes in behavior regarding duplicate-named children in both real and virtual DOM contexts.
- Adjusted assertions to ensure that duplicate-named children are correctly removed or added based on the new matching logic.
- Refactored `hydrate.lua` and `matching.lua` to streamline the matching process by removing unnecessary parameters, enhancing clarity and maintainability.
- Updated metadata references in various test files to include the `.model.json5` extension for consistency in path handling.

These changes improve the accuracy of reconciliation tests and enhance the overall robustness of the Reconciler's path handling logic.
2026-02-17 04:46:54 -06:00
Joe
1a1d9ce886 Implement new recursive change-count matching algorithm for instance reconciliation
- Introduced a comprehensive rewrite of the matching algorithm, replacing the previous multi-pass approach with a recursive change-count scoring function and greedy assignment strategy.
- Updated the `hydrate.lua` and `matching.lua` files to utilize the new algorithm, enhancing the efficiency of instance matching during hydration.
- Documented the new algorithm's structure, including fast-paths for Ref pins and (Name, ClassName) grouping, which significantly improve matching accuracy and performance.
- Ensured that the new implementation maintains compatibility with existing systems while providing a more robust solution for handling ambiguous instances.

These changes enhance the clarity and efficiency of the reconciliation process, paving the way for improved project management and user experience.
2026-02-17 03:54:06 -06:00
Joe
d6162ac210 Complete implementation of the 3-pass matching algorithm for ambiguous path handling
- Finalized the transition from a duplicate-name blacklist system to a comprehensive 3-pass matching algorithm for both syncback and forward operations, enhancing the handling of ambiguous paths.
- Integrated a new `matching.lua` module to facilitate instance matching during hydration, utilizing unique name matching, class name narrowing, and structural fingerprinting.
- Updated various components, including `ChangeBatcher`, `Reconciler`, and `snapshot` modules, to leverage the new matching logic, ensuring robust file management and improved performance.
- Removed legacy duplicate handling code and streamlined the encoding process for instances with duplicate names.

These changes significantly enhance the clarity and efficiency of file operations, paving the way for improved project management and user experience.
2026-02-17 02:43:25 -06:00
Joe
d05ad92c84 Enhance path handling in ChangeProcessor and snapshot_project
- Canonicalized the project_root in ChangeProcessor to ensure consistent path comparisons on Windows, addressing issues with the \\?\ prefix.
- Updated snapshot_project to push canonicalized paths to relevant_paths, improving path accuracy in project snapshots.

These changes enhance the reliability of path handling in the project and improve overall consistency in file operations.
2026-02-15 02:54:07 -06:00
Joe
434a3c458a Refactor CI documentation and improve change processing logic
- Updated CI documentation to enforce running the full suite without skipping the rbx-dom section, clarifying the execution steps.
- Refactored `change_processor.rs` for improved readability by enhancing formatting and structure in name deduplication logic.
- Cleaned up imports in `project.rs` and improved formatting in `file_names.rs` to enhance code clarity and maintainability.
- Adjusted test assertions in `two_way_sync.rs` for better readability and consistency in checking file existence after renaming operations.
2026-02-10 21:44:29 -06:00
Joe
2ff5c3936c Implement fixes and enhancements from audit pass 2
- Introduced a new plan addressing critical bugs, correctness issues, and refactors identified during the audit, including improvements to metadata handling and slugification logic.
- Replaced inline removal logic in `handle_api_write` with a new `remove_instance_at_path` method to ensure correct deletion of instances.
- Updated pre-seeding logic in `dir.rs` and `project.rs` to derive deduplication keys from actual filesystem paths instead of slugified names.
- Enhanced rename handling in `change_processor.rs` to prevent collisions by deduplicating names against existing siblings.
- Added tests for various scenarios, including handling of forbidden characters and ensuring correct deduplication behavior.
- Refactored child iteration in `dir.rs` and `project.rs` to sort children alphabetically for deterministic deduplication ordering.
2026-02-10 21:35:26 -06:00
Joe
baf05dcb3d Update slugify branch audit plan and implement fixes for identified issues
- Marked all todos in the slugify branch audit plan as completed, confirming the successful verification of core invariants and sync processes.
- Introduced a new plan to address critical bugs, correctness issues, and refactors identified during the audit, including enhancements to metadata handling and slugification logic.
- Added tests to ensure proper handling of filenames with forbidden characters and deduplication of instances with colliding names.
- Refactored metadata operations to improve clarity and maintainability, consolidating logic for reading and writing the `name` field in `.meta.json5` files.
2026-02-10 20:18:33 -06:00
Joe
f38c1f1311 Refactor metadata name handling for improved clarity
- Updated comments in `json_model.rs` and `meta_file.rs` to clarify the logic for storing the real name of instances when the filesystem name differs due to slugification or deduplication.
- Simplified the conditions under which the real name is preserved, ensuring better readability and understanding of the code's intent.
2026-02-10 17:34:12 -06:00
Joe
f496bee347 Refactor CI pipeline scripts and documentation
- Consolidated CI pipeline steps into PowerShell and Bash scripts for improved usability and consistency.
- Updated documentation in `ci.md` to clarify the execution process and options for skipping the rbx-dom section.
- Enhanced error handling and reporting in the scripts to provide clearer feedback on CI step results.
- Streamlined formatting and linting processes for both Rojo and rbx-dom, ensuring a more efficient CI workflow.
2026-02-10 16:46:26 -06:00
Joe
9688224cb4 Enhance slugification process and deduplication logic
- Updated the `slugify_name` function to strip trailing spaces and dots before checking for Windows reserved names, ensuring correct handling of inputs like "CON.".
- Added tests to validate the new slugification behavior, particularly for Windows reserved names with trailing characters.
- Refactored `syncback_dir_no_meta` to pre-seed `taken_names` from old children's slugified instance names, improving deduplication against existing siblings regardless of iteration order.
2026-02-10 04:10:00 -06:00
Joe
cc2eb2b66d Refactor syncback instance processing for improved efficiency
- Consolidated repetitive calls to `syncback_instance_to_path_with_stats` into a new method `process_children_incremental` for better code organization and readability.
- Streamlined the handling of sibling slug computation by removing redundant logic, enhancing maintainability in `api.rs`.
- Updated function signatures to reflect changes in return types and improve clarity in the syncback process.
2026-02-10 03:46:08 -06:00
Joe
7a090b104f Refactor syncback instance handling for improved readability
- Consolidated function calls in `syncback_instance_to_path_with_stats` across multiple files for better clarity and maintainability.
- Streamlined the formatting of function arguments to enhance code readability in `api.rs`, `file_names.rs`, and `project.rs`.
- Updated tests to reflect changes in function call structure while ensuring existing functionality remains intact.
2026-02-10 03:34:08 -06:00
Joe
560fce1352 Implement fix for stem-level deduplication in file handling
- Introduced a migration plan to address the deduplication issue where bare slugs were compared against full filenames, leading to undetected file-format collisions.
- Updated `name_for_inst` to return a deduplication key alongside the filename and metadata flag, ensuring callers use the bare slug for collision detection.
- Replaced disk-based seeding of `taken_names` with tree-based slugified names to maintain consistency and avoid extension mismatches.
- Refactored relevant modules including `dir.rs`, `project.rs`, and `api.rs` to utilize the new deduplication logic.
- Updated tests to validate the new deduplication behavior and ensure correct handling of file-format instances.
2026-02-10 03:32:17 -06:00
Joe
126c8e66a4 Update vscode-rojo submodule reference and complete migration plan for path encoding
- Updated the vscode-rojo submodule to the latest commit.
- Marked all phases of the path encoding migration plan as completed, transitioning to a slugify and deduplication approach.
- Updated documentation to reflect changes in filename handling and metadata management.
2026-02-10 03:14:18 -06:00
Joe
64d21dc6d6 Update submodule reference and refactor match statements for improved clarity
- Updated the rbx-dom submodule to the latest commit.
- Refactored match statements in resolution.rs, sourcemap.rs, project.rs, and api.rs to dereference names directly for better readability.
2026-02-10 00:33:47 -06:00
Joe
5298664bad Remove NetworkClient from visibleServices in multiple snapshot test files 2026-02-01 21:05:00 -06:00
Joe
76e4bc4f25 Add Int64 conversion functions to EncodedValue.lua and enhance script middleware selection in project.rs
- Implemented int64FromPod and int64ToPod functions for converting between Lua numbers and msgpack Int64 representation, addressing precision concerns for large integers.
- Updated the middleware_for_script function to determine the correct script directory based on the RunContext property, improving middleware selection for Script instances.
- Adjusted syncback logic to utilize the new middleware selection for Scripts, ensuring accurate handling of different script types during project synchronization.
2026-02-01 20:51:13 -06:00
Joe
411f79ee81 Enhance syncback functionality and improve code readability
- Refactored the syncback process to streamline directory management and improve handling of class transitions in clean mode.
- Updated the `clone_tree_filtered` function to accept a tuple of compiled Globs and original pattern strings, enhancing flexibility in ignore pattern handling.
- Simplified logic for checking if paths are inside removed directories, improving code clarity.
- Added detailed comments to clarify the purpose of new logic for handling script directory transitions, ensuring better maintainability.
- Improved logging for better traceability during syncback operations.
2026-02-01 20:10:11 -06:00
Joe
1534a54808 Refactor syncback process to improve clean mode handling and directory management
- Removed the ensure_project_paths_exist function, streamlining the syncback process by directly managing directory creation and cleanup within the syncback command.
- Enhanced error handling for sourcemap generation, logging warnings when clean mode removes directories that only contain meta files.
- Updated the logic for middleware selection in clean mode to better handle directory and script transitions based on the filesystem structure.
- Added utility functions to ensure base project directories exist before syncback, addressing the need for directory structure in clean mode.
- Improved test coverage for clean mode scenarios, ensuring that directory existence checks are performed and documented.
2026-02-01 19:55:54 -06:00
Joe
2e6f909a71 Fix syncback process to handle directory paths and class transitions in clean mode
- Resolved an issue where syncback failed for projects with `$path` pointing to single files by ensuring only directory paths are scanned for orphans.
- Implemented a function to create missing directories in clean mode, allowing restoration of deleted directories.
- Enhanced class transition handling in clean mode to allow recoverable transitions between Folder and Script types.
- Updated logging and error handling to improve clarity during the syncback process.
2026-02-01 19:27:53 -06:00
Joe
8fe908740f Add statistics tracking for syncback operations
- Introduced a new `SyncbackStats` module to track and report issues during syncback, including duplicate names, unknown classes, and properties.
- Updated the `syncback_loop` and related functions to utilize the stats tracker, logging summaries of issues encountered.
- Enhanced property filtering and instance syncing to record statistics on unknown classes and properties.
- Modified tests to ensure consistent output by normalizing temporary paths in snapshots.
- Added `regex` dependency for path normalization in tests.
2026-02-01 08:29:00 -06:00
Joe
5f91a0777a Implement new oneshot Vfs for CLI commands and enhance JSON5 serialization
- Added `new_oneshot` method to the Vfs struct, providing a backend suitable for one-time operations without file watching.
- Updated CLI commands (`fmt_project` and `syncback`) to utilize the new oneshot Vfs, improving error handling during file operations.
- Enhanced JSON5 serialization by introducing a custom serializer that directly builds a tree structure, improving performance and maintaining sorted keys.
- Improved error logging in JSON serialization to provide more context on failures.
2026-01-31 13:06:37 -06:00
Joe
0b243b4ca9 Add CSV input files and enhance syncback warning logic
- Introduced a new CSV input file with translation data for Spanish.
- Updated the `init.csv` file to include a new entry for "Rojo" with its context and example.
- Enhanced the syncback logic to conditionally emit warnings for duplicate child names based on the new `warn_duplicate_names` configuration in `SyncbackRules`.
- Added documentation for the new warning behavior in the `SyncbackRules` and `SyncbackSnapshot` implementations.
2026-01-30 10:42:38 -06:00
Joe
18a74c8e4d Enhance script handling and UI consistency
- Added support for encoding the RunContext property in the Script class to determine file suffixes (.server.luau, .client.luau, .plugin.luau, .legacy.luau).
- Updated the Confirming.lua file to improve UI consistency in the confirmation message display.
- Modified snapshot middleware and syncback logic to accommodate new file extensions and ensure proper handling of plugin scripts.
- Refactored tests to align with the new naming conventions for script suffixes and improved path uniqueness checks.
2026-01-30 10:31:36 -06:00
Joe
5a85c5145c Refactor project configuration to use .json format
- Replaced the legacy .json5 project file with a new .json format for `plugin.project.json`.
- Updated `build.rs` to prioritize the new .json format while maintaining backward compatibility with .json5.
- Made adjustments to the Confirming.lua file to enhance UI consistency.
- Improved metadata handling in snapshot middleware to support both modern and legacy file extensions.
2026-01-30 10:06:32 -06:00
Joe
5f059dbc16 Add back legacy support for .json and .lua 2026-01-30 09:38:43 -06:00
Joe
84f76d48fc Add project configuration and test files in JSON5 format
- Introduced new configuration files for testing, including `default.project.json5`, `client_in_folder`, and others.
- Updated existing project files to use JSON5 format, ensuring consistency across configurations.
- Added new Lua scripts and model files to support various test cases.
- Adjusted snapshot files to align with the new project structure and properties.
2026-01-30 09:16:46 -06:00
Joe
779e3de0ae Update project configuration and snapshot files
- Changed the default project file from `plugin.project.json` to `plugin.project.json5` in `build.rs`.
- Added new project configuration files in JSON5 format for various test cases, including `default.project.json5`, `client_in_folder`, `csv_bug_145`, and others.
- Updated snapshot files to reflect changes in instance structure and properties, ensuring consistency with the new project configurations.
- Removed unnecessary script and module instances from snapshot files to streamline the testing process.
2026-01-30 08:32:42 -06:00
Joe
d3cc11659a RunContext refactor + sync updates 2026-01-30 08:12:04 -06:00
Joe
8ede87e1cf Update dependencies and improve CLI argument handling
- Updated various dependencies in Cargo.toml and Cargo.lock, including `env_logger`, `fs-err`, and `clap`, to their latest versions for enhanced functionality and security.
- Refactored CLI argument handling in `build.rs` and `mod.rs` to align with the new `clap` version, improving error reporting and verbosity options.
- Added new packages such as `anstream`, `colorchoice`, and `jiff` to enhance project capabilities.
2026-01-30 05:22:39 -06:00
Joe
a7f36fab5e Preserve and restore instigating source in Lua snapshot initialization
- Added logic to preserve the init script's instigating source before copying directory metadata, ensuring accurate file path references.
- Updated snapshot files to reflect the correct instigating source path for improved synchronization and consistency in tests.
2026-01-24 10:04:47 -06:00
Joe
fdd1d145cf json5 work 2026-01-22 22:19:00 -06:00
Joe
cd63b5c9c0 json5 2026-01-22 21:27:34 -06:00
Joe
074e62951f revert unresolved value patches 2026-01-22 19:51:44 -06:00
Joe
69774fd546 Update .gitmodules for rbx-dom URL and enhance UnresolvedValue handling
- Changed the rbx-dom submodule URL in .gitmodules from SSH to HTTPS for easier access.
- Introduced a new method to handle OptionalCFrame variants in UnresolvedValue, allowing for better serialization by returning `None` for `OptionalCFrame(None)`.
- Updated various files to utilize the new handling logic, ensuring properties are only inserted when resolved values are available.
2026-01-21 16:07:24 -06:00
Joe
4a09689090 Enhance WhitespaceUtil and Reconciler to handle duplicate names and improve diffing logic
- Updated WhitespaceUtil to return distinct values for totalDiff and whitespaceDiff when differences exceed 100 lines, based on content changes.
- Introduced findDuplicateNames function in Reconciler to identify and skip instances with duplicate names, ensuring reliable syncing.
- Enhanced diffing logic to account for ambiguous paths and log skipped duplicates during the sync process.
- Improved error handling in syncback_dir_no_meta to log and skip children with duplicate names in the file system.
2026-01-20 22:10:19 -06:00
Joe
2dfcac4ef7 Refactor code for improved readability by consolidating multi-line statements into single lines across various modules, including JSON serialization, snapshot middleware, and file name validation. 2026-01-20 20:48:52 -06:00
CoderJoey
c36421626d
Merge branch 'rojo-rbx:master' into master 2026-01-20 20:10:14 -06:00
Ivan Matthew
2a1102fc55
Implement VFS Path normalization for improved cross-platform tree synchronization (#1201) 2026-01-19 15:04:59 -08:00
Joe
033dec4e18 Refactor metadata file name generation in syncback_lua and AdjacentMetadata to strip known script type suffixes (.server, .client, .plugin) for improved compatibility. This change ensures accurate base name extraction for meta files while maintaining handling of Windows-invalid characters. 2026-01-15 16:08:53 -06:00
Joe
5ad50e65d4 Enhance syncback error handling by including full instance paths in duplicate name error messages. Updated the syncback_project function to provide clearer context when instances with duplicate names are detected. 2025-12-29 00:40:13 -06:00
Joe
f6dc37e400 Enhance syncback functionality by implementing tree filtering based on ignoreTrees patterns. Added support for glob pattern matching to filter instances during serialization in both rbxm and rbxmx. Updated SyncbackSnapshot to compile and utilize these patterns efficiently. 2025-12-29 00:02:40 -06:00
Joe
978dd41047 Refactor metadata file name generation in syncback_lua to utilize the file stem from the script path, ensuring proper handling of Windows-invalid characters while maintaining compatibility with existing naming conventions. 2025-12-28 23:24:41 -06:00
Joe
f54367e5e0 Refactor metadata file name generation in read_and_apply_all to use the file stem from the actual path, ensuring correct handling of encoded characters for Windows-invalid filenames. 2025-12-28 23:04:59 -06:00
Joe
8c776c09b3 Implement encoding for Windows-invalid characters in syncback metadata file names across CSV, Lua, and TXT handlers. Updated file name generation to conditionally encode names based on the presence of invalid characters. 2025-12-28 22:43:21 -06:00
Joe
6d8c85682d Implement encoding and decoding for Windows-invalid characters in file names
- Added `encode_path_name` and `decode_path_name` functions to handle encoding and decoding of invalid characters.
- Updated `SyncbackRules` to include an option for encoding Windows-invalid characters during syncback.
- Modified `name_for_inst` to conditionally encode invalid characters based on the new option.
- Adjusted `ServeSession` and `InstanceContext` to support the new encoding feature.
- Updated tests and snapshots to reflect changes in encoding behavior.
2025-12-19 22:50:53 -06:00
Joe
db87fa010f ignore classes feature 2025-12-19 01:15:14 -06:00
Joe
929bc254fa Sorted keys 2025-12-09 00:28:13 -06:00