Commit graph

139 commits

Author SHA1 Message Date
Joe
2ff5c3936c Implement fixes and enhancements from audit pass 2
- Introduced a new plan addressing critical bugs, correctness issues, and refactors identified during the audit, including improvements to metadata handling and slugification logic.
- Replaced inline removal logic in `handle_api_write` with a new `remove_instance_at_path` method to ensure correct deletion of instances.
- Updated pre-seeding logic in `dir.rs` and `project.rs` to derive deduplication keys from actual filesystem paths instead of slugified names.
- Enhanced rename handling in `change_processor.rs` to prevent collisions by deduplicating names against existing siblings.
- Added tests for various scenarios, including handling of forbidden characters and ensuring correct deduplication behavior.
- Refactored child iteration in `dir.rs` and `project.rs` to sort children alphabetically for deterministic deduplication ordering.
2026-02-10 21:35:26 -06:00
Joe
baf05dcb3d Update slugify branch audit plan and implement fixes for identified issues
- Marked all todos in the slugify branch audit plan as completed, confirming the successful verification of core invariants and sync processes.
- Introduced a new plan to address critical bugs, correctness issues, and refactors identified during the audit, including enhancements to metadata handling and slugification logic.
- Added tests to ensure proper handling of filenames with forbidden characters and deduplication of instances with colliding names.
- Refactored metadata operations to improve clarity and maintainability, consolidating logic for reading and writing the `name` field in `.meta.json5` files.
2026-02-10 20:18:33 -06:00
Joe
126c8e66a4 Update vscode-rojo submodule reference and complete migration plan for path encoding
- Updated the vscode-rojo submodule to the latest commit.
- Marked all phases of the path encoding migration plan as completed, transitioning to a slugify and deduplication approach.
- Updated documentation to reflect changes in filename handling and metadata management.
2026-02-10 03:14:18 -06:00
Joe
7172ae2d09 Enhance syncback functionality to manage .gitkeep files
- Updated the syncback loop to skip hidden files/directories, with an exception for .gitkeep, which is necessary for directory management.
- Added comprehensive tests to ensure proper cleanup of stale .gitkeep files in various scenarios, including non-empty directories and init-style projects.
2026-02-08 16:23:41 -06:00
Joe
250c7db84a Add macOS compatibility check for extreme filesystem rename tests
- Updated the `extreme_filesystem_5_file_rename_chains` test to ignore execution on macOS due to kqueue limitations with high rename pressure.
- Added documentation to clarify the reason for the macOS-specific ignore condition, ensuring better understanding of the test's behavior across different platforms.
2026-02-08 12:13:04 -06:00
Joe
6e323660e0 Update snapshot files to standardize messageCursor redaction
- Replaced specific messageCursor values with a placeholder "[message-cursor]" across multiple snapshot files to ensure consistency in test outputs.
- Adjusted the test utility to add redaction for messageCursor, addressing non-deterministic behavior in macOS environments.
2026-02-08 12:07:20 -06:00
Joe
e934912b4a Add polling functions for file existence and content checks in two-way sync tests
- Introduced `poll_file_exists`, `poll_not_exists`, and `poll_file_contains` functions to enhance the reliability of assertions in asynchronous file operations.
- Replaced direct assertions with polling functions in various test cases to ensure proper synchronization with filesystem changes.
- Improved error handling and timeout messages for better debugging during test failures.
2026-02-08 11:55:10 -06:00
Joe
50b9e18514 Refactor and rebrand Rojo to Atlas
- Renamed the project from Rojo to Atlas, updating all relevant files, scripts, and documentation to reflect the new branding.
- Bumped version to 7.7.0-rc.2 and updated protocol version from 5 to 6.
- Modified installation scripts and CI workflows to use the new binary name and paths.
- Adjusted plugin settings and internal identifiers to prevent conflicts with the original Rojo.
- Enhanced session management and API response handling to support the new server fork identifier.
- Updated project templates and README files to guide users on using Atlas.
2026-02-07 13:21:52 -06:00
Joe
aeec3a962b Ignore certain tests on macOS due to kqueue event overload
- Added `#[cfg_attr(target_os = "macos", ignore)]` to three tests to prevent them from running on macOS, where rapid rename operations can overwhelm the ChangeProcessor.
- This change aims to improve test reliability across different operating systems.
2026-02-06 23:26:41 -06:00
Joe
4f3e66bec9 Add tests for PatchSet and ServeSession functionality
- Introduced comprehensive tests for the `PatchSet` module, covering methods such as `hasRemoves`, `hasAdditions`, `hasUpdates`, and various ID and instance checks.
- Added a new test suite for `ServeSession`, focusing on WebSocket message handling and confirmation behavior, ensuring correct merging and application of patches.
- Implemented edge case tests in `ChangeBatcher` to verify data integrity during rapid changes and instance removals.
- Enhanced `applyPatch` tests to handle scenarios involving missing instances and ignored class names, ensuring robust patch application logic.
2026-02-06 22:23:14 -06:00
Joe
fa5393121e Enhance CloneCommand and Syncback functionality
- Added a check in `CloneCommand` to prevent cloning into an existing directory, providing a clearer error message for users.
- Updated the `SyncbackCommand` to support a configurable working directory for resolving relative paths, improving flexibility in project structure.
- Refactored the commit message in the `CloneCommand` to accurately reflect the syncback operation.
- Improved path resolution logic in `SyncbackCommand` to handle both absolute and relative paths more effectively.
2026-02-06 19:29:37 -06:00
Joe
5442c02f67 Enhance redaction handling and update snapshot references
- Added support for redacting tagged values in the `RedactionMap` implementation, ensuring that nested tagged values are processed correctly.
- Updated multiple snapshot files to replace specific reference IDs with deterministic `id-N` values, improving consistency across tests and snapshots.
- Refactored internment logic in `internable.rs` to include variant references, ensuring proper redaction of property values in snapshots.
2026-02-06 19:20:16 -06:00
Joe
4ee1e79a90 Refactor FSEvents handling in serve_util.rs for improved reliability
- Removed unnecessary sleep duration for FSEvents on macOS, reducing it from 500ms to 100ms to enhance responsiveness while still allowing time for event processing.
- Updated comments to clarify the rationale behind the timing adjustments and noted the need for a more robust solution in the future.
2026-02-06 13:10:00 -06:00
Joe
67537070db Refactor path suppression handling in ChangeProcessor and ApiService
- Updated the `suppressed_paths` structure in `ChangeProcessor` and `ApiService` to store counts for both remove and create/write events, enhancing the accuracy of event suppression.
- Refactored methods for suppressing and unsuppressing paths to accommodate the new structure, ensuring that the system correctly manages VFS events based on the type of operation.
- Improved documentation and comments to clarify the purpose and behavior of the suppression logic, contributing to better maintainability and understanding of the code.
2026-02-06 13:01:07 -06:00
Joe
84771840e1 Refactor port number generation in tests for improved reliability
- Replaced the static port number generation with a method that binds a temporary TcpListener to an ephemeral port assigned by the OS, enhancing the reliability of port allocation during tests.
- Updated documentation to clarify the new approach for obtaining a free port, ensuring better understanding and maintainability of the test code.
2026-02-06 12:24:50 -06:00
Joe
acb3b18fa1 Refactor test code for consistency and readability
- Streamlined multiple test functions by consolidating calls to `send_update_no_wait` and `get_format_transitions_instance` into single lines, enhancing code clarity.
- Improved consistency in handling session and instance IDs across various rename scenarios, contributing to better maintainability of the test suite.
2026-02-06 12:24:34 -06:00
Joe
4bfb52d299 Refactor rename handling in two-way sync tests for improved stability
- Updated multiple test functions to retrieve session IDs and instance IDs before loops, enhancing the stability of rename operations during rapid updates.
- Removed redundant calls to fetch instance information within loops, streamlining the code and reducing potential race conditions.
- Ensured consistent handling of session IDs across various rename scenarios, contributing to more reliable test outcomes in high-load situations.
2026-02-06 12:18:50 -06:00
Joe
a6fb256192 Update extreme_no_wait_directory_all_three test to reduce iterations from 20 to 15
- Adjusted the test case `extreme_no_wait_directory_all_three_20x` to `extreme_no_wait_directory_all_three_15x`, reflecting a change in the number of iterations to better manage system load without timing out.
- Updated assertions and file checks to align with the new iteration count, ensuring the test remains valid and accurately reflects the expected outcomes for the reduced load scenario.
2026-02-06 12:01:20 -06:00
Joe
dd3466b225 Implement unsuppress_path method in JobThreadContext for better VFS event handling
- Added a new `unsuppress_path` method to the `JobThreadContext` to remove suppressions on paths when filesystem operations fail, ensuring that future events for those paths are processed correctly.
- Updated various sections of `change_processor.rs` to call `unsuppress_path` in relevant error handling scenarios during rename and write operations, improving the accuracy of event processing.
- Enhanced the logic for managing suppressed paths, contributing to a more robust change processing mechanism in the VFS.
2026-02-06 11:23:31 -06:00
Joe
6fd9037cd1 Add stress tests for VFS and Change Processor under maximum load
- Introduced a comprehensive suite of stress tests targeting various combinations of rename, ClassName changes, and source updates to evaluate the robustness of the VFS and Change Processor.
- Implemented multiple test cases that simulate rapid, concurrent operations with zero wait times, ensuring the system can handle high-pressure scenarios without race conditions.
- Enhanced the existing test framework to support fire-and-forget operations, validating the correct handling of file existence and metadata updates after intensive changes.
- Documented the purpose and structure of the new tests to clarify their role in stress testing the two-way sync functionality.
2026-02-06 10:41:46 -06:00
Joe
9bed6dfaa2 Complete todos for stress tests and add syncback_stress fixture
- Marked all todos in the stress test plan as completed, including the implementation of timing helpers, a PRNG, and various fuzz tests for two-way sync operations.
- Introduced a new `syncback_stress` fixture with a project configuration and multiple Luau modules (Alpha, Bravo, Charlie, Delta, Echo) to support stress testing.
- Enhanced the `api.rs` file with improved path handling logic for better integration with the stress testing framework.
2026-02-05 22:36:19 -06:00
Joe
206ffd3720 Refactor path handling in tests for encoded names consistency
- Consolidated multiple lines of code into single statements for improved readability in test cases related to encoded names.
- Updated the logic in various tests to ensure consistent handling of file paths, particularly for encoded module and script names.
- Enhanced clarity in test assertions by streamlining path construction, contributing to better maintainability and understanding of the test suite.
2026-02-05 21:26:18 -06:00
Joe
5b2ea2a23f Refactor file path handling in JobThreadContext to support combined rename and ClassName changes
- Updated the logic to use the effective file path when handling script-to-script transitions, ensuring that the correct path is used even after a rename operation.
- Enhanced the test suite with a new test case for combined rename and ClassName changes, verifying that the extension change is applied correctly after a file rename.
- Improved comments for clarity on the renaming process and its implications for file handling during updates.
2026-02-05 21:19:18 -06:00
Joe
9c92204956 Add syncback_encoded_names fixture and related tests for encoded name handling
- Introduced a new fixture `syncback_encoded_names` with various scripts and models demonstrating encoded characters in names, including `What%QUESTION%Module` and `Key%COLON%Script`.
- Added tests to validate the correct handling of encoded names during directory conversions, ensuring that the filesystem names match the expected encoded formats.
- Updated `change_processor.rs` and `api.rs` to improve file handling and metadata management related to encoded names, enhancing overall robustness and consistency in operations.
2026-02-05 21:12:06 -06:00
Joe
1edca38c34 Add directory name derivation functions in ApiService for improved file handling
- Introduced `dir_name_from_script_path` and `dir_name_from_instance_path` functions to derive directory names from script and instance paths, respectively, enhancing consistency in file operations.
- Updated the `convert_standalone_script_to_directory` and related methods to utilize the new functions for accurate directory naming, ensuring proper handling of various script suffixes.
- Added comprehensive unit tests for the new functions to validate their behavior across different file types and edge cases, improving overall test coverage and reliability.
2026-02-05 21:02:39 -06:00
Joe
6ce3717663 Complete integration tests for two-way sync operations
- Marked all planned integration tests for the two-way sync functionality as completed, ensuring coverage for critical operations such as source property writes, property persistence, instance renames, and lifecycle management.
- Added a new test module specifically for two-way sync, enhancing overall test coverage and reliability of the sync system.
- Updated the change processor to handle file path overrides during instance updates, improving the accuracy of source writes and file management.
2026-02-05 19:51:14 -06:00
Joe
f4f07744c7 Refactor suppression mechanism for file watcher events
- Updated the suppression logic for file watcher events to use a HashMap instead of a HashSet, allowing for count-based suppression. Each API write increments the count, while each suppressed VFS event decrements it.
- Introduced a new method to register paths in the suppression map, improving the handling of redundant re-snapshots and WebSocket messages.
- Enhanced logging for suppressed events to provide better context during debugging.
- Adjusted related structures and methods across ChangeProcessor, ServeSession, and ApiService to accommodate the new suppression mechanism.
2026-02-05 18:46:36 -06:00
Joe
4a03d4bab5 Refactor serve tests to use recv_socket_packet for file operations
- Replaced direct file operations with the new recv_socket_packet method in multiple test functions to ensure WebSocket listeners are active before changes are made.
- This change helps avoid race conditions and improves the reliability of test outcomes by ensuring that actions triggering file changes are executed after the WebSocket connection is established.
2026-02-05 17:12:07 -06:00
Joe
b1aa2a384f Improve error handling in TestServeSession for Rojo process
- Enhanced the error handling logic when the Rojo process exits unexpectedly by capturing and displaying stderr output. This provides better context for debugging failures during test execution.
- Updated the process spawning to pipe stderr, allowing for more informative panic messages when the process fails.
2026-02-05 16:13:09 -06:00
Joe
df06fd0852 Add sleep duration in serve tests for stability
- Introduced a 100ms sleep in multiple test functions to ensure stability during file operations and socket packet retrieval.
- This change aims to mitigate timing issues that may arise from rapid file changes and improve the reliability of test outcomes.
2026-02-04 15:59:19 -06:00
Joe
fe2bf358e7 Enhance syncback logic for orphan file handling and improve test coverage
- Updated the syncback process to prevent scanning the project root folder, ensuring that unrelated files like .gitignore and README.md are not deleted.
- Introduced functionality to collect alternate file representations for paths, allowing orphan files to be identified and removed correctly.
- Enhanced the logic for handling ignored paths, particularly for directories, to improve accuracy in clean mode operations.
- Updated tests to reflect changes in orphan file handling, ensuring that only orphans within $path directories are removed, and clarified documentation regarding clean mode behavior.
2026-02-01 20:18:45 -06:00
Joe
411f79ee81 Enhance syncback functionality and improve code readability
- Refactored the syncback process to streamline directory management and improve handling of class transitions in clean mode.
- Updated the `clone_tree_filtered` function to accept a tuple of compiled Globs and original pattern strings, enhancing flexibility in ignore pattern handling.
- Simplified logic for checking if paths are inside removed directories, improving code clarity.
- Added detailed comments to clarify the purpose of new logic for handling script directory transitions, ensuring better maintainability.
- Improved logging for better traceability during syncback operations.
2026-02-01 20:10:11 -06:00
Joe
1534a54808 Refactor syncback process to improve clean mode handling and directory management
- Removed the ensure_project_paths_exist function, streamlining the syncback process by directly managing directory creation and cleanup within the syncback command.
- Enhanced error handling for sourcemap generation, logging warnings when clean mode removes directories that only contain meta files.
- Updated the logic for middleware selection in clean mode to better handle directory and script transitions based on the filesystem structure.
- Added utility functions to ensure base project directories exist before syncback, addressing the need for directory structure in clean mode.
- Improved test coverage for clean mode scenarios, ensuring that directory existence checks are performed and documented.
2026-02-01 19:55:54 -06:00
Joe
fe1ac1dd95 Fix bugs in syncback process and enhance error handling
- Resolved issues with nested orphan file removal, ensuring that both top-level and nested orphan files are correctly handled.
- Improved directory removal logic to gracefully handle cases where directories may have already been removed, preventing crashes.
- Enhanced the path scanning algorithm to accurately collect all source directories defined in the project, addressing missing paths in the syncback process.
- Updated logging to provide clearer debug information during directory scanning and error handling in the syncback operations.
2026-02-01 19:14:12 -06:00
Joe
4a66eff938 Add comprehensive syncback roundtrip and clean mode stress tests
- Completed the implementation of roundtrip tests to validate the syncback process, ensuring that building a project, syncing back, and rebuilding produces semantically identical outputs.
- Created a new utility module for roundtrip testing, including functions for running builds and asserting directory equality.
- Developed clean mode stress tests to verify that syncing back to a dirty directory yields identical results to syncing back to a fresh directory, covering various mutation scenarios.
- Updated existing test modules to incorporate new test cases and ensure consistency in handling standalone scripts transitioning to directory format when children are added.
2026-02-01 19:01:42 -06:00
Joe
b95b28179f Update LocalScript references in syncback format transition tests
- Changed references from `init.client.luau` to `init.local.luau` in the test for directory LocalScript to accurately reflect the script type being tested.
- Updated assertions to match the new file naming convention, ensuring consistency in the test's expectations for standalone files.
2026-02-01 18:35:22 -06:00
Joe
c08f701f0e Add syncback roundtrip and clean mode stress tests
- Introduced a new testing plan for comprehensive build-syncback roundtrip tests and mutation stress tests to ensure clean mode produces consistent results.
- Created utility functions for running builds and syncbacks, and for asserting directory equality.
- Updated existing tests to reflect changes in behavior when standalone scripts receive children, ensuring they convert to directory format.
- Added new test cases to validate the conversion of standalone scripts and modules when children are added, enhancing coverage of syncback functionality.
2026-02-01 18:34:02 -06:00
Joe
0eb135ee15 Enhance service visibility configuration in snapshot tests
- Added `ignoreHiddenServices: true` and a comprehensive list of `visibleServices` to multiple snapshot test files to improve service visibility management during testing.
- Refactored logging in the `diff.lua` file to streamline the trace output for hidden services, enhancing clarity in the sync process.
2026-02-01 18:28:31 -06:00
Joe
b368eb2401 Add CI pipeline documentation and enhance syncback architecture
- Introduced a new CI pipeline documentation file outlining steps for auto-fixing formatting, running linters, building, and testing.
- Documented the Rojo syncback architecture, detailing improvements to prevent duplicate file creation and format mismatches.
- Updated syncback logic to ensure existing instances are updated instead of creating duplicates, enhancing project file management.
- Added tests for encoding instances and handling nested children to improve reliability in syncback operations.
2026-02-01 18:18:31 -06:00
Joe
2b2ada219e Implement auto-download feature for syncback command
- Enhanced the `syncback` command to automatically download Roblox place files from the asset delivery API, allowing for zero-argument usage.
- Updated `Cargo.toml` to include compression features for `reqwest` and moved `tempfile` to dependencies.
- Introduced a new `download` argument in `SyncbackCommand` to specify place IDs for downloading.
- Implemented logic to resolve input file paths, prioritizing downloads when necessary.
- Added sourcemap generation after successful syncback and refreshed the git index if in a repository.
- Updated tests to validate the new download functionality and sourcemap creation.
2026-02-01 09:46:34 -06:00
Joe
8fe908740f Add statistics tracking for syncback operations
- Introduced a new `SyncbackStats` module to track and report issues during syncback, including duplicate names, unknown classes, and properties.
- Updated the `syncback_loop` and related functions to utilize the stats tracker, logging summaries of issues encountered.
- Enhanced property filtering and instance syncing to record statistics on unknown classes and properties.
- Modified tests to ensure consistent output by normalizing temporary paths in snapshots.
- Added `regex` dependency for path normalization in tests.
2026-02-01 08:29:00 -06:00
Joe
58399bb863 Implement incremental syncback mode to preserve existing file structure
- Added an `incremental` flag to the `SyncbackCommand` to allow for preserving existing file structure and middleware formats during syncback operations.
- Enhanced the `syncback_loop` function to differentiate between clean mode and incremental mode, ensuring orphaned files are removed only in clean mode.
- Updated logging to reflect the mode of operation during syncback.
- Introduced new tests to validate the behavior of clean mode versus incremental mode, ensuring proper handling of orphaned files and preservation of existing structures.
2026-02-01 08:14:20 -06:00
Joe
5f91a0777a Implement new oneshot Vfs for CLI commands and enhance JSON5 serialization
- Added `new_oneshot` method to the Vfs struct, providing a backend suitable for one-time operations without file watching.
- Updated CLI commands (`fmt_project` and `syncback`) to utilize the new oneshot Vfs, improving error handling during file operations.
- Enhanced JSON5 serialization by introducing a custom serializer that directly builds a tree structure, improving performance and maintaining sorted keys.
- Improved error logging in JSON serialization to provide more context on failures.
2026-01-31 13:06:37 -06:00
Joe
8c6aa36027 Enhance error handling in TestServeSession for interrupted reads
- Updated the error handling logic in the TestServeSession implementation to account for `std::io::ErrorKind::Interrupted`, ensuring that the session can properly retry on interrupted read operations, in addition to handling timeouts and would-block scenarios.
2026-01-30 11:40:01 -06:00
Joe
f08a4d21e7 Update file extensions from .lua to .luau and adjust README for glob ignore tests
- Changed return statements in `init.client.lua` and `init.lua` files to reflect the new .luau extension.
- Updated README in `ignore_glob_spec` to specify that glob ignores now work with .spec.luau files instead of .spec.lua.
- Modified various test files to transition from .json to .json5 format, ensuring consistency across project configurations.
2026-01-30 09:19:35 -06:00
Joe
84f76d48fc Add project configuration and test files in JSON5 format
- Introduced new configuration files for testing, including `default.project.json5`, `client_in_folder`, and others.
- Updated existing project files to use JSON5 format, ensuring consistency across configurations.
- Added new Lua scripts and model files to support various test cases.
- Adjusted snapshot files to align with the new project structure and properties.
2026-01-30 09:16:46 -06:00
Joe
d3cc11659a RunContext refactor + sync updates 2026-01-30 08:12:04 -06:00
Joe
bda2af3dfb Use msgpack for API 2026-01-27 07:13:02 -06:00
Micah
a4e80eb114 Fix tests running infinitely 2026-01-27 06:46:00 -06:00
Joe
fdd1d145cf json5 work 2026-01-22 22:19:00 -06:00