Commit graph

150 commits

Author SHA1 Message Date
Joe
4070b48308 Add audit documentation and enhance two-way sync handling
- Introduced a comprehensive audit guide for the production-grade sync feature, detailing the evaluation criteria for round-trip identity and code quality.
- Added a new plan for auditing the two-way sync Ref property feature, outlining specific tasks and expected behaviors.
- Updated `ServeSession.lua` to include the instance map in patch updates, ensuring accurate property encoding.
- Enhanced `encodeInstance.lua` to prevent encoding Ref properties during instance addition, delegating this to the patch update process.
- Implemented regression tests to highlight known limitations in Ref handling, ensuring better coverage and identification of existing bugs.

These changes improve the clarity of the sync feature's audit process and strengthen the handling of Ref properties in the system.
2026-02-14 12:08:12 -06:00
Joe
c181172460 Refactor test cases and improve snapshot handling
- Simplified the formatting of function calls in `encodePatchUpdate.spec.lua` for better readability.
- Added multiple new snapshot files to enhance end-to-end testing for reference path changes, including scenarios for adding, removing, and handling nonexistent attributes.
- Updated test cases in `apply.rs`, `compute.rs`, and `api.rs` to ensure consistent handling of reference properties and improve clarity in property updates.

These changes enhance the maintainability of the test suite and improve the robustness of snapshot testing.
2026-02-14 11:36:48 -06:00
Joe
bbd9c0e0fa test fixtures 2026-02-14 11:04:28 -06:00
Joe
2e49763b69 Update snapshot files and enhance VFS event handling
- Updated snapshot files to reflect changes in IDs for consistency across connected mode tests.
- Improved VFS event processing in ChangeProcessor to handle multiple events more efficiently, ensuring consistent message structure across platforms.
- Added merging functionality for patch sets to consolidate changes from various VFS events, enhancing synchronization reliability.

These changes improve the accuracy of test snapshots and optimize VFS event management.
2026-02-12 02:13:58 -06:00
Joe
8462c06b82 Enhance VFS initialization and error handling
- Introduced a new method `new_default_with_errors` in the VFS implementation to create a VFS instance with a critical error receiver for monitoring watcher health.
- Updated the `ServeSession` constructor to accept an optional critical error receiver, allowing for better error management during session initialization.
- Adjusted various calls to `ServeSession::new` across the codebase to accommodate the new parameter, ensuring consistent error handling.
- Improved logging in the API context and serve session files for better clarity during server operations.

These changes enhance the robustness of the VFS and session management, providing better tools for error detection and handling.
2026-02-11 20:13:02 -06:00
Joe
ec7f91017b Refactor API file handling and improve echo suppression documentation
- Updated the `write_init_meta_json` method to include a new parameter for better control over initialization behavior.
- Removed unnecessary calls to `suppress_path` and `suppress_path_remove` in various sections to streamline file handling.
- Revised comments in the `two_way_sync` tests to clarify the behavior of echo suppression during API interactions, ensuring accurate expectations for VFS event handling.

These changes enhance the clarity and efficiency of the API service and improve the documentation for testing echo suppression.
2026-02-11 19:08:26 -06:00
Joe
e2f515d8eb Update dependencies and enhance VFS staleness testing
- Updated several dependencies in Cargo.lock, including `clap`, `env_logger`, and `rand`, to their latest versions for improved functionality and security.
- Introduced a new API endpoint `/api/validate-tree` for checking tree freshness without applying corrections, enhancing the validation process.
- Added a `TreeFreshnessReport` struct to encapsulate the results of the tree freshness check, providing detailed insights into discrepancies.
- Completed multiple todos in the VFS staleness test plan, ensuring comprehensive coverage for various scenarios and improving the robustness of the testing framework.

These changes enhance the project's dependency management and testing capabilities, ensuring better alignment with the latest implementation standards.
2026-02-11 18:50:43 -06:00
Joe
f798f3f5e8 Enhance connected mode test suite and introduce VFS staleness fixtures
- Updated the connected mode test expansion plan, marking all todos as completed, including multi-step sequence tests and various phase implementations.
- Added a new plan for VFS staleness test fixtures, outlining the creation of a validation API, test helpers, and multiple staleness tests to improve detection of VFS event issues.
- Introduced new rules for the Atlas API and project file format specifications, enhancing the overall structure and documentation for better clarity and maintainability.

This update significantly improves testing coverage and addresses potential VFS synchronization issues.
2026-02-11 17:56:37 -06:00
Joe
f037ec3b7c Refactor filename handling in ApiService to streamline slugification logic
- Updated the mapping of entries to directly return lowercase slugified names, improving clarity and efficiency.
- Enhanced test stability by adding a conditional ignore for flaky tests on macOS related to rapid rename and write operations.
2026-02-11 02:06:07 -06:00
Joe
bb266789c0 Add audit plan for slugify branch pass 3 and implement critical fixes
- Introduced a comprehensive audit plan for the slugify branch, detailing findings from a holistic review of 110 files, including 1 critical bug and 2 correctness issues.
- Completed todos addressing model JSON renaming, syncback updates, and test enhancements for roundtrip fidelity.
- Added tests for reserved names and ensured correct handling of slugified instances, preserving compound extensions in model files.
- Refactored change processing logic to correctly manage the `name` field in model files, enhancing metadata handling and ensuring compliance with round-trip identity standards.
2026-02-11 01:31:09 -06:00
Joe
434a3c458a Refactor CI documentation and improve change processing logic
- Updated CI documentation to enforce running the full suite without skipping the rbx-dom section, clarifying the execution steps.
- Refactored `change_processor.rs` for improved readability by enhancing formatting and structure in name deduplication logic.
- Cleaned up imports in `project.rs` and improved formatting in `file_names.rs` to enhance code clarity and maintainability.
- Adjusted test assertions in `two_way_sync.rs` for better readability and consistency in checking file existence after renaming operations.
2026-02-10 21:44:29 -06:00
Joe
2ff5c3936c Implement fixes and enhancements from audit pass 2
- Introduced a new plan addressing critical bugs, correctness issues, and refactors identified during the audit, including improvements to metadata handling and slugification logic.
- Replaced inline removal logic in `handle_api_write` with a new `remove_instance_at_path` method to ensure correct deletion of instances.
- Updated pre-seeding logic in `dir.rs` and `project.rs` to derive deduplication keys from actual filesystem paths instead of slugified names.
- Enhanced rename handling in `change_processor.rs` to prevent collisions by deduplicating names against existing siblings.
- Added tests for various scenarios, including handling of forbidden characters and ensuring correct deduplication behavior.
- Refactored child iteration in `dir.rs` and `project.rs` to sort children alphabetically for deterministic deduplication ordering.
2026-02-10 21:35:26 -06:00
Joe
baf05dcb3d Update slugify branch audit plan and implement fixes for identified issues
- Marked all todos in the slugify branch audit plan as completed, confirming the successful verification of core invariants and sync processes.
- Introduced a new plan to address critical bugs, correctness issues, and refactors identified during the audit, including enhancements to metadata handling and slugification logic.
- Added tests to ensure proper handling of filenames with forbidden characters and deduplication of instances with colliding names.
- Refactored metadata operations to improve clarity and maintainability, consolidating logic for reading and writing the `name` field in `.meta.json5` files.
2026-02-10 20:18:33 -06:00
Joe
126c8e66a4 Update vscode-rojo submodule reference and complete migration plan for path encoding
- Updated the vscode-rojo submodule to the latest commit.
- Marked all phases of the path encoding migration plan as completed, transitioning to a slugify and deduplication approach.
- Updated documentation to reflect changes in filename handling and metadata management.
2026-02-10 03:14:18 -06:00
Joe
7172ae2d09 Enhance syncback functionality to manage .gitkeep files
- Updated the syncback loop to skip hidden files/directories, with an exception for .gitkeep, which is necessary for directory management.
- Added comprehensive tests to ensure proper cleanup of stale .gitkeep files in various scenarios, including non-empty directories and init-style projects.
2026-02-08 16:23:41 -06:00
Joe
250c7db84a Add macOS compatibility check for extreme filesystem rename tests
- Updated the `extreme_filesystem_5_file_rename_chains` test to ignore execution on macOS due to kqueue limitations with high rename pressure.
- Added documentation to clarify the reason for the macOS-specific ignore condition, ensuring better understanding of the test's behavior across different platforms.
2026-02-08 12:13:04 -06:00
Joe
6e323660e0 Update snapshot files to standardize messageCursor redaction
- Replaced specific messageCursor values with a placeholder "[message-cursor]" across multiple snapshot files to ensure consistency in test outputs.
- Adjusted the test utility to add redaction for messageCursor, addressing non-deterministic behavior in macOS environments.
2026-02-08 12:07:20 -06:00
Joe
e934912b4a Add polling functions for file existence and content checks in two-way sync tests
- Introduced `poll_file_exists`, `poll_not_exists`, and `poll_file_contains` functions to enhance the reliability of assertions in asynchronous file operations.
- Replaced direct assertions with polling functions in various test cases to ensure proper synchronization with filesystem changes.
- Improved error handling and timeout messages for better debugging during test failures.
2026-02-08 11:55:10 -06:00
Joe
50b9e18514 Refactor and rebrand Rojo to Atlas
- Renamed the project from Rojo to Atlas, updating all relevant files, scripts, and documentation to reflect the new branding.
- Bumped version to 7.7.0-rc.2 and updated protocol version from 5 to 6.
- Modified installation scripts and CI workflows to use the new binary name and paths.
- Adjusted plugin settings and internal identifiers to prevent conflicts with the original Rojo.
- Enhanced session management and API response handling to support the new server fork identifier.
- Updated project templates and README files to guide users on using Atlas.
2026-02-07 13:21:52 -06:00
Joe
aeec3a962b Ignore certain tests on macOS due to kqueue event overload
- Added `#[cfg_attr(target_os = "macos", ignore)]` to three tests to prevent them from running on macOS, where rapid rename operations can overwhelm the ChangeProcessor.
- This change aims to improve test reliability across different operating systems.
2026-02-06 23:26:41 -06:00
Joe
4f3e66bec9 Add tests for PatchSet and ServeSession functionality
- Introduced comprehensive tests for the `PatchSet` module, covering methods such as `hasRemoves`, `hasAdditions`, `hasUpdates`, and various ID and instance checks.
- Added a new test suite for `ServeSession`, focusing on WebSocket message handling and confirmation behavior, ensuring correct merging and application of patches.
- Implemented edge case tests in `ChangeBatcher` to verify data integrity during rapid changes and instance removals.
- Enhanced `applyPatch` tests to handle scenarios involving missing instances and ignored class names, ensuring robust patch application logic.
2026-02-06 22:23:14 -06:00
Joe
fa5393121e Enhance CloneCommand and Syncback functionality
- Added a check in `CloneCommand` to prevent cloning into an existing directory, providing a clearer error message for users.
- Updated the `SyncbackCommand` to support a configurable working directory for resolving relative paths, improving flexibility in project structure.
- Refactored the commit message in the `CloneCommand` to accurately reflect the syncback operation.
- Improved path resolution logic in `SyncbackCommand` to handle both absolute and relative paths more effectively.
2026-02-06 19:29:37 -06:00
Joe
5442c02f67 Enhance redaction handling and update snapshot references
- Added support for redacting tagged values in the `RedactionMap` implementation, ensuring that nested tagged values are processed correctly.
- Updated multiple snapshot files to replace specific reference IDs with deterministic `id-N` values, improving consistency across tests and snapshots.
- Refactored internment logic in `internable.rs` to include variant references, ensuring proper redaction of property values in snapshots.
2026-02-06 19:20:16 -06:00
Joe
4ee1e79a90 Refactor FSEvents handling in serve_util.rs for improved reliability
- Removed unnecessary sleep duration for FSEvents on macOS, reducing it from 500ms to 100ms to enhance responsiveness while still allowing time for event processing.
- Updated comments to clarify the rationale behind the timing adjustments and noted the need for a more robust solution in the future.
2026-02-06 13:10:00 -06:00
Joe
67537070db Refactor path suppression handling in ChangeProcessor and ApiService
- Updated the `suppressed_paths` structure in `ChangeProcessor` and `ApiService` to store counts for both remove and create/write events, enhancing the accuracy of event suppression.
- Refactored methods for suppressing and unsuppressing paths to accommodate the new structure, ensuring that the system correctly manages VFS events based on the type of operation.
- Improved documentation and comments to clarify the purpose and behavior of the suppression logic, contributing to better maintainability and understanding of the code.
2026-02-06 13:01:07 -06:00
Joe
84771840e1 Refactor port number generation in tests for improved reliability
- Replaced the static port number generation with a method that binds a temporary TcpListener to an ephemeral port assigned by the OS, enhancing the reliability of port allocation during tests.
- Updated documentation to clarify the new approach for obtaining a free port, ensuring better understanding and maintainability of the test code.
2026-02-06 12:24:50 -06:00
Joe
acb3b18fa1 Refactor test code for consistency and readability
- Streamlined multiple test functions by consolidating calls to `send_update_no_wait` and `get_format_transitions_instance` into single lines, enhancing code clarity.
- Improved consistency in handling session and instance IDs across various rename scenarios, contributing to better maintainability of the test suite.
2026-02-06 12:24:34 -06:00
Joe
4bfb52d299 Refactor rename handling in two-way sync tests for improved stability
- Updated multiple test functions to retrieve session IDs and instance IDs before loops, enhancing the stability of rename operations during rapid updates.
- Removed redundant calls to fetch instance information within loops, streamlining the code and reducing potential race conditions.
- Ensured consistent handling of session IDs across various rename scenarios, contributing to more reliable test outcomes in high-load situations.
2026-02-06 12:18:50 -06:00
Joe
a6fb256192 Update extreme_no_wait_directory_all_three test to reduce iterations from 20 to 15
- Adjusted the test case `extreme_no_wait_directory_all_three_20x` to `extreme_no_wait_directory_all_three_15x`, reflecting a change in the number of iterations to better manage system load without timing out.
- Updated assertions and file checks to align with the new iteration count, ensuring the test remains valid and accurately reflects the expected outcomes for the reduced load scenario.
2026-02-06 12:01:20 -06:00
Joe
dd3466b225 Implement unsuppress_path method in JobThreadContext for better VFS event handling
- Added a new `unsuppress_path` method to the `JobThreadContext` to remove suppressions on paths when filesystem operations fail, ensuring that future events for those paths are processed correctly.
- Updated various sections of `change_processor.rs` to call `unsuppress_path` in relevant error handling scenarios during rename and write operations, improving the accuracy of event processing.
- Enhanced the logic for managing suppressed paths, contributing to a more robust change processing mechanism in the VFS.
2026-02-06 11:23:31 -06:00
Joe
6fd9037cd1 Add stress tests for VFS and Change Processor under maximum load
- Introduced a comprehensive suite of stress tests targeting various combinations of rename, ClassName changes, and source updates to evaluate the robustness of the VFS and Change Processor.
- Implemented multiple test cases that simulate rapid, concurrent operations with zero wait times, ensuring the system can handle high-pressure scenarios without race conditions.
- Enhanced the existing test framework to support fire-and-forget operations, validating the correct handling of file existence and metadata updates after intensive changes.
- Documented the purpose and structure of the new tests to clarify their role in stress testing the two-way sync functionality.
2026-02-06 10:41:46 -06:00
Joe
9bed6dfaa2 Complete todos for stress tests and add syncback_stress fixture
- Marked all todos in the stress test plan as completed, including the implementation of timing helpers, a PRNG, and various fuzz tests for two-way sync operations.
- Introduced a new `syncback_stress` fixture with a project configuration and multiple Luau modules (Alpha, Bravo, Charlie, Delta, Echo) to support stress testing.
- Enhanced the `api.rs` file with improved path handling logic for better integration with the stress testing framework.
2026-02-05 22:36:19 -06:00
Joe
206ffd3720 Refactor path handling in tests for encoded names consistency
- Consolidated multiple lines of code into single statements for improved readability in test cases related to encoded names.
- Updated the logic in various tests to ensure consistent handling of file paths, particularly for encoded module and script names.
- Enhanced clarity in test assertions by streamlining path construction, contributing to better maintainability and understanding of the test suite.
2026-02-05 21:26:18 -06:00
Joe
5b2ea2a23f Refactor file path handling in JobThreadContext to support combined rename and ClassName changes
- Updated the logic to use the effective file path when handling script-to-script transitions, ensuring that the correct path is used even after a rename operation.
- Enhanced the test suite with a new test case for combined rename and ClassName changes, verifying that the extension change is applied correctly after a file rename.
- Improved comments for clarity on the renaming process and its implications for file handling during updates.
2026-02-05 21:19:18 -06:00
Joe
9c92204956 Add syncback_encoded_names fixture and related tests for encoded name handling
- Introduced a new fixture `syncback_encoded_names` with various scripts and models demonstrating encoded characters in names, including `What%QUESTION%Module` and `Key%COLON%Script`.
- Added tests to validate the correct handling of encoded names during directory conversions, ensuring that the filesystem names match the expected encoded formats.
- Updated `change_processor.rs` and `api.rs` to improve file handling and metadata management related to encoded names, enhancing overall robustness and consistency in operations.
2026-02-05 21:12:06 -06:00
Joe
1edca38c34 Add directory name derivation functions in ApiService for improved file handling
- Introduced `dir_name_from_script_path` and `dir_name_from_instance_path` functions to derive directory names from script and instance paths, respectively, enhancing consistency in file operations.
- Updated the `convert_standalone_script_to_directory` and related methods to utilize the new functions for accurate directory naming, ensuring proper handling of various script suffixes.
- Added comprehensive unit tests for the new functions to validate their behavior across different file types and edge cases, improving overall test coverage and reliability.
2026-02-05 21:02:39 -06:00
Joe
6ce3717663 Complete integration tests for two-way sync operations
- Marked all planned integration tests for the two-way sync functionality as completed, ensuring coverage for critical operations such as source property writes, property persistence, instance renames, and lifecycle management.
- Added a new test module specifically for two-way sync, enhancing overall test coverage and reliability of the sync system.
- Updated the change processor to handle file path overrides during instance updates, improving the accuracy of source writes and file management.
2026-02-05 19:51:14 -06:00
Joe
f4f07744c7 Refactor suppression mechanism for file watcher events
- Updated the suppression logic for file watcher events to use a HashMap instead of a HashSet, allowing for count-based suppression. Each API write increments the count, while each suppressed VFS event decrements it.
- Introduced a new method to register paths in the suppression map, improving the handling of redundant re-snapshots and WebSocket messages.
- Enhanced logging for suppressed events to provide better context during debugging.
- Adjusted related structures and methods across ChangeProcessor, ServeSession, and ApiService to accommodate the new suppression mechanism.
2026-02-05 18:46:36 -06:00
Joe
4a03d4bab5 Refactor serve tests to use recv_socket_packet for file operations
- Replaced direct file operations with the new recv_socket_packet method in multiple test functions to ensure WebSocket listeners are active before changes are made.
- This change helps avoid race conditions and improves the reliability of test outcomes by ensuring that actions triggering file changes are executed after the WebSocket connection is established.
2026-02-05 17:12:07 -06:00
Joe
b1aa2a384f Improve error handling in TestServeSession for Rojo process
- Enhanced the error handling logic when the Rojo process exits unexpectedly by capturing and displaying stderr output. This provides better context for debugging failures during test execution.
- Updated the process spawning to pipe stderr, allowing for more informative panic messages when the process fails.
2026-02-05 16:13:09 -06:00
Joe
df06fd0852 Add sleep duration in serve tests for stability
- Introduced a 100ms sleep in multiple test functions to ensure stability during file operations and socket packet retrieval.
- This change aims to mitigate timing issues that may arise from rapid file changes and improve the reliability of test outcomes.
2026-02-04 15:59:19 -06:00
Joe
fe2bf358e7 Enhance syncback logic for orphan file handling and improve test coverage
- Updated the syncback process to prevent scanning the project root folder, ensuring that unrelated files like .gitignore and README.md are not deleted.
- Introduced functionality to collect alternate file representations for paths, allowing orphan files to be identified and removed correctly.
- Enhanced the logic for handling ignored paths, particularly for directories, to improve accuracy in clean mode operations.
- Updated tests to reflect changes in orphan file handling, ensuring that only orphans within $path directories are removed, and clarified documentation regarding clean mode behavior.
2026-02-01 20:18:45 -06:00
Joe
411f79ee81 Enhance syncback functionality and improve code readability
- Refactored the syncback process to streamline directory management and improve handling of class transitions in clean mode.
- Updated the `clone_tree_filtered` function to accept a tuple of compiled Globs and original pattern strings, enhancing flexibility in ignore pattern handling.
- Simplified logic for checking if paths are inside removed directories, improving code clarity.
- Added detailed comments to clarify the purpose of new logic for handling script directory transitions, ensuring better maintainability.
- Improved logging for better traceability during syncback operations.
2026-02-01 20:10:11 -06:00
Joe
1534a54808 Refactor syncback process to improve clean mode handling and directory management
- Removed the ensure_project_paths_exist function, streamlining the syncback process by directly managing directory creation and cleanup within the syncback command.
- Enhanced error handling for sourcemap generation, logging warnings when clean mode removes directories that only contain meta files.
- Updated the logic for middleware selection in clean mode to better handle directory and script transitions based on the filesystem structure.
- Added utility functions to ensure base project directories exist before syncback, addressing the need for directory structure in clean mode.
- Improved test coverage for clean mode scenarios, ensuring that directory existence checks are performed and documented.
2026-02-01 19:55:54 -06:00
Joe
fe1ac1dd95 Fix bugs in syncback process and enhance error handling
- Resolved issues with nested orphan file removal, ensuring that both top-level and nested orphan files are correctly handled.
- Improved directory removal logic to gracefully handle cases where directories may have already been removed, preventing crashes.
- Enhanced the path scanning algorithm to accurately collect all source directories defined in the project, addressing missing paths in the syncback process.
- Updated logging to provide clearer debug information during directory scanning and error handling in the syncback operations.
2026-02-01 19:14:12 -06:00
Joe
4a66eff938 Add comprehensive syncback roundtrip and clean mode stress tests
- Completed the implementation of roundtrip tests to validate the syncback process, ensuring that building a project, syncing back, and rebuilding produces semantically identical outputs.
- Created a new utility module for roundtrip testing, including functions for running builds and asserting directory equality.
- Developed clean mode stress tests to verify that syncing back to a dirty directory yields identical results to syncing back to a fresh directory, covering various mutation scenarios.
- Updated existing test modules to incorporate new test cases and ensure consistency in handling standalone scripts transitioning to directory format when children are added.
2026-02-01 19:01:42 -06:00
Joe
b95b28179f Update LocalScript references in syncback format transition tests
- Changed references from `init.client.luau` to `init.local.luau` in the test for directory LocalScript to accurately reflect the script type being tested.
- Updated assertions to match the new file naming convention, ensuring consistency in the test's expectations for standalone files.
2026-02-01 18:35:22 -06:00
Joe
c08f701f0e Add syncback roundtrip and clean mode stress tests
- Introduced a new testing plan for comprehensive build-syncback roundtrip tests and mutation stress tests to ensure clean mode produces consistent results.
- Created utility functions for running builds and syncbacks, and for asserting directory equality.
- Updated existing tests to reflect changes in behavior when standalone scripts receive children, ensuring they convert to directory format.
- Added new test cases to validate the conversion of standalone scripts and modules when children are added, enhancing coverage of syncback functionality.
2026-02-01 18:34:02 -06:00
Joe
0eb135ee15 Enhance service visibility configuration in snapshot tests
- Added `ignoreHiddenServices: true` and a comprehensive list of `visibleServices` to multiple snapshot test files to improve service visibility management during testing.
- Refactored logging in the `diff.lua` file to streamline the trace output for hidden services, enhancing clarity in the sync process.
2026-02-01 18:28:31 -06:00
Joe
b368eb2401 Add CI pipeline documentation and enhance syncback architecture
- Introduced a new CI pipeline documentation file outlining steps for auto-fixing formatting, running linters, building, and testing.
- Documented the Rojo syncback architecture, detailing improvements to prevent duplicate file creation and format mismatches.
- Updated syncback logic to ensure existing instances are updated instead of creating duplicates, enhancing project file management.
- Added tests for encoding instances and handling nested children to improve reliability in syncback operations.
2026-02-01 18:18:31 -06:00