Commit graph

61 commits

Author SHA1 Message Date
Joe
3fd50df0b2 Update logging and enhance .gitignore for Atlas directory
- Introduced a logging system using `tracing` and related crates to improve log management and output formatting.
- Updated `.gitignore` to include the `.atlas/` directory, ensuring that log files and other generated content are not tracked by Git.
- Enhanced project templates to include the new `.gitignore` settings for Atlas in various project types.

These changes aim to improve logging capabilities and maintain a cleaner repository by ignoring unnecessary files.
2026-03-01 13:16:00 -06:00
Joe
0a359ccaf1 Update dependencies and refactor git integration
- Added new dependencies in `Cargo.lock` and updated `Cargo.toml` to include `gix` for improved git functionality.
- Refactored git-related functions to utilize the `gix` library, enhancing performance and reliability in handling git operations.
- Simplified the process of initializing git repositories and committing changes within the CLI commands.
- Updated tests to reflect changes in git integration and ensure proper functionality.

These changes aim to modernize the git handling in the project and improve overall code quality.
2026-03-01 07:49:03 -06:00
Joe
eb114ac76e Refactor snapshot attribute handling in patch computation
- Removed redundant Rojo reference attributes from snapshot comparisons to prevent spurious diffs during VFS events.
- Updated the logic for interned instance additions to ensure deterministic ordering by redacted parent and name.

These changes aim to improve the accuracy of snapshot comparisons and enhance the handling of interned instances.
2026-02-19 07:33:32 -06:00
Joe
d252b13820 Increase backoff duration and maximum retry attempts in server initialization
- Updated the `BASE_DURATION_MS` constant from 30.0 to 100.0 to allow for longer wait times during server startup.
- Increased the `MAX_TRIES` constant from 5 to 10 to provide more attempts for the server to come online before failing.

These changes aim to improve the reliability of server initialization by allowing for more extended backoff periods and retries.
2026-02-19 05:35:13 -06:00
Joe
d9fa62c88e Add server-side and test-side port retry mechanisms to handle race conditions
- Introduced a new plan to address port-in-use race conditions in both server and test environments.
- Implemented a retry mechanism with exponential backoff for binding to known ports in `LiveServer::start()`.
- Added a `respawn_with_new_port()` method in `TestServeSession` to handle transient port conflicts during tests.
- Modified `wait_to_come_online()` to retry with a new port if the server process exits due to `AddrInUse`.
- Updated `fresh_rebuild_read()` to include port retry logic, enhancing robustness against port conflicts.

These changes aim to improve the reliability of server and test interactions by mitigating race conditions related to port usage.
2026-02-19 01:34:53 -06:00
Joe
0450dabfce Refactor encoding logic in ChangeBatcher for improved readability and consistency
- Simplified the `isReadable` condition in `encodeService.lua` for better clarity.
- Updated float formatting functions in `json.rs` to enhance precision and consistency in numeric representation.
- Refactored handling of `AmbiguousValue` in `resolution.rs` to eliminate unnecessary type casting, improving data integrity.
- Cleaned up formatting in `serve.rs` and test files for better code readability.

These changes enhance the clarity and efficiency of the encoding and serialization processes, contributing to overall performance improvements.
2026-02-18 09:47:50 -06:00
Joe
39c8bd41f4 Enhance live syncback process with improved reference handling and float precision
- Implemented ObjectValue carriers for service-level references, allowing for accurate serialization of non-direct child references during syncback.
- Updated `encodeService` to include explicit handling of attributes and tags, ensuring they are properly encoded and preventing data loss.
- Refactored `build_dom_from_chunks` to manage ObjectValue carriers and resolve references correctly, enhancing data integrity.
- Truncated float values to f32 precision in `UnresolvedValue::from_variant`, aligning with rbxl storage requirements and improving consistency.

These changes address critical issues in the live syncback process, enhancing reliability and data accuracy.
2026-02-18 09:11:59 -06:00
Joe
a75a1443c0 Enhance syncback process by integrating serialization of service children
- Introduced `SerializationService` to serialize child instances during syncback, improving data handling.
- Updated `performSyncback` to include serialized children data in the syncback request.
- Refactored `ServiceChunk` structure to store child count instead of serialized data, optimizing memory usage.
- Adjusted related functions and tests to accommodate the new data structure, ensuring consistency and accuracy in syncback operations.

These changes significantly enhance the efficiency and clarity of the syncback process, allowing for better management of service data.
2026-02-18 08:21:25 -06:00
Joe
e0fd9077e8 Refactor service encoding to streamline properties handling
- Removed the encoding of attributes and tags from the `encodeService` function, consolidating their inclusion into the properties map as `Variant::Attributes(...)` and `Variant::Tags(...)`.
- Updated the `ServiceChunk` struct to reflect these changes, ensuring a more efficient serialization process during syncback operations.
- Adjusted related tests to align with the new structure, enhancing overall data management and consistency in service encoding.

These modifications improve the clarity and performance of the service encoding process.
2026-02-18 08:00:49 -06:00
Joe
72ad84b5a7 Implement live syncback service properties encoding and enhance syncback functionality
- Introduced a new `encodeService` helper in the plugin to serialize service properties, attributes, tags, and references alongside children data.
- Updated the `performSyncback` function to utilize the new encoding method, ensuring all services are processed without child-only restrictions.
- Extended the `ServiceChunk` struct to include fields for properties, attributes, tags, and references, improving data handling during syncback.
- Modified the `build_dom_from_chunks` function to apply service properties and resolve references when reconstructing the service instances.
- Added comprehensive tests to validate the new service properties handling, ensuring accurate synchronization and data integrity.

These enhancements significantly improve the live syncback process, allowing for better service data management and consistency.
2026-02-18 07:22:29 -06:00
Joe
b687c8862d Enhance matching module with reference identity and caching improvements
- Introduced a new `RefIdentity` type to store reference details (name and className) in the matching process.
- Updated `VCache` and `SCache` structures to include references, improving the handling of reference comparisons during matching.
- Modified `cacheVirtual` and `cacheStudio` functions to support reference caching, enhancing performance and accuracy in matching.
- Improved `countOwnDiffs` function to account for reference mismatches, ensuring more precise cost calculations.
- Added comprehensive tests to validate the new reference handling and caching mechanisms.

These changes significantly enhance the efficiency and accuracy of the matching process in the application.
2026-02-18 06:16:33 -06:00
Joe
93ea0bdef6 Refactor syncback process and improve code readability
- Reformatted the `SYNCBACK_SERVICES` list for better readability by aligning entries vertically.
- Simplified the `performSyncback` function by removing unnecessary Promise wrapping, enhancing clarity and maintainability.
- Updated the `NotConnectedPage` component to improve the structure of button elements and their associated tooltips.
- Refactored the `hydrate` and `matching` modules to streamline function calls and improve code organization.

These changes enhance the overall readability and maintainability of the syncback process and related components.
2026-02-18 05:58:17 -06:00
Joe
b39c01e5e9 Add matching session caching and native codegen improvements
- Introduced a new matching session caching mechanism in the Lua matching module to enhance performance with native code generation.
- Rewrote the Lua matching algorithm to utilize session-based caching, reducing redundant computations and improving efficiency.
- Updated Rust implementations to include session caching, ensuring consistency across both Lua and Rust.
- Enhanced the `hydrate` function to support session management during instance hydration.
- Added comprehensive tests to validate the new caching mechanism and its integration with existing functionality.

These changes significantly improve the performance and maintainability of the matching process in the application.
2026-02-18 04:55:31 -06:00
Joe
307ac81aec Implement live syncback feature and enhance API for service data transfer
- Added a new `performSyncback` function in the `App` module to handle the synchronization of service data from the game to the server.
- Introduced a `SyncbackConfirm` UI component to confirm syncback actions from the user.
- Updated the API to include a new `/api/syncback` endpoint for handling syncback requests, validating protocol and server versions.
- Enhanced the `SyncbackSignal` mechanism to facilitate communication between the server and the syncback process.
- Added tests to ensure the functionality of the live syncback feature and its integration with the CLI.

These changes improve the overall synchronization capabilities of the application, allowing for real-time updates and better project management.
2026-02-18 04:36:57 -06:00
Joe
6c0098d839 Add git metadata handling and integration tests for sync direction defaults
- Introduced new tests for git-based sync direction defaults, verifying the presence of git metadata and correct mappings of changed files to instance references.
- Enhanced the `compute_git_metadata` function to include canonical paths for repository roots.
- Added multiple test cases to ensure accurate behavior of `changedIds` and `scriptCommittedHashes` based on git status.
- Created a new project structure for testing, including various script and model files to facilitate comprehensive testing of git integration.

These changes improve the robustness of the git synchronization features and ensure reliable behavior in various scenarios.
2026-02-17 20:46:14 -06:00
Joe
df22500102 Refactor code for improved clarity and maintainability
- Added `#[allow(clippy::too_many_arguments)]` annotations in `change_processor.rs` and `matching.rs` to suppress warnings for functions with multiple parameters.
- Updated `serve_session.rs` to include `clippy::type_complexity` in the `#[allow(dead_code)]` annotation for better code quality.
- Simplified array initialization in `file_names.rs` for consistency.
- Enhanced texture filtering logic in `matching_fixtures.rs` by replacing `map_or` with `is_some_and` for improved readability.

These changes contribute to a cleaner codebase and address static analysis warnings, ensuring better maintainability.
2026-02-17 20:04:52 -06:00
Joe
a6341783ef Update Luau LSP and project configurations
- Upgraded `luau-lsp` dependency to version 1.62.0 for improved features and bug fixes.
- Modified sourcemap generator command in VSCode settings for better output handling.
- Enabled `fillCallArguments` in project template settings to enhance code completion.
- Added new configuration options for fragment autocomplete and quote conversion in project template settings.
- Introduced a `sourcemap` flag in `SyncbackCommand` to allow optional sourcemap generation during syncback operations.

These changes enhance the development experience and improve the functionality of the Luau language server integration.
2026-02-15 16:19:35 -06:00
Joe
37d2ca8986 Update settle duration in TestServeSession to improve message handling
- Adjusted the settle duration from 300ms to 100ms to ensure it is shorter than the 200ms reconciliation timer. This change prevents capturing tree-correction messages as part of the same batch, enhancing the accuracy of message processing in the test environment.

This update improves the reliability of message handling during test sessions.
2026-02-15 02:18:30 -06:00
Joe
9a70bf65e9 Update snapshot files to standardize messageCursor values
- Replaced numeric messageCursor values with a placeholder "[message-cursor]" across multiple snapshot files in the connected mode tests.
- This change ensures consistency in the handling of message cursors, facilitating easier updates and maintenance of test snapshots.

These updates improve the clarity and reliability of the snapshot files used in end-to-end testing.
2026-02-15 01:33:31 -06:00
Joe
8462c06b82 Enhance VFS initialization and error handling
- Introduced a new method `new_default_with_errors` in the VFS implementation to create a VFS instance with a critical error receiver for monitoring watcher health.
- Updated the `ServeSession` constructor to accept an optional critical error receiver, allowing for better error management during session initialization.
- Adjusted various calls to `ServeSession::new` across the codebase to accommodate the new parameter, ensuring consistent error handling.
- Improved logging in the API context and serve session files for better clarity during server operations.

These changes enhance the robustness of the VFS and session management, providing better tools for error detection and handling.
2026-02-11 20:13:02 -06:00
Joe
e2f515d8eb Update dependencies and enhance VFS staleness testing
- Updated several dependencies in Cargo.lock, including `clap`, `env_logger`, and `rand`, to their latest versions for improved functionality and security.
- Introduced a new API endpoint `/api/validate-tree` for checking tree freshness without applying corrections, enhancing the validation process.
- Added a `TreeFreshnessReport` struct to encapsulate the results of the tree freshness check, providing detailed insights into discrepancies.
- Completed multiple todos in the VFS staleness test plan, ensuring comprehensive coverage for various scenarios and improving the robustness of the testing framework.

These changes enhance the project's dependency management and testing capabilities, ensuring better alignment with the latest implementation standards.
2026-02-11 18:50:43 -06:00
Joe
f798f3f5e8 Enhance connected mode test suite and introduce VFS staleness fixtures
- Updated the connected mode test expansion plan, marking all todos as completed, including multi-step sequence tests and various phase implementations.
- Added a new plan for VFS staleness test fixtures, outlining the creation of a validation API, test helpers, and multiple staleness tests to improve detection of VFS event issues.
- Introduced new rules for the Atlas API and project file format specifications, enhancing the overall structure and documentation for better clarity and maintainability.

This update significantly improves testing coverage and addresses potential VFS synchronization issues.
2026-02-11 17:56:37 -06:00
Joe
6e323660e0 Update snapshot files to standardize messageCursor redaction
- Replaced specific messageCursor values with a placeholder "[message-cursor]" across multiple snapshot files to ensure consistency in test outputs.
- Adjusted the test utility to add redaction for messageCursor, addressing non-deterministic behavior in macOS environments.
2026-02-08 12:07:20 -06:00
Joe
50b9e18514 Refactor and rebrand Rojo to Atlas
- Renamed the project from Rojo to Atlas, updating all relevant files, scripts, and documentation to reflect the new branding.
- Bumped version to 7.7.0-rc.2 and updated protocol version from 5 to 6.
- Modified installation scripts and CI workflows to use the new binary name and paths.
- Adjusted plugin settings and internal identifiers to prevent conflicts with the original Rojo.
- Enhanced session management and API response handling to support the new server fork identifier.
- Updated project templates and README files to guide users on using Atlas.
2026-02-07 13:21:52 -06:00
Joe
fa5393121e Enhance CloneCommand and Syncback functionality
- Added a check in `CloneCommand` to prevent cloning into an existing directory, providing a clearer error message for users.
- Updated the `SyncbackCommand` to support a configurable working directory for resolving relative paths, improving flexibility in project structure.
- Refactored the commit message in the `CloneCommand` to accurately reflect the syncback operation.
- Improved path resolution logic in `SyncbackCommand` to handle both absolute and relative paths more effectively.
2026-02-06 19:29:37 -06:00
Joe
5442c02f67 Enhance redaction handling and update snapshot references
- Added support for redacting tagged values in the `RedactionMap` implementation, ensuring that nested tagged values are processed correctly.
- Updated multiple snapshot files to replace specific reference IDs with deterministic `id-N` values, improving consistency across tests and snapshots.
- Refactored internment logic in `internable.rs` to include variant references, ensuring proper redaction of property values in snapshots.
2026-02-06 19:20:16 -06:00
Joe
4ee1e79a90 Refactor FSEvents handling in serve_util.rs for improved reliability
- Removed unnecessary sleep duration for FSEvents on macOS, reducing it from 500ms to 100ms to enhance responsiveness while still allowing time for event processing.
- Updated comments to clarify the rationale behind the timing adjustments and noted the need for a more robust solution in the future.
2026-02-06 13:10:00 -06:00
Joe
67537070db Refactor path suppression handling in ChangeProcessor and ApiService
- Updated the `suppressed_paths` structure in `ChangeProcessor` and `ApiService` to store counts for both remove and create/write events, enhancing the accuracy of event suppression.
- Refactored methods for suppressing and unsuppressing paths to accommodate the new structure, ensuring that the system correctly manages VFS events based on the type of operation.
- Improved documentation and comments to clarify the purpose and behavior of the suppression logic, contributing to better maintainability and understanding of the code.
2026-02-06 13:01:07 -06:00
Joe
84771840e1 Refactor port number generation in tests for improved reliability
- Replaced the static port number generation with a method that binds a temporary TcpListener to an ephemeral port assigned by the OS, enhancing the reliability of port allocation during tests.
- Updated documentation to clarify the new approach for obtaining a free port, ensuring better understanding and maintainability of the test code.
2026-02-06 12:24:50 -06:00
Joe
f4f07744c7 Refactor suppression mechanism for file watcher events
- Updated the suppression logic for file watcher events to use a HashMap instead of a HashSet, allowing for count-based suppression. Each API write increments the count, while each suppressed VFS event decrements it.
- Introduced a new method to register paths in the suppression map, improving the handling of redundant re-snapshots and WebSocket messages.
- Enhanced logging for suppressed events to provide better context during debugging.
- Adjusted related structures and methods across ChangeProcessor, ServeSession, and ApiService to accommodate the new suppression mechanism.
2026-02-05 18:46:36 -06:00
Joe
4a03d4bab5 Refactor serve tests to use recv_socket_packet for file operations
- Replaced direct file operations with the new recv_socket_packet method in multiple test functions to ensure WebSocket listeners are active before changes are made.
- This change helps avoid race conditions and improves the reliability of test outcomes by ensuring that actions triggering file changes are executed after the WebSocket connection is established.
2026-02-05 17:12:07 -06:00
Joe
b1aa2a384f Improve error handling in TestServeSession for Rojo process
- Enhanced the error handling logic when the Rojo process exits unexpectedly by capturing and displaying stderr output. This provides better context for debugging failures during test execution.
- Updated the process spawning to pipe stderr, allowing for more informative panic messages when the process fails.
2026-02-05 16:13:09 -06:00
Joe
411f79ee81 Enhance syncback functionality and improve code readability
- Refactored the syncback process to streamline directory management and improve handling of class transitions in clean mode.
- Updated the `clone_tree_filtered` function to accept a tuple of compiled Globs and original pattern strings, enhancing flexibility in ignore pattern handling.
- Simplified logic for checking if paths are inside removed directories, improving code clarity.
- Added detailed comments to clarify the purpose of new logic for handling script directory transitions, ensuring better maintainability.
- Improved logging for better traceability during syncback operations.
2026-02-01 20:10:11 -06:00
Joe
1534a54808 Refactor syncback process to improve clean mode handling and directory management
- Removed the ensure_project_paths_exist function, streamlining the syncback process by directly managing directory creation and cleanup within the syncback command.
- Enhanced error handling for sourcemap generation, logging warnings when clean mode removes directories that only contain meta files.
- Updated the logic for middleware selection in clean mode to better handle directory and script transitions based on the filesystem structure.
- Added utility functions to ensure base project directories exist before syncback, addressing the need for directory structure in clean mode.
- Improved test coverage for clean mode scenarios, ensuring that directory existence checks are performed and documented.
2026-02-01 19:55:54 -06:00
Joe
fe1ac1dd95 Fix bugs in syncback process and enhance error handling
- Resolved issues with nested orphan file removal, ensuring that both top-level and nested orphan files are correctly handled.
- Improved directory removal logic to gracefully handle cases where directories may have already been removed, preventing crashes.
- Enhanced the path scanning algorithm to accurately collect all source directories defined in the project, addressing missing paths in the syncback process.
- Updated logging to provide clearer debug information during directory scanning and error handling in the syncback operations.
2026-02-01 19:14:12 -06:00
Joe
4a66eff938 Add comprehensive syncback roundtrip and clean mode stress tests
- Completed the implementation of roundtrip tests to validate the syncback process, ensuring that building a project, syncing back, and rebuilding produces semantically identical outputs.
- Created a new utility module for roundtrip testing, including functions for running builds and asserting directory equality.
- Developed clean mode stress tests to verify that syncing back to a dirty directory yields identical results to syncing back to a fresh directory, covering various mutation scenarios.
- Updated existing test modules to incorporate new test cases and ensure consistency in handling standalone scripts transitioning to directory format when children are added.
2026-02-01 19:01:42 -06:00
Joe
b368eb2401 Add CI pipeline documentation and enhance syncback architecture
- Introduced a new CI pipeline documentation file outlining steps for auto-fixing formatting, running linters, building, and testing.
- Documented the Rojo syncback architecture, detailing improvements to prevent duplicate file creation and format mismatches.
- Updated syncback logic to ensure existing instances are updated instead of creating duplicates, enhancing project file management.
- Added tests for encoding instances and handling nested children to improve reliability in syncback operations.
2026-02-01 18:18:31 -06:00
Joe
2b2ada219e Implement auto-download feature for syncback command
- Enhanced the `syncback` command to automatically download Roblox place files from the asset delivery API, allowing for zero-argument usage.
- Updated `Cargo.toml` to include compression features for `reqwest` and moved `tempfile` to dependencies.
- Introduced a new `download` argument in `SyncbackCommand` to specify place IDs for downloading.
- Implemented logic to resolve input file paths, prioritizing downloads when necessary.
- Added sourcemap generation after successful syncback and refreshed the git index if in a repository.
- Updated tests to validate the new download functionality and sourcemap creation.
2026-02-01 09:46:34 -06:00
Joe
8fe908740f Add statistics tracking for syncback operations
- Introduced a new `SyncbackStats` module to track and report issues during syncback, including duplicate names, unknown classes, and properties.
- Updated the `syncback_loop` and related functions to utilize the stats tracker, logging summaries of issues encountered.
- Enhanced property filtering and instance syncing to record statistics on unknown classes and properties.
- Modified tests to ensure consistent output by normalizing temporary paths in snapshots.
- Added `regex` dependency for path normalization in tests.
2026-02-01 08:29:00 -06:00
Joe
58399bb863 Implement incremental syncback mode to preserve existing file structure
- Added an `incremental` flag to the `SyncbackCommand` to allow for preserving existing file structure and middleware formats during syncback operations.
- Enhanced the `syncback_loop` function to differentiate between clean mode and incremental mode, ensuring orphaned files are removed only in clean mode.
- Updated logging to reflect the mode of operation during syncback.
- Introduced new tests to validate the behavior of clean mode versus incremental mode, ensuring proper handling of orphaned files and preservation of existing structures.
2026-02-01 08:14:20 -06:00
Joe
5f91a0777a Implement new oneshot Vfs for CLI commands and enhance JSON5 serialization
- Added `new_oneshot` method to the Vfs struct, providing a backend suitable for one-time operations without file watching.
- Updated CLI commands (`fmt_project` and `syncback`) to utilize the new oneshot Vfs, improving error handling during file operations.
- Enhanced JSON5 serialization by introducing a custom serializer that directly builds a tree structure, improving performance and maintaining sorted keys.
- Improved error logging in JSON serialization to provide more context on failures.
2026-01-31 13:06:37 -06:00
Joe
8c6aa36027 Enhance error handling in TestServeSession for interrupted reads
- Updated the error handling logic in the TestServeSession implementation to account for `std::io::ErrorKind::Interrupted`, ensuring that the session can properly retry on interrupted read operations, in addition to handling timeouts and would-block scenarios.
2026-01-30 11:40:01 -06:00
Joe
84f76d48fc Add project configuration and test files in JSON5 format
- Introduced new configuration files for testing, including `default.project.json5`, `client_in_folder`, and others.
- Updated existing project files to use JSON5 format, ensuring consistency across configurations.
- Added new Lua scripts and model files to support various test cases.
- Adjusted snapshot files to align with the new project structure and properties.
2026-01-30 09:16:46 -06:00
Micah
a4e80eb114 Fix tests running infinitely 2026-01-27 06:46:00 -06:00
Joe
fdd1d145cf json5 work 2026-01-22 22:19:00 -06:00
Joe
cd63b5c9c0 json5 2026-01-22 21:27:34 -06:00
Ken Loeffler
02b41133f8
Use post for ref patch and serialize (#1192) 2026-01-19 22:44:42 +00:00
boatbomber
87f58e0a55
Use WebSocket instead of Long Polling (#1142) 2025-11-26 19:57:01 -08:00
Micah
9b5a07191b
Implement Syncback to support converting Roblox files to a Rojo project (#937)
This is a very large commit.
Consider checking the linked PR for more information.
2025-11-19 09:21:33 -08:00
boatbomber
d0b029f995
Add JSONC Support for Project, Meta, and Model JSON files (#1144)
Replaces `serde_json` parsing with `jsonc-parser` throughout the
codebase, enabling support for **comments** and **trailing commas** in
all JSON files including `.project.json`, `.model.json`, and
`.meta.json` files.
MSRV bumps from `1.83.0` to `1.88.0` in order to
use the jsonc_parser dependency.
2025-10-28 17:29:57 -07:00