Updated logging levels in the syncback loop to debug for orphan scan performance metrics, reducing verbosity and improving performance during normal operations.
- Replaced the previous greedy matching approach with the Hungarian (Kuhn-Munkres) algorithm for improved accuracy in instance pairing, especially in ambiguous groups with multiple same-named instances.
- Introduced a new Hungarian.lua module to encapsulate the algorithm's logic.
- Updated the reconciler to utilize the new optimal assignment function, enhancing overall matching performance.
- Documented changes in the CHANGELOG for clarity on the new implementation.
- Added a new function for minimum-cost bipartite matching via the Hungarian algorithm to improve the efficiency of the matching process.
- Refactored existing matching logic to utilize the new algorithm, replacing the previous greedy assignment approach.
- Updated relevant documentation to reflect changes in the matching strategy.
- Enhanced performance in ambiguous group matching scenarios.
Bugbot flagged that the pre-walked fast path bypassed collect_alternate_files
and instigating_source checks. Fixed by extracting both into shared helpers
called before the if/else branch, so both paths use the same logic.
The `refresh_git_index` function is now a no-op, as the previous implementation added unnecessary latency without meaningful benefits. Additionally, logging levels for performance metrics have been changed from info to debug across various modules to reduce verbosity and improve performance during normal operations.
Bugbot flagged two issues with the pre-walked orphan paths:
1. cache.is_file.keys() included shallow project-folder entries (README.md,
rokit.toml, etc.) that would be incorrectly flagged as orphans.
Fixed by filtering to only paths under walked $path roots.
2. syncbackRules.ignorePaths was not applied to pre-walked paths.
Fixed by running is_valid_path() on the pre-walked set before use.
Syncback on a 132K-file project dropped from ~32s to ~18s by:
- Skipping hash_tree computation in clean (non-incremental) mode,
since hashes are only consulted when incremental=true (-6s)
- Reusing prefetch walkdir results for orphan detection instead of
walking the filesystem a second time (-1.7s)
- Rewriting collect_all_paths from O(n*d) bottom-up ancestor walks
to O(n) top-down BFS (-0.7s)
- Adding file-size-first comparison in write_to_vfs_parallel to
avoid reading files that differ in size
- Scoping git ls-files --stage to project directory prefix
- Adding [PERF] instrumentation at info level for every pipeline phase
- Eliminated the `canonicalize` method from the `VfsBackend` trait and its implementations in `InMemoryFs`, `NoopBackend`, and `StdBackend`, simplifying the interface and reducing complexity.
- Updated various parts of the codebase to remove references to canonicalization, including tests and prefetch cache handling, ensuring consistency across the project.
- Adjusted logging and comments to reflect the removal of path normalization logic, enhancing clarity and maintainability.
These changes aim to streamline the VFS implementation by focusing on core functionalities without the overhead of path canonicalization.
- Expanded the TODO list with new tasks related to MCP modes, including standard, manual, fastfail, and dryrun options for better user control during sync operations.
- Introduced a new `get_script` command to allow agents to read scripts directly from Roblox Studio, facilitating conflict resolution during sync.
- Implemented a `GitIndexCache` to optimize file writes by skipping unchanged files based on blob SHA1 comparisons, improving syncback efficiency.
- Added logging for agent connections and commands to enhance traceability and debugging capabilities.
These changes aim to improve the overall functionality and performance of the MCP integration and syncback process.
- Updated the TODO list to consolidate and clarify tasks, removing outdated entries while retaining relevant ones.
- Enhanced the slugification logic to treat "init" and its case variations as reserved names, ensuring they are properly suffixed with an underscore.
- Added tests to verify the slugification behavior for reserved names, improving code reliability.
These changes aim to streamline task management and ensure consistent handling of reserved names in the application.
- Moved the `is_script_class` function to the `snapshot` module for better organization and reusability.
- Updated imports in `git.rs` and `ref_properties.rs` to reflect the new location of `is_script_class`, enhancing code clarity and maintainability.
These changes aim to streamline script class management and improve the overall structure of the codebase.
- Updated the `vscode-rojo` submodule to indicate a dirty state.
- Modified `DomLabel.lua` to adjust size and position calculations for better layout.
- Removed redundant code in `init.lua` related to visible nodes.
- Introduced new functions in `Confirming.lua` to create statistical tags and bars for change information, improving the visual representation of changes in the confirmation page.
These updates aim to enhance the user interface and provide clearer insights into changes made during the patch visualization process.
- Updated the `vscode-rojo` submodule to the latest commit.
- Added a new plan for addressing issues found during the audit of parallelization and optimization commits, including fixes for a data loss bug in the orphan scanner, a duplicate-children bug in the prefetch cache, and a nested-detection correctness issue.
- Implemented a small DRY refactor by extracting a shared helper function for UNC path stripping.
- Enhanced the `syncback` logic to ensure hidden directories are preserved during clean mode operations and added tests to validate these changes.
These updates aim to improve the reliability and correctness of the parallel I/O functionality while maintaining code clarity and performance.
- Changed several logging statements from `info` to `debug` across various modules, including `StdBackend`, `RefPathIndex`, `ServeSession`, and `syncback_loop_with_stats`.
- This adjustment aims to reduce log verbosity during normal operations while retaining detailed logs for debugging purposes, enhancing overall performance and maintainability.
- Deleted the `project_port` and `serve_address` methods from the `ServeSession` implementation as they were no longer needed.
- Removed the `scan_directory` and `normalize_existing_path` helper functions from the `syncback_loop_with_stats` function, streamlining the code and improving maintainability.
These changes aim to enhance code clarity and reduce unnecessary complexity in the session and syncback functionalities.
- Updated the `ServeCommand` to load the initial project directly, improving clarity and reducing unnecessary session handling.
- Introduced a new `replace_placeholders` function in `FsSnapshot` to efficiently replace `__ROJO_REF_<hex>__` placeholders using a HashMap, optimizing the substitution process.
- Enhanced the syncback logic to handle both `__ROJO_REF_` and `Rojo_Ref_` markers, ensuring comprehensive path replacements during the syncback process.
These changes aim to improve the performance and maintainability of the serve and syncback functionalities.
- Simplified the comparison logic for `Font` variants in the `variant_eq` and `variant_eq_disk` functions by consolidating multiple conditions into single lines for improved readability.
- Removed the handling of `cached_face_id` from the `hash_variant` function, streamlining the hashing process for `Font` variants.
These changes aim to enhance code clarity and maintainability while preserving the existing functionality.
- Introduced a new plan to automatically strip `~N` dedup suffixes from filenames during forward sync, eliminating the need for redundant `.meta.json5` files.
- Implemented the `strip_dedup_suffix()` utility in `dedup_suffix.rs` to handle suffix removal.
- Updated `file_name_for_path()` and `get_dir_middleware()` to utilize the new stripping functionality.
- Modified syncback logic to skip writing meta `name` fields for dedup-only cases, ensuring cleaner instance naming.
- Enhanced existing tests and added new tests to cover the updated dedup behavior, ensuring robustness and correctness.
These changes aim to streamline the handling of deduplicated instances, improving overall system efficiency and clarity in naming conventions.
- Made `format_f32` and `format_f64` functions public to allow external access.
- Introduced new constants for disk options and buffer sizes for `f32` and `f64`.
- Added `cleanup_f32` function to clean up float values before comparison.
- Implemented `disk_eq_f32` and `disk_eq_f64` functions to compare float values based on their disk representation.
- Updated `variant_eq_disk` to utilize the new disk equality functions, improving accuracy in matching algorithms.
These changes aim to enhance the precision of float comparisons and ensure consistent behavior across different representations in the system.
- Updated versions of several dependencies in `Cargo.lock`, including `async-compression` (0.4.40), `aws-lc-rs` (1.16.0), `bumpalo` (3.20.1), and `compression-codecs` (0.4.37), ensuring compatibility and access to the latest features and fixes.
- Refactored the `fire` method in `SyncbackSignal` to return a boolean indicating whether a syncback was successfully initiated, preventing concurrent syncbacks.
- Enhanced error handling in the API syncback process to return a conflict response if a syncback is already in progress.
- Improved the `serve.rs` file by replacing `forget` with `drop` for better memory management and added checks to ensure service chunk integrity during live syncback operations.
- Added new tests for edge cases and resilience in live syncback functionality, ensuring robustness against malformed data and rapid sequential requests.
These changes aim to enhance the reliability and performance of the syncback feature while keeping dependencies up to date.
- Eliminated the `float-cmp` crate from the project, replacing its functionality with custom float comparison functions that match Lua's epsilon-based equality checks.
- Updated the `variant_eq` function to utilize the new fuzzy equality methods for `Float32` and `Float64` variants, ensuring consistent behavior across Rust and Lua implementations.
- Enhanced the matching algorithms in both forward sync and syncback processes to account for refined scoring of `Ref`, `Tags`, and `Attributes`, improving scoring accuracy and parity between implementations.
- Introduced a new plan document detailing the changes made to the matching algorithms, including the rationale for the updates and the expected impact on performance and correctness.
These changes aim to improve the reliability and maintainability of the matching algorithms while ensuring parity with Lua's behavior.
- Introduced a new endpoint `/api/syncback` for one-click full sync from Roblox Studio to the filesystem, eliminating the need for an active WebSocket connection.
- Implemented `SyncbackRequest` and `ServiceChunk` structures to handle service-level properties and child instances during syncback.
- Enhanced `encodeService` to include ObjectValue carriers for accurate reference handling and prevent data loss.
- Updated the syncback process to utilize Luau-style relative paths for reference attributes, improving path management and consistency.
- Integrated session caching in the matching algorithm to optimize performance during syncback operations.
These changes significantly improve the live syncback experience, ensuring reliable data synchronization and enhanced performance during development.
- Added `lexical-write-float` and its dependencies to `Cargo.toml` and `Cargo.lock` for improved float formatting capabilities.
- Enhanced float formatting functions in `json.rs` to utilize the new lexical library, ensuring more precise and consistent numeric representation.
- Updated tests to validate the new formatting behavior, including handling of small values and special cases.
These changes improve the accuracy and efficiency of float serialization in JSON, addressing previous inconsistencies and enhancing overall data integrity.
- Added explicit encoding for Workspace attributes and tags in `encodeService.lua` to prevent data loss during syncback.
- Filtered non-serializable properties, including `ClockTime`, in `build_dom_from_chunks` to ensure accurate data handling.
- Resolved a leak of `Rojo_Ref_CurrentCamera` by filtering deferred referents after camera destruction in `syncback_loop`.
- Truncated float values to f32 precision in `UnresolvedValue::from_variant` to match rbxl storage format, improving data consistency.
These changes enhance the reliability and accuracy of the live syncback process, addressing critical issues identified in previous implementations.
- Simplified the `isReadable` condition in `encodeService` for better readability.
- Updated the `ServiceChunk` structure to conditionally include properties and references only if they exist, enhancing data efficiency.
- Introduced a `MAX_SCORING_DEPTH` constant in the matching module to limit recursion depth, improving performance and preventing potential stack overflow issues.
- Refactored the `computePairCost` and `matchChildren` functions to incorporate the new depth parameter, ensuring accurate cost calculations during matching.
These changes enhance the clarity and efficiency of the encoding and matching processes, contributing to overall performance improvements.
- Implemented live syncback via the Studio plugin, enabling Roblox to filesystem synchronization during `atlas serve` sessions.
- Added `SyncbackRequest` structure and API protocol validation for plugin-initiated syncback.
- Introduced `SyncbackConfirm` UI component for user confirmation of live syncback operations.
- Enhanced matching module with reference identity checks, caching, and valid children tracking.
- Updated deduplication logic to start suffix numbering at `~2` and improved attribute indexing in `RefPathIndex`.
- Refactored animation goal settings for instant transitions and updated color handling in the Reconciler.
These changes significantly enhance the syncback process and improve the overall performance and maintainability of the matching module.
- Introduced a new `encodeService` helper in the plugin to serialize service properties, attributes, tags, and references alongside children data.
- Updated the `performSyncback` function to utilize the new encoding method, ensuring all services are processed without child-only restrictions.
- Extended the `ServiceChunk` struct to include fields for properties, attributes, tags, and references, improving data handling during syncback.
- Modified the `build_dom_from_chunks` function to apply service properties and resolve references when reconstructing the service instances.
- Added comprehensive tests to validate the new service properties handling, ensuring accurate synchronization and data integrity.
These enhancements significantly improve the live syncback process, allowing for better service data management and consistency.
- Added a `validChildren` field to the `VCache` structure to track valid child instances during the matching process.
- Refactored the `cacheVirtual` function to improve readability by formatting parameters across multiple lines.
- Updated the `computePairCost` function to utilize the new `validChildren` field for more accurate cost calculations.
- Simplified the `matchChildren` function by removing redundant code and improving the handling of matched instances.
- Enhanced test cases for the `matchChildren` function to ensure consistent behavior and validate the new changes.
These updates improve the efficiency and clarity of the matching process, ensuring better performance and maintainability.
- Introduced a new `RefIdentity` type to store reference details (name and className) in the matching process.
- Updated `VCache` and `SCache` structures to include references, improving the handling of reference comparisons during matching.
- Modified `cacheVirtual` and `cacheStudio` functions to support reference caching, enhancing performance and accuracy in matching.
- Improved `countOwnDiffs` function to account for reference mismatches, ensuring more precise cost calculations.
- Added comprehensive tests to validate the new reference handling and caching mechanisms.
These changes significantly enhance the efficiency and accuracy of the matching process in the application.
- Reformatted the `SYNCBACK_SERVICES` list for better readability by aligning entries vertically.
- Simplified the `performSyncback` function by removing unnecessary Promise wrapping, enhancing clarity and maintainability.
- Updated the `NotConnectedPage` component to improve the structure of button elements and their associated tooltips.
- Refactored the `hydrate` and `matching` modules to streamline function calls and improve code organization.
These changes enhance the overall readability and maintainability of the syncback process and related components.
- Introduced a new matching session caching mechanism in the Lua matching module to enhance performance with native code generation.
- Rewrote the Lua matching algorithm to utilize session-based caching, reducing redundant computations and improving efficiency.
- Updated Rust implementations to include session caching, ensuring consistency across both Lua and Rust.
- Enhanced the `hydrate` function to support session management during instance hydration.
- Added comprehensive tests to validate the new caching mechanism and its integration with existing functionality.
These changes significantly improve the performance and maintainability of the matching process in the application.
- Simplified function calls and removed unnecessary line breaks in `change_processor.rs`, `rojo_ref.rs`, `syncback/meta.rs`, `syncback/mod.rs`, `syncback/ref_properties.rs`, and `web/api.rs` to enhance code clarity.
- Updated test assertions in `two_way_sync.rs` for better readability by consolidating multiple lines into single assertions.
These changes improve the overall maintainability and readability of the codebase.
- Introduced a new migration plan for converting `Rojo_Ref_*` attribute paths to Luau require-by-string style, enhancing portability with relative paths.
- Updated existing test snapshots to reflect the new path formats, ensuring consistency across various scenarios.
- Enhanced the `compute_relative_ref_path` and `resolve_ref_path` functions to support the new path semantics, improving path resolution logic.
- Refactored related code in `change_processor`, `syncback`, and `snapshot` modules to accommodate the new reference path handling.
These changes improve the robustness and maintainability of reference path management in the project.
- Simplified test assertions in `fs_snapshot.rs` and `ref_properties.rs` for better readability by removing unnecessary line breaks.
- Enhanced the formatting of code in `rojo_ref.rs` to maintain consistency and improve clarity.
These changes improve the overall readability and maintainability of the test code and related implementations.
- Introduced tests to verify that the new approach for handling Rojo_Ref_* attributes uses unique placeholders, preventing path corruption from chaining substitutions.
- Added a test to document the old failure mode where duplicate-named instances would lead to corrupted paths due to sequential substitutions.
- Enhanced the `collect_referents` function to populate a mapping of placeholders to target references, ensuring accurate path resolution during syncback.
These changes improve the reliability of reference path handling and prevent future regressions related to path substitutions.
- Changed the default value of `sync_unscriptable` in `SyncbackRules` from `true` to `false` to better reflect intended behavior.
- Adjusted the property filter logic to align with the updated default, ensuring consistent handling of unscriptable properties.
These changes improve the clarity and functionality of the syncback feature, enhancing its overall reliability.
- Removed the `luau` dependency from `rokit.toml`.
- Updated deduplication logic to append `~2`, `~3`, etc. instead of `~1`, `~2` for naming collisions in various files, ensuring consistent suffix management.
- Adjusted comments and documentation across multiple files to reflect the new suffix handling rules.
- Added new test cases to validate the updated deduplication behavior, ensuring accurate naming for instances with collisions.
These changes enhance the clarity and functionality of the deduplication process, improving overall consistency in naming conventions.
- Added `#[allow(clippy::too_many_arguments)]` annotations in `change_processor.rs` and `matching.rs` to suppress warnings for functions with multiple parameters.
- Updated `serve_session.rs` to include `clippy::type_complexity` in the `#[allow(dead_code)]` annotation for better code quality.
- Simplified array initialization in `file_names.rs` for consistency.
- Enhanced texture filtering logic in `matching_fixtures.rs` by replacing `map_or` with `is_some_and` for improved readability.
These changes contribute to a cleaner codebase and address static analysis warnings, ensuring better maintainability.
- Updated the `rbx-dom` subproject to the latest commit for improved functionality.
- Enhanced the `/audit` command documentation to emphasize the importance of entering Plan mode before executing any actions.
- Improved the `atlas-dedup.mdc` file by adding details on user-initiated renames and ensuring ref path integrity during deduplication.
- Cleared out deprecated material color entries in `allValues.json` for better clarity.
- Updated `database.json` to reflect new properties and classes, including `AssetQualityService` and `BaseCoreGuiConfiguration`.
- Introduced new scripts for updating the reflection database, streamlining the process for generating necessary files.
These changes improve the overall functionality and maintainability of the project, ensuring better handling of deduplication and reflection updates.
- Simplified the formatting of path handling in `change_processor.rs` and `fs_snapshot.rs` by consolidating multiple lines into single lines for better clarity.
- Updated test assertions in `two_way_sync.rs` to streamline the code and enhance maintainability.
These changes enhance the overall readability of the codebase, making it easier to manage and understand.
- Updated the audit plan for the ambiguous paths follow-up, marking all critical fixes as completed, including dedup cleanup, added paths, and ref path finalization.
- Introduced a new `atlas-dedup.mdc` file detailing the deduplication system, including matching algorithms, suffix cleanup, and integration for duplicate-named instances.
- Enhanced the `syncback` process to ensure proper handling of ref paths and deduplication, improving metadata consistency and overall system reliability.
- Implemented attribute-scoped path fixes in the `FsSnapshot` to ensure only relevant lines are modified during ref path updates.
These changes improve the clarity and effectiveness of the audit process while enhancing the deduplication and synchronization mechanisms within the project.
- Introduced a new `trueEquals` function in `trueEquals.lua` for shared value equality checks, improving consistency between `diff.lua` and `matching.lua`.
- Removed redundant equality logic from `diff.lua` and `matching.lua`, streamlining the codebase.
- Updated path handling in various test snapshots to ensure correct reference values, enhancing the accuracy of reconciliation tests.
- Implemented a post-processing step in `syncback` to fix `Rojo_Ref_*` attribute paths with correct dedup suffixes, improving path resolution.
These changes enhance the clarity and efficiency of the Reconciler's equality checks and path handling, paving the way for improved project management and user experience.
- Updated test descriptions in `diff.spec.lua` and `diff.stress.spec.lua` to reflect changes in behavior regarding duplicate-named children in both real and virtual DOM contexts.
- Adjusted assertions to ensure that duplicate-named children are correctly removed or added based on the new matching logic.
- Refactored `hydrate.lua` and `matching.lua` to streamline the matching process by removing unnecessary parameters, enhancing clarity and maintainability.
- Updated metadata references in various test files to include the `.model.json5` extension for consistency in path handling.
These changes improve the accuracy of reconciliation tests and enhance the overall robustness of the Reconciler's path handling logic.
- Introduced a comprehensive rewrite of the matching algorithm, replacing the previous multi-pass approach with a recursive change-count scoring function and greedy assignment strategy.
- Updated the `hydrate.lua` and `matching.lua` files to utilize the new algorithm, enhancing the efficiency of instance matching during hydration.
- Documented the new algorithm's structure, including fast-paths for Ref pins and (Name, ClassName) grouping, which significantly improve matching accuracy and performance.
- Ensured that the new implementation maintains compatibility with existing systems while providing a more robust solution for handling ambiguous instances.
These changes enhance the clarity and efficiency of the reconciliation process, paving the way for improved project management and user experience.
- Introduced a comprehensive audit plan detailing critical issues, code quality items, and test coverage gaps identified in the Ambiguous Path Handling feature.
- Documented specific todos addressing various correctness and performance issues, including case sensitivity, path formatting, and dead code removal.
- Established standards for round-trip identity and CLI syncback parity to ensure consistent behavior across the system.
These changes enhance the clarity and accountability of the auditing process, paving the way for improved feature reliability and maintainability.
- Updated the `encodeInstance.stress.spec.lua` tests to eliminate checks for skipped counts, as duplicates are now included alongside unique children.
- Adjusted expectations in the tests to reflect the new behavior of the encoding process, ensuring accurate validation of encoded instances.
- Removed legacy assertions related to skipped counts, streamlining the test cases for better clarity and maintainability.
- Finalized the transition from a duplicate-name blacklist system to a comprehensive 3-pass matching algorithm for both syncback and forward operations, enhancing the handling of ambiguous paths.
- Integrated a new `matching.lua` module to facilitate instance matching during hydration, utilizing unique name matching, class name narrowing, and structural fingerprinting.
- Updated various components, including `ChangeBatcher`, `Reconciler`, and `snapshot` modules, to leverage the new matching logic, ensuring robust file management and improved performance.
- Removed legacy duplicate handling code and streamlined the encoding process for instances with duplicate names.
These changes significantly enhance the clarity and efficiency of file operations, paving the way for improved project management and user experience.
- Introduced a new function `write_sourcemap_from_syncback` to generate sourcemaps directly from in-memory data, improving performance by eliminating the need for file reads during sourcemap creation.
- Updated the `syncback_loop` to return a `SyncbackResult` struct, encapsulating filesystem operations and instance paths for better post-processing.
- Modified the `SyncbackCommand` to utilize the new sourcemap generation method, allowing for parallel execution of file writes and sourcemap creation.
These changes streamline the syncback process and enhance the efficiency of sourcemap generation.
- Added a detailed section in the audit documentation outlining the sync system layout for both CLI syncback and two-way sync pipelines, including file mappings and processing steps.
- Updated `init.meta.json5` to include the `Rojo_Ref_PrimaryPart` attribute for better reference tracking.
- Improved the `ChangeProcessor` to populate the `RefPathIndex` from existing meta/model files at startup, ensuring accurate handling of pre-existing `Rojo_Ref_*` attributes during renames.
- Refactored the `remove_file` method in `RefPathIndex` to `remove_all_for_file`, enhancing the re-indexing process after attribute removals.
- Added tests to verify the correct behavior of the `RefPathIndex` and ensure robustness against edge cases.
These changes improve the clarity of the audit process and enhance the reliability of reference path management in the synchronization system.
- Introduced a new audit plan detailing fixes for four issues identified in the second audit of two-way sync Ref properties, including stale comments, slugified name mismatches, immediate ID assignments, and performance optimizations.
- Updated CI scripts to run Rust tests with increased parallelism by specifying `--test-threads=16`, improving test execution efficiency.
- Enhanced error handling in `update_ref_paths_in_file` to provide clearer context on failures during file reading and JSON parsing.
These changes improve the documentation for Ref property audits and optimize the CI testing process.
