Root cause: notify-debouncer-full with macos_kqueue caused 25K+
directory watch registrations to take >170s on large projects.
Changes:
- Replace notify-debouncer-full with notify-debouncer-mini to avoid
FSEvents Create+Remove event cancellation and heavy per-watch overhead
- Remove macos_kqueue feature; use FSEvents (matches upstream Rojo)
- Fix VFS read guard: watch_enabled && watch_recursive -> watch_enabled
so recording mechanism works when watch_recursive is false
- Build RefPathIndex from patch_apply data instead of redundant 4.8s
WalkDir + 5,854 file re-reads (populate_from_dir)
- Use ServeSession::new_oneshot() for sourcemap without --watch
- Restore simple synchronous watch setup (implicit during reads)
Serve: >180s (never completed) -> ~8s to Listening
Sourcemap: 14.3s -> ~9s
All 489 tests passing.
- Introduced a new .gitignore file to exclude log files from version control.
- Modified the `init_tree` function in `ServeSession` to return `path_roots`, enhancing the function's utility.
- Cleaned up commented-out code in `sourcemap.rs`, `patch_apply.rs`, and `tree.rs` for better readability and maintainability.
Implemented asynchronous log compression in the logging initialization process. A new function, `lower_thread_priority`, adjusts the thread priority to below normal for the compression task, improving overall application performance during logging operations.
Updated logging levels in the syncback loop to debug for orphan scan performance metrics, reducing verbosity and improving performance during normal operations.
- Replaced the previous greedy matching approach with the Hungarian (Kuhn-Munkres) algorithm for improved accuracy in instance pairing, especially in ambiguous groups with multiple same-named instances.
- Introduced a new Hungarian.lua module to encapsulate the algorithm's logic.
- Updated the reconciler to utilize the new optimal assignment function, enhancing overall matching performance.
- Documented changes in the CHANGELOG for clarity on the new implementation.
- Added a new function for minimum-cost bipartite matching via the Hungarian algorithm to improve the efficiency of the matching process.
- Refactored existing matching logic to utilize the new algorithm, replacing the previous greedy assignment approach.
- Updated relevant documentation to reflect changes in the matching strategy.
- Enhanced performance in ambiguous group matching scenarios.
Bugbot flagged that the pre-walked fast path bypassed collect_alternate_files
and instigating_source checks. Fixed by extracting both into shared helpers
called before the if/else branch, so both paths use the same logic.
The `refresh_git_index` function is now a no-op, as the previous implementation added unnecessary latency without meaningful benefits. Additionally, logging levels for performance metrics have been changed from info to debug across various modules to reduce verbosity and improve performance during normal operations.
Bugbot flagged two issues with the pre-walked orphan paths:
1. cache.is_file.keys() included shallow project-folder entries (README.md,
rokit.toml, etc.) that would be incorrectly flagged as orphans.
Fixed by filtering to only paths under walked $path roots.
2. syncbackRules.ignorePaths was not applied to pre-walked paths.
Fixed by running is_valid_path() on the pre-walked set before use.
Syncback on a 132K-file project dropped from ~32s to ~18s by:
- Skipping hash_tree computation in clean (non-incremental) mode,
since hashes are only consulted when incremental=true (-6s)
- Reusing prefetch walkdir results for orphan detection instead of
walking the filesystem a second time (-1.7s)
- Rewriting collect_all_paths from O(n*d) bottom-up ancestor walks
to O(n) top-down BFS (-0.7s)
- Adding file-size-first comparison in write_to_vfs_parallel to
avoid reading files that differ in size
- Scoping git ls-files --stage to project directory prefix
- Adding [PERF] instrumentation at info level for every pipeline phase
- Changed VFS event logging from INFO to DEBUG level for better performance and reduced verbosity during normal operations.
- Removed unnecessary project folder variable and redundant watch calls in ServeSession, enhancing clarity and efficiency in the watch setup process.
- Introduced `glob_ignore_rules` to manage instance removal based on glob patterns, allowing for more granular control over which instances are retained.
- Updated the logic in `compute_patch_set_internal` and `compute_children_patches` to check against these rules before removing unmatched instances.
- Refactored related tests to ensure correct behavior when instances are matched or ignored based on their paths, improving the robustness of instance synchronization.
- Renamed `ignore_unknown_instances` to `suppress_removals` in patch computation logic for clarity.
- Updated tests to reflect the new naming and behavior, ensuring correct handling of instance removals based on `glob_ignored_children`.
- Enhanced API and interface definitions to include `glob_ignored_children`, improving metadata management for instance synchronization.
- Added `globIgnoredChildren` to `ApiInstanceMetadata` to manage child deletion behavior based on glob patterns.
- Updated `shouldDeleteChild` function to suppress deletions when `globIgnoredChildren` is true, differentiating from scripts-only mode.
- Enhanced tests to verify correct behavior of instance deletion based on the new metadata, ensuring robust handling of unknown scripts.
- Refactored snapshot handling to incorporate `globIgnoredChildren` in metadata, improving clarity and maintainability.
- Enhanced the VFS watch setup in ServeSession to differentiate between macOS and other platforms, ensuring proper handling of file and directory watches.
- Improved clarity by restructuring the logic for watching paths, addressing platform-specific requirements and optimizing resource usage.
- Maintained non-recursive watching for ignored directories while ensuring all relevant files are monitored effectively.
- Added a new entry in TODO.txt to follow up with Coelho on outstanding tasks.
- Simplified VFS watch logic to ensure both files and directories are monitored, addressing potential conflicts on macOS and improving overall watch reliability.
- Updated snapshot_dir_no_meta to set ignore_unknown_instances based on the presence of glob-ignored children.
- Added tests to verify that directories with glob-ignored children correctly flag ignore_unknown_instances as true, while those without do not.
- Improved clarity and maintainability of the snapshot logic by incorporating child filtering rules.
- Simplified the VFS watch setup by removing the directory type check, allowing all entries to be watched directly.
- Improved code clarity and reduced complexity in the watching mechanism for project files.
- Added a task to address the luau-lsp sourcemap generator command configuration.
- Documented a problem with the plugin not respecting two-way sync for glob ignored scripts, highlighting the need for a fix.
- Updated Cargo.toml to conditionally configure the notify dependency for macOS and other platforms.
- Improved ServeSession's VFS watch logic to filter out ignored directories based on glob patterns, optimizing resource usage and watch handle count.
- Introduced PathIgnoreRule handling to streamline directory watching, enhancing overall performance and maintainability.
- Introduced a new method `path_ignore_rules` in the Project struct to streamline the creation of PathIgnoreRule instances.
- Updated ServeSession and snapshot middleware to utilize the new method, improving code clarity and reducing duplication.
- Removed redundant PathIgnoreRule mapping from multiple locations, enhancing maintainability.
- Updated various dependencies in Cargo.lock and Cargo.toml, including aws-lc-rs, aws-lc-sys, tokio, and uuid to their latest versions.
- Refactored the memofs crate to simplify the notify dependency configuration.
- Introduced path ignore rules in ChangeProcessor and ServeSession to filter VFS events based on glob patterns, improving event handling efficiency.
- Updated CloneCommand to support multiple place IDs, allowing the creation of multi-place projects with individual project files for each place.
- Refactored the initialization logic in InitCommand to streamline project setup and ensure proper handling of project templates.
- Added a new function to fetch place names from the Roblox API, improving the user experience during project creation.
- Updated TODO.txt with a new entry for fixing the luau-lsp sourcemap generator command.
- Added a new entry in TODO.txt to remove atlas-project.mdc from shared agent rules and migrate it to manual cloning for new projects.
- Updated init.rs to include atlas-project.mdc in the project initialization process, ensuring it is created in the appropriate rules directory.
- Introduced `is_watch_enabled` method to check the status of automatic file watching in the VFS.
- Added a new test for prefetch cache to validate metadata retrieval and error handling for walked roots and external paths.
- Updated the `watch` method in `StdBackend` for cleaner logic.
- Enhanced the `dir_init` field in `PrefetchCache` to store resolved init-file information for directories, improving prefetch operations.
- Updated the `prefetch_project_files` function to perform a shallow walk of the project folder, limiting the depth to enhance performance and avoid unnecessary recursion into specific directories.
- Improved the logic for handling $path roots, ensuring correct canonicalization and efficient traversal based on the relationship to the project folder.
- Adjusted the VFS watch setup to watch the project folder non-recursively while allowing recursive watching for $path roots, optimizing resource usage during file monitoring.
These changes enhance the accuracy and performance of file prefetching and watching in the VFS.
- Updated the `prefetch_project_files` function to canonicalize folder paths before checking if roots start with the folder, preventing false positives with relative paths.
- Enhanced the logic to skip non-existent roots early, improving efficiency during the prefetching process.
These changes enhance the accuracy and performance of the file prefetching mechanism in the VFS.
- Added a new field `dir_init` to `PrefetchCache` to store pre-resolved init-file information for directories, allowing for efficient retrieval during prefetch operations.
- Introduced the `prefetch_dir_init` method in the `Vfs` struct to look up init-file info from the prefetch cache.
- Updated the `prefetch_project_files` function to populate the `dir_init` map based on the defined `INIT_FILE_PRIORITY`.
- Adjusted related tests to ensure proper initialization of the `dir_init` field in various scenarios.
These changes improve the performance of directory handling in the VFS by reducing the need for repeated metadata probes.
- Modified the `check_tree_freshness` method to set `sync_scripts_only` in the `InstanceContext`, ensuring that the correct synchronization behavior is applied during tree freshness checks.
- This change enhances the functionality of the `ServeSession` by allowing for more precise control over script synchronization.
- Replaced the selective watching logic with a more efficient recursive watching setup in the `ServeSession` implementation.
- Enhanced logging to include the number of recursive watches set up and the elapsed time for the setup process, providing better insights into performance.
- Removed the previous recorded paths logic to streamline the watching process.
These changes aim to optimize the VFS's directory watching capabilities and improve monitoring during the setup phase.
- Added timing measurement for the selective watching process in the `ServeSession` implementation, improving visibility into performance during directory watching.
- Updated the debug log message to include the elapsed time for better monitoring of the watching behavior.
These changes aim to provide more detailed insights into the VFS's directory watching performance.
- Updated the `watch` method in `VfsBackend` to accept a `recursive` parameter, allowing for more flexible directory watching.
- Implemented a recording mode in the `Vfs` struct to track accessed paths without immediately watching them, improving performance during read operations.
- Added tests to verify the behavior of the new recording mode and ensure correct path tracking.
- Enhanced the `StdBackend` and `NoopBackend` implementations to support the updated watch functionality.
These changes aim to improve the VFS's ability to manage file system events and enhance user control over watching behavior.
- Removed outdated git configuration command from TODO.txt for clarity.
- Added a command to set the `core.longpaths` configuration in the `git_init_repo` function, improving support for long file paths in the repository initialization process.
These changes aim to streamline the TODO list and enhance the git initialization workflow.
- Updated the `InitCommand` to conditionally add and commit all changes after initializing a git repository, improving project setup efficiency.
- Modified the API service to include requested IDs in the instance retrieval process, ensuring accurate instance management.
These changes aim to streamline the initialization process and enhance the API's functionality for better user experience.
- Introduced a new `SKILL.md` file for the `atlas-push` skill, detailing the automated workflow for pushing filesystem changes into Roblox Studio, including prerequisites, workflow steps, and conflict resolution strategies.
- Implemented `git_submodule_add` and `git_config_set` functions to manage git submodules and configuration settings, enhancing the git integration within the project.
- Updated the `InitCommand` to add agent submodules in parallel, improving the efficiency of the initialization process.
These changes aim to improve user guidance for the `atlas-push` skill and streamline git submodule handling in the project.
- Updated the `clap` dependency in `Cargo.toml` to include the "env" feature for improved command-line argument handling.
- Introduced a new `roblox_api` module to encapsulate Roblox API interactions, including authentication and asset downloading.
- Refactored the `CloneCommand` and `StudioCommand` to utilize the new `roblox_api` functions for fetching experience names and universe IDs, enhancing code clarity and reducing duplication.
- Removed obsolete functions related to Roblox API calls from the CLI commands, streamlining the codebase.
- Added error handling for authentication failures, ensuring better user feedback when required credentials are missing.
These changes aim to improve the overall structure and functionality of the CLI while enhancing the integration with the Roblox API.
- Introduced a new `onSyncbackCommand` option in `McpStream` to handle syncback requests.
- Implemented the `handle_mcp_syncback` function in the API to process syncback requests, including validation and logging.
- Updated the `run_live_syncback` function to return statistics on added and removed files during the syncback process.
- Enhanced the MCP tools list to include the new syncback tool, with corresponding documentation added.
These changes aim to facilitate a complete syncback of the Roblox Studio state to the filesystem, improving project management capabilities.
- Eliminated the `canonicalize` method from the `VfsBackend` trait and its implementations in `InMemoryFs`, `NoopBackend`, and `StdBackend`, simplifying the interface and reducing complexity.
- Updated various parts of the codebase to remove references to canonicalization, including tests and prefetch cache handling, ensuring consistency across the project.
- Adjusted logging and comments to reflect the removal of path normalization logic, enhancing clarity and maintainability.
These changes aim to streamline the VFS implementation by focusing on core functionalities without the overhead of path canonicalization.
- Replaced the previous implementation of tracking changes with a more efficient approach using `git status` and `git diff`, reducing the number of command calls and enhancing performance.
- Consolidated the handling of tracked and untracked files, improving code readability and maintainability.
- Updated logging to provide more detailed timing information for git operations, aiding in performance monitoring.
These changes aim to streamline git interactions and enhance the overall efficiency of the git module.
- Replaced the use of the gix API with direct command-line calls to `git` for retrieving the repository root and current HEAD commit SHA, improving compatibility and reducing dependency on the gix library.
- Updated the `git_changed_files_impl` function to use command-line `git diff` and `git ls-files` for tracking changes, enhancing performance and flexibility in detecting file modifications.
- Improved error handling and logging for git operations, ensuring better feedback during execution.
These changes aim to streamline git interactions and enhance the overall efficiency of the git module.
- Changed the logging initialization to enable target logging, improving the detail and context of log messages. This adjustment enhances the overall logging output, making it more informative for debugging and monitoring purposes.
- Introduced a constant to enable or disable periodic tree reconciliation after VFS events, addressing a persistent drift bug that caused infinite re-snapshot loops on large projects.
- Updated the `ChangeProcessor` to conditionally execute tree reconciliation based on the new constant, improving performance during file modifications.
- Enhanced the git module by adding new functions for shallow cloning and checking if files are staged, streamlining git operations.
- Refactored the `InitCommand` to utilize the new git functions for repository initialization and cloning, improving clarity and reducing redundancy.
These changes aim to optimize file change handling and enhance git integration within the project.
- Conducted a complete audit of the git module to leverage the gix API for improved performance, aiming for significant speedups in git operations.
- Updated the `git_changed_files_impl` function to accept project prefixes as a vector of strings, enhancing the flexibility of change detection.
- Made the `collect_path_roots` function public to allow external access for collecting project directory paths.
These changes aim to optimize git operations and improve the overall efficiency of path handling in the project.
- Removed the direct initialization of the Project within the ServeCommand's run method, replacing it with a more streamlined session management approach using ServeSession.
- Enhanced error handling by creating a new Vfs instance with error tracking for each session, ensuring better management of critical errors during server operation.
- Improved the overall flow of the run method by reducing redundancy in session creation and enhancing clarity.
These changes aim to optimize the server's session handling and improve error management during execution.
- Updated the `flatten_tree` function in `git.rs` to clarify the repository parameter name.
- Changed logging level from `debug` to `trace` in `finalize_patch_application` for more detailed output.
- Modified the `TestServeSession` initialization to persist git configuration to disk, ensuring user identity is accessible for subsequent operations.
These changes aim to improve code clarity, enhance logging detail, and ensure proper git configuration handling in tests.
