Commit graph

47 commits

Author SHA1 Message Date
Joe
8acbfc0cf8 Refactor notify dependency configuration and enhance VFS watch setup
- Updated Cargo.toml to conditionally configure the notify dependency for macOS and other platforms.
- Improved ServeSession's VFS watch logic to filter out ignored directories based on glob patterns, optimizing resource usage and watch handle count.
- Introduced PathIgnoreRule handling to streamline directory watching, enhancing overall performance and maintainability.
2026-03-06 23:22:54 -06:00
Joe
6fcb1b9cf2 Update dependencies and enhance path ignore functionality
- Updated various dependencies in Cargo.lock and Cargo.toml, including aws-lc-rs, aws-lc-sys, tokio, and uuid to their latest versions.
- Refactored the memofs crate to simplify the notify dependency configuration.
- Introduced path ignore rules in ChangeProcessor and ServeSession to filter VFS events based on glob patterns, improving event handling efficiency.
2026-03-06 23:01:20 -06:00
Joe
5275d45adc Add is_watch_enabled method to VFS and enhance prefetch cache tests
- Introduced `is_watch_enabled` method to check the status of automatic file watching in the VFS.
- Added a new test for prefetch cache to validate metadata retrieval and error handling for walked roots and external paths.
- Updated the `watch` method in `StdBackend` for cleaner logic.
- Enhanced the `dir_init` field in `PrefetchCache` to store resolved init-file information for directories, improving prefetch operations.
2026-03-06 21:51:22 -06:00
Joe
5ff0ed3f99 Refactor cache lookup logic in VFS to streamline path checks
- Simplified the condition for checking if a path is within the walked roots by removing unnecessary variable assignments.
- This change enhances the clarity and efficiency of the metadata retrieval process in the VFS.
2026-03-06 20:55:48 -06:00
Joe
dc788711c7 Fix walked_roots to only include recursively walked directories 2026-03-06 20:53:28 -06:00
Joe
6168597acb Enhance PrefetchCache with directory init-file resolution
- Added a new field `dir_init` to `PrefetchCache` to store pre-resolved init-file information for directories, allowing for efficient retrieval during prefetch operations.
- Introduced the `prefetch_dir_init` method in the `Vfs` struct to look up init-file info from the prefetch cache.
- Updated the `prefetch_project_files` function to populate the `dir_init` map based on the defined `INIT_FILE_PRIORITY`.
- Adjusted related tests to ensure proper initialization of the `dir_init` field in various scenarios.

These changes improve the performance of directory handling in the VFS by reducing the need for repeated metadata probes.
2026-03-06 19:58:07 -06:00
Joe
a1825f7c4a Enhance VFS watch functionality to support upgrade from non-recursive to recursive watching
- Modified the `watch` method in `StdBackend` to allow upgrading a non-recursive watch to a recursive one, improving flexibility in file watching.
- Added a new test to verify that a non-recursive watch can be upgraded to recursive, ensuring correct behavior in directory watching scenarios.

These changes aim to enhance the VFS's ability to manage file system events more effectively.
2026-03-06 17:49:50 -06:00
Joe
9ddd9379ab Refine VFS watch behavior to support recursive watching
- Updated the `read_raw` method in `VfsInner` to ensure that the file watching behavior is only triggered when both `watch_enabled` and `watch_recursive` are true. This change enhances the control over file watching, particularly in scenarios involving nested directories.

These modifications aim to improve the efficiency and accuracy of the VFS's file watching capabilities.
2026-03-06 17:43:08 -06:00
Joe
2ea1d49316 Enhance VFS watch functionality and introduce recording mode
- Updated the `watch` method in `VfsBackend` to accept a `recursive` parameter, allowing for more flexible directory watching.
- Implemented a recording mode in the `Vfs` struct to track accessed paths without immediately watching them, improving performance during read operations.
- Added tests to verify the behavior of the new recording mode and ensure correct path tracking.
- Enhanced the `StdBackend` and `NoopBackend` implementations to support the updated watch functionality.

These changes aim to improve the VFS's ability to manage file system events and enhance user control over watching behavior.
2026-03-06 17:25:38 -06:00
Joe
fc8a6e9b9c Remove canonicalization functionality from VFS and related components
- Eliminated the `canonicalize` method from the `VfsBackend` trait and its implementations in `InMemoryFs`, `NoopBackend`, and `StdBackend`, simplifying the interface and reducing complexity.
- Updated various parts of the codebase to remove references to canonicalization, including tests and prefetch cache handling, ensuring consistency across the project.
- Adjusted logging and comments to reflect the removal of path normalization logic, enhancing clarity and maintainability.

These changes aim to streamline the VFS implementation by focusing on core functionalities without the overhead of path canonicalization.
2026-03-01 18:42:41 -06:00
Joe
d8d6a53fce Update dependencies and add CLI completions support
- Updated several dependencies in `Cargo.lock` and `Cargo.toml`, including `anyhow`, `tempfile`, `clap`, and others to their latest versions for improved functionality and security.
- Introduced a new `CompletionsCommand` in the CLI to support shell completions, enhancing user experience by allowing users to generate command completions for various shells.

These changes aim to keep the project dependencies up-to-date and improve the command-line interface usability.
2026-03-01 07:14:55 -06:00
Joe
d360ebe66f Enhance file and directory removal logic in VFS
- Added a check for `watch_enabled` before calling `unwatch` in the `remove_file` and `remove_dir_all` methods to prevent unnecessary operations when watching is disabled.
- Updated error handling in the `StdBackend` to include a new condition for `Generic` errors related to watch status, improving logging clarity.

These changes aim to optimize the file system operations and enhance error handling in the virtual file system.
2026-02-19 05:24:56 -06:00
Joe
1c25b928a8 Refactor logging levels to debug for improved performance and clarity
- Changed several logging statements from `info` to `debug` across various modules, including `StdBackend`, `RefPathIndex`, `ServeSession`, and `syncback_loop_with_stats`.
- This adjustment aims to reduce log verbosity during normal operations while retaining detailed logs for debugging purposes, enhancing overall performance and maintainability.
2026-02-19 04:12:44 -06:00
Joe
3ccaaa63db Enhance PrefetchCache and improve directory handling in VFS
- Added a `children` field to the `PrefetchCache` structure to store sorted child paths for directories, optimizing directory read operations.
- Updated the `read_dir` method in `VfsInner` to utilize the new `children` cache, improving performance by reducing backend calls for cached directories.
- Refactored the `prefetch_project_files` function to populate the `children` map during prefetching, enhancing the efficiency of path collection.
- Adjusted tests to accommodate the new `children` field in `PrefetchCache`, ensuring comprehensive coverage of the updated functionality.

These changes aim to improve the performance and reliability of the VFS by refining the prefetching mechanism and enhancing directory handling.
2026-02-19 02:39:23 -06:00
Joe
0462b91a6c Enhance PrefetchCache functionality and improve project file prefetching
- Updated the `PrefetchCache` structure to include metadata about whether paths are files or directories, optimizing the cache's utility during VFS operations.
- Refactored the `prefetch_project_files` function to accept a `Project` object, allowing for more flexible path collection and improved logging of the prefetching process.
- Added tests to validate the behavior of the prefetch cache, ensuring correct handling of file reads, canonicalization, and cache depletion scenarios.
- Enhanced existing tests to cover new functionality, including concurrent reads and stress tests for snapshot determinism.

These changes aim to improve the performance and reliability of the VFS by refining the prefetching mechanism and ensuring accurate metadata handling.
2026-02-19 02:26:04 -06:00
Joe
1ffed1212d Add prefetch cache for VFS to optimize file reads
- Introduced a `PrefetchCache` structure to store pre-read file contents and canonical paths, enhancing the speed of initial reads during snapshot building.
- Implemented parallel file reading using `rayon` and `walkdir` to populate the cache efficiently.
- Updated the `VfsInner` struct to include the prefetch cache and modified read methods to utilize it, reducing I/O operations.
- Added functions to set and clear the prefetch cache in the `Vfs` API, improving memory management during live operations.
- Enhanced tests to validate the functionality of the prefetch cache, ensuring correct behavior during file reads and canonicalization.

These changes aim to significantly improve the performance of the VFS by minimizing file access times during initial operations.
2026-02-19 02:04:15 -06:00
Joe
dea60eb4cf Add formatting and static analysis scripts with usage instructions
- Introduced new scripts for auto-fixing formatting and performing static analysis across the project, including support for Lua, Rust, and TypeScript.
- Created `format.md` documentation to guide users on running the formatting scripts.
- Updated CI scripts to utilize the number of processor threads for improved performance during linting and testing.
- Enhanced logging in the `unwatch` function to provide clearer feedback on path watching status.

These changes streamline the formatting and analysis process, ensuring consistent code quality across the project.
2026-02-17 19:07:50 -06:00
Joe
8462c06b82 Enhance VFS initialization and error handling
- Introduced a new method `new_default_with_errors` in the VFS implementation to create a VFS instance with a critical error receiver for monitoring watcher health.
- Updated the `ServeSession` constructor to accept an optional critical error receiver, allowing for better error management during session initialization.
- Adjusted various calls to `ServeSession::new` across the codebase to accommodate the new parameter, ensuring consistent error handling.
- Improved logging in the API context and serve session files for better clarity during server operations.

These changes enhance the robustness of the VFS and session management, providing better tools for error detection and handling.
2026-02-11 20:13:02 -06:00
Joe
181a003ad2 Improves sourcemap path handling with pathdiff 2026-02-09 23:31:44 -06:00
Joe
3cb9e16889 Update Atlas to version 8.0.0
- Bumped version from 7.7.0-rc.2 to 8.0.0 across all relevant files.
- Updated repository links to reflect the new UserGeneratedLLC branding.
- Enhanced CHANGELOG with new features and fixes for the 8.0.0 release.
- Updated plugin version and internal references to align with the new branding.
- Modified dependencies in Cargo files to use the latest version of Atlas.
2026-02-07 13:52:45 -06:00
Joe
a2c8a583a6 Refactor StdBackend error handling for file watcher reliability
- Updated the `new` method in `StdBackend` to improve error handling for file watching.
- Introduced specific handling for `WatcherCriticalError::RescanRequired`, allowing the watcher to recover from rapid changes while logging a warning.
- Ensured that all other critical errors will terminate the process, maintaining reliability in file watching operations.
2026-02-06 19:51:46 -06:00
Joe
0a4496a6c9 Update serde dependency in Cargo.toml for memofs
- Moved the `serde` dependency under the macOS-specific target configuration to ensure proper conditional inclusion based on the operating system.
- This change enhances the clarity of dependency management and maintains compatibility across different platforms.
2026-02-06 18:28:31 -06:00
Joe
8abb16e4a5 Update dependencies in Cargo.lock and Cargo.toml
- Bumped versions for several dependencies, including `anyhow`, `async-compression`, `jiff`, `memchr`, `pest`, and `time`, ensuring compatibility with the latest features and fixes.
- Adjusted the `notify` dependency in `memofs` to conditionally include platform-specific features for macOS, enhancing cross-platform support.
- Updated checksums in `Cargo.lock` to reflect the new versions, maintaining integrity and security of the dependency tree.
2026-02-06 18:22:50 -06:00
Joe
cf33a93640 Refactor StdBackend to improve file watcher reliability
- Updated the `StdBackend` to utilize `notify::RecommendedWatcher` instead of a platform-specific watcher type, enhancing cross-platform event delivery.
- Simplified the debouncer initialization by removing platform-specific configurations, streamlining the code for better maintainability.
- Improved documentation to reflect changes in the watcher implementation and clarify the debouncing mechanism.
2026-02-06 14:17:14 -06:00
Joe
0605d5cfee Refactor file watcher and path suppression handling for improved reliability
- Updated the `StdBackend` to use a platform-specific watcher type, enhancing event delivery on macOS by utilizing `PollWatcher`.
- Refactored the path suppression logic in `JobThreadContext` to handle any VFS event type, improving the accuracy of event processing during renames and writes.
- Enhanced documentation to clarify the changes in event handling and suppression mechanisms, contributing to better maintainability and understanding of the code.
2026-02-06 13:58:59 -06:00
Joe
0968e74208 Implement comprehensive fixes for sync system issues
- Introduced a new plan to address eight identified issues in the sync system, including improvements to standalone script handling, property persistence, and file watcher behavior.
- Key changes include extracting standalone-to-directory conversion logic, enhancing property update persistence, and implementing a recovery mechanism for rapid delete+recreate scenarios.
- Added integration tests to ensure coverage for each fix and improve overall system reliability.
- Enhanced logging for VFS events to aid in debugging file watcher desynchronization issues.

This commit significantly improves the robustness of the sync system and addresses critical issues identified in the audit.
2026-02-05 18:25:47 -06:00
Joe
1e92d36dac Upgrade notify crate from v4 to v8 with stress tests
- Update notify dependency from 4.0.18 to 8.2

- Add notify-debouncer-full 0.7 for event debouncing

- Rewrite StdBackend to use new debouncer API and EventKind system

- Add 13 stress tests including undo/redo simulation

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-04 15:10:02 -06:00
Joe
5f91a0777a Implement new oneshot Vfs for CLI commands and enhance JSON5 serialization
- Added `new_oneshot` method to the Vfs struct, providing a backend suitable for one-time operations without file watching.
- Updated CLI commands (`fmt_project` and `syncback`) to utilize the new oneshot Vfs, improving error handling during file operations.
- Enhanced JSON5 serialization by introducing a custom serializer that directly builds a tree structure, improving performance and maintaining sorted keys.
- Improved error logging in JSON serialization to provide more context on failures.
2026-01-31 13:06:37 -06:00
Joe
8ede87e1cf Update dependencies and improve CLI argument handling
- Updated various dependencies in Cargo.toml and Cargo.lock, including `env_logger`, `fs-err`, and `clap`, to their latest versions for enhanced functionality and security.
- Refactored CLI argument handling in `build.rs` and `mod.rs` to align with the new `clap` version, improving error reporting and verbosity options.
- Added new packages such as `anstream`, `colorchoice`, and `jiff` to enhance project capabilities.
2026-01-30 05:22:39 -06:00
Joe
41363473b2 Enhance StdBackend with critical error handling and improve file watcher reliability
- Introduced `WatcherCriticalError` enum to categorize critical errors from the file watcher.
- Added a `CriticalErrorHandler` callback type for custom error handling in the watcher thread.
- Updated `StdBackend` to include a receiver for critical errors, allowing for better error management and logging.
- Refactored the watcher thread to handle errors gracefully instead of terminating the process, improving overall stability.
2026-01-30 05:02:06 -06:00
Joe
d002c4e821 Update dependencies and enhance logging in memofs
- Bump versions of `clap` and `zerocopy` packages in Cargo.lock.
- Add `log` dependency to memofs Cargo.toml.
- Improve logging in the file watcher thread to capture errors and trace events, enhancing reliability and debugging capabilities.
2026-01-29 11:37:21 -06:00
Joe
84ca8c4809 Update dependencies in Cargo.toml and Cargo.lock for improved compatibility
- Bumped versions of several dependencies, including `anyhow`, `backtrace`, `crossbeam-channel`, `csv`, `futures`, `hyper`, `log`, `num_cpus`, `rayon`, `reqwest`, `serde`, `serde_json`, `thiserror`, `tokio`, `profiling`, `yaml-rust2`, `blake3`, and `indexmap` to their latest releases for enhanced functionality and security.
- Updated submodule reference for `rbx-dom` to the latest commit.
- Adjusted dependencies in `crates/memofs` and `crates/rojo-insta-ext` to align with the updated versions, ensuring consistency across the project.
2026-01-21 19:46:27 -06:00
Ivan Matthew
2a1102fc55
Implement VFS Path normalization for improved cross-platform tree synchronization (#1201) 2026-01-19 15:04:59 -08:00
Micah
b89cc7f398
Release memofs v0.3.1 (#1175) 2025-11-27 12:32:57 -08:00
Ken Loeffler
93e9c51204
Fix rojo plugin install by adding Vfs::exists (#1169) 2025-11-21 07:04:34 -08:00
Micah
9b5a07191b
Implement Syncback to support converting Roblox files to a Rojo project (#937)
This is a very large commit.
Consider checking the linked PR for more information.
2025-11-19 09:21:33 -08:00
Micah
f3c423d77d
Fix the various lints (#1124) 2025-10-10 13:00:56 -07:00
Micah
227042d6b1
Add current maintainers to author field of Cargo.toml files (#1053) 2025-05-21 20:55:39 -07:00
Kenneth Loeffler
4b3470d30b
Fix removing trailing newlines by using str::replace in memofs (#903) 2024-04-17 11:55:23 -07:00
Kenneth Loeffler
87920964d7
Release memofs 0.3.0, bump Rojo dependency (#894) 2024-03-25 10:48:27 -07:00
Micah
856d43ce69
Update Cargo dependencies (#887) 2024-03-04 15:20:58 -08:00
Kenneth Loeffler
cf25eb0833
Normalize line endings to LF in Lua middleware (#854) 2024-02-12 14:58:03 -08:00
Kenneth Loeffler
3369b0d429
Downgrade to Notify 4, use FSEvents, use minimal recursive watches (#830) 2024-01-02 09:26:06 -08:00
Jack T
b43b45be8f
Upgrade to Notify 6 (#816) 2023-11-23 16:16:43 -08:00
Kenneth Loeffler
3cafbf7f1a
Use PollWatcher in memofs StdBackend on macOS (#783) 2023-09-26 15:41:33 -07:00
Kenneth Loeffler
0dabd8a1f6
Fix memofs clippy lints (#682)
The third in a series of PRs that aim to get CI passing
2023-06-30 11:04:04 -07:00
Lucien Greathouse
79b57b3359 Move memofs and rojo-insta-ext into crates folder 2022-05-26 04:23:44 -04:00