- Added `globIgnoredChildren` to `ApiInstanceMetadata` to manage child deletion behavior based on glob patterns.
- Updated `shouldDeleteChild` function to suppress deletions when `globIgnoredChildren` is true, differentiating from scripts-only mode.
- Enhanced tests to verify correct behavior of instance deletion based on the new metadata, ensuring robust handling of unknown scripts.
- Refactored snapshot handling to incorporate `globIgnoredChildren` in metadata, improving clarity and maintainability.
- Updated snapshot_dir_no_meta to set ignore_unknown_instances based on the presence of glob-ignored children.
- Added tests to verify that directories with glob-ignored children correctly flag ignore_unknown_instances as true, while those without do not.
- Improved clarity and maintainability of the snapshot logic by incorporating child filtering rules.
- Eliminated the `canonicalize` method from the `VfsBackend` trait and its implementations in `InMemoryFs`, `NoopBackend`, and `StdBackend`, simplifying the interface and reducing complexity.
- Updated various parts of the codebase to remove references to canonicalization, including tests and prefetch cache handling, ensuring consistency across the project.
- Adjusted logging and comments to reflect the removal of path normalization logic, enhancing clarity and maintainability.
These changes aim to streamline the VFS implementation by focusing on core functionalities without the overhead of path canonicalization.
- Simplified the iteration over directory entries in the `prefetch_project_files` function by using `into_iter().flatten()` instead of a nested loop.
- This change enhances code clarity while maintaining the existing functionality of collecting valid directory entries.
These modifications aim to streamline the code and improve maintainability in the prefetching process.
- Updated the `PrefetchCache` structure to include metadata about whether paths are files or directories, optimizing the cache's utility during VFS operations.
- Refactored the `prefetch_project_files` function to accept a `Project` object, allowing for more flexible path collection and improved logging of the prefetching process.
- Added tests to validate the behavior of the prefetch cache, ensuring correct handling of file reads, canonicalization, and cache depletion scenarios.
- Enhanced existing tests to cover new functionality, including concurrent reads and stress tests for snapshot determinism.
These changes aim to improve the performance and reliability of the VFS by refining the prefetching mechanism and ensuring accurate metadata handling.
- Introduced a `PrefetchCache` structure to store pre-read file contents and canonical paths, enhancing the speed of initial reads during snapshot building.
- Implemented parallel file reading using `rayon` and `walkdir` to populate the cache efficiently.
- Updated the `VfsInner` struct to include the prefetch cache and modified read methods to utilize it, reducing I/O operations.
- Added functions to set and clear the prefetch cache in the `Vfs` API, improving memory management during live operations.
- Enhanced tests to validate the functionality of the prefetch cache, ensuring correct behavior during file reads and canonicalization.
These changes aim to significantly improve the performance of the VFS by minimizing file access times during initial operations.
- Introduced a new matching session caching mechanism in the Lua matching module to enhance performance with native code generation.
- Rewrote the Lua matching algorithm to utilize session-based caching, reducing redundant computations and improving efficiency.
- Updated Rust implementations to include session caching, ensuring consistency across both Lua and Rust.
- Enhanced the `hydrate` function to support session management during instance hydration.
- Added comprehensive tests to validate the new caching mechanism and its integration with existing functionality.
These changes significantly improve the performance and maintainability of the matching process in the application.
- Introduced a comprehensive rewrite of the matching algorithm, replacing the previous multi-pass approach with a recursive change-count scoring function and greedy assignment strategy.
- Updated the `hydrate.lua` and `matching.lua` files to utilize the new algorithm, enhancing the efficiency of instance matching during hydration.
- Documented the new algorithm's structure, including fast-paths for Ref pins and (Name, ClassName) grouping, which significantly improve matching accuracy and performance.
- Ensured that the new implementation maintains compatibility with existing systems while providing a more robust solution for handling ambiguous instances.
These changes enhance the clarity and efficiency of the reconciliation process, paving the way for improved project management and user experience.
- Finalized the transition from a duplicate-name blacklist system to a comprehensive 3-pass matching algorithm for both syncback and forward operations, enhancing the handling of ambiguous paths.
- Integrated a new `matching.lua` module to facilitate instance matching during hydration, utilizing unique name matching, class name narrowing, and structural fingerprinting.
- Updated various components, including `ChangeBatcher`, `Reconciler`, and `snapshot` modules, to leverage the new matching logic, ensuring robust file management and improved performance.
- Removed legacy duplicate handling code and streamlined the encoding process for instances with duplicate names.
These changes significantly enhance the clarity and efficiency of file operations, paving the way for improved project management and user experience.
- Introduced a new plan addressing critical bugs, correctness issues, and refactors identified during the audit, including improvements to metadata handling and slugification logic.
- Replaced inline removal logic in `handle_api_write` with a new `remove_instance_at_path` method to ensure correct deletion of instances.
- Updated pre-seeding logic in `dir.rs` and `project.rs` to derive deduplication keys from actual filesystem paths instead of slugified names.
- Enhanced rename handling in `change_processor.rs` to prevent collisions by deduplicating names against existing siblings.
- Added tests for various scenarios, including handling of forbidden characters and ensuring correct deduplication behavior.
- Refactored child iteration in `dir.rs` and `project.rs` to sort children alphabetically for deterministic deduplication ordering.
- Consolidated CI pipeline steps into PowerShell and Bash scripts for improved usability and consistency.
- Updated documentation in `ci.md` to clarify the execution process and options for skipping the rbx-dom section.
- Enhanced error handling and reporting in the scripts to provide clearer feedback on CI step results.
- Streamlined formatting and linting processes for both Rojo and rbx-dom, ensuring a more efficient CI workflow.
- Updated the `slugify_name` function to strip trailing spaces and dots before checking for Windows reserved names, ensuring correct handling of inputs like "CON.".
- Added tests to validate the new slugification behavior, particularly for Windows reserved names with trailing characters.
- Refactored `syncback_dir_no_meta` to pre-seed `taken_names` from old children's slugified instance names, improving deduplication against existing siblings regardless of iteration order.
- Introduced a migration plan to address the deduplication issue where bare slugs were compared against full filenames, leading to undetected file-format collisions.
- Updated `name_for_inst` to return a deduplication key alongside the filename and metadata flag, ensuring callers use the bare slug for collision detection.
- Replaced disk-based seeding of `taken_names` with tree-based slugified names to maintain consistency and avoid extension mismatches.
- Refactored relevant modules including `dir.rs`, `project.rs`, and `api.rs` to utilize the new deduplication logic.
- Updated tests to validate the new deduplication behavior and ensure correct handling of file-format instances.
- Updated the vscode-rojo submodule to the latest commit.
- Marked all phases of the path encoding migration plan as completed, transitioning to a slugify and deduplication approach.
- Updated documentation to reflect changes in filename handling and metadata management.
- Introduced a new `SyncbackStats` module to track and report issues during syncback, including duplicate names, unknown classes, and properties.
- Updated the `syncback_loop` and related functions to utilize the stats tracker, logging summaries of issues encountered.
- Enhanced property filtering and instance syncing to record statistics on unknown classes and properties.
- Modified tests to ensure consistent output by normalizing temporary paths in snapshots.
- Added `regex` dependency for path normalization in tests.
- Introduced a new CSV input file with translation data for Spanish.
- Updated the `init.csv` file to include a new entry for "Rojo" with its context and example.
- Enhanced the syncback logic to conditionally emit warnings for duplicate child names based on the new `warn_duplicate_names` configuration in `SyncbackRules`.
- Added documentation for the new warning behavior in the `SyncbackRules` and `SyncbackSnapshot` implementations.
- Introduced new configuration files for testing, including `default.project.json5`, `client_in_folder`, and others.
- Updated existing project files to use JSON5 format, ensuring consistency across configurations.
- Added new Lua scripts and model files to support various test cases.
- Adjusted snapshot files to align with the new project structure and properties.
- Updated WhitespaceUtil to return distinct values for totalDiff and whitespaceDiff when differences exceed 100 lines, based on content changes.
- Introduced findDuplicateNames function in Reconciler to identify and skip instances with duplicate names, ensuring reliable syncing.
- Enhanced diffing logic to account for ambiguous paths and log skipped duplicates during the sync process.
- Improved error handling in syncback_dir_no_meta to log and skip children with duplicate names in the file system.
- Added `encode_path_name` and `decode_path_name` functions to handle encoding and decoding of invalid characters.
- Updated `SyncbackRules` to include an option for encoding Windows-invalid characters during syncback.
- Modified `name_for_inst` to conditionally encode invalid characters based on the new option.
- Adjusted `ServeSession` and `InstanceContext` to support the new encoding feature.
- Updated tests and snapshots to reflect changes in encoding behavior.
* Mostly mechanical port bits
* Almost there
* It builds again!
* Turn on all the code again
* Tests compiling but not passing
* Stub work for value resolution
* Implement resolution minus enums and derived properties
* Implement property descriptor resolution
* Update referent snapshots
* Update unions test project
Using a place file instead of a model yields better
error messages in Roblox Studio.
* Add easy shortcut to testing with local rbx-dom
* Update rbx-dom
* Add enum resolution
* Update init.meta.json to use UnresolvedValue
* Expand value resolution support, add test
* Filter SharedString values from web API
* Add 'property' builder method to InstanceSnapshot
* Change InstanceSnapshot/InstanceBuilder boundary
* Fix remove_file crash
* rustfmt
* Update to latest rbx_dom_lua
* Update dependencies, including rbx_dom_weak
* Update to latest rbx-dom
* Update dependencies
* Update rbx-dom, fixing more bugs
* Remove experimental warning on binary place builds
* Remove unused imports
* First take at flattening middleware for simpler code and better perf
* Undo debug prints
* Fix using wrong path in snapshot_from_vfs
* Disable some broken tests
* Re-enable (mistakenly?) disabled CSV test
* Fix some tests
* Update project file tests
* Fix benchmark
* vroom
* Port dir middleware
* Filter rules
* Directory metadata
* Project support
* Enable Lua support
* StringValue support
* CSV
* rbxm, rbxmx, and rbxlx
* JSON models
* Clean up some warnings
* Strip out PathMap
* Unwatch paths when they're reported as removed
* Fix 'rojo upload' behavior
* Upgrade to Insta 0.13.1
* Update dependencies
* Release 0.6.0-alpha.2
* Fix bad merge
* Replace MemoryBackend with InMemoryFs
* Sledgehammer tests into passing for now
* Txt middleware
* Update easy snapshot tests
* Lua tests
* Project middleware tests
* Try to fix test failures by sorting
* Port first set of serve session tests
* Add InMemoryFs::raise_event
* Finish porting serve session tests
* Remove UI code for introspecting VFS for now
* VFS docs
* Add Glob wrapper type with better serialization
* Introduce PathIgnoreRule struct
* Implement equality for Glob type
* Add PathIgnoreRule to InstanceContext
* Implement glob ignores in directory middleware
* Fix up filters
* Use Iterator::all instead of loop
* Add project-level configuration for glob ignores
* Add test project for glob ignores
* Wire up project file and snapshots to make glob ignores work
* Better codepaths for adding ignore rules with empty iterators
* Add test for globs inherited from parent projects
* Add test details, support glob ignores in nested projects
* Add feature flag for globs
* Switch to use ExactSizeIterator instead of size_hint
* Remove glob visitor
* Drop plugin context on the floor
* Remove redirect from old context to new context
* Pass InstanceContext via & instead of &mut reference
* Re-use context value in ChangeProcessor from metadata
This doesn't feel ideal. Though it's true that all directories are influenced by
any init scripts they have, the directory middleware shouldn't need to know
about Lua.
I don't really want to go back into working on the middleware chain since it
mostly feels like busywork when there are other things to build on in Rojo.
also all of this feels really complicated
This PR refactors all of the methods on `Vfs` from accepting `&mut self` to
accepting `&self` and keeping data wrapped in a mutex. This builds on previous
changes to make reference count file contents and cleans up the last places
where we're returning borrowed data out of the VFS interface.
Once this change lands, there are two possible directions we can go that I see:
* Conservative: Refactor all remaining `&mut Vfs` handles to `&Vfs`
* Interesting: Embrace ref counting by changing `Vfs` methods to accept `self:
Arc<Self>`, which makes the `VfsEntry` API no longer need an explicit `Vfs`
argument for its operations.
* Change VfsFetcher to be immutable with internal locking
* Refactor Vfs::would_be_resident
* Refactor Vfs::read_if_not_exists
* Refactor Vfs::raise_file_removed
* Refactor Vfs::raise_file_changed
* Add Vfs::get_internal as bits of Vfs::get
* Switch Vfs to use internal locking
* Migrate all Vfs methods from &mut self to &self
* Make VfsEntry access Vfs immutably
* Remove outer VFS locking (#260)
* Refactor all snapshot middleware to accept &Vfs instead of &mut Vfs
* Remove outer VFS Mutex across the board
