- Expanded the stress test plan to include file watcher stress tests alongside existing randomized fuzzing and deterministic tests, improving coverage of the two-way sync functionality. - Updated todos to reflect the addition of filesystem polling helpers and a FuzzState tracker for better test management. - Refined path handling in `api.rs` to ensure encoded names are consistently used across file operations, enhancing filesystem safety and reliability during syncback processes. - Improved documentation in the rojo rules to clarify the long-term goals and current state of the two-way sync strategy, emphasizing the need for feature parity with the existing CLI syncback command.
20 KiB
| name | overview | todos | isProject | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Stress Tests Two-Way Sync | Nuclear-grade stress tests, randomized fuzzing, and file watcher stress tests. Deterministic tests at 10+ cycles. Randomized fuzzer generates hundreds of random operation sequences. File watcher tests hammer the VFS pipeline with direct filesystem operations, init file shenanigans, singular/directory format flips, and editor save pattern simulations. |
|
false |
Nuclear-Grade Stress Tests + Fuzzing + File Watcher Stress
Context
16 commits ahead of origin. Major changes to:
- src/change_processor.rs -- event suppression (canonical+raw matching), pending recovery, rename/ClassName/Source handling,
overridden_source_path, canonicalize retry, init file detection - src/web/api.rs -- syncback deletion (two-phase,
actually_removedtracking),suppress_path(single canonical entry), standalone-to-directory conversion, encoded name handling - crates/memofs/src/lib.rs -- watches NOT removed on delete (for rapid recreate)
Existing tests: 25 in two_way_sync.rs (API-driven), ~10 in serve.rs (filesystem-driven). No tests exercise:
- Rapid multi-cycle operations
- Recovery mechanism under stress
- Init file type cycling on disk
- Singular/directory format conversion via filesystem
- Editor save patterns (atomic write, backup-rename-write)
- Concurrent filesystem + API operations
Files Changed
- tests/tests/two_way_sync.rs -- ALL new tests go here
- rojo-test/serve-tests/syncback_stress/ -- new fixture (5 ModuleScripts)
New Fixture
**rojo-test/serve-tests/syncback_stress/**
default.project.json5(DataModel with ReplicatedStorage ->src/)src/Alpha.luau,src/Bravo.luau,src/Charlie.luau,src/Delta.luau,src/Echo.luau
Each file: -- {Name} module\nreturn {}
New Infrastructure
Timing Helpers
fn send_update_fast(session, session_id, update) // 50ms sleep
fn send_update_no_wait(session, session_id, update) // 0ms sleep
fn send_removal_fast(session, session_id, ids) // 50ms sleep
fn wait_for_settle() // 800ms sleep
Tree Polling Helpers (for file-watcher tests)
/// Poll tree until an instance with `name` under `parent_id` has Source containing `expected`.
/// Panics after `timeout_ms` if not found.
fn poll_tree_source(
session: &TestServeSession,
parent_id: Ref,
instance_name: &str,
expected_content: &str,
timeout_ms: u64,
)
/// Poll tree until an instance with `name` exists under `parent_id`.
fn poll_tree_has_instance(
session: &TestServeSession,
parent_id: Ref,
instance_name: &str,
timeout_ms: u64,
) -> Ref
/// Poll tree until NO instance with `name` exists under `parent_id`.
fn poll_tree_no_instance(
session: &TestServeSession,
parent_id: Ref,
instance_name: &str,
timeout_ms: u64,
)
/// Poll tree until instance `name` has the expected ClassName.
fn poll_tree_class(
session: &TestServeSession,
parent_id: Ref,
instance_name: &str,
expected_class: &str,
timeout_ms: u64,
)
These all use a 50ms polling interval with get_api_read(). The typical timeout for file watcher propagation is 2000ms (50ms debounce + processing time + safety margin).
Deterministic PRNG
struct XorShift64(u64);
impl XorShift64 {
fn new(seed: u64) -> Self { ... }
fn next(&mut self) -> u64 { ... }
fn range(&mut self, min: u64, max: u64) -> u64 { ... }
fn choose<'a, T>(&mut self, items: &'a [T]) -> &'a T { ... }
}
FuzzOp / FuzzState / verify_instance_file
(Same as previous plan revision -- FuzzOp enum with 7 variants, FuzzState tracking current_name/class/source, verify_instance_file checking file existence + extension + content + no stale files)
PART 1: API-Driven Stress Tests (Tests 26-41)
All tests use send_update_fast (50ms) or send_update_no_wait (0ms) and verify final state.
A. Rapid Source (26-27)
- **26:
rapid_source_writes_10x**-- 10 Source writes at 50ms gaps. Fixture:syncback_write - **27:
rapid_source_writes_no_wait_10x**-- 10 Source writes with 0ms gaps. Fixture:syncback_write
B. Rapid Rename (28-29)
- **28:
rapid_rename_chain_10x**-- 10 sequential renames (re-read tree each time). Fixture:syncback_write - **29:
rapid_rename_chain_directory_10x**-- 10 renames of directory-format script, verify children intact. Fixture:syncback_format_transitions
C. Rapid ClassName (30-31)
- **30:
rapid_classname_cycle_10x**-- 10 class transitions (Module->Script->Local->...). Fixture:syncback_write - **31:
rapid_classname_cycle_directory_init_10x**-- 10 class transitions on directory init, verify children survive. Fixture:syncback_format_transitions
D. Combined Blitz (32-34)
- **32:
combined_rename_classname_source_blitz_10x**-- All 3 in one update, 10 rounds cycling classes. Fixture:syncback_write - **33:
combined_rename_and_source_rapid_10x**-- Rename + Source, 10 rounds. Fixture:syncback_write - **34:
combined_classname_and_source_rapid_10x**-- ClassName + Source, 10 rounds. Fixture:syncback_write
E. Multi-Instance (35-36)
- **35:
multi_instance_source_update_single_request**-- Single WriteRequest updating 5 instances. Fixture:syncback_stress - **36:
multi_instance_rename_single_request**-- Single WriteRequest renaming 5 instances. Fixture:syncback_stress
F. Delete + Recreate Race (37-38)
- **37:
delete_and_recreate_via_filesystem_recovery**-- API delete, thenfs::writefile back, wait for recovery. Fixture:syncback_write - **38:
rapid_delete_recreate_cycle_5x**-- 5 cycles of API delete + filesystem recreate. Fixture:syncback_write
G. Echo Suppression (39-40)
- **39:
echo_suppression_rapid_adds_10x**-- 10 rapid instance additions, verify cursor doesn't explode. Fixture:syncback_write - **40:
echo_suppression_mixed_operations**-- 2 adds + 2 updates + 1 removal in one request. Fixture:syncback_stress
H. Encoded Names (41)
- **41:
encoded_name_rapid_rename_chain**-- RenameWhat?Modulethrough 5 special-char names. Fixture:syncback_encoded_names
PART 2: Randomized Fuzzing (Tests 42-46)
Seeded XorShift64 PRNG. On failure, prints seed + iteration for exact replay.
- **42:
fuzz_source_and_rename_200_iterations**-- Random Source writes and renames, 3-8 ops per iteration. Fixture:syncback_write - **43:
fuzz_classname_cycling_200_iterations**-- Random ClassName transitions, 3-8 ops per iteration. Fixture:syncback_write - **44:
fuzz_combined_operations_200_iterations**-- ALL 7 FuzzOp variants random, 3-8 ops per iteration. Fixture:syncback_write - **45:
fuzz_multi_instance_100_iterations**-- Random ops across 1-5 instances per WriteRequest. Fixture:syncback_stress - **46:
fuzz_directory_format_operations_100_iterations**-- Random rename/class on directory-format script. Fixture:syncback_format_transitions
PART 3: File Watcher Stress Tests (Tests 47-66)
These exercise the filesystem -> VFS -> ChangeProcessor -> Tree pipeline by directly manipulating files with fs::write, fs::rename, fs::remove_file, fs::create_dir, etc. Verification uses the poll_tree_* helpers to wait for the tree to update via get_api_read().
Pattern for all watcher tests:
- Get the ReplicatedStorage ID via API
- Directly manipulate files in
session.path().join("src") - Poll the tree until it reflects the change (2000ms timeout)
- Assert final state
I. Rapid Source Edits on Disk (47-48)
**Test 47: watcher_rapid_source_edits_on_disk_10x**
- Write
existing.luaudirectly 10 times viafs::writewith 30ms gaps - Each write has distinct content:
"-- disk v1"through"-- disk v10" - Poll tree until Source contains
"-- disk v10" - Exercises: debouncer coalescing, snapshot regeneration, no lost updates
- Fixture:
syncback_write
**Test 48: watcher_burst_writes_100x**
- Write
existing.luau100 times with ZERO delay (as fast as possible) - Final content:
"-- burst final" - Poll tree until Source contains
"-- burst final" - Exercises: debouncer under extreme load, final state correctness
- Fixture:
syncback_write
J. Filesystem Rename (49-50)
**Test 49: watcher_filesystem_rename_chain_10x**
- Rename
existing.luauon disk through 10 names:R1.luau->R2.luau-> ... ->R10.luau - 30ms between renames
- Poll tree until instance named
R10exists - Verify: no instances with intermediate names, content preserved
- Exercises: VFS Remove+Create events from renames, tree path tracking
- Fixture:
syncback_write
**Test 50: watcher_rename_with_content_change**
- Write new content to a temp file, then
fs::renametemp ->existing.luau(atomic overwrite) - Poll tree until Source reflects new content
- Exercises: editor-style atomic writes via rename
- Fixture:
syncback_write
K. Delete + Recreate via Filesystem (51-53)
**Test 51: watcher_delete_recreate_immediate**
fs::remove_file("existing.luau"), immediatelyfs::write("existing.luau", new_content)- Poll tree until Source contains new content
- Exercises:
pending_recoverymechanism -- VFS Remove may arrive but file already exists on disk - Fixture:
syncback_write
**Test 52: watcher_delete_recreate_cycle_5x**
- 5 cycles of:
fs::remove_file-> 50ms delay ->fs::writewith new content - Each cycle uses distinct content
- After each cycle, poll tree for the new content
- Exercises: repeated recovery under stress (the recovery delay is 200ms, sweep is 500ms)
- Fixture:
syncback_write
**Test 53: watcher_delete_recreate_different_content**
- Delete
existing.luau, wait 300ms (let removal propagate), recreate with completely different content - Poll tree until Source matches the new content AND instance still exists
- Exercises: instance removal + recreation via recovery mechanism
- Fixture:
syncback_write
L. Init File Shenanigans (54-57)
All use syncback_format_transitions fixture which has DirModuleWithChildren/init.luau + ChildA.luau + ChildB.luau.
**Test 54: watcher_edit_init_file**
fs::write("DirModuleWithChildren/init.luau", "-- edited init")- Poll tree until
DirModuleWithChildreninstance Source contains"-- edited init" - Verify children still exist
- Exercises: init file instigating_source -> parent directory snapshot
**Test 55: watcher_init_type_cycling_10x**
- 10 cycles of renaming the init file:
init.luau->init.server.luau->init.local.luau->init.luau-> ... - After each rename, poll tree until ClassName matches expected (ModuleScript/Script/LocalScript)
- Verify children (ChildA, ChildB) survive all 10 transitions
- Exercises: init file detection (
find_init_file), snapshot_path resolution, tree ClassName updates
**Test 56: watcher_delete_init_file**
- Delete
DirModuleWithChildren/init.luau - Poll tree until
DirModuleWithChildrenClassName changes (no longer a ModuleScript since init file is gone -- becomes a Folder) - Verify children still exist (directory still exists on disk)
- Exercises: init file removal edge case
**Test 57: watcher_replace_init_file_type**
- Delete
init.luau, immediately createinit.server.luauwith same content - Poll tree until ClassName is "Script"
- Verify children unaffected
- Exercises: atomic init file type swap
M. Singular <-> Directory Format Conversion (58-60)
**Test 58: watcher_standalone_to_directory_conversion**
- Start with
StandaloneModule.luau(standalone ModuleScript) - On disk:
fs::remove_file("StandaloneModule.luau"),fs::create_dir("StandaloneModule"),fs::write("StandaloneModule/init.luau", original_content),fs::write("StandaloneModule/ChildNew.luau", child_content) - Poll tree until
StandaloneModuleinstance has a child namedChildNew - Verify
StandaloneModuleSource matches original content - Exercises: complete format transition via filesystem, VFS picks up directory + init + children
- Fixture:
syncback_format_transitions
**Test 59: watcher_directory_to_standalone_conversion**
- Start with
DirModuleWithChildren/(directory with init.luau + children) - Read original init.luau content
- On disk:
fs::remove_dir_all("DirModuleWithChildren"),fs::write("DirModuleWithChildren.luau", original_content) - Poll tree until
DirModuleWithChildrenhas NO children (standalone script) - Verify Source matches
- Exercises: directory removal + standalone file creation detected by watcher
- Fixture:
syncback_format_transitions
**Test 60: watcher_format_flip_flop_5x**
- 5 cycles of: standalone -> directory -> standalone -> directory -> standalone
- Start with
StandaloneModule.luau - Each "to directory" step: remove file, create dir + init.luau + child
- Each "to standalone" step: remove dir, create .luau file
- Verify final state correct after all 5 cycles
- Exercises: the ultimate format conversion stress test
- Fixture:
syncback_format_transitions
N. Editor Save Patterns (61-62)
**Test 61: watcher_atomic_save_pattern**
- Simulate VSCode/Vim atomic save: write to
existing.luau.tmp, thenfs::rename("existing.luau.tmp", "existing.luau") - Poll tree until Source reflects new content
- Do this 5 times with different content
- Exercises: the most common real-world save pattern
- Fixture:
syncback_write
**Test 62: watcher_backup_rename_write_pattern**
- Simulate editor backup-rename-write:
fs::rename("existing.luau", "existing.luau.bak"), thenfs::write("existing.luau", new_content) - Poll tree until Source reflects new content
- Clean up .bak file
- Exercises: rename-away + create-new pattern (generates Remove + Create events)
- Fixture:
syncback_write
O. Parent Directory Operations (63-64)
**Test 63: watcher_parent_directory_rename**
- Rename
DirModuleWithChildrendirectory toRenamedDiron disk - Poll tree until instance
RenamedDirappears with children intact - Verify
DirModuleWithChildreninstance is gone - Exercises: directory rename generates Remove(old/init.luau) + Create(new/init.luau), tree rebuilt
- Fixture:
syncback_format_transitions
**Test 64: watcher_parent_directory_delete_all**
fs::remove_dir_all("DirModuleWithChildren")- Poll tree until
DirModuleWithChildreninstance is gone - Verify no orphaned children remain
- Exercises: recursive delete generates Remove events for all children + init file
- Fixture:
syncback_format_transitions
P. Concurrent Filesystem + API (65-66)
**Test 65: watcher_filesystem_and_api_concurrent**
- Simultaneously:
fs::writetoStandaloneModule.luauon disk AND send API Source update toStandaloneScript - Both operations target different instances to avoid data races
- Poll tree until BOTH instances reflect their respective changes
- Exercises: VFS events and tree mutation channel both active at same time
- Fixture:
syncback_format_transitions
**Test 66: watcher_multi_file_simultaneous_edits**
- Write to 5 different files on disk simultaneously (Alpha through Echo) with distinct content
- Poll tree until ALL 5 instances reflect new Source
- Exercises: multiple VFS events in quick succession, all processed correctly
- Fixture:
syncback_stress
Implementation Details
Polling Timeout Strategy
File watcher tests use 2000ms timeout (generous for: 50ms debounce + processing + recovery + Windows slowness). Polling interval is 50ms.
Test Isolation
Each test uses run_serve_test which copies the fixture to a temp directory and spawns a fresh rojo serve process. Tests can freely mutate files without affecting other tests.
PRNG Reproducibility
Fuzz tests print "Fuzz test failed at seed {seed}. Replay with this seed to reproduce." on failure.
State Reset Between Fuzz Iterations
After each fuzz iteration: rename instance back to original name, restore original class, write original source, wait 300ms. Avoids restarting the serve session (which would be slow).
File Watcher Test Timing
- After
fs::write: poll immediately (debouncer needs ~50-100ms) - After
fs::rename: poll immediately (rename events come fast) - After
fs::remove_file+fs::write(delete+recreate): poll with 800ms initial wait (recovery delay 200ms + sweep 500ms) - After directory operations: poll with extra buffer (multiple events to process)
Total Test Count
- Part 1 (API stress): 16 tests (26-41)
- Part 2 (Fuzzing): 5 tests (42-46) with 900 total iterations
- Part 3 (File watcher): 20 tests (47-66)
- Total: 41 new tests