rojo/.cursor/plans/stress_tests_two-way_sync_baeda112.plan.md
Joe bc77b26f07 Enhance stress tests and path handling for two-way sync
- Expanded the stress test plan to include file watcher stress tests alongside existing randomized fuzzing and deterministic tests, improving coverage of the two-way sync functionality.
- Updated todos to reflect the addition of filesystem polling helpers and a FuzzState tracker for better test management.
- Refined path handling in `api.rs` to ensure encoded names are consistently used across file operations, enhancing filesystem safety and reliability during syncback processes.
- Improved documentation in the rojo rules to clarify the long-term goals and current state of the two-way sync strategy, emphasizing the need for feature parity with the existing CLI syncback command.
2026-02-05 22:02:37 -06:00

20 KiB

name overview todos isProject
Stress Tests Two-Way Sync Nuclear-grade stress tests, randomized fuzzing, and file watcher stress tests. Deterministic tests at 10+ cycles. Randomized fuzzer generates hundreds of random operation sequences. File watcher tests hammer the VFS pipeline with direct filesystem operations, init file shenanigans, singular/directory format flips, and editor save pattern simulations.
id content status
fixture Create syncback_stress fixture (default.project.json5 + 5 .luau files) pending
id content status
helpers Add timing helpers (send_update_fast, send_update_no_wait, wait_for_settle), lookup helpers (get_stress_instances), and filesystem polling helpers (poll_tree_source, poll_tree_has_instance, poll_tree_no_instance) pending
id content status
prng Add XorShift64 PRNG, FuzzOp enum, FuzzState tracker, and verify_instance_file helper pending
id content status
rapid-source Tests 26-27: rapid_source_writes_10x, rapid_source_writes_no_wait_10x pending
id content status
rapid-rename Tests 28-29: rapid_rename_chain_10x, rapid_rename_chain_directory_10x pending
id content status
rapid-classname Tests 30-31: rapid_classname_cycle_10x, rapid_classname_cycle_directory_init_10x pending
id content status
combined-blitz Tests 32-34: combined_rename_classname_source_blitz_10x, combined_rename_and_source_rapid_10x, combined_classname_and_source_rapid_10x pending
id content status
multi-instance Tests 35-36: multi_instance_source_update_single_request, multi_instance_rename_single_request pending
id content status
delete-recreate Tests 37-38: delete_and_recreate_via_filesystem_recovery, rapid_delete_recreate_cycle_5x pending
id content status
echo-stress Tests 39-40: echo_suppression_rapid_adds_10x, echo_suppression_mixed_operations pending
id content status
encoded-stress Test 41: encoded_name_rapid_rename_chain pending
id content status
fuzz-source-rename Test 42: fuzz_source_and_rename_200_iterations pending
id content status
fuzz-classname Test 43: fuzz_classname_cycling_200_iterations pending
id content status
fuzz-combined Test 44: fuzz_combined_operations_200_iterations pending
id content status
fuzz-multi-instance Test 45: fuzz_multi_instance_100_iterations pending
id content status
fuzz-directory Test 46: fuzz_directory_format_operations_100_iterations pending
id content status
watcher-rapid-edits Tests 47-48: watcher_rapid_source_edits_on_disk_10x, watcher_burst_writes_100x pending
id content status
watcher-rename Tests 49-50: watcher_filesystem_rename_chain_10x, watcher_rename_with_content_change pending
id content status
watcher-delete-recreate Tests 51-53: watcher_delete_recreate_immediate, watcher_delete_recreate_cycle_5x, watcher_delete_recreate_different_content pending
id content status
watcher-init Tests 54-57: watcher_edit_init_file, watcher_init_type_cycling_10x, watcher_delete_init_file, watcher_replace_init_file_type pending
id content status
watcher-format-flip Tests 58-60: watcher_standalone_to_directory_conversion, watcher_directory_to_standalone_conversion, watcher_format_flip_flop_5x pending
id content status
watcher-editor-patterns Tests 61-62: watcher_atomic_save_pattern, watcher_backup_rename_write_pattern pending
id content status
watcher-parent-dir Tests 63-64: watcher_parent_directory_rename, watcher_parent_directory_delete_all pending
id content status
watcher-concurrent Tests 65-66: watcher_filesystem_and_api_concurrent, watcher_multi_file_simultaneous_edits pending
false

Nuclear-Grade Stress Tests + Fuzzing + File Watcher Stress

Context

16 commits ahead of origin. Major changes to:

  • src/change_processor.rs -- event suppression (canonical+raw matching), pending recovery, rename/ClassName/Source handling, overridden_source_path, canonicalize retry, init file detection
  • src/web/api.rs -- syncback deletion (two-phase, actually_removed tracking), suppress_path (single canonical entry), standalone-to-directory conversion, encoded name handling
  • crates/memofs/src/lib.rs -- watches NOT removed on delete (for rapid recreate)

Existing tests: 25 in two_way_sync.rs (API-driven), ~10 in serve.rs (filesystem-driven). No tests exercise:

  • Rapid multi-cycle operations
  • Recovery mechanism under stress
  • Init file type cycling on disk
  • Singular/directory format conversion via filesystem
  • Editor save patterns (atomic write, backup-rename-write)
  • Concurrent filesystem + API operations

Files Changed

New Fixture

**rojo-test/serve-tests/syncback_stress/**

  • default.project.json5 (DataModel with ReplicatedStorage -> src/)
  • src/Alpha.luau, src/Bravo.luau, src/Charlie.luau, src/Delta.luau, src/Echo.luau

Each file: -- {Name} module\nreturn {}


New Infrastructure

Timing Helpers

fn send_update_fast(session, session_id, update)  // 50ms sleep
fn send_update_no_wait(session, session_id, update) // 0ms sleep
fn send_removal_fast(session, session_id, ids)     // 50ms sleep
fn wait_for_settle()                                // 800ms sleep

Tree Polling Helpers (for file-watcher tests)

/// Poll tree until an instance with `name` under `parent_id` has Source containing `expected`.
/// Panics after `timeout_ms` if not found.
fn poll_tree_source(
    session: &TestServeSession,
    parent_id: Ref,
    instance_name: &str,
    expected_content: &str,
    timeout_ms: u64,
)

/// Poll tree until an instance with `name` exists under `parent_id`.
fn poll_tree_has_instance(
    session: &TestServeSession,
    parent_id: Ref,
    instance_name: &str,
    timeout_ms: u64,
) -> Ref

/// Poll tree until NO instance with `name` exists under `parent_id`.
fn poll_tree_no_instance(
    session: &TestServeSession,
    parent_id: Ref,
    instance_name: &str,
    timeout_ms: u64,
)

/// Poll tree until instance `name` has the expected ClassName.
fn poll_tree_class(
    session: &TestServeSession,
    parent_id: Ref,
    instance_name: &str,
    expected_class: &str,
    timeout_ms: u64,
)

These all use a 50ms polling interval with get_api_read(). The typical timeout for file watcher propagation is 2000ms (50ms debounce + processing time + safety margin).

Deterministic PRNG

struct XorShift64(u64);
impl XorShift64 {
    fn new(seed: u64) -> Self { ... }
    fn next(&mut self) -> u64 { ... }
    fn range(&mut self, min: u64, max: u64) -> u64 { ... }
    fn choose<'a, T>(&mut self, items: &'a [T]) -> &'a T { ... }
}

FuzzOp / FuzzState / verify_instance_file

(Same as previous plan revision -- FuzzOp enum with 7 variants, FuzzState tracking current_name/class/source, verify_instance_file checking file existence + extension + content + no stale files)


PART 1: API-Driven Stress Tests (Tests 26-41)

All tests use send_update_fast (50ms) or send_update_no_wait (0ms) and verify final state.

A. Rapid Source (26-27)

  • **26: rapid_source_writes_10x** -- 10 Source writes at 50ms gaps. Fixture: syncback_write
  • **27: rapid_source_writes_no_wait_10x** -- 10 Source writes with 0ms gaps. Fixture: syncback_write

B. Rapid Rename (28-29)

  • **28: rapid_rename_chain_10x** -- 10 sequential renames (re-read tree each time). Fixture: syncback_write
  • **29: rapid_rename_chain_directory_10x** -- 10 renames of directory-format script, verify children intact. Fixture: syncback_format_transitions

C. Rapid ClassName (30-31)

  • **30: rapid_classname_cycle_10x** -- 10 class transitions (Module->Script->Local->...). Fixture: syncback_write
  • **31: rapid_classname_cycle_directory_init_10x** -- 10 class transitions on directory init, verify children survive. Fixture: syncback_format_transitions

D. Combined Blitz (32-34)

  • **32: combined_rename_classname_source_blitz_10x** -- All 3 in one update, 10 rounds cycling classes. Fixture: syncback_write
  • **33: combined_rename_and_source_rapid_10x** -- Rename + Source, 10 rounds. Fixture: syncback_write
  • **34: combined_classname_and_source_rapid_10x** -- ClassName + Source, 10 rounds. Fixture: syncback_write

E. Multi-Instance (35-36)

  • **35: multi_instance_source_update_single_request** -- Single WriteRequest updating 5 instances. Fixture: syncback_stress
  • **36: multi_instance_rename_single_request** -- Single WriteRequest renaming 5 instances. Fixture: syncback_stress

F. Delete + Recreate Race (37-38)

  • **37: delete_and_recreate_via_filesystem_recovery** -- API delete, then fs::write file back, wait for recovery. Fixture: syncback_write
  • **38: rapid_delete_recreate_cycle_5x** -- 5 cycles of API delete + filesystem recreate. Fixture: syncback_write

G. Echo Suppression (39-40)

  • **39: echo_suppression_rapid_adds_10x** -- 10 rapid instance additions, verify cursor doesn't explode. Fixture: syncback_write
  • **40: echo_suppression_mixed_operations** -- 2 adds + 2 updates + 1 removal in one request. Fixture: syncback_stress

H. Encoded Names (41)

  • **41: encoded_name_rapid_rename_chain** -- Rename What?Module through 5 special-char names. Fixture: syncback_encoded_names

PART 2: Randomized Fuzzing (Tests 42-46)

Seeded XorShift64 PRNG. On failure, prints seed + iteration for exact replay.

  • **42: fuzz_source_and_rename_200_iterations** -- Random Source writes and renames, 3-8 ops per iteration. Fixture: syncback_write
  • **43: fuzz_classname_cycling_200_iterations** -- Random ClassName transitions, 3-8 ops per iteration. Fixture: syncback_write
  • **44: fuzz_combined_operations_200_iterations** -- ALL 7 FuzzOp variants random, 3-8 ops per iteration. Fixture: syncback_write
  • **45: fuzz_multi_instance_100_iterations** -- Random ops across 1-5 instances per WriteRequest. Fixture: syncback_stress
  • **46: fuzz_directory_format_operations_100_iterations** -- Random rename/class on directory-format script. Fixture: syncback_format_transitions

PART 3: File Watcher Stress Tests (Tests 47-66)

These exercise the filesystem -> VFS -> ChangeProcessor -> Tree pipeline by directly manipulating files with fs::write, fs::rename, fs::remove_file, fs::create_dir, etc. Verification uses the poll_tree_* helpers to wait for the tree to update via get_api_read().

Pattern for all watcher tests:

  1. Get the ReplicatedStorage ID via API
  2. Directly manipulate files in session.path().join("src")
  3. Poll the tree until it reflects the change (2000ms timeout)
  4. Assert final state

I. Rapid Source Edits on Disk (47-48)

**Test 47: watcher_rapid_source_edits_on_disk_10x**

  • Write existing.luau directly 10 times via fs::write with 30ms gaps
  • Each write has distinct content: "-- disk v1" through "-- disk v10"
  • Poll tree until Source contains "-- disk v10"
  • Exercises: debouncer coalescing, snapshot regeneration, no lost updates
  • Fixture: syncback_write

**Test 48: watcher_burst_writes_100x**

  • Write existing.luau 100 times with ZERO delay (as fast as possible)
  • Final content: "-- burst final"
  • Poll tree until Source contains "-- burst final"
  • Exercises: debouncer under extreme load, final state correctness
  • Fixture: syncback_write

J. Filesystem Rename (49-50)

**Test 49: watcher_filesystem_rename_chain_10x**

  • Rename existing.luau on disk through 10 names: R1.luau -> R2.luau -> ... -> R10.luau
  • 30ms between renames
  • Poll tree until instance named R10 exists
  • Verify: no instances with intermediate names, content preserved
  • Exercises: VFS Remove+Create events from renames, tree path tracking
  • Fixture: syncback_write

**Test 50: watcher_rename_with_content_change**

  • Write new content to a temp file, then fs::rename temp -> existing.luau (atomic overwrite)
  • Poll tree until Source reflects new content
  • Exercises: editor-style atomic writes via rename
  • Fixture: syncback_write

K. Delete + Recreate via Filesystem (51-53)

**Test 51: watcher_delete_recreate_immediate**

  • fs::remove_file("existing.luau"), immediately fs::write("existing.luau", new_content)
  • Poll tree until Source contains new content
  • Exercises: pending_recovery mechanism -- VFS Remove may arrive but file already exists on disk
  • Fixture: syncback_write

**Test 52: watcher_delete_recreate_cycle_5x**

  • 5 cycles of: fs::remove_file -> 50ms delay -> fs::write with new content
  • Each cycle uses distinct content
  • After each cycle, poll tree for the new content
  • Exercises: repeated recovery under stress (the recovery delay is 200ms, sweep is 500ms)
  • Fixture: syncback_write

**Test 53: watcher_delete_recreate_different_content**

  • Delete existing.luau, wait 300ms (let removal propagate), recreate with completely different content
  • Poll tree until Source matches the new content AND instance still exists
  • Exercises: instance removal + recreation via recovery mechanism
  • Fixture: syncback_write

L. Init File Shenanigans (54-57)

All use syncback_format_transitions fixture which has DirModuleWithChildren/init.luau + ChildA.luau + ChildB.luau.

**Test 54: watcher_edit_init_file**

  • fs::write("DirModuleWithChildren/init.luau", "-- edited init")
  • Poll tree until DirModuleWithChildren instance Source contains "-- edited init"
  • Verify children still exist
  • Exercises: init file instigating_source -> parent directory snapshot

**Test 55: watcher_init_type_cycling_10x**

  • 10 cycles of renaming the init file: init.luau -> init.server.luau -> init.local.luau -> init.luau -> ...
  • After each rename, poll tree until ClassName matches expected (ModuleScript/Script/LocalScript)
  • Verify children (ChildA, ChildB) survive all 10 transitions
  • Exercises: init file detection (find_init_file), snapshot_path resolution, tree ClassName updates

**Test 56: watcher_delete_init_file**

  • Delete DirModuleWithChildren/init.luau
  • Poll tree until DirModuleWithChildren ClassName changes (no longer a ModuleScript since init file is gone -- becomes a Folder)
  • Verify children still exist (directory still exists on disk)
  • Exercises: init file removal edge case

**Test 57: watcher_replace_init_file_type**

  • Delete init.luau, immediately create init.server.luau with same content
  • Poll tree until ClassName is "Script"
  • Verify children unaffected
  • Exercises: atomic init file type swap

M. Singular <-> Directory Format Conversion (58-60)

**Test 58: watcher_standalone_to_directory_conversion**

  • Start with StandaloneModule.luau (standalone ModuleScript)
  • On disk: fs::remove_file("StandaloneModule.luau"), fs::create_dir("StandaloneModule"), fs::write("StandaloneModule/init.luau", original_content), fs::write("StandaloneModule/ChildNew.luau", child_content)
  • Poll tree until StandaloneModule instance has a child named ChildNew
  • Verify StandaloneModule Source matches original content
  • Exercises: complete format transition via filesystem, VFS picks up directory + init + children
  • Fixture: syncback_format_transitions

**Test 59: watcher_directory_to_standalone_conversion**

  • Start with DirModuleWithChildren/ (directory with init.luau + children)
  • Read original init.luau content
  • On disk: fs::remove_dir_all("DirModuleWithChildren"), fs::write("DirModuleWithChildren.luau", original_content)
  • Poll tree until DirModuleWithChildren has NO children (standalone script)
  • Verify Source matches
  • Exercises: directory removal + standalone file creation detected by watcher
  • Fixture: syncback_format_transitions

**Test 60: watcher_format_flip_flop_5x**

  • 5 cycles of: standalone -> directory -> standalone -> directory -> standalone
  • Start with StandaloneModule.luau
  • Each "to directory" step: remove file, create dir + init.luau + child
  • Each "to standalone" step: remove dir, create .luau file
  • Verify final state correct after all 5 cycles
  • Exercises: the ultimate format conversion stress test
  • Fixture: syncback_format_transitions

N. Editor Save Patterns (61-62)

**Test 61: watcher_atomic_save_pattern**

  • Simulate VSCode/Vim atomic save: write to existing.luau.tmp, then fs::rename("existing.luau.tmp", "existing.luau")
  • Poll tree until Source reflects new content
  • Do this 5 times with different content
  • Exercises: the most common real-world save pattern
  • Fixture: syncback_write

**Test 62: watcher_backup_rename_write_pattern**

  • Simulate editor backup-rename-write: fs::rename("existing.luau", "existing.luau.bak"), then fs::write("existing.luau", new_content)
  • Poll tree until Source reflects new content
  • Clean up .bak file
  • Exercises: rename-away + create-new pattern (generates Remove + Create events)
  • Fixture: syncback_write

O. Parent Directory Operations (63-64)

**Test 63: watcher_parent_directory_rename**

  • Rename DirModuleWithChildren directory to RenamedDir on disk
  • Poll tree until instance RenamedDir appears with children intact
  • Verify DirModuleWithChildren instance is gone
  • Exercises: directory rename generates Remove(old/init.luau) + Create(new/init.luau), tree rebuilt
  • Fixture: syncback_format_transitions

**Test 64: watcher_parent_directory_delete_all**

  • fs::remove_dir_all("DirModuleWithChildren")
  • Poll tree until DirModuleWithChildren instance is gone
  • Verify no orphaned children remain
  • Exercises: recursive delete generates Remove events for all children + init file
  • Fixture: syncback_format_transitions

P. Concurrent Filesystem + API (65-66)

**Test 65: watcher_filesystem_and_api_concurrent**

  • Simultaneously: fs::write to StandaloneModule.luau on disk AND send API Source update to StandaloneScript
  • Both operations target different instances to avoid data races
  • Poll tree until BOTH instances reflect their respective changes
  • Exercises: VFS events and tree mutation channel both active at same time
  • Fixture: syncback_format_transitions

**Test 66: watcher_multi_file_simultaneous_edits**

  • Write to 5 different files on disk simultaneously (Alpha through Echo) with distinct content
  • Poll tree until ALL 5 instances reflect new Source
  • Exercises: multiple VFS events in quick succession, all processed correctly
  • Fixture: syncback_stress

Implementation Details

Polling Timeout Strategy

File watcher tests use 2000ms timeout (generous for: 50ms debounce + processing + recovery + Windows slowness). Polling interval is 50ms.

Test Isolation

Each test uses run_serve_test which copies the fixture to a temp directory and spawns a fresh rojo serve process. Tests can freely mutate files without affecting other tests.

PRNG Reproducibility

Fuzz tests print "Fuzz test failed at seed {seed}. Replay with this seed to reproduce." on failure.

State Reset Between Fuzz Iterations

After each fuzz iteration: rename instance back to original name, restore original class, write original source, wait 300ms. Avoids restarting the serve session (which would be slow).

File Watcher Test Timing

  • After fs::write: poll immediately (debouncer needs ~50-100ms)
  • After fs::rename: poll immediately (rename events come fast)
  • After fs::remove_file + fs::write (delete+recreate): poll with 800ms initial wait (recovery delay 200ms + sweep 500ms)
  • After directory operations: poll with extra buffer (multiple events to process)

Total Test Count

  • Part 1 (API stress): 16 tests (26-41)
  • Part 2 (Fuzzing): 5 tests (42-46) with 900 total iterations
  • Part 3 (File watcher): 20 tests (47-66)
  • Total: 41 new tests