---
name: Parallel IO audit fixes
overview: "Fix three issues found during audit of the parallelization/optimization commits: a data loss bug in the orphan scanner, a duplicate-children bug in the prefetch cache, and a nested-detection correctness issue, plus a small DRY refactor."
todos:
  - id: fix1-hidden-dirs
    content: Fix orphan scanner walkdir to prune hidden directories using filter_entry()
    status: completed
  - id: fix2-dedup-roots
    content: Deduplicate overlapping $path roots in prefetch_project_files before walking
    status: completed
  - id: fix3-nested-detection
    content: Replace broken binary search nested detection with O(n log n) ancestor-tracking approach
    status: completed
  - id: fix4-unc-refactor
    content: Extract shared strip_unc_prefix helper from 3 inline copies
    status: completed
  - id: test-hidden-dir
    content: Add integration test for hidden directory survival in clean syncback
    status: completed
  - id: test-overlap
    content: Add test for overlapping $path roots with prefetch cache
    status: completed
  - id: test-nested
    content: Add unit test for nested path detection correctness
    status: completed
  - id: run-ci
    content: Run full CI pipeline (cargo test, clippy, fmt)
    status: completed
isProject: false
---

# Parallel I/O Audit Fixes

> This plan was generated by `/audit` from a Plan-mode session.
> Refer to `.cursor/rules/atlas.mdc` for full project standards.

## Standards

These standards govern every fix in this plan. An implementer MUST read `.cursor/rules/atlas.mdc` before starting.

### Round-Trip Identity (from `atlas.mdc` - Quality Standard)

Syncback (or two-way sync) writes a directory tree. Building an rbxl from that directory tree and forward-syncing it back must produce a **bit-identical instance tree** -- same names, same classes, same properties, same hierarchy, same ref targets. Any deviation is a bug.

### CLI Syncback Parity (from `atlas.mdc` - Two-Way Sync Strategy)

Plugin-based sync must produce **exactly the same filesystem output** that `atlas syncback` would give for the same input. Byte-for-byte identical files, identical directory structures, identical naming. Any divergence is a bug until proven otherwise.

### Code Quality (from `atlas.mdc` - Code Quality Standard)

DRY code that is easy to reason about and hard to get wrong. The same slugify/dedup/meta-update pattern should not be copy-pasted across files. Small refactors (2-3 call sites) are included as fixes. Major rewrites (10+ call sites) are deferred.

## Context

Audited the last 7 commits on the current branch (from `1ffed121` through `56e1ab4a`), which add:

- VFS prefetch cache for parallel file reading at startup
- Parallel snapshot building via rayon in `dir.rs`
- Parallel RefPathIndex population via rayon + walkdir
- Parallel tree hashing in syncback
- Orphan scanner rewrite using walkdir
- Placeholder-based ref path substitution optimization
- ServeCommand refactor and dead code cleanup

## Fixes

### Fix 1: Prune hidden directories from walkdir orphan scanner

- **Status:** Approved
- **Finding:** Orphan scanner walkdir replacement doesn't prune hidden directories
- **Files:** `[src/syncback/mod.rs](src/syncback/mod.rs)`
- **Problem:** Lines ~406-424. The `walkdir::WalkDir` iterator yields ALL entries including children of hidden directories. The hidden-name filter only checks `file_name().starts_with('.')`, so `settings.json` inside `.vscode/` passes the filter and gets added to `existing_paths`. In clean mode, these files would be deleted.
The old recursive `scan_directory` function skipped hidden directories AND never recursed into them. The walkdir replacement must replicate this behavior.
- **Solution:** Replace the flat `.flatten()` + post-hoc hidden check with `filter_entry()` to prune the walk at hidden directories:

```rust
  for entry in walkdir::WalkDir::new(&canonical_dir)
      .follow_links(true)
      .into_iter()
      .filter_entry(|e| {
          if e.depth() == 0 {
              return true;
          }
          e.file_name()
              .to_str()
              .map_or(true, |n| !n.starts_with('.') || n == ".gitkeep")
      })
      .flatten()
  {
      if entry.depth() == 0 {
          continue;
      }
      let path = entry.path().to_path_buf();
      if !is_valid_path(&ignore_patterns, project_path, &path) {
          continue;
      }
      // Hidden check no longer needed here -- filter_entry handles it
      paths.insert(path);
  }
  

```

  `filter_entry` controls which directories walkdir descends into. When it returns `false` for a directory, walkdir skips the directory AND all its descendants. This exactly matches the old recursive behavior.

- **Risk:** Low -- only touches the orphan scanner loop. `filter_entry` is walkdir's standard pruning mechanism.
- **Verify round-trip:** Clean mode syncback with a `.vscode/` directory inside a `$path` source dir should NOT delete `.vscode/` contents.
- **Verify syncback parity:** N/A -- this only affects clean mode orphan removal, not the sync pipeline.
- **Tests required:**
  - Integration test: create a temp project with `src/.hidden/keep.txt`, run clean-mode syncback, verify `.hidden/keep.txt` survives.

### Fix 2: Deduplicate overlapping roots in prefetch cache

- **Status:** Approved
- **Finding:** Overlapping `$path` roots cause duplicate children in prefetch cache
- **Files:** `[src/serve_session.rs](src/serve_session.rs)`
- **Problem:** Lines ~136-156 in `prefetch_project_files`. `collect_path_roots` collects ALL `$path` entries including nested ones. For a project with `$path: "src"` and `$path: "src/shared"`, walkdir walks `src/shared` twice -- once as part of `src` and once as its own root. The `children_map` accumulates duplicate entries for `src/shared`'s children. When consumed by VFS `read_dir`, these duplicates cause duplicate instance snapshots.
- **Solution:** After collecting roots in `prefetch_project_files`, deduplicate by removing any root that is a descendant of another root:

```rust
  let mut roots: Vec<std::path::PathBuf> = Vec::new();
  collect_path_roots(&project.tree, folder, &mut roots);

  // Remove roots that are descendants of other roots to avoid
  // walking the same directory twice and producing duplicate children.
  roots.sort();
  let mut deduped: Vec<PathBuf> = Vec::new();
  for root in roots {
      if !deduped.iter().any(|existing| root.starts_with(existing)) {
          deduped.push(root);
      }
  }
  let roots = deduped;
  

```

  After sorting, a parent path always sorts before its descendants. By iterating in order and only keeping roots that are NOT descendants of already-kept roots, we eliminate all overlaps in O(n^2) worst case but typically O(n) since most projects have few `$path` entries.

- **Risk:** Low -- only modifies which directories are walked during prefetch. The dedup logic is simple and well-understood.
- **Verify round-trip:** Build a project with overlapping `$path` entries. Verify snapshot tree has no duplicate instances. Run build 5 times, verify deterministic output.
- **Verify syncback parity:** N/A -- prefetch cache only affects forward sync initial load.
- **Tests required:**
  - Unit test in `tests/tests/build.rs`: create a project with `$path: "src"` and a child with `$path: "src/shared"`. Run `snapshot_from_vfs` with prefetch cache. Verify no duplicate children under the `shared` directory instance.

### Fix 3: Fix nested path detection in orphan removal

- **Status:** Approved (O(n log n) approach)
- **Finding:** Binary search nested detection misses sibling paths
- **Files:** `[src/syncback/mod.rs](src/syncback/mod.rs)`
- **Problem:** Lines ~869-888. The optimization uses `binary_search` to find the current path's position, then checks only `sorted_removals[idx - 1]`. This misses cases where a sibling path sits between the parent and the current path in sorted order.
Example: `sorted = ["a/b", "a/b/c", "a/b/d"]`. For `a/b/d` at idx 2, the code checks `sorted[1] = "a/b/c"` (a sibling, not an ancestor). The actual ancestor `"a/b"` at idx 0 is not checked.
- **Solution:** Replace binary search with a sorted single-pass using an `current_ancestor` tracker:

```rust
  let mut sorted_removals: Vec<_> = paths_to_remove.iter().cloned().collect();
  sorted_removals.sort();

  let mut current_ancestor: Option<&PathBuf> = None;
  for old_path in &sorted_removals {
      if let Some(ancestor) = current_ancestor {
          if old_path.starts_with(ancestor) && old_path != ancestor {
              continue; // nested under current ancestor, skip
          }
      }
      current_ancestor = Some(old_path);

      let relative_path = old_path
          .strip_prefix(&canonical_project_path)
          .unwrap_or(old_path);

      log::debug!("Removing orphaned path: {}", relative_path.display());
      if old_path.is_dir() {
          fs_snapshot.remove_dir(relative_path);
      } else {
          fs_snapshot.remove_file(relative_path);
      }
  }
  

```

  In sorted path order, all descendants of a path appear consecutively after it (because `Path` comparison is component-based). So a single variable tracking the current top-level ancestor is sufficient. When we encounter a non-descendant, it becomes the new ancestor. This is O(n log n) for the sort + O(n) for the walk.

  This also removes the intermediate `paths_vec` and the `binary_search` code, simplifying the function.

- **Risk:** Low -- the logic is simpler than what it replaces. The correctness argument is straightforward: sorted paths guarantee all descendants appear consecutively.
- **Verify round-trip:** Clean mode syncback with nested directories. Verify parent-only removal (no redundant child removals).
- **Verify syncback parity:** N/A -- orphan removal is internal to clean mode.
- **Tests required:**
  - Unit test: create paths `["a/b", "a/b/c", "a/b/d", "x/y", "x/y/z"]`, verify only `"a/b"` and `"x/y"` are top-level removals.

### Fix 4: Extract shared `strip_unc_prefix` helper

- **Status:** Approved
- **Finding:** Duplicated UNC prefix stripping logic in 3 locations
- **Files:** `[src/syncback/mod.rs](src/syncback/mod.rs)`
- **Problem:** The same UNC prefix stripping pattern appears at:
  1. Lines ~396-404 (orphan scanner: canonicalize `dirs_to_scan` entries)
  2. Lines ~649-656 (`strip_unc_prefix` helper definition inside orphan removal block)
  3. Lines ~703-711 (inline canonicalize in `collect_all_path_refs_and_mappings`)
  Location 2 defines a proper `fn strip_unc_prefix`, but locations 1 and 3 inline the same logic.
- **Solution:** Move `strip_unc_prefix` to module level (or just above the clean-mode block), and replace all 3 inline instances with calls to it. The function is already defined at location 2:

```rust
  fn strip_unc_prefix(p: PathBuf) -> PathBuf {
      let path_str = p.to_string_lossy();
      if path_str.starts_with(r"\\?\") || path_str.starts_with("//?/") {
          PathBuf::from(&path_str[4..])
      } else {
          p
      }
  }
  

```

  Move this definition before the `existing_paths` block (around line ~258), then replace the inline code at lines ~396-404 with `strip_unc_prefix(std::fs::canonicalize(dir).unwrap_or_else(|_| dir.clone()))` and at lines ~703-711 with `strip_unc_prefix(p)`.

- **Risk:** Zero -- pure refactor, no behavior change.
- **Verify round-trip:** N/A -- refactor only.
- **Verify syncback parity:** N/A -- refactor only.
- **Tests required:** None -- existing tests cover this code path.

## Skipped Fixes

### Skipped: Serve command loads project twice

- **Finding:** `src/cli/serve.rs` loads the project file via `Project::load_initial_project` for port/address, then `ServeSession::new` loads it again in `init_tree`.
- **Problem:** Minor performance -- project file read twice from disk at startup.
- **Reason skipped:** Acceptable overhead. Project files are small and this only happens once.

### Skipped: Commented-out git config in init.rs

- **Finding:** Four commented-out git config lines in `src/cli/init.rs` lines 134-137.
- **Reason skipped:** User indicated these are planned for future enablement.

## Accepted Limitations

- **Limitation:** Symlinks within project `$path` directories may cause path mismatches
- **Scenario:** If a symlink inside a `$path` source directory points elsewhere, walkdir-collected paths won't match `added_paths` during clean-mode orphan removal. Files behind the symlink could be incorrectly removed.
- **User decision:** Accepted for this release
- **Mitigation:** Symlinks within source directories are not a supported configuration in Roblox development. Root-level symlinks (project folder itself) still work because `canonicalize` is called on the root.

## Deferred Refactors

None identified beyond the small `strip_unc_prefix` extraction (included as Fix 4).

## Test Plan

### Rust Integration Tests (`tests/tests/`)

- **Hidden directory survival in clean syncback** (Fix 1): Create temp project with `src/.hidden/keep.txt`, run clean-mode syncback, assert `.hidden/keep.txt` still exists after syncback.
- **Overlapping `$path` roots don't create duplicates** (Fix 2): Create project with overlapping `$path` entries (`"src"` and `"src/shared"`), run `snapshot_from_vfs` with prefetch cache, verify no duplicate children.

### Rust Unit Tests

- **Nested path detection correctness** (Fix 3): Test the ancestor-tracking algorithm with paths `["a/b", "a/b/c", "a/b/d", "x/y", "x/y/z"]`, verify only `"a/b"` and `"x/y"` are emitted as top-level.

## Test Rules

- Every bug fix MUST have a test that would have FAILED before the fix and PASSES after.
- Fix 4 (refactor) does not need new tests -- existing test coverage is sufficient.

## Final Step: Run CI

After ALL fixes and tests are implemented, run the full CI pipeline (`cargo test --locked`, `cargo clippy`, `cargo fmt --check`). Every fix must pass CI before the plan is considered complete.