rojo/.cursor/rules/atlas-syncback.mdc

---
description: "Syncback architecture: Roblox→filesystem synchronization, incremental vs clean modes, hash-based change detection, middleware selection, and property filtering."
globs:
  - "src/syncback/**/*.rs"
  - "src/cli/syncback.rs"
---

# Atlas Syncback Architecture

> Atlas is a fork of Rojo. Many internal identifiers, API endpoints, and struct names still reference the original "Rojo" naming.

## Quality Standard

This is a production-grade synchronization system. The filesystem must represent a 1:1 copy of the Studio place. The core invariant:

**Round-trip identity:** Syncback (or two-way sync) writes a directory tree. Building an rbxl from that directory tree and forward-syncing it back must produce a bit-identical instance tree -- same names, same classes, same properties, same hierarchy, same ref targets. Any deviation is a bug.

Every audit item must be evaluated against this standard. A "works most of the time" finding is not acceptable. If there is ANY code path where an instance name, property, or hierarchy relationship can be lost, mangled, or silently altered through a syncback→rebuild cycle, flag it as critical.

## Code Quality Standard

We are building a maintainable codebase. During audits, watch for duplicated logic -- the same slugify/dedup/meta-update pattern copy-pasted across `change_processor.rs`, `api.rs`, `dir.rs`, `project.rs`, etc. If you find duplicated code that could be consolidated into a shared helper:

- **Small refactor** (extracting a helper function, consolidating 2-3 call sites): flag it AND fix it as part of the audit.
- **Major system rewrite** (restructuring data flow, changing function signatures across 10+ call sites, redesigning the snapshot pipeline): flag it clearly in a "Deferred Refactors" section at the bottom of the report. We will evaluate whether it's worth doing after the audit is complete.

The goal is clean, DRY code that is easy to reason about and hard to get wrong. If the same "compute meta path, check if name needs slugify, upsert/remove name field" sequence appears in 3+ places, that's a refactor candidate.

**Syncback** is the reverse operation of building: it takes a Roblox binary file (rbxl/rbxm) and writes instances back to the filesystem according to project structure.

**Entry point:** `atlas syncback <project> --input <file>`

## Core Concept

```
Roblox File (rbxl/rbxm)
  ↓ Parse with rbx_binary
WeakDom (in-memory instances)
  ↓ Match to project structure
Filesystem (Lua files, models, directories)
```

**Key challenge:** Determining which instances map to which files, and preserving existing file structure when possible.

## Architecture Overview (`src/syncback/`)

### Module Structure

```
syncback/
  mod.rs              - Main loop, orchestration
  snapshot.rs         - Convert instances → snapshots
  fs_snapshot.rs      - Filesystem representation
  file_names.rs       - Generate valid filenames
  property_filter.rs  - Filter properties per syncback rules
  ref_properties.rs   - Handle Ref properties (cross-references)
  hash/               - Content hashing for change detection
    mod.rs            - Hash variants
    variant.rs        - Variant-specific hashing
  stats.rs            - Statistics tracking
```

### Main Loop (`syncback_loop`)

**Flow:**

1. **Parse input file** - Read rbxl/rbxm with `rbx_binary`
2. **Prune tree** - Remove instances not in original project
3. **Filter hidden services** - Remove internal Roblox services (AdService, Analytics, etc.)
4. **Collect pre-prune paths** - Track all instances before pruning (for external refs)
5. **Hash comparison** - Compare hashes of old tree vs new tree
6. **Recursive syncback** - Walk trees, matching instances to files
7. **Write filesystem** - Create/update files based on changes
8. **Link Ref properties** - Resolve cross-references between instances

**Signature:**
```rust
pub fn syncback_loop(
    vfs: &Vfs,
    old_tree: &mut RojoTree,      // Existing project tree
    new_tree: WeakDom,             // Roblox file contents
    project: &Project,             // Project configuration
    incremental: bool,             // Preserve structure vs rebuild
) -> anyhow::Result<FsSnapshot>
```

## Syncback Modes

### Incremental Mode (`incremental: true`)

**Default behavior.** Preserves existing file structure:

- If `src/MyScript.luau` exists, update it
- If `src/MyFolder/init.luau` exists (directory structure), preserve it
- Only creates new files when instances don't match anything
- Compares hashes to detect actual changes (avoids rewriting unchanged files)

**Use case:** Normal workflow where you sync changes from Studio back to disk.

### Clean Mode (`incremental: false`)

**Rebuild from scratch.** Treats all children as new:

- Ignores existing file structure
- Generates new filenames for everything
- Chooses best middleware for each instance
- Useful when project structure changed significantly

**Use case:** Major refactoring, or when incremental mode produces wrong results.

**Implementation detail:** `old_ref` is ignored in `SyncbackSnapshot::with_joined_path` when `incremental=false`.

## Syncback Rules

Configured in `syncbackRules` section of project file:

```rust
pub struct SyncbackRules {
    /// Roblox tree paths to ignore (e.g., "ServerStorage/Secrets")
    pub ignore_trees: Vec<String>,
    
    /// Filesystem paths to ignore (e.g., "src/client/*")
    pub ignore_paths: Vec<PathBuf>,
    
    /// Properties to not sync back per class
    /// Example: {"BasePart": ["Color"], "Script": ["Disabled"]}
    pub ignore_properties: HashMap<String, Vec<String>>,
    
    /// Whether to sync Workspace.CurrentCamera (default: false)
    pub sync_current_camera: bool,
    
    /// Whether to sync unscriptable properties (default: true)
    pub sync_unscriptable: bool,
}
```

**Property filtering** (`property_filter.rs`):
- Filters out `Name`, `Parent` (always)
- Filters out properties in `ignore_properties`
- Filters out unscriptable properties if `sync_unscriptable: false`
- Filters out middleware-specific properties (e.g., `Script.Source` when writing to `.luau` file)

### Hidden Services

**Default behavior (`ignoreHiddenServices: true`):**

Only sync "visible" services:
```rust
const VISIBLE_SERVICES: &[&str] = &[
    "Lighting", "MaterialService", "ReplicatedFirst",
    "ReplicatedStorage", "ServerScriptService", "ServerStorage",
    "SoundService", "StarterGui", "StarterPack", "StarterPlayer",
    "Teams", "TextChatService", "VoiceChatService", "Workspace",
];
```

**Rationale:** Prevents plugin from trying to delete services like `HttpService`, `AnalyticsService`, `Chat` that were never synced to disk.

## Hash-Based Change Detection (`hash/`)

### Why Hashing?

**Problem:** Writing every file on every syncback is slow and creates noise in git diffs.

**Solution:** Hash both old and new instance trees. Only write files where hashes differ.

### Hash Algorithm

Hashing uses **blake3** directly - there's no `HashVariant` struct.

```rust
/// Returns HashMap<Ref, Hash> mapping each instance to its content hash
pub fn hash_tree(project: &Project, dom: &WeakDom, root_ref: Ref) -> HashMap<Ref, Hash>
```

**Hashing strategy:**
1. Process instances bottom-up (children first)
2. Hash class name, then filtered properties
3. Sort and hash children's hashes
4. Finalize with blake3

**Special cases:**
- Ref properties are hashed as paths (since Refs change between trees)
- Some properties are normalized (e.g., line endings in scripts)
- Unscriptable properties can be excluded from hash

### Hash Comparison

```rust
if old_hash == new_hash {
    // No changes - skip writing file
    return Ok(None);
} else {
    // Changed - write file
    write_instance_to_file(snapshot)?;
}
```

## SyncbackSnapshot (`snapshot.rs`)

**Core type representing instance to sync back:**

```rust
pub struct SyncbackSnapshot<'sync> {
    pub data: SyncbackData<'sync>,  // Context (VFS, trees, project)
    pub old: Option<Ref>,            // Old tree instance (if exists)
    pub new: Ref,                    // New tree instance (Roblox file)
    pub path: PathBuf,               // Target filesystem path
    pub middleware: Option<Middleware>, // How to write (Lua, Rbxm, etc.)
}
```

### Snapshot Creation

**Methods:**

```rust
// Join child to current path
snapshot.with_joined_path(new_ref, old_ref)?
// Result: parent_path/ChildName.luau

// Use base path
snapshot.with_base_path(&base, new_ref, old_ref)?
// Result: base/ChildName.luau

// Explicit path
snapshot.with_new_path(path, new_ref, old_ref)
// Result: path (exactly as provided)
```

**Filename generation** (`file_names.rs`):
- Slugifies instance name (`slugify_name()`: forbidden chars → `_`)
- Deduplicates against taken names (`deduplicate_name()`: appends `~1`, `~2`, etc.)
- Adds appropriate extension based on middleware
- Stores real name in `.meta.json5` `name` field when slug differs
- Example: `Script` named "My Script" → `My Script.server.luau`
- Example: `Script` named "Hey/Bro" → `Hey_Bro.server.luau` + meta with `name: "Hey/Bro"`

### Property Filtering

```rust
snapshot.get_path_filtered_properties(new_ref)
```

**Filters out:**
- `Source` property for scripts (content goes in file body)
- `Value` property for StringValue (content goes in `.txt` file)
- Properties specified in `ignoreProperties` rules
- Name/Parent (structural, not data)

## Middleware Selection

**Decision tree:**

1. **Old instance exists?** → Use same middleware as before (preserve structure)
2. **Script type?** → Choose appropriate Lua middleware (`.server.luau`, `.client.luau`, etc.)
3. **Model/container?** → Choose directory or rbxm based on complexity
4. **Special types?** → CSV for LocalizationTable, TXT for StringValue
5. **Fallback** → Rbxm or JsonModel

**Debug override:** `ROJO_SYNCBACK_DEBUG` environment variable
- `1` = Use Rbxmx instead of Rbxm
- `2` = Use JsonModel

## Filesystem Snapshot (`fs_snapshot.rs`)

**Represents filesystem operations to perform:**

```rust
pub struct FsSnapshot {
    added_files: HashMap<PathBuf, Vec<u8>>,   // New files to create
    added_dirs: HashSet<PathBuf>,             // New directories
    removed_files: HashSet<PathBuf>,          // Files to delete
    removed_dirs: HashSet<PathBuf>,           // Directories to delete
}
```

**Key methods:**
```rust
// Create snapshot with operations
fs_snapshot.with_added_file(path, contents)
fs_snapshot.with_added_dir(path)
fs_snapshot.merge(other)  // Combine snapshots

// Apply to filesystem
fs_snapshot.write_to_vfs(base_path, vfs)?;

// Query
fs_snapshot.is_empty()
fs_snapshot.added_paths()
fs_snapshot.removed_paths()
```

## Ref Properties (`ref_properties.rs`)

**Challenge:** Ref properties (like `ObjectValue.Value`) point to other instances by Ref, but Refs are different between old and new trees.

**Solution:**

1. **Collect paths** - Map every Ref → full instance path (e.g., `Workspace.Part1`)
2. **Collect referents** - Find all properties that are Refs, store as (source_path, target_path)
3. **Link referents** - After syncback, use Atlas's `$id` / `Rojo_Target_PropertyName` system to link instances

**Atlas linking system:**
```json
{
    "id": "some-unique-id",
    "properties": {},
    "attributes": {
        "Rojo_Target_Value": "some-unique-id"
    }
}
```

Plugin reads `Rojo_Target_PropertyName` attributes and sets corresponding Ref properties.

## Statistics Tracking (`stats.rs`)

```rust
pub struct SyncbackStats {
    duplicate_name_count: AtomicUsize,     // Instances skipped (name collision)
    rbxm_fallback_count: AtomicUsize,      // Instances using rbxm fallback
    unknown_class_count: AtomicUsize,      // Unknown classes encountered
    unknown_property_count: AtomicUsize,   // Unknown properties encountered
    unknown_classes: Mutex<HashSet<String>>,
    unknown_properties: Mutex<HashSet<String>>,
}
```

**Key methods:**
```rust
stats.record_duplicate_name(path, name)
stats.record_rbxm_fallback(path, reason)
stats.record_unknown_class(class_name)
stats.record_unknown_property(class, property)
stats.has_issues()      // Any warnings?
stats.has_unknown_types()  // Unknown classes/properties?
stats.log_summary()     // Log final report
```

**Logged at end of syncback:**
```
Syncback complete. Warnings: 2 duplicate names, 1 unknown class
```

## Common Patterns

### Syncing a Script

**Roblox file has:**
```
Script "MyScript"
  Source = "print('hello')"
  Disabled = false
```

**Syncback produces:**
```luau
-- src/MyScript.server.luau
print('hello')
```

**Meta file (if needed):**
```json5
// src/MyScript.server.meta.json5 (for MyScript.server.luau)
{
    "$properties": {
        "Disabled": false
    }
}
```

### Syncing a Model

**Simple model (few children):** → `.rbxm` or `.rbxmx` file
**Complex model (many children):** → Directory with `init.meta.json5` or `init.luau`

### Syncing Nested Folders

```
Roblox:
  Folder "Modules"
    ModuleScript "Utils"
    Folder "Math"
      ModuleScript "Add"

Filesystem:
  src/Modules/
    Utils.luau
    Math/
      Add.luau
```

## Error Handling

**Common errors:**

1. **Middleware mismatch** - Instance type changed but file structure can't accommodate
   - Solution: Use clean mode
   
2. **Invalid filename** - Instance name contains invalid chars
   - Solution: Handled automatically by `slugify_name()` + metadata `name` field
   
3. **Ref to pruned instance** - Property points to instance not in project
   - Logged as warning, property not synced
   
4. **Write permission** - Can't write to file
   - Error reported, syncback continues for other files

**Error collection:**
```rust
stats.record_error(&format!("Failed to sync {}: {}", path.display(), error));
```

All errors collected and reported at end.

## Integration with Plugin

**Plugin does NOT handle syncback.** It only handles:
1. Downloading place file from Studio
2. Sending file to server
3. Server runs syncback command

**Two-way sync (experimental)** is different - see `atlas-api.mdc`.

## Performance Considerations

**Hashing overhead:** Hashing is fast (~50µs per instance), negligible compared to disk I/O.

**Incremental benefit:** On large projects, incremental mode with hashing can be 10-100x faster than rewriting everything.

**Parallelization:** Not currently parallelized - filesystem writes are sequential.

## Debugging

**Enable verbose logging:**
```bash
RUST_LOG=rojo::syncback=debug atlas syncback project/ --input place.rbxl
```

**Shows:**
- Which instances are being processed
- Hash comparisons
- File write decisions
- Middleware selection

**Test syncback:**
```bash
# Syncback once
atlas syncback project/ --input place.rbxl

# Build back to verify roundtrip
atlas build project/ -o rebuilt.rbxl

# Compare
diff place.rbxl rebuilt.rbxl
```

## Future Improvements

**From code comments:**
- Better detection of when root is directory vs file
- More sophisticated middleware selection
- Parallel file writes
- Better handling of Roblox format changes
- Support for partial syncback (only changed instances)

## Key Files to Reference

- `src/syncback/mod.rs` - Main orchestration loop
- `src/syncback/snapshot.rs` - SyncbackSnapshot implementation
- `src/syncback/hash/` - Hashing algorithm
- `src/syncback/property_filter.rs` - Property filtering logic
- `src/cli/syncback.rs` - CLI command implementation