rojo/.cursor/rules/rojo-api.mdc

---
description: "Web API and plugin communication: WebSocket protocol, live sync architecture, two-way sync, patch format, ServeSession state management, and ChangeBatcher."
globs:
  - "src/web/**/*.rs"
  - "src/serve_session.rs"
  - "plugin/src/ServeSession.lua"
  - "plugin/src/ChangeBatcher/**/*.lua"
  - "plugin/src/ApiContext.lua"
---

# Rojo API & Communication

**Protocol Version:** 5 | Server/plugin MAJOR.MINOR must match

## Architecture

```
Filesystem → Server (serve_session.rs) → WebSocket (web/api.rs) → Plugin (ServeSession.lua) → Studio

Two-way (experimental):
Studio → ChangeBatcher → Plugin → WebSocket → Server → Filesystem
```

## HTTP Endpoints (`src/web/api.rs`)

| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/api/rojo` | GET | Server info (version, sessionId, projectName, rootInstanceId, servePlaceIds) |
| `/api/read/:id` | GET | Read instance tree from ID |
| `/api/serialize/:id` | GET | Instance tree as msgpack |
| `/api/write` | POST | Two-way sync: plugin sends changes |
| `/api/open/:id` | POST | Open script in editor (`{"cursor": [line, col]}`) |
| `/api/ref-patch/:id` | GET | Fix Ref properties after syncback |

### Server Info Response (`GET /api/rojo`)

```json
{
    "serverVersion": "7.7.0-rc.1",
    "protocolVersion": 5,
    "sessionId": "uuid",
    "projectName": "MyProject",
    "expectedPlaceIds": [123456],
    "rootInstanceId": "00000000-0000-0000-0000-000000000000",
    "syncSourceOnly": true,
    "visibleServices": ["Workspace", "ServerScriptService"]
}
```

### Write Request (`POST /api/write`)

```json
{
    "removed": ["id-1"],
    "added": { "id-2": { "id": "id-2", "className": "Folder", "name": "New", "properties": {}, "children": [] } },
    "updated": [{ "id": "id-3", "changedName": "Renamed", "changedProperties": { "Source": "print('hi')" } }]
}
```

**Format detection:** `Name/init.luau` → directory | `Name.luau` → file | neither → use `has_children` (`.lua` also supported)

## WebSocket Protocol (v5)

**Connection:** `ws://host:port/api/rojo`

1. Connect → Receive `ServerInfo` handshake → Send `{type = "Subscribe"}` → Receive patches

### Patch Message (`ApplyPatch`)

```lua
{
    type = "ApplyPatch",
    sessionId = "uuid",
    patch = {
        removed = {"id-1", "id-2"},
        added = {
            ["id-3"] = {
                id = "id-3", className = "Folder", name = "New",
                properties = {}, metadata = { ignoreUnknownInstances = false },
                children = {"child-1"},
            },
        },
        updated = {
            { id = "id-4", changedName = "Renamed", changedProperties = { Source = "code" } },
        },
    }
}
```

**Format:** MessagePack (~30% smaller than JSON, ~2x faster)

## Server ServeSession (`src/serve_session.rs`)

```rust
pub struct ServeSession {
    session_id: Uuid,
    tree_handle: Arc<Mutex<RojoTree>>,
    message_queue: MessageQueue,
    project: Project,
    vfs: Vfs,
}
```

**Lifecycle:** Create → Watch filesystem → Generate patches on changes → Broadcast to plugins → Shutdown

**Message Queue:** Batches changes (~100ms window), cursor system for reconnect recovery

**WebSocket loop:**
```rust
loop {
    select! {
        msg = receiver.next() => handle_plugin_message(msg),
        patch = message_queue.recv() => sender.send(patch).await,
    }
}
```

## Plugin ServeSession (`plugin/src/ServeSession.lua`)

```lua
ServeSession = {
    __status = "NotStarted",  -- NotStarted → Connecting → Connected → Disconnected
    __apiContext = ApiContext,
    __twoWaySync = false,
    __reconciler = Reconciler,
    __instanceMap = InstanceMap,
    __changeBatcher = ChangeBatcher,
    __confirmingPatch = nil,
    __isConfirming = false,  -- Prevents stacking dialogs, changes during review
}
```

### Patch Confirmation

**Setting `confirmationBehavior`:** `"Always"` (default) | `"Initial"` | `"Large Changes"` | `"Unlisted PlaceId"` | `"Never"`

**Flow:** Receive patch → Check setting → Show PatchVisualizer (pause ChangeBatcher) → User confirms → Apply via Reconciler → Resume ChangeBatcher

### Patch Application

```lua
-- Remove
for _, id in patch.removed do instanceMap:fromId(id):Destroy(); instanceMap:removeId(id) end
-- Add
for id, data in patch.added do instanceMap:insert(id, Reconciler.reify(data)) end
-- Update
for _, update in patch.updated do Reconciler.hydrate(instanceMap:fromId(update.id), update) end
```

## Two-Way Sync (Experimental)

**Setting:** `twoWaySync` (default: true) | Syncs Studio changes → filesystem in real-time

### Complete Data Flow

```
┌─────────────────────────────────────────────────────────────────────────────┐
│ PLUGIN SIDE                                                                 │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  Studio Instance                                                            │
│       │ PropertyChanged / DescendantAdded / etc.                            │
│       ▼                                                                     │
│  InstanceMap.__connectSignals()                                             │
│       │ Fires onInstanceChanged(instance, propertyName)                     │
│       │ (skipped if RunService:IsRunning() or instance is paused)           │
│       ▼                                                                     │
│  ServeSession.onInstanceChanged()                                           │
│       │ Guards: if not self.__twoWaySync then return end                    │
│       ▼                                                                     │
│  ChangeBatcher:add(instance, propertyName)                                  │
│       │ Stores in __pendingPropertyChanges[instance][propertyName] = true   │
│       ▼                                                                     │
│  ChangeBatcher:__cycle(dt)  [RenderStepped, every frame]                    │
│       │ Accumulates dt until >= BATCH_INTERVAL (0.2s)                       │
│       │ Guards: if __paused then return end                                 │
│       ▼                                                                     │
│  ChangeBatcher:__flush()                                                    │
│       │ Calls createPatchSet(instanceMap, pendingChanges, syncSourceOnly)   │
│       │ Clears __pendingPropertyChanges                                     │
│       ▼                                                                     │
│  onChangesFlushed(patch)                                                    │
│       │ Guards: if Settings:get("oneShotSync") then return end              │
│       ▼                                                                     │
│  ApiContext:write(patch)  → HTTP POST /api/write (MessagePack)              │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ SERVER SIDE                                                                 │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  handle_api_write() [src/web/api.rs]                                        │
│       │ Validates session_id                                                │
│       │ Deserializes MessagePack → WriteRequest                             │
│       ▼                                                                     │
│  For each removed: syncback_removed_instance()                              │
│       │ Looks up InstigatingSource::Path → fs::remove_file()                │
│       ▼                                                                     │
│  For each added: syncback_added_instance()                                  │
│       │ Determines file format, writes to filesystem                        │
│       ▼                                                                     │
│  Sends PatchSet to tree_mutation_sender channel                             │
│       ▼                                                                     │
│  ChangeProcessor::handle_tree_event() [src/change_processor.rs]             │
│       │ For removed: fs::remove_file(instigating_source.path)               │
│       │ For updated: if "Source" property → fs::write(path, value)          │
│       ▼                                                                     │
│  apply_patch_set() → Updates RojoTree                                       │
│       ▼                                                                     │
│  message_queue.push_messages() → Broadcasts to all plugins                  │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
```

### InstanceMap: Change Detection Foundation (`plugin/src/InstanceMap.lua`)

The InstanceMap is a bidirectional map (ID ↔ Instance) that also hooks property change signals:

```lua
InstanceMap = {
    fromIds = {},              -- Server ID → Studio Instance
    fromInstances = {},        -- Studio Instance → Server ID
    pausedUpdateInstances = {}, -- Set of instances to ignore changes from
    instancesToSignal = {},    -- Instance → RBXScriptConnection(s)
    onInstanceChanged = fn,    -- Callback: (instance, propertyName) → void
}
```

**Signal Connection:**
```lua
function InstanceMap:__connectSignals(instance)
    if instance:IsA("ValueBase") then
        -- ValueBase.Changed only fires for Value, need explicit signals
        signals = {
            instance:GetPropertyChangedSignal("Name"):Connect(...),
            instance:GetPropertyChangedSignal("Value"):Connect(...),
            instance:GetPropertyChangedSignal("Parent"):Connect(...),
        }
    else
        -- Regular instances use Changed event (fires for any property)
        instance.Changed:Connect(function(propertyName)
            self:__maybeFireInstanceChanged(instance, propertyName)
        end)
    end
end
```

**Change Guards:**
```lua
function InstanceMap:__maybeFireInstanceChanged(instance, propertyName)
    if self.pausedUpdateInstances[instance] then return end  -- Paused during reconciliation
    if self.onInstanceChanged == nil then return end         -- No callback registered
    if RunService:IsRunning() then return end                -- Don't sync during Play mode
    self.onInstanceChanged(instance, propertyName)
end
```

**Pause Mechanism:** When applying incoming patches, instances are paused to prevent their changes from being sent back to the server (feedback loop prevention).

### ChangeBatcher (`plugin/src/ChangeBatcher/`)

| File | Purpose |
|------|---------|
| `init.lua` | Main batcher: accumulator, RenderStepped loop, pause/resume |
| `createPatchSet.lua` | Converts pending changes → PatchSet |
| `encodePatchUpdate.lua` | Encodes property updates for existing instances |
| `encodeInstance.lua` | Full instance encoding (for additions/syncback) |
| `encodeProperty.lua` | Single property value encoding via RbxDom |

**Core State:**
```lua
ChangeBatcher = {
    __accumulator = 0,                    -- Time since last flush
    __pendingPropertyChanges = {},        -- Instance → { PropertyName → true }
    __syncSourceOnly = false,             -- Only sync Source property
    __paused = false,                     -- Pause during confirmation
    __renderSteppedConnection = conn,     -- Frame loop connection
    __instanceMap = instanceMap,
    __onChangesFlushed = callback,
}
```

**Batch Cycle (every frame):**
```lua
local BATCH_INTERVAL = 0.2  -- 200ms

function ChangeBatcher:__cycle(dt)
    if self.__paused then return end
    
    self.__accumulator += dt
    if self.__accumulator >= BATCH_INTERVAL then
        self.__accumulator -= BATCH_INTERVAL
        local patch = self:__flush()
        if patch then
            self.__onChangesFlushed(patch)
        end
    end
    
    self.__instanceMap:unpauseAllInstances()  -- Resume paused instances
end
```

**Why 200ms batch interval?**
- Studio fires many rapid property changes (e.g., dragging a part fires Position.X, Y, Z separately)
- Batching combines related changes into single patch
- Reduces server load and WebSocket traffic
- Still feels responsive to users

### createPatchSet.lua: Patch Generation

```lua
function createPatchSet(instanceMap, propertyChanges, syncSourceOnly)
    local patch = PatchSet.newEmpty()
    
    for instance, properties in propertyChanges do
        local instanceId = instanceMap.fromInstances[instance]
        if instanceId == nil then
            Log.warn("Ignoring change for unknown instance")
            continue
        end
        
        if properties.Parent then
            if instance.Parent == nil then
                -- Instance was deleted in Studio
                table.insert(patch.removed, instanceId)
            else
                Log.warn("Cannot sync non-nil Parent changes yet")  -- Reparenting not supported
            end
        else
            if syncSourceOnly then
                -- Server in source-only mode: only sync Source property
                if properties.Source then
                    local update = encodePatchUpdate(instance, instanceId, { Source = true })
                    if update then table.insert(patch.updated, update) end
                end
            else
                local update = encodePatchUpdate(instance, instanceId, properties)
                if update then table.insert(patch.updated, update) end
            end
        end
    end
    
    return patch
end
```

### encodePatchUpdate.lua: Update Encoding

```lua
function encodePatchUpdate(instance, instanceId, properties)
    local update = {
        id = instanceId,
        changedProperties = {},
    }
    
    for propertyName in properties do
        if propertyName == "Name" then
            update.changedName = instance.Name
        else
            local descriptor = RbxDom.findCanonicalPropertyDescriptor(instance.ClassName, propertyName)
            if not descriptor then continue end
            
            local success, encoded = encodeProperty(instance, propertyName, descriptor)
            if success then
                update.changedProperties[propertyName] = encoded
            end
        end
    end
    
    if next(update.changedProperties) == nil and update.changedName == nil then
        return nil  -- No actual changes
    end
    
    return update
end
```

### encodeInstance.lua: Full Instance Encoding (Syncback/Additions)

Used when pulling new instances from Studio to filesystem. **Critical: handles duplicate detection.**

```lua
-- Duplicate handling (matches server's ref_properties.rs::is_path_unique)
local function isPathUnique(instance)
    local current = instance
    while current do
        if hasDuplicateSiblings(current) then
            return false  -- Any duplicate in ancestor chain = ambiguous path
        end
        current = current.Parent
        if current == game then break end
    end
    return true
end

function encodeInstance(instance, parentId, _skipPathCheck)
    -- Skip instances with ambiguous paths (duplicate-named siblings)
    if not _skipPathCheck and not isPathUnique(instance) then
        Log.warn("Skipped '{}' - path contains duplicate-named siblings", instance:GetFullName())
        return nil
    end
    
    local properties = {}
    
    -- Encode Attributes and Tags (work on all instance types)
    encodeAttributes(instance, properties)
    encodeTags(instance, properties)
    
    -- For scripts, Source is required
    if SCRIPT_CLASS_NAMES[instance.ClassName] then
        local success, encoded = encodeProperty(instance, "Source", descriptor)
        if not success then return nil end
        properties.Source = encoded
    end
    
    -- Encode all serializable properties
    local classDescriptor = RbxDom.findClassDescriptor(instance.ClassName)
    for propertyName, meta in classDescriptor.properties do
        if SKIP_PROPERTIES[propertyName] then continue end           -- Parent, Name, Archivable
        if SKIP_PROPERTIES_BY_CLASS[instance.ClassName]?[propertyName] then continue end  -- Source, ScriptGuid, etc.
        
        local isReadable = meta.scriptability == "ReadWrite" or meta.scriptability == "Read"
        local doesSerialize = meta.serialization ~= "DoesNotSerialize"
        
        if isReadable and doesSerialize then
            local success, encoded = encodeProperty(instance, propertyName, descriptor)
            if success and encoded ~= nil then
                properties[propertyName] = encoded
            end
        end
    end
    
    -- Recursively encode children (skipping duplicates at each level)
    local children = {}
    local childDuplicates = findDuplicateNames(instance:GetChildren())
    for _, child in instance:GetChildren() do
        if childDuplicates[child.Name] then continue end
        local encoded = encodeInstance(child, nil, true)  -- Skip path check, already validated
        if encoded then table.insert(children, encoded) end
    end
    
    return {
        parent = parentId,
        name = instance.Name,
        className = instance.ClassName,
        properties = properties,
        children = children,
    }
end
```

**Properties skipped per class:**
| Class | Skipped Properties | Reason |
|-------|-------------------|--------|
| Script/LocalScript/ModuleScript | Source, ScriptGuid, RunContext | Encoded in file contents/suffix |
| LocalizationTable | Contents | Encoded in .csv file |
| StringValue | Value | Encoded in .txt file |

### encodeProperty.lua: Value Encoding

```lua
function encodeProperty(instance, propertyName, descriptor)
    local readSuccess, value = descriptor:read(instance)
    if not readSuccess then return false, nil end
    
    local dataType = descriptor.dataType
    local encodeSuccess, encoded = RbxDom.EncodedValue.encode(value, dataType)
    if not encodeSuccess then return false, nil end
    
    return true, encoded
end
```

**Encoded value format (via RbxDom):**
```lua
-- Primitives
true, false, 42, "string"

-- Roblox types
{ Vector3 = {1, 2, 3} }
{ CFrame = { position = {0,0,0}, orientation = {...} } }
{ UDim2 = {{0, 100}, {0, 50}} }
{ Color3 = {1, 0.5, 0} }
{ BrickColor = 194 }
{ Ref = "server-instance-id" }  -- Object references
{ Attributes = { key = value, ... } }
{ Tags = {"tag1", "tag2"} }
```

### Server-Side: ChangeProcessor (`src/change_processor.rs`)

Receives patches from plugin via `tree_mutation_receiver` channel:

```rust
fn handle_tree_event(&self, patch_set: PatchSet) {
    let mut tree = self.tree.lock().unwrap();
    
    // Process removals
    for &id in &patch_set.removed_instances {
        if let Some(instance) = tree.get_instance(id) {
            match &instance.metadata().instigating_source {
                Some(InstigatingSource::Path(path)) => {
                    log::info!("Two-way sync: Removing file {}", path.display());
                    fs::remove_file(path)?;
                }
                Some(InstigatingSource::ProjectNode { .. }) => {
                    log::warn!("Cannot remove instance from project file");
                }
                None => {
                    log::warn!("Cannot remove instance without instigating source");
                }
            }
        }
    }
    
    // Process updates
    for update in &patch_set.updated_instances {
        if let Some(instance) = tree.get_instance(update.id) {
            // Currently only Source property is supported
            if let Some(Variant::String(source)) = update.changed_properties.get("Source") {
                match &instance.metadata().instigating_source {
                    Some(InstigatingSource::Path(path)) => {
                        log::info!("Two-way sync: Writing Source to {}", path.display());
                        fs::write(path, source)?;
                    }
                    _ => log::warn!("Cannot update instance without path source"),
                }
            }
            
            // Other property changes logged but not written
            if update.changed_name.is_some() { log::warn!("Cannot rename instances yet"); }
            if update.changed_class_name.is_some() { log::warn!("Cannot change ClassName yet"); }
        }
    }
    
    apply_patch_set(&mut tree, patch_set);  // Update DOM
    self.message_queue.push_messages(&[applied_patch]);  // Broadcast to plugins
}
```

**InstigatingSource:** Every instance in RojoTree tracks where it came from:
- `InstigatingSource::Path(PathBuf)` - Created from a file, can be written back
- `InstigatingSource::ProjectNode { path, name, node, parent_class }` - Defined in project file, **cannot be modified via two-way sync**

### Pause Mechanism: Critical for Consistency

**Why pause is essential:**
1. **During incoming patch confirmation:** User is reviewing changes. If ChangeBatcher sends Studio changes mid-review, it creates race conditions.
2. **During patch application:** Reconciler modifies instances, triggering property change events. Without pause, these would be sent back as "changes" (feedback loop).
3. **Defense-in-depth:** `__isConfirming` flag in ServeSession prevents multiple dialogs and ensures atomicity.

```lua
-- In ServeSession, when showing confirmation dialog:
self.__changeBatcher:pause()
-- ... show PatchVisualizer, wait for user ...
self.__changeBatcher:resume()
```

```lua
-- In Reconciler, when modifying instances:
instanceMap:pauseInstance(instance)
instance.SomeProperty = newValue  -- Won't trigger ChangeBatcher
-- Later, in ChangeBatcher:__cycle():
instanceMap:unpauseAllInstances()
```

### One-Shot Mode Integration

**Setting:** `oneShotSync` (default: true)

One-shot mode adds an extra guard in `onChangesFlushed`:

```lua
local function onChangesFlushed(patch)
    if Settings:get("oneShotSync") then
        Log.info("One-shot mode: blocking automatic outgoing write from ChangeBatcher")
        return  -- Silently discard patch
    end
    self.__apiContext:write(patch)
end
```

**Behavior:**
- ChangeBatcher still collects changes (needed for diff visualization)
- Automatic writes are blocked
- Only explicit "pull" selections in confirmation dialog can write
- Disconnects after initial sync completes
- No sync lock acquired (multiple users can view same place)

**Use cases:**
- Quick preview of changes without risk of accidental writes
- Safe for production environments
- Team viewing same project simultaneously

### Known Limitations (Detailed)

| Issue | Cause | Impact |
|-------|-------|--------|
| **Undo/redo triggers spurious changes** | Studio's ChangeHistoryService fires property changes during undo | Can cause unexpected writes or conflicts |
| **Rapid changes interleave** | 200ms batch window may split logically related changes | Partial updates can leave inconsistent state |
| **Reparenting not supported** | `Parent` property changes to non-nil values are ignored | Moving instances in Studio doesn't sync |
| **Renaming not supported** | `changedName` logged but not written | Renaming in Studio doesn't sync |
| **ClassName changes ignored** | Would require file type change | Changing script type doesn't sync |
| **Only Source property writes** | Other properties logged but not written to filesystem | Only script content syncs reliably |
| **Project node instances immutable** | InstigatingSource::ProjectNode cannot be modified | Instances from project file can't be changed |
| **Duplicate-named siblings skipped** | Path ambiguity makes reliable sync impossible | Instances with same-name siblings ignored |
| **Ref properties complex** | Instance references may become invalid when instances move | ObjectValue.Value may not sync correctly |

**Recommended workflow:** Use one-way sync (server → Studio) as primary. Use two-way sync only for script Source changes with careful attention to undo/redo behavior.

## Protocol Enforcement

```rust
// Server
if client_protocol != PROTOCOL_VERSION { return error("Protocol mismatch") }
```

```lua
-- Plugin
if serverInfo.protocolVersion ~= Config.protocolVersion then showError("Protocol mismatch") end
```

**Breaking changes:** Bump `protocolVersion` in `src/web/interface.rs` and `plugin/src/Config.lua`

## Session Management

1. Server starts → Generate UUID
2. Plugin connects → Receives session ID in handshake
3. All requests include session ID
4. Server restart → New session ID → Plugin auto-reconnects on mismatch

## Debugging

```bash
# Server
RUST_LOG=rojo::web=debug,rojo::serve_session=debug rojo serve
```

```lua
-- Plugin
Settings:set("logLevel", "Trace")
```

Browser: `http://localhost:34872` → Network tab → WebSocket messages

## Key Files

| Server | Plugin |
|--------|--------|
| `src/web/api.rs` - Endpoints | `plugin/src/ServeSession.lua` - Connection |
| `src/web/interface.rs` - Packets | `plugin/src/ApiContext.lua` - HTTP/WS client |
| `src/serve_session.rs` - Session | `plugin/src/ChangeBatcher/` - Two-way sync |
| | `plugin/src/Reconciler/` - Patch application |