--- description: "Web API and plugin communication: WebSocket protocol, live sync architecture, two-way sync, patch format, ServeSession state management, and ChangeBatcher." globs: - "src/web/**/*.rs" - "src/serve_session.rs" - "plugin/src/ServeSession.lua" - "plugin/src/ChangeBatcher/**/*.lua" - "plugin/src/ApiContext.lua" --- # Rojo API & Communication **Protocol Version:** 5 | Server/plugin MAJOR.MINOR must match ## Architecture ``` Filesystem → Server (serve_session.rs) → WebSocket (web/api.rs) → Plugin (ServeSession.lua) → Studio Two-way (experimental): Studio → ChangeBatcher → Plugin → WebSocket → Server → Filesystem ``` ## HTTP Endpoints (`src/web/api.rs`) | Endpoint | Method | Purpose | |----------|--------|---------| | `/api/rojo` | GET | Server info (version, sessionId, projectName, rootInstanceId, servePlaceIds) | | `/api/read/:id` | GET | Read instance tree from ID | | `/api/serialize/:id` | GET | Instance tree as msgpack | | `/api/write` | POST | Two-way sync: plugin sends changes | | `/api/open/:id` | POST | Open script in editor (`{"cursor": [line, col]}`) | | `/api/ref-patch/:id` | GET | Fix Ref properties after syncback | ### Server Info Response (`GET /api/rojo`) ```json { "serverVersion": "7.7.0-rc.1", "protocolVersion": 5, "sessionId": "uuid", "projectName": "MyProject", "expectedPlaceIds": [123456], "rootInstanceId": "00000000-0000-0000-0000-000000000000", "syncSourceOnly": true, "visibleServices": ["Workspace", "ServerScriptService"] } ``` ### Write Request (`POST /api/write`) ```json { "removed": ["id-1"], "added": { "id-2": { "id": "id-2", "className": "Folder", "name": "New", "properties": {}, "children": [] } }, "updated": [{ "id": "id-3", "changedName": "Renamed", "changedProperties": { "Source": "print('hi')" } }] } ``` **Format detection:** `Name/init.luau` → directory | `Name.luau` → file | neither → use `has_children` (`.lua` also supported) ## WebSocket Protocol (v5) **Connection:** `ws://host:port/api/rojo` 1. Connect → Receive `ServerInfo` handshake → Send `{type = "Subscribe"}` → Receive patches ### Patch Message (`ApplyPatch`) ```lua { type = "ApplyPatch", sessionId = "uuid", patch = { removed = {"id-1", "id-2"}, added = { ["id-3"] = { id = "id-3", className = "Folder", name = "New", properties = {}, metadata = { ignoreUnknownInstances = false }, children = {"child-1"}, }, }, updated = { { id = "id-4", changedName = "Renamed", changedProperties = { Source = "code" } }, }, } } ``` **Format:** MessagePack (~30% smaller than JSON, ~2x faster) ## Server ServeSession (`src/serve_session.rs`) ```rust pub struct ServeSession { session_id: Uuid, tree_handle: Arc>, message_queue: MessageQueue, project: Project, vfs: Vfs, } ``` **Lifecycle:** Create → Watch filesystem → Generate patches on changes → Broadcast to plugins → Shutdown **Message Queue:** Batches changes (~100ms window), cursor system for reconnect recovery **WebSocket loop:** ```rust loop { select! { msg = receiver.next() => handle_plugin_message(msg), patch = message_queue.recv() => sender.send(patch).await, } } ``` ## Plugin ServeSession (`plugin/src/ServeSession.lua`) ```lua ServeSession = { __status = "NotStarted", -- NotStarted → Connecting → Connected → Disconnected __apiContext = ApiContext, __twoWaySync = false, __reconciler = Reconciler, __instanceMap = InstanceMap, __changeBatcher = ChangeBatcher, __confirmingPatch = nil, __isConfirming = false, -- Prevents stacking dialogs, changes during review } ``` ### Patch Confirmation **Setting `confirmationBehavior`:** `"Always"` (default) | `"Initial"` | `"Large Changes"` | `"Unlisted PlaceId"` | `"Never"` **Flow:** Receive patch → Check setting → Show PatchVisualizer (pause ChangeBatcher) → User confirms → Apply via Reconciler → Resume ChangeBatcher ### Patch Application ```lua -- Remove for _, id in patch.removed do instanceMap:fromId(id):Destroy(); instanceMap:removeId(id) end -- Add for id, data in patch.added do instanceMap:insert(id, Reconciler.reify(data)) end -- Update for _, update in patch.updated do Reconciler.hydrate(instanceMap:fromId(update.id), update) end ``` ## Two-Way Sync (Experimental) **Setting:** `twoWaySync` (default: true) | Syncs Studio changes → filesystem in real-time ### Complete Data Flow ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ PLUGIN SIDE │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ Studio Instance │ │ │ PropertyChanged / DescendantAdded / etc. │ │ ▼ │ │ InstanceMap.__connectSignals() │ │ │ Fires onInstanceChanged(instance, propertyName) │ │ │ (skipped if RunService:IsRunning() or instance is paused) │ │ ▼ │ │ ServeSession.onInstanceChanged() │ │ │ Guards: if not self.__twoWaySync then return end │ │ ▼ │ │ ChangeBatcher:add(instance, propertyName) │ │ │ Stores in __pendingPropertyChanges[instance][propertyName] = true │ │ ▼ │ │ ChangeBatcher:__cycle(dt) [RenderStepped, every frame] │ │ │ Accumulates dt until >= BATCH_INTERVAL (0.2s) │ │ │ Guards: if __paused then return end │ │ ▼ │ │ ChangeBatcher:__flush() │ │ │ Calls createPatchSet(instanceMap, pendingChanges, syncSourceOnly) │ │ │ Clears __pendingPropertyChanges │ │ ▼ │ │ onChangesFlushed(patch) │ │ │ Guards: if Settings:get("oneShotSync") then return end │ │ ▼ │ │ ApiContext:write(patch) → HTTP POST /api/write (MessagePack) │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────────┐ │ SERVER SIDE │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ handle_api_write() [src/web/api.rs] │ │ │ Validates session_id │ │ │ Deserializes MessagePack → WriteRequest │ │ ▼ │ │ For each removed: syncback_removed_instance() │ │ │ Looks up InstigatingSource::Path → fs::remove_file() │ │ ▼ │ │ For each added: syncback_added_instance() │ │ │ Determines file format, writes to filesystem │ │ ▼ │ │ Sends PatchSet to tree_mutation_sender channel │ │ ▼ │ │ ChangeProcessor::handle_tree_event() [src/change_processor.rs] │ │ │ For removed: fs::remove_file(instigating_source.path) │ │ │ For updated: if "Source" property → fs::write(path, value) │ │ ▼ │ │ apply_patch_set() → Updates RojoTree │ │ ▼ │ │ message_queue.push_messages() → Broadcasts to all plugins │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ ``` ### InstanceMap: Change Detection Foundation (`plugin/src/InstanceMap.lua`) The InstanceMap is a bidirectional map (ID ↔ Instance) that also hooks property change signals: ```lua InstanceMap = { fromIds = {}, -- Server ID → Studio Instance fromInstances = {}, -- Studio Instance → Server ID pausedUpdateInstances = {}, -- Set of instances to ignore changes from instancesToSignal = {}, -- Instance → RBXScriptConnection(s) onInstanceChanged = fn, -- Callback: (instance, propertyName) → void } ``` **Signal Connection:** ```lua function InstanceMap:__connectSignals(instance) if instance:IsA("ValueBase") then -- ValueBase.Changed only fires for Value, need explicit signals signals = { instance:GetPropertyChangedSignal("Name"):Connect(...), instance:GetPropertyChangedSignal("Value"):Connect(...), instance:GetPropertyChangedSignal("Parent"):Connect(...), } else -- Regular instances use Changed event (fires for any property) instance.Changed:Connect(function(propertyName) self:__maybeFireInstanceChanged(instance, propertyName) end) end end ``` **Change Guards:** ```lua function InstanceMap:__maybeFireInstanceChanged(instance, propertyName) if self.pausedUpdateInstances[instance] then return end -- Paused during reconciliation if self.onInstanceChanged == nil then return end -- No callback registered if RunService:IsRunning() then return end -- Don't sync during Play mode self.onInstanceChanged(instance, propertyName) end ``` **Pause Mechanism:** When applying incoming patches, instances are paused to prevent their changes from being sent back to the server (feedback loop prevention). ### ChangeBatcher (`plugin/src/ChangeBatcher/`) | File | Purpose | |------|---------| | `init.lua` | Main batcher: accumulator, RenderStepped loop, pause/resume | | `createPatchSet.lua` | Converts pending changes → PatchSet | | `encodePatchUpdate.lua` | Encodes property updates for existing instances | | `encodeInstance.lua` | Full instance encoding (for additions/syncback) | | `encodeProperty.lua` | Single property value encoding via RbxDom | **Core State:** ```lua ChangeBatcher = { __accumulator = 0, -- Time since last flush __pendingPropertyChanges = {}, -- Instance → { PropertyName → true } __syncSourceOnly = false, -- Only sync Source property __paused = false, -- Pause during confirmation __renderSteppedConnection = conn, -- Frame loop connection __instanceMap = instanceMap, __onChangesFlushed = callback, } ``` **Batch Cycle (every frame):** ```lua local BATCH_INTERVAL = 0.2 -- 200ms function ChangeBatcher:__cycle(dt) if self.__paused then return end self.__accumulator += dt if self.__accumulator >= BATCH_INTERVAL then self.__accumulator -= BATCH_INTERVAL local patch = self:__flush() if patch then self.__onChangesFlushed(patch) end end self.__instanceMap:unpauseAllInstances() -- Resume paused instances end ``` **Why 200ms batch interval?** - Studio fires many rapid property changes (e.g., dragging a part fires Position.X, Y, Z separately) - Batching combines related changes into single patch - Reduces server load and WebSocket traffic - Still feels responsive to users ### createPatchSet.lua: Patch Generation ```lua function createPatchSet(instanceMap, propertyChanges, syncSourceOnly) local patch = PatchSet.newEmpty() for instance, properties in propertyChanges do local instanceId = instanceMap.fromInstances[instance] if instanceId == nil then Log.warn("Ignoring change for unknown instance") continue end if properties.Parent then if instance.Parent == nil then -- Instance was deleted in Studio table.insert(patch.removed, instanceId) else Log.warn("Cannot sync non-nil Parent changes yet") -- Reparenting not supported end else if syncSourceOnly then -- Server in source-only mode: only sync Source property if properties.Source then local update = encodePatchUpdate(instance, instanceId, { Source = true }) if update then table.insert(patch.updated, update) end end else local update = encodePatchUpdate(instance, instanceId, properties) if update then table.insert(patch.updated, update) end end end end return patch end ``` ### encodePatchUpdate.lua: Update Encoding ```lua function encodePatchUpdate(instance, instanceId, properties) local update = { id = instanceId, changedProperties = {}, } for propertyName in properties do if propertyName == "Name" then update.changedName = instance.Name else local descriptor = RbxDom.findCanonicalPropertyDescriptor(instance.ClassName, propertyName) if not descriptor then continue end local success, encoded = encodeProperty(instance, propertyName, descriptor) if success then update.changedProperties[propertyName] = encoded end end end if next(update.changedProperties) == nil and update.changedName == nil then return nil -- No actual changes end return update end ``` ### encodeInstance.lua: Full Instance Encoding (Syncback/Additions) Used when pulling new instances from Studio to filesystem. **Critical: handles duplicate detection.** ```lua -- Duplicate handling (matches server's ref_properties.rs::is_path_unique) local function isPathUnique(instance) local current = instance while current do if hasDuplicateSiblings(current) then return false -- Any duplicate in ancestor chain = ambiguous path end current = current.Parent if current == game then break end end return true end function encodeInstance(instance, parentId, _skipPathCheck) -- Skip instances with ambiguous paths (duplicate-named siblings) if not _skipPathCheck and not isPathUnique(instance) then Log.warn("Skipped '{}' - path contains duplicate-named siblings", instance:GetFullName()) return nil end local properties = {} -- Encode Attributes and Tags (work on all instance types) encodeAttributes(instance, properties) encodeTags(instance, properties) -- For scripts, Source is required if SCRIPT_CLASS_NAMES[instance.ClassName] then local success, encoded = encodeProperty(instance, "Source", descriptor) if not success then return nil end properties.Source = encoded end -- Encode all serializable properties local classDescriptor = RbxDom.findClassDescriptor(instance.ClassName) for propertyName, meta in classDescriptor.properties do if SKIP_PROPERTIES[propertyName] then continue end -- Parent, Name, Archivable if SKIP_PROPERTIES_BY_CLASS[instance.ClassName]?[propertyName] then continue end -- Source, ScriptGuid, etc. local isReadable = meta.scriptability == "ReadWrite" or meta.scriptability == "Read" local doesSerialize = meta.serialization ~= "DoesNotSerialize" if isReadable and doesSerialize then local success, encoded = encodeProperty(instance, propertyName, descriptor) if success and encoded ~= nil then properties[propertyName] = encoded end end end -- Recursively encode children (skipping duplicates at each level) local children = {} local childDuplicates = findDuplicateNames(instance:GetChildren()) for _, child in instance:GetChildren() do if childDuplicates[child.Name] then continue end local encoded = encodeInstance(child, nil, true) -- Skip path check, already validated if encoded then table.insert(children, encoded) end end return { parent = parentId, name = instance.Name, className = instance.ClassName, properties = properties, children = children, } end ``` **Properties skipped per class:** | Class | Skipped Properties | Reason | |-------|-------------------|--------| | Script/LocalScript/ModuleScript | Source, ScriptGuid, RunContext | Encoded in file contents/suffix | | LocalizationTable | Contents | Encoded in .csv file | | StringValue | Value | Encoded in .txt file | ### encodeProperty.lua: Value Encoding ```lua function encodeProperty(instance, propertyName, descriptor) local readSuccess, value = descriptor:read(instance) if not readSuccess then return false, nil end local dataType = descriptor.dataType local encodeSuccess, encoded = RbxDom.EncodedValue.encode(value, dataType) if not encodeSuccess then return false, nil end return true, encoded end ``` **Encoded value format (via RbxDom):** ```lua -- Primitives true, false, 42, "string" -- Roblox types { Vector3 = {1, 2, 3} } { CFrame = { position = {0,0,0}, orientation = {...} } } { UDim2 = {{0, 100}, {0, 50}} } { Color3 = {1, 0.5, 0} } { BrickColor = 194 } { Ref = "server-instance-id" } -- Object references { Attributes = { key = value, ... } } { Tags = {"tag1", "tag2"} } ``` ### Server-Side: ChangeProcessor (`src/change_processor.rs`) Receives patches from plugin via `tree_mutation_receiver` channel: ```rust fn handle_tree_event(&self, patch_set: PatchSet) { let mut tree = self.tree.lock().unwrap(); // Process removals for &id in &patch_set.removed_instances { if let Some(instance) = tree.get_instance(id) { match &instance.metadata().instigating_source { Some(InstigatingSource::Path(path)) => { log::info!("Two-way sync: Removing file {}", path.display()); fs::remove_file(path)?; } Some(InstigatingSource::ProjectNode { .. }) => { log::warn!("Cannot remove instance from project file"); } None => { log::warn!("Cannot remove instance without instigating source"); } } } } // Process updates for update in &patch_set.updated_instances { if let Some(instance) = tree.get_instance(update.id) { // Currently only Source property is supported if let Some(Variant::String(source)) = update.changed_properties.get("Source") { match &instance.metadata().instigating_source { Some(InstigatingSource::Path(path)) => { log::info!("Two-way sync: Writing Source to {}", path.display()); fs::write(path, source)?; } _ => log::warn!("Cannot update instance without path source"), } } // Other property changes logged but not written if update.changed_name.is_some() { log::warn!("Cannot rename instances yet"); } if update.changed_class_name.is_some() { log::warn!("Cannot change ClassName yet"); } } } apply_patch_set(&mut tree, patch_set); // Update DOM self.message_queue.push_messages(&[applied_patch]); // Broadcast to plugins } ``` **InstigatingSource:** Every instance in RojoTree tracks where it came from: - `InstigatingSource::Path(PathBuf)` - Created from a file, can be written back - `InstigatingSource::ProjectNode { path, name, node, parent_class }` - Defined in project file, **cannot be modified via two-way sync** ### Pause Mechanism: Critical for Consistency **Why pause is essential:** 1. **During incoming patch confirmation:** User is reviewing changes. If ChangeBatcher sends Studio changes mid-review, it creates race conditions. 2. **During patch application:** Reconciler modifies instances, triggering property change events. Without pause, these would be sent back as "changes" (feedback loop). 3. **Defense-in-depth:** `__isConfirming` flag in ServeSession prevents multiple dialogs and ensures atomicity. ```lua -- In ServeSession, when showing confirmation dialog: self.__changeBatcher:pause() -- ... show PatchVisualizer, wait for user ... self.__changeBatcher:resume() ``` ```lua -- In Reconciler, when modifying instances: instanceMap:pauseInstance(instance) instance.SomeProperty = newValue -- Won't trigger ChangeBatcher -- Later, in ChangeBatcher:__cycle(): instanceMap:unpauseAllInstances() ``` ### One-Shot Mode Integration **Setting:** `oneShotSync` (default: true) One-shot mode adds an extra guard in `onChangesFlushed`: ```lua local function onChangesFlushed(patch) if Settings:get("oneShotSync") then Log.info("One-shot mode: blocking automatic outgoing write from ChangeBatcher") return -- Silently discard patch end self.__apiContext:write(patch) end ``` **Behavior:** - ChangeBatcher still collects changes (needed for diff visualization) - Automatic writes are blocked - Only explicit "pull" selections in confirmation dialog can write - Disconnects after initial sync completes - No sync lock acquired (multiple users can view same place) **Use cases:** - Quick preview of changes without risk of accidental writes - Safe for production environments - Team viewing same project simultaneously ### Known Limitations (Detailed) | Issue | Cause | Impact | |-------|-------|--------| | **Undo/redo triggers spurious changes** | Studio's ChangeHistoryService fires property changes during undo | Can cause unexpected writes or conflicts | | **Rapid changes interleave** | 200ms batch window may split logically related changes | Partial updates can leave inconsistent state | | **Reparenting not supported** | `Parent` property changes to non-nil values are ignored | Moving instances in Studio doesn't sync | | **Renaming not supported** | `changedName` logged but not written | Renaming in Studio doesn't sync | | **ClassName changes ignored** | Would require file type change | Changing script type doesn't sync | | **Only Source property writes** | Other properties logged but not written to filesystem | Only script content syncs reliably | | **Project node instances immutable** | InstigatingSource::ProjectNode cannot be modified | Instances from project file can't be changed | | **Duplicate-named siblings skipped** | Path ambiguity makes reliable sync impossible | Instances with same-name siblings ignored | | **Ref properties complex** | Instance references may become invalid when instances move | ObjectValue.Value may not sync correctly | **Recommended workflow:** Use one-way sync (server → Studio) as primary. Use two-way sync only for script Source changes with careful attention to undo/redo behavior. ## Protocol Enforcement ```rust // Server if client_protocol != PROTOCOL_VERSION { return error("Protocol mismatch") } ``` ```lua -- Plugin if serverInfo.protocolVersion ~= Config.protocolVersion then showError("Protocol mismatch") end ``` **Breaking changes:** Bump `protocolVersion` in `src/web/interface.rs` and `plugin/src/Config.lua` ## Session Management 1. Server starts → Generate UUID 2. Plugin connects → Receives session ID in handshake 3. All requests include session ID 4. Server restart → New session ID → Plugin auto-reconnects on mismatch ## Debugging ```bash # Server RUST_LOG=rojo::web=debug,rojo::serve_session=debug rojo serve ``` ```lua -- Plugin Settings:set("logLevel", "Trace") ``` Browser: `http://localhost:34872` → Network tab → WebSocket messages ## Key Files | Server | Plugin | |--------|--------| | `src/web/api.rs` - Endpoints | `plugin/src/ServeSession.lua` - Connection | | `src/web/interface.rs` - Packets | `plugin/src/ApiContext.lua` - HTTP/WS client | | `src/serve_session.rs` - Session | `plugin/src/ChangeBatcher/` - Two-way sync | | | `plugin/src/Reconciler/` - Patch application |