mirror of
https://github.com/rojo-rbx/rojo.git
synced 2026-10-23 05:51:39 +00:00
Update logging levels in Rojo plugin and enhance test plans
- Promoted sync-related logging from Debug/Trace to Info level across multiple files, ensuring visibility of changes in the Studio output console. - Added a new plan for expanding the connected mode test suite to cover multi-step scenarios and round-trip identity verification. - Introduced a plan to fix stale tree desynchronization by implementing a validation mechanism on plugin connection. - Created a plan to bump sync logging to Info level, detailing todos for various components in the plugin. This update improves the clarity of logs and enhances testing coverage for connected mode scenarios.
This commit is contained in:
Joe
parent
2cc1726975
commit
316d7ea70e
17 changed files with 2443 additions and 1553 deletions
119
.cursor/plans/bump_sync_logging_to_info_3e39ff81.plan.md
Normal file
119
.cursor/plans/bump_sync_logging_to_info_3e39ff81.plan.md
Normal file
|
|
@ -0,0 +1,119 @@
|
|||
---
|
||||
name: Bump sync logging to info
|
||||
overview: Promote all sync-related change logging in the Rojo Studio plugin from Debug/Trace to Info level, so that every change received, confirmed, sent, and applied is visible in the Studio output console at the default log level.
|
||||
todos:
|
||||
- id: log-padding
|
||||
content: Remove 15-space padding from log tags in plugin/log/init.lua (TRACE_TAG, INFO_TAG, DEBUG_TAG)
|
||||
status: completed
|
||||
- id: serve-session
|
||||
content: Bump Debug/Trace to Info in ServeSession.lua (received, confirmed, applied summaries + per-item decisions). Add combined patch summary after WebSocket message combining.
|
||||
status: completed
|
||||
- id: change-batcher
|
||||
content: Bump batch summary in ChangeBatcher/init.lua line 115 from Debug to Info
|
||||
status: completed
|
||||
- id: api-context
|
||||
content: Bump sending-to-server logs in ApiContext.lua from Debug/Trace to Info (summary + per-item operations)
|
||||
status: completed
|
||||
- id: apply-patch
|
||||
content: Bump patch application logs in Reconciler/applyPatch.lua from Debug to Info (summary + per-update details)
|
||||
status: completed
|
||||
isProject: false
|
||||
---
|
||||
|
||||
# Bump Sync Change Logging to Info
|
||||
|
||||
## Current State
|
||||
|
||||
The plugin's `Log` module (at `plugin/log/init.lua`) supports levels: Error(0), Warning(1), Info(2), Debug(3), Trace(4). Default log level is `"Info"`, meaning Debug and Trace messages are hidden unless the user manually changes the setting.
|
||||
|
||||
Most sync-flow messages are currently at Debug or Trace, so users can't see what changes are being received/sent/confirmed without digging into settings.
|
||||
|
||||
## Files to Change
|
||||
|
||||
### 0. [plugin/log/init.lua](plugin/log/init.lua) -- Fix excessive log padding
|
||||
|
||||
The log tags have 15 hardcoded spaces prepended for alignment with Studio's timestamp column, but it creates absurdly wide output -- especially for multi-line messages where `addTags` replaces every `\n` with `\n` + the full padded tag.
|
||||
|
||||
```lua
|
||||
-- Current (lines 15-16) -- repeats tag on every line of multi-line messages:
|
||||
local function addTags(tag, message)
|
||||
return tag .. message:gsub("\n", "\n" .. tag)
|
||||
end
|
||||
|
||||
-- Fix: just prepend the tag once
|
||||
local function addTags(tag, message)
|
||||
return tag .. message
|
||||
end
|
||||
```
|
||||
|
||||
```lua
|
||||
-- Current (lines 19-21) -- 15 spaces of padding before each tag:
|
||||
local TRACE_TAG = (" "):rep(15) .. "[Rojo-Trace] "
|
||||
local INFO_TAG = (" "):rep(15) .. "[Rojo-Info] "
|
||||
local DEBUG_TAG = (" "):rep(15) .. "[Rojo-Debug] "
|
||||
|
||||
-- Fix: remove the padding
|
||||
local TRACE_TAG = "[Rojo-Trace] "
|
||||
local INFO_TAG = "[Rojo-Info] "
|
||||
local DEBUG_TAG = "[Rojo-Debug] "
|
||||
```
|
||||
|
||||
Note: `WARN_TAG` on line 22 already has no padding (it goes through `warn()` which has its own formatting).
|
||||
|
||||
### 1. [plugin/src/ServeSession.lua](plugin/src/ServeSession.lua) -- Received / Confirmed / Applied
|
||||
|
||||
**Received from server (WebSocket):**
|
||||
|
||||
- Line 222: `Log.debug` -> `Log.info` -- "Received {} messages from Rojo server"
|
||||
- After line 228 (after combining messages into `combinedPatch`): add an `Log.info` summary of what's IN the combined patch (removals, additions, updates counts), similar to how `applyPatch.lua` does it. Currently the contents are only visible at Trace via the raw packet dump.
|
||||
|
||||
**Confirmation selections:**
|
||||
|
||||
- Line 699: `Log.debug` -> `Log.info` -- "User selections: {} push, {} pull, {} ignored"
|
||||
- Line 713: `Log.trace` -> `Log.info` -- "[Push] Update: {}"
|
||||
- Line 726: `Log.trace` -> `Log.info` -- "[Pull] Update: {}"
|
||||
- Line 757: `Log.trace` -> `Log.info` -- "[Push] Delete: {}"
|
||||
- Line 792: `Log.trace` -> `Log.info` -- "[Push] Add: {}.{}"
|
||||
- Lines 772, 797, 828 are already Info -- no change needed
|
||||
|
||||
**Applied to Studio (summary):**
|
||||
|
||||
- Line 813: `Log.debug` -> `Log.info` -- "Applying to Studio: {} additions, {} removals, {} updates"
|
||||
- Line 583: `Log.debug` -> `Log.info` -- unapplied patch summary (important for diagnosing sync issues)
|
||||
|
||||
### 2. [plugin/src/ChangeBatcher/init.lua](plugin/src/ChangeBatcher/init.lua) -- Outgoing batched changes
|
||||
|
||||
- Line 115: `Log.debug` -> `Log.info` -- "Two-way sync: {} updates, {} additions, {} removals"
|
||||
- Lines 70, 73 stay at Trace -- these fire per-property-per-frame and would flood the console if promoted
|
||||
|
||||
### 3. [plugin/src/ApiContext.lua](plugin/src/ApiContext.lua) -- Sent to server
|
||||
|
||||
- Line 202: `Log.debug` -> `Log.info` -- "Sending to server: {} removals, {} additions, {} updates"
|
||||
- Line 212: `Log.trace` -> `Log.info` -- "[Syncback] Remove ID: {}"
|
||||
- Line 218: `Log.trace` -> `Log.info` -- "[Syncback] Add {} ({})"
|
||||
- Line 228: `Log.trace` -> `Log.info` -- "[Syncback] Update ID {} ({} properties)"
|
||||
|
||||
### 4. [plugin/src/Reconciler/applyPatch.lua](plugin/src/Reconciler/applyPatch.lua) -- Applied to Studio (per-item)
|
||||
|
||||
- Line 36: `Log.debug` -> `Log.info` -- "Applying patch to Studio: {} removals, {} additions, {} updates"
|
||||
- Line 141: `Log.debug` -> `Log.info` -- "[Studio] Update: {} - {}"
|
||||
- Lines 56, 106 are already Info -- no change needed
|
||||
|
||||
## What stays at Debug/Trace (intentionally)
|
||||
|
||||
- ChangeBatcher per-property detection (lines 70, 73) -- fires every frame, too noisy
|
||||
- Raw WebSocket packet dump (ApiContext line 299) -- huge payloads, not useful for comparison
|
||||
- Precommit/postcommit hook registration (ServeSession lines 182, 189, 203, 210) -- lifecycle plumbing, not sync data
|
||||
- Internal fallback/replace logic (ServeSession lines 561, 566) -- internal recovery mechanism
|
||||
|
||||
## Summary of Sync Flow Visibility After Changes
|
||||
|
||||
```
|
||||
Server → Plugin: "Received N messages" + combined patch summary (counts)
|
||||
Confirmation: "User selections: X push, Y pull, Z ignored" + per-item decisions
|
||||
Apply to Studio: "Applying patch: N removals, M additions, K updates" + per-item details
|
||||
Studio → Server: "Two-way sync: X updates, Y additions, Z removals" (batch summary)
|
||||
Send to Server: "Sending to server: N removals, M additions, K updates" + per-item details
|
||||
```
|
||||
|
||||
All of these will now appear in the Studio output console at the default Info log level.
|
||||
489
.cursor/plans/connected_mode_test_expansion_5b0c39a3.plan.md
Normal file
489
.cursor/plans/connected_mode_test_expansion_5b0c39a3.plan.md
Normal file
|
|
@ -0,0 +1,489 @@
|
|||
---
|
||||
name: Connected Mode Test Expansion
|
||||
overview: Expand the serve test suite to cover multi-step connected mode scenarios -- format transitions, init file operations, directory conversions, slugify/dedup forward-sync, meta name field lifecycle, and sequential change sequences -- with full WebSocket patch verification AND round-trip identity verification. 53 tests total across 8 phases.
|
||||
todos:
|
||||
- id: fixtures
|
||||
content: "Create 4 new test fixtures in rojo-test/serve-tests/: connected_scripts, connected_init, connected_models, connected_slugify"
|
||||
status: in_progress
|
||||
- id: harness
|
||||
content: "Add harness helpers to tests/rojo_test/serve_util.rs: get_message_cursor for chaining multi-step tests, fresh_rebuild_read for round-trip verification"
|
||||
status: pending
|
||||
- id: phase1-format-transitions
|
||||
content: "Implement Phase 1: Format transition tests (standalone<->directory, init type changes) with round-trip verification -- 8 tests"
|
||||
status: pending
|
||||
- id: phase2-multi-step
|
||||
content: "Implement Phase 2: Multi-step connected sequence tests (create/edit/rename/delete lifecycle) with round-trip verification -- 5 tests"
|
||||
status: pending
|
||||
- id: phase3-init
|
||||
content: "Implement Phase 3: Init file operation tests during connected mode with round-trip verification -- 4 tests"
|
||||
status: pending
|
||||
- id: phase4-meta
|
||||
content: "Implement Phase 4: Property/meta file change tests -- 4 tests"
|
||||
status: pending
|
||||
- id: phase5-models
|
||||
content: "Implement Phase 5: Model JSON file change tests -- 5 tests"
|
||||
status: pending
|
||||
- id: phase6-cursor
|
||||
content: "Implement Phase 6: Cursor tracking and reconnection tests -- 3 tests"
|
||||
status: pending
|
||||
- id: phase7-mixed
|
||||
content: "Implement Phase 7: Mixed API + filesystem verification tests -- 2 tests"
|
||||
status: pending
|
||||
- id: phase8-slugify-fixture
|
||||
content: Create connected_slugify fixture with slugified files, dedup ~1 files, clean files, and directory-format slugified instances
|
||||
status: pending
|
||||
- id: phase8-slugify-forward
|
||||
content: "Implement Phase 8a: Slugify forward-sync tests -- editing/creating/deleting slugified files and verifying patches show real names from meta -- 6 tests"
|
||||
status: pending
|
||||
- id: phase8-dedup-forward
|
||||
content: "Implement Phase 8b: Dedup (~N suffix) forward-sync tests -- collision handling, ~1 with and without meta, add/remove colliding files -- 5 tests"
|
||||
status: pending
|
||||
- id: phase8-slugdir
|
||||
content: "Implement Phase 8c: Slugified directory format tests -- init type changes, child adds inside slugified dirs -- 3 tests"
|
||||
status: pending
|
||||
- id: phase8-meta-name-lifecycle
|
||||
content: "Implement Phase 8d: Meta name field lifecycle tests -- create/update/delete meta name, model.json5 name field -- 5 tests"
|
||||
status: pending
|
||||
- id: phase8-slug-multistep
|
||||
content: "Implement Phase 8e: Multi-step slugify scenarios with round-trip verification -- slug rename chains, collision evolution -- 3 tests"
|
||||
status: pending
|
||||
- id: run-and-review
|
||||
content: Run cargo test, cargo insta review, accept snapshots, verify clean pass
|
||||
status: pending
|
||||
isProject: false
|
||||
---
|
||||
|
||||
# Connected Mode Test Expansion
|
||||
|
||||
## Quality Standard
|
||||
|
||||
This test expansion is held to the same standard as the [slugify branch audit](audit_slugify_branch_e913b7dc.plan.md):
|
||||
|
||||
> **Round-trip identity**: Syncback (or two-way sync) writes a directory tree. Building an rbxl from that directory tree and forward-syncing it back must produce a **bit-identical instance tree** -- same names, same classes, same properties, same hierarchy, same ref targets. Any deviation is a bug.
|
||||
|
||||
Every test in this plan must verify not just "did the WebSocket patch look right" but also "if I killed the server and rebuilt from the filesystem right now, would I get the same tree?" This is verified by starting a fresh `rojo serve` on the same project directory after changes and comparing the resulting tree.
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
The current test suite has a major asymmetry:
|
||||
|
||||
- `**two_way_sync.rs**` (plugin -> filesystem via API): 119 tests, covering stress, format transitions, encoded names, concurrent ops, failure recovery
|
||||
- `**serve.rs**` (filesystem -> plugin via WebSocket patches): 22 tests, each making only ONE change. No multi-step sequences, no format transitions, no directory conversions, no round-trip verification
|
||||
|
||||
Connected mode means the plugin is connected and receiving patches over time as the filesystem evolves. The `serve.rs` tests only verify a single atomic change, never simulating a session where the tree evolves through multiple states. They also never verify the round-trip invariant -- that the filesystem state after changes would produce an identical tree if rebuilt from scratch.
|
||||
|
||||
---
|
||||
|
||||
## Architecture of a Connected Mode Test
|
||||
|
||||
Each test follows this pattern in [tests/tests/serve.rs](tests/tests/serve.rs):
|
||||
|
||||
```rust
|
||||
run_serve_test("fixture_name", |session, mut redactions| {
|
||||
// 1. Snapshot initial state
|
||||
let info = session.get_api_rojo().unwrap();
|
||||
let root_id = info.root_instance_id;
|
||||
let read = session.get_api_read(root_id).unwrap();
|
||||
assert_yaml_snapshot!("test_initial", read.intern_and_redact(&mut redactions, root_id));
|
||||
|
||||
// 2. Change 1 -> verify patch + state
|
||||
let packet = session.recv_socket_packet(SocketPacketType::Messages, 0, || {
|
||||
fs::write(&path, "new content").unwrap();
|
||||
}).unwrap();
|
||||
assert_yaml_snapshot!("test_patch1", packet.intern_and_redact(&mut redactions, ()));
|
||||
let read = session.get_api_read(root_id).unwrap();
|
||||
assert_yaml_snapshot!("test_state1", read.intern_and_redact(&mut redactions, root_id));
|
||||
|
||||
// 3. Change 2 -> verify patch + state (using cursor from patch 1)
|
||||
let cursor = get_message_cursor(&packet);
|
||||
let packet = session.recv_socket_packet(SocketPacketType::Messages, cursor, || {
|
||||
fs::rename(&old, &new).unwrap();
|
||||
}).unwrap();
|
||||
assert_yaml_snapshot!("test_patch2", packet.intern_and_redact(&mut redactions, ()));
|
||||
|
||||
// 4. Round-trip verification: fresh rebuild must match live tree
|
||||
let final_read = session.get_api_read(root_id).unwrap();
|
||||
let fresh_read = fresh_rebuild_read(session.path());
|
||||
assert_trees_match(&final_read, &fresh_read);
|
||||
});
|
||||
```
|
||||
|
||||
The critical addition over existing serve.rs tests:
|
||||
|
||||
- **Multi-step sequences** with cursor chaining
|
||||
- **Round-trip verification** at the end -- a fresh `rojo serve` on the same filesystem must produce the same instance tree
|
||||
|
||||
Key files involved:
|
||||
|
||||
- [tests/tests/serve.rs](tests/tests/serve.rs) -- where new tests go
|
||||
- [tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs) -- test harness
|
||||
- [rojo-test/serve-tests/](rojo-test/serve-tests/) -- test fixtures
|
||||
- [rojo-test/serve-test-snapshots/](rojo-test/serve-test-snapshots/) -- insta snapshots
|
||||
|
||||
---
|
||||
|
||||
## Harness Changes
|
||||
|
||||
### `get_message_cursor` -- cursor extraction for chaining
|
||||
|
||||
[tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs) needs a helper to extract `messageCursor` from a `SocketPacket` for chaining multi-step tests:
|
||||
|
||||
```rust
|
||||
/// Extract the message cursor from a SocketPacket for use in subsequent subscriptions.
|
||||
pub fn get_message_cursor(packet: &SocketPacket) -> u32 {
|
||||
match &packet.body {
|
||||
SocketPacketBody::Messages(msg) => msg.message_cursor,
|
||||
_ => panic!("Expected Messages packet"),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `fresh_rebuild_read` -- round-trip identity verification
|
||||
|
||||
This is the core addition for enforcing the round-trip standard. After all changes, start a **fresh** `rojo serve` on the same project directory and read the full tree. If the live DOM and the fresh rebuild disagree, the filesystem state is inconsistent.
|
||||
|
||||
```rust
|
||||
/// Start a fresh rojo serve on the given project path, read the full tree,
|
||||
/// and return a normalized representation for comparison.
|
||||
///
|
||||
/// This verifies the round-trip invariant: the filesystem state written by
|
||||
/// the live session, when read from scratch, must produce the same tree.
|
||||
pub fn fresh_rebuild_read(project_path: &Path) -> NormalizedTree {
|
||||
// Find the project file in the directory
|
||||
let project_file = find_project_file(project_path);
|
||||
|
||||
let port = get_port_number();
|
||||
let port_string = port.to_string();
|
||||
let working_dir = get_working_dir_path();
|
||||
|
||||
let mut process = Command::new(ROJO_PATH)
|
||||
.args(["serve", project_file.to_str().unwrap(), "--port", &port_string])
|
||||
.current_dir(working_dir)
|
||||
.stderr(Stdio::piped())
|
||||
.spawn()
|
||||
.expect("Couldn't start fresh Rojo for round-trip check");
|
||||
|
||||
let kill_guard = KillOnDrop(process);
|
||||
|
||||
// Wait for fresh server to come online (same backoff as TestServeSession)
|
||||
let info = wait_for_server(port);
|
||||
let read = get_read(port, info.root_instance_id);
|
||||
|
||||
// Normalize: strip Refs (non-deterministic), sort children, extract
|
||||
// (name, class, properties, hierarchy) tuples for structural comparison
|
||||
normalize_tree(&read)
|
||||
}
|
||||
```
|
||||
|
||||
### `NormalizedTree` -- structural comparison ignoring Refs
|
||||
|
||||
Since two separate `rojo serve` sessions produce different `Ref` values, we need structural comparison. A `NormalizedTree` is a recursive structure of `(name, class_name, sorted_properties, sorted_children)` that can be compared with `==` or snapshotted.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, PartialEq, Eq, serde::Serialize)]
|
||||
pub struct NormalizedInstance {
|
||||
pub name: String,
|
||||
pub class_name: String,
|
||||
pub properties: BTreeMap<String, String>, // property name -> debug representation
|
||||
pub children: Vec<NormalizedInstance>, // sorted by (name, class_name)
|
||||
}
|
||||
```
|
||||
|
||||
### `assert_round_trip` -- convenience assertion
|
||||
|
||||
```rust
|
||||
/// Assert that the live session tree matches a fresh rebuild from the filesystem.
|
||||
pub fn assert_round_trip(session: &TestServeSession, root_id: Ref) {
|
||||
let live_read = session.get_api_read(root_id).unwrap();
|
||||
let live_tree = normalize_read_response(&live_read, root_id);
|
||||
|
||||
let fresh_tree = fresh_rebuild_read(session.path());
|
||||
|
||||
assert_eq!(
|
||||
live_tree, fresh_tree,
|
||||
"Round-trip identity violation: live tree and fresh rebuild differ. \
|
||||
The filesystem state does not faithfully represent the instance tree."
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## New Test Fixtures
|
||||
|
||||
### `rojo-test/serve-tests/connected_scripts/`
|
||||
|
||||
Fixture with both standalone and directory format scripts, covering all three script types:
|
||||
|
||||
```
|
||||
connected_scripts/
|
||||
default.project.json5 # name: "connected_scripts", tree: { ReplicatedStorage: { $path: "src" } }
|
||||
src/
|
||||
standalone.luau # ModuleScript (standalone format)
|
||||
server.server.luau # Script (standalone format)
|
||||
local.client.luau # LocalScript (standalone format)
|
||||
DirModule/
|
||||
init.luau # ModuleScript (directory format)
|
||||
ChildA.luau # child ModuleScript
|
||||
ChildB.luau # child ModuleScript
|
||||
DirScript/
|
||||
init.server.luau # Script (directory format)
|
||||
Handler.luau # child ModuleScript
|
||||
DirLocal/
|
||||
init.local.luau # LocalScript (directory format)
|
||||
Helper.luau # child ModuleScript
|
||||
```
|
||||
|
||||
### `rojo-test/serve-tests/connected_init/`
|
||||
|
||||
Focused init file fixture for init-specific tests:
|
||||
|
||||
```
|
||||
connected_init/
|
||||
default.project.json5 # name: "connected_init", tree: { ReplicatedStorage: { $path: "src" } }
|
||||
src/
|
||||
init.luau # Root is a ModuleScript
|
||||
Child.luau # A child module
|
||||
```
|
||||
|
||||
### `rojo-test/serve-tests/connected_models/`
|
||||
|
||||
Model file fixture:
|
||||
|
||||
```
|
||||
connected_models/
|
||||
default.project.json5 # name: "connected_models", tree: { ReplicatedStorage: { $path: "src" } }
|
||||
src/
|
||||
SimpleModel.model.json5 # { "className": "Configuration", "properties": { "Name": "SimpleModel" } }
|
||||
DirModel/
|
||||
init.meta.json5 # { "className": "Folder" }
|
||||
Child.luau # child ModuleScript
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## New Tests -- Organized by Category
|
||||
|
||||
### Phase 1: Format Transitions (filesystem-driven, patch + round-trip verified)
|
||||
|
||||
Highest priority. Tests what the plugin receives when files transform between standalone and directory formats, AND verifies the filesystem is consistent after each transformation.
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
|
||||
| 1 | `standalone_to_directory_add_child` | Create `ChildNew.luau` next to `standalone.luau`. The snapshot system should convert `standalone.luau` -> `standalone/init.luau` + `standalone/ChildNew.luau`. Verify the patch shows the correct tree transformation. | Yes |
|
||||
| 2 | `directory_to_standalone_remove_children` | Remove `ChildA.luau` and `ChildB.luau` from `DirModule/`. With no children left, verify whether `DirModule/init.luau` collapses to `DirModule.luau`. Verify patch. | Yes |
|
||||
| 3 | `standalone_to_directory_and_back` | Multi-step: (a) add child next to standalone script (standalone->dir), (b) remove child (dir->standalone). Two patches verified, plus round-trip after each step. | Yes |
|
||||
| 4 | `init_type_change_module_to_server` | Rename `DirModule/init.luau` -> `DirModule/init.server.luau`. Verify patch contains ClassName change from ModuleScript to Script. | Yes |
|
||||
| 5 | `init_type_change_server_to_local` | Rename `DirScript/init.server.luau` -> `DirScript/init.local.luau`. Verify ClassName change from Script to LocalScript. | Yes |
|
||||
| 6 | `init_type_change_cycle` | Multi-step: `init.luau` -> `init.server.luau` -> `init.local.luau` -> `init.luau`. Three patches, three state snapshots, round-trip at each step. | Yes |
|
||||
| 7 | `init_delete_children_survive` | Delete `DirModule/init.luau` while `ChildA.luau` and `ChildB.luau` remain. Verify children survive and parent becomes Folder (or whatever the correct behavior is). | Yes |
|
||||
| 8 | `init_create_in_existing_folder` | Given a folder with only children (no init file), create `init.luau`. Verify the Folder instance becomes a ModuleScript. | Yes |
|
||||
|
||||
|
||||
**Important for tests 1-2**: The snapshot middleware may or may not automatically convert between standalone and directory format when children are added/removed via the filesystem. If it doesn't (i.e., the conversion only happens through the API/change processor), the test should document that and verify the actual behavior.
|
||||
|
||||
### Phase 2: Multi-Step Connected Sequences
|
||||
|
||||
Core connected mode pattern: the tree evolves through multiple states. Each intermediate state is verified.
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ |
|
||||
| 9 | `create_edit_rename_delete_sequence` | Create `src/NewScript.luau` -> edit its contents -> rename to `Renamed.luau` -> delete it. Four patches, four state snapshots. Verify each patch contains exactly the expected delta. | Yes (at end) |
|
||||
| 10 | `directory_lifecycle` | Create `src/NewDir/` -> add `init.luau` + `Child.luau` -> edit `init.luau` -> remove `Child.luau` -> remove directory. Full directory lifecycle with 5 patches. | Yes (at end) |
|
||||
| 11 | `multi_file_sequential_edits` | Edit `standalone.luau`, then `server.server.luau`, then `local.client.luau`. Verify each patch only contains the changed file (no cross-contamination). | No |
|
||||
| 12 | `add_multiple_files_sequentially` | Add 3 new `.luau` files one at a time. Verify each patch adds exactly one instance with correct name and class. | Yes (at end) |
|
||||
| 13 | `remove_multiple_files_sequentially` | Remove 3 files one at a time. Verify each patch removes exactly one instance. | No |
|
||||
|
||||
|
||||
### Phase 3: Init File Operations (connected mode)
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
|
||||
| 14 | `edit_init_then_edit_child` | Edit `DirModule/init.luau`, then edit `DirModule/ChildA.luau`. Two separate patches, each containing only the changed instance's Source property. | No |
|
||||
| 15 | `replace_init_file_type` | Delete `DirModule/init.luau`, then create `DirModule/init.server.luau`. Verify the ClassName changes from ModuleScript to Script, children preserved. | Yes |
|
||||
| 16 | `add_init_to_plain_folder` | Start with a directory containing only children (create by removing init from fixture). Add `init.luau`. Verify Folder -> ModuleScript. | Yes |
|
||||
| 17 | `delete_init_from_directory` | Delete init file from directory with children. Verify parent becomes Folder, children survive. | Yes |
|
||||
|
||||
|
||||
### Phase 4: Property/Meta Changes (connected mode)
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
|
||||
| 18 | `adjacent_meta_creation` | Create `standalone.meta.json5` with `{ "properties": { "Disabled": true } }` next to `standalone.luau`. Verify property appears in patch. | Yes |
|
||||
| 19 | `adjacent_meta_edit` | Edit existing `.meta.json5` to change a property value. Verify update patch. | No |
|
||||
| 20 | `adjacent_meta_delete` | Delete `.meta.json5`. Verify properties revert to defaults. | Yes |
|
||||
| 21 | `init_meta_creation` | Create `DirModule/init.meta.json5` with properties. Verify directory instance gains properties. | Yes |
|
||||
|
||||
|
||||
### Phase 5: Model File Changes (connected mode)
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | ---------------------------- | ------------------------------------------------------------------------------------------ | ---------- |
|
||||
| 22 | `model_json_edit_properties` | Edit `SimpleModel.model.json5` to change/add properties. Verify property changes in patch. | Yes |
|
||||
| 23 | `model_json_add_children` | Edit model JSON to add child instances. Verify additions in patch. | Yes |
|
||||
| 24 | `model_json_remove_children` | Edit model JSON to remove child instances. Verify removals in patch. | No |
|
||||
| 25 | `model_json_create` | Create new `NewModel.model.json5`. Verify instance addition patch. | Yes |
|
||||
| 26 | `model_json_delete` | Delete `SimpleModel.model.json5`. Verify instance removal patch. | No |
|
||||
|
||||
|
||||
### Phase 6: Cursor Tracking and Reconnection
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | ---------- |
|
||||
| 27 | `cursor_advances_correctly` | Make 3 filesystem changes. After each, verify the cursor returned in the WebSocket packet increments monotonically. | No |
|
||||
| 28 | `reconnect_with_old_cursor` | Make 2 changes with cursor advancement. Reconnect with cursor 0. Verify ALL patches are received in the catch-up response. | No |
|
||||
| 29 | `reconnect_with_partial_cursor` | Make 3 changes. Reconnect with cursor from after change 1. Verify only patches 2+3 are received. | No |
|
||||
|
||||
|
||||
### Phase 7: Mixed API + Filesystem Verification
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------- |
|
||||
| 30 | `api_write_then_filesystem_change` | POST `/api/write` to update Source, then edit a different file on disk. Verify only the filesystem change produces a WebSocket patch (echo suppression working). | Yes |
|
||||
| 31 | `filesystem_change_then_api_write` | Edit file on disk, receive patch. Then POST `/api/write` on a different instance. Verify the API write produces the correct filesystem change and does NOT echo back as a duplicate patch. | Yes |
|
||||
|
||||
|
||||
### Phase 8: Slugify / Dedup / Meta Name (connected mode, round-trip critical)
|
||||
|
||||
This is the most critical phase for round-trip identity. The slugify system replaces forbidden filesystem chars with `_`, deduplicates collisions with `~1`/`~2` suffixes, and stores the real instance name in `.meta.json5` / `.model.json5` `name` fields. If ANY part of this chain is broken in the forward-sync direction (filesystem -> plugin patches), instance names are silently corrupted.
|
||||
|
||||
The existing `two_way_sync.rs` has 17+ tests for slugify/dedup via the API write path (plugin -> filesystem). **None** of those verify the forward-sync direction: what WebSocket patch does the plugin receive when slugified files are changed on the filesystem? This phase fills that gap completely.
|
||||
|
||||
#### New Fixture: `rojo-test/serve-tests/connected_slugify/`
|
||||
|
||||
Richer than `syncback_encoded_names` -- includes dedup collisions and directory-format slugified instances:
|
||||
|
||||
```
|
||||
connected_slugify/
|
||||
default.project.json5 # name: "connected_slugify", tree: { ReplicatedStorage: { $path: "src" } }
|
||||
src/
|
||||
# Clean-named files (no meta needed)
|
||||
Normal.luau # ModuleScript "Normal"
|
||||
CleanScript.server.luau # Script "CleanScript"
|
||||
|
||||
# Slugified files (meta has real name)
|
||||
What_Module.luau # ModuleScript, meta: "What?Module"
|
||||
What_Module.meta.json5 # { "name": "What?Module" }
|
||||
Key_Script.server.luau # Script, meta: "Key:Script"
|
||||
Key_Script.meta.json5 # { "name": "Key:Script" }
|
||||
|
||||
# Dedup collision: two instances whose slugs both = "Hey_Bro"
|
||||
Hey_Bro.luau # ModuleScript "Hey_Bro" (natural name, no meta)
|
||||
Hey_Bro~1.luau # ModuleScript, meta: "Hey/Bro"
|
||||
Hey_Bro~1.meta.json5 # { "name": "Hey/Bro" }
|
||||
|
||||
# Model JSON with name field
|
||||
What_Model.model.json5 # { "name": "What?Model", "className": "Configuration" }
|
||||
|
||||
# Slugified directory format
|
||||
Slug_Dir/ # Directory, meta: "Slug:Dir"
|
||||
init.luau # ModuleScript init
|
||||
init.meta.json5 # { "name": "Slug:Dir" }
|
||||
DirChild.luau # child ModuleScript
|
||||
```
|
||||
|
||||
#### Phase 8a: Slugify Forward-Sync (filesystem -> plugin patches)
|
||||
|
||||
Tests that editing/creating/deleting slugified files produces patches with correct **real** instance names (from meta), not filesystem names (slugs).
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
|
||||
| 32 | `edit_slugified_script_source` | Edit `What_Module.luau` content. Verify WebSocket patch contains `changedProperties.Source` for instance named `What?Module` (NOT `What_Module`). The meta `name` field must be respected during forward-sync. | Yes |
|
||||
| 33 | `edit_slugified_server_script_source` | Edit `Key_Script.server.luau` content. Verify patch contains Source change for instance named `Key:Script`. Same principle, different script type. | No |
|
||||
| 34 | `create_new_slugified_file_with_meta` | Create `New_Slug.luau` + `New_Slug.meta.json5` with `{"name": "New/Slug"}`. Verify the patch adds an instance named `New/Slug` (not `New_Slug`). | Yes |
|
||||
| 35 | `delete_slugified_file_and_meta` | Delete both `What_Module.luau` and `What_Module.meta.json5`. Verify the patch removes exactly one instance (the one named `What?Module`). | No |
|
||||
| 36 | `edit_only_meta_name_field` | Edit `What_Module.meta.json5` to change `{"name": "What?Module"}` -> `{"name": "What*Module"}`. Verify the patch contains a name change (or re-snapshot). The slug stays the same but the real name changes. This is the "Foo/Bar -> Foo | Bar, both slugify to Foo_Bar" case from the audit. |
|
||||
| 37 | `delete_meta_name_reverts_to_stem` | Delete `What_Module.meta.json5` (but leave `What_Module.luau`). Verify the patch shows the instance name changing from `What?Module` to `What_Module` (the filename stem becomes the name when no meta exists). | Yes |
|
||||
|
||||
|
||||
#### Phase 8b: Dedup (~N Suffix) Forward-Sync
|
||||
|
||||
Tests that `~N` suffixed files are handled correctly: the suffix is just part of the filename stem, the real name comes from meta. Without meta, the full stem (including `~N`) IS the instance name.
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
|
||||
| 38 | `edit_dedup_file_with_meta` | Edit `Hey_Bro~1.luau` content. Verify patch shows Source change for instance named `Hey/Bro` (from meta), not `Hey_Bro~1`. | Yes |
|
||||
| 39 | `dedup_file_without_meta_uses_stem` | Create `Foo~1.luau` with NO meta file. Verify the instance name in the patch is literally `Foo~1` -- the tilde is NOT parsed as a dedup marker during forward-sync. This is the backward compatibility guarantee. | Yes |
|
||||
| 40 | `add_second_colliding_file` | Start with `Hey_Bro.luau` (name: `Hey_Bro`). Create `Hey_Bro~1.luau` + `Hey_Bro~1.meta.json5` with `{"name": "Hey:Bro"}`. Verify the patch adds a NEW instance named `Hey:Bro` as a sibling, without disturbing `Hey_Bro`. | Yes |
|
||||
| 41 | `remove_one_of_two_colliding_files` | Delete `Hey_Bro~1.luau` + `Hey_Bro~1.meta.json5`. Verify the patch removes exactly the instance named `Hey/Bro`, leaving `Hey_Bro` untouched. | Yes |
|
||||
| 42 | `add_third_collision` | With `Hey_Bro.luau` and `Hey_Bro~1.luau` already present, create `Hey_Bro~2.luau` + `Hey_Bro~2.meta.json5` with `{"name": "Hey*Bro"}`. Verify patch adds instance named `Hey*Bro`. All three coexist. | Yes |
|
||||
|
||||
|
||||
#### Phase 8c: Slugified Directory Format
|
||||
|
||||
Tests that directory-format instances with slugified names work correctly during connected mode. The real name is in `init.meta.json5`.
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
|
||||
| 43 | `edit_init_in_slugified_directory` | Edit `Slug_Dir/init.luau` content. Verify patch shows Source change for instance named `Slug:Dir` (from `init.meta.json5`), not `Slug_Dir`. | Yes |
|
||||
| 44 | `init_type_change_in_slugified_directory` | Rename `Slug_Dir/init.luau` -> `Slug_Dir/init.server.luau`. Verify ClassName changes from ModuleScript to Script. Instance name must remain `Slug:Dir` (from meta). | Yes |
|
||||
| 45 | `add_child_to_slugified_directory` | Create `Slug_Dir/NewChild.luau`. Verify patch adds a new ModuleScript child under the instance named `Slug:Dir`. | Yes |
|
||||
|
||||
|
||||
#### Phase 8d: Meta Name Field Lifecycle (connected mode)
|
||||
|
||||
Tests the full lifecycle of the meta `name` field -- the bridge between filesystem names and real instance names. Any bug here silently corrupts names.
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------- |
|
||||
| 46 | `add_meta_name_to_clean_file` | `Normal.luau` has no meta (name = `Normal`). Create `Normal.meta.json5` with `{"name": "Nor/mal"}`. Verify the patch shows the instance name changing from `Normal` to `Nor/mal`. | Yes |
|
||||
| 47 | `update_meta_name_field` | `What_Module.meta.json5` has `{"name": "What?Module"}`. Change to `{"name": "What:Module"}`. Verify the patch shows name change. The slug (`What_Module`) doesn't change, only the real name does. | Yes |
|
||||
| 48 | `meta_name_with_other_properties` | Create meta file with BOTH `name` AND `$properties`: `{"name": "Real/Name", "$properties": {"Disabled": true}}`. Verify both the name override AND the property appear in the patch. | Yes |
|
||||
| 49 | `model_json_name_field_edit` | Edit `What_Model.model.json5` to change `"name": "What?Model"` -> `"name": "What:Model"`. Verify name change in patch. The `name` field in `.model.json5` serves the same purpose as in `.meta.json5`. | Yes |
|
||||
| 50 | `model_json_remove_name_field` | Edit `What_Model.model.json5` to remove the `"name"` field entirely. Verify instance name reverts from `What?Model` to `What_Model` (derived from filename stem). | Yes |
|
||||
|
||||
|
||||
#### Phase 8e: Multi-Step Slugify Scenarios (round-trip proofs)
|
||||
|
||||
Multi-step sequences that exercise the slugify system through state transitions, with round-trip verification at each step. These are the highest-value tests for catching subtle bugs.
|
||||
|
||||
|
||||
| # | Test Name | What It Tests | Round-Trip |
|
||||
| --- | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
|
||||
| 51 | `slugify_rename_chain` | Multi-step: (1) Edit `What_Module.luau` content, (2) Update meta name `What?Module` -> `What*Module`, (3) Delete meta (name reverts to `What_Module`), (4) Re-create meta with `{"name": "What/Module"}`. Four patches, four state snapshots. Round-trip verified at each step. | Yes (each step) |
|
||||
| 52 | `collision_evolution` | Multi-step: (1) Add `Hey_Bro~2.luau` + meta `{"name": "Hey*Bro"}` (third collision), (2) Delete `Hey_Bro~1.luau` + meta (remove `Hey/Bro`), (3) Add `Hey_Bro~1.luau` + meta with different name `{"name": "Hey | Bro"}`(reuse the`~1` slot). Three patches. Verify correct instances exist at each step. |
|
||||
| 53 | `clean_to_slug_to_clean_via_filesystem` | Multi-step: (1) `Normal.luau` exists (clean name). Add `Normal.meta.json5` with `{"name": "Nor:mal"}` -- name changes to `Nor:mal`. (2) Delete the meta -- name reverts to `Normal`. (3) Rename `Normal.luau` -> `Has_Slash.luau` + create `Has_Slash.meta.json5` with `{"name": "Has/Slash"}` -- completely different slug+name. Round-trip verified at each step. | Yes (each step) |
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Execution Order
|
||||
|
||||
1. **Fixtures**: Create 4 new fixture directories: `connected_scripts`, `connected_init`, `connected_models`, `connected_slugify`
|
||||
2. **Harness**: Add `get_message_cursor`, `NormalizedTree`/`NormalizedInstance`, `fresh_rebuild_read`, `assert_round_trip` to [tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs)
|
||||
3. **Phase 1** (format transitions): Highest value, most likely to catch bugs in the snapshot pipeline
|
||||
4. **Phase 2** (multi-step sequences): Core connected mode pattern, validates cursor chaining
|
||||
5. **Phase 3** (init file ops): Critical for directory handling correctness
|
||||
6. **Phases 4-7**: Property/meta, model JSON, cursor tracking, mixed API+filesystem
|
||||
7. **Phase 8a-e** (slugify/dedup): 22 tests covering the entire forward-sync direction of the name system, with round-trip proofs
|
||||
8. **Run**: `cargo test`, then `cargo insta review` to accept snapshots
|
||||
9. **Verify**: All tests pass on a clean run, no snapshot drift
|
||||
|
||||
---
|
||||
|
||||
## Notes on Test Design
|
||||
|
||||
### Filesystem events are OS-dependent
|
||||
|
||||
Some operations (like rename) may fire different events on macOS (kqueue/FSEvents) vs Windows (ReadDirectoryChangesW) vs Linux (inotify). Tests that depend on exact patch structure may need `#[cfg_attr(target_os = "macos", ignore)]` annotations if event ordering differs. The existing `ref_properties_remove` test already does this.
|
||||
|
||||
### The snapshot system may not auto-convert formats
|
||||
|
||||
When adding a child file next to a standalone script via the filesystem, the snapshot system may NOT automatically convert to directory format (that conversion might only happen through the change processor during API writes). Tests should verify and document the actual behavior rather than assuming conversion happens.
|
||||
|
||||
### Echo suppression is critical for mixed tests
|
||||
|
||||
Phase 7 tests depend on echo suppression working correctly. The API write path suppresses VFS events for paths it wrote, preventing feedback loops. If echo suppression is broken, these tests will see duplicate patches.
|
||||
|
|
@ -229,15 +229,18 @@ All in [tests/tests/two_way_sync.rs](tests/tests/two_way_sync.rs) and [tests/tes
|
|||
|
||||
### Files created
|
||||
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `src/syncback/meta.rs` | Shared `upsert_meta_name()` and `remove_meta_name()` -- pure JSON operations without filesystem event suppression. Returns `RemoveNameOutcome` enum so callers handle suppression themselves. |
|
||||
| `rojo-test/build-tests/tilde_no_meta/` | Build fixture: `Foo~1.luau` with no meta file. Verifies forward sync produces instance name `Foo~1`. |
|
||||
|
||||
|
||||
### Files modified
|
||||
|
||||
|
||||
| File | What changed |
|
||||
|------|-------------|
|
||||
| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `src/syncback/file_names.rs` | Added `KNOWN_SCRIPT_SUFFIXES`, `strip_script_suffix()`, `adjacent_meta_path()`. Added 19 new unit tests (suffix stripping, meta path derivation, tilde collision, stress, idempotency). |
|
||||
| `src/syncback/mod.rs` | Added `pub mod meta;`. Exported `adjacent_meta_path`. Removed `deny_unknown_fields` from `SyncbackRules`. |
|
||||
| `src/change_processor.rs` | Rewrote `upsert_meta_name_field` / `remove_meta_name_field` to delegate to shared `syncback::meta` module. **C1 fix**: moved meta name update logic OUTSIDE `if new_path != *path` for both init-file and regular-file rename paths. Introduced `effective_dir_path` / `effective_meta_base` tracking variables. |
|
||||
|
|
@ -246,6 +249,7 @@ All in [tests/tests/two_way_sync.rs](tests/tests/two_way_sync.rs) and [tests/tes
|
|||
| `tests/tests/build.rs` | Added `tilde_no_meta` to `gen_build_tests!` macro. |
|
||||
| `tests/tests/two_way_sync.rs` | Added 2 integration tests: `add_instance_with_forbidden_chars_creates_slug_and_meta`, `add_two_colliding_instances_deduplicates`. |
|
||||
|
||||
|
||||
### Test results
|
||||
|
||||
- **646 unit tests pass** (15 new in `file_names.rs`)
|
||||
|
|
@ -255,10 +259,8 @@ All in [tests/tests/two_way_sync.rs](tests/tests/two_way_sync.rs) and [tests/tes
|
|||
|
||||
### Key design decisions
|
||||
|
||||
1. **`meta.rs` returns outcomes, doesn't suppress** -- The shared `upsert_meta_name` / `remove_meta_name` functions are pure I/O (read JSON, modify, write/delete). Filesystem event suppression stays in the caller (`change_processor.rs`) because it depends on the `JobThreadContext` API. This keeps the shared module framework-agnostic.
|
||||
|
||||
2. **`effective_dir_path` / `effective_meta_base` pattern** -- For the C1 fix, rather than duplicating the meta update logic in both branches of the path-change condition, we track which path ended up being used (original or renamed) and run the meta update once afterward. This avoids the "nested inside conditional" bug class entirely.
|
||||
|
||||
3. **`adjacent_meta_path` derives from filesystem name** -- The helper strips the file extension and script suffix from the actual filesystem path, never from the instance name. This ensures the meta path always matches the script file regardless of slugification.
|
||||
|
||||
1. `**meta.rs` returns outcomes, doesn't suppress** -- The shared `upsert_meta_name` / `remove_meta_name` functions are pure I/O (read JSON, modify, write/delete). Filesystem event suppression stays in the caller (`change_processor.rs`) because it depends on the `JobThreadContext` API. This keeps the shared module framework-agnostic.
|
||||
2. `**effective_dir_path` / `effective_meta_base` pattern** -- For the C1 fix, rather than duplicating the meta update logic in both branches of the path-change condition, we track which path ended up being used (original or renamed) and run the meta update once afterward. This avoids the "nested inside conditional" bug class entirely.
|
||||
3. `**adjacent_meta_path` derives from filesystem name** -- The helper strips the file extension and script suffix from the actual filesystem path, never from the instance name. This ensures the meta path always matches the script file regardless of slugification.
|
||||
4. **Alphabetical sort for project.rs determinism** -- Chose alphabetical sort over referent-ID sort because it's human-predictable and matches what a developer would expect when looking at the filesystem.
|
||||
|
||||
|
|
|
|||
130
.cursor/plans/fix_stale_tree_desync_9e74f718.plan.md
Normal file
130
.cursor/plans/fix_stale_tree_desync_9e74f718.plan.md
Normal file
|
|
@ -0,0 +1,130 @@
|
|||
---
|
||||
name: Fix stale tree desync
|
||||
overview: Add a tree validation mechanism that re-snapshots the project tree from the real filesystem when a plugin connects, ensuring the in-memory RojoTree never serves stale data even when VFS watcher events are missed.
|
||||
todos:
|
||||
- id: validate-method
|
||||
content: Add `validate_tree()` method to ServeSession that re-snapshots from disk and patches the tree
|
||||
status: completed
|
||||
- id: trigger-on-connect
|
||||
content: Call `validate_tree()` in the WebSocket connection handler before serving data
|
||||
status: completed
|
||||
- id: broadcast-corrections
|
||||
content: Push any corrections from validation to the message queue for all connected plugins
|
||||
status: completed
|
||||
- id: test-validation
|
||||
content: "Test: start serve, externally delete/modify a tracked file, connect plugin, verify tree is corrected"
|
||||
status: completed
|
||||
isProject: false
|
||||
---
|
||||
|
||||
# Fix Stale Tree Desync on Plugin Connect
|
||||
|
||||
## Problem
|
||||
|
||||
When a plugin connects (especially in one-shot mode), the server's in-memory `RojoTree` can be stale if the VFS file watcher missed external filesystem changes. The tree is built once on startup and only updated via VFS events through `ChangeProcessor`. If events are missed (watcher buffer overflow, rapid batch operations like git, OS-level quirks), the tree silently drifts from reality. The plugin reads this stale tree via `GET /api/read/:id`, computes its diff against Studio, and sees "no changes" even though the filesystem is different.
|
||||
|
||||
Rebooting `rojo serve` forces a fresh read and fixes it, confirming the issue is purely stale in-memory state.
|
||||
|
||||
## Solution: Tree Validation on Plugin Connect
|
||||
|
||||
When a new plugin connection is established (WebSocket handshake), trigger a validation pass that re-snapshots the root project path from the real filesystem, diffs against the current tree, and applies any corrections. This is essentially what `ServeSession::new` does on startup, but incremental and without restart.
|
||||
|
||||
## Key Files
|
||||
|
||||
- `[src/serve_session.rs](src/serve_session.rs)` -- Owns the tree, VFS, and project. Add `validate_tree()` method here.
|
||||
- `[src/change_processor.rs](src/change_processor.rs)` -- Has `apply_patches()` which re-snapshots a path and patches the tree. Reuse this pattern.
|
||||
- `[src/web/api.rs](src/web/api.rs)` -- WebSocket connection handler. Call validation before serving the tree.
|
||||
- `[src/snapshot/patch_compute.rs](src/snapshot/patch_compute.rs)` -- `compute_patch_set` used to diff snapshots.
|
||||
|
||||
## Implementation
|
||||
|
||||
### 1. Add `validate_tree()` to ServeSession
|
||||
|
||||
In `[src/serve_session.rs](src/serve_session.rs)`, add a public method that:
|
||||
|
||||
- Takes the root project's start path (`self.root_project.folder_location()`)
|
||||
- Locks the VFS and tree
|
||||
- Calls `snapshot_from_vfs(&instance_context, &vfs, start_path)` to get a fresh snapshot from disk
|
||||
- Calls `compute_patch_set(snapshot, &tree, root_id)` to diff against the current tree
|
||||
- If the patch is non-empty, calls `apply_patch_set(&mut tree, patch_set)` and pushes the result to the message queue
|
||||
|
||||
This reuses the same functions that `ServeSession::new` uses for initial tree construction. The VFS reads directly from the filesystem (no cache), so the snapshot will reflect the real state.
|
||||
|
||||
A compile-time constant at the top of `serve_session.rs` controls whether validation runs:
|
||||
|
||||
```rust
|
||||
/// Set to `false` to skip tree validation on plugin connect (useful for testing).
|
||||
const VALIDATE_TREE_ON_CONNECT: bool = true;
|
||||
```
|
||||
|
||||
The method itself:
|
||||
|
||||
```rust
|
||||
pub fn validate_tree(&self) -> Vec<AppliedPatchSet> {
|
||||
if !VALIDATE_TREE_ON_CONNECT {
|
||||
log::debug!("Tree validation skipped (VALIDATE_TREE_ON_CONNECT = false)");
|
||||
return Vec::new();
|
||||
}
|
||||
|
||||
let start = std::time::Instant::now();
|
||||
let start_path = self.root_project.folder_location();
|
||||
let instance_context = InstanceContext::new();
|
||||
|
||||
let snapshot = match snapshot_from_vfs(&instance_context, &self.vfs, start_path) {
|
||||
Ok(s) => s,
|
||||
Err(e) => {
|
||||
log::error!("Tree validation snapshot error: {:?}", e);
|
||||
return Vec::new();
|
||||
}
|
||||
};
|
||||
|
||||
let mut tree = self.tree.lock().unwrap();
|
||||
let root_id = tree.get_root_id();
|
||||
let patch_set = compute_patch_set(snapshot, &tree, root_id);
|
||||
|
||||
if patch_set.is_empty() {
|
||||
log::info!("Tree validation complete (no corrections needed) in {:.1?}", start.elapsed());
|
||||
return Vec::new();
|
||||
}
|
||||
|
||||
log::info!("Tree validation found stale state, applying corrections");
|
||||
let applied = apply_patch_set(&mut tree, patch_set);
|
||||
log::info!("Tree validation complete in {:.1?}", start.elapsed());
|
||||
vec![applied]
|
||||
}
|
||||
```
|
||||
|
||||
Note: The `Vfs` struct wraps the backend with a mutex internally (`self.inner.lock()`), so we access it via `self.vfs` (which is `Arc<Vfs>`) without needing an explicit lock. The exact locking pattern will follow the existing code style.
|
||||
|
||||
### 2. Trigger Validation on WebSocket Connect
|
||||
|
||||
In `[src/web/api.rs](src/web/api.rs)`, the WebSocket handler establishes a new connection. Before entering the message loop (or right after the handshake), call `validate_tree()` and broadcast any corrections:
|
||||
|
||||
```rust
|
||||
// In the WebSocket handler, after handshake:
|
||||
let corrections = self.serve_session.validate_tree();
|
||||
if !corrections.is_empty() {
|
||||
self.serve_session.message_queue().push_messages(&corrections);
|
||||
}
|
||||
```
|
||||
|
||||
This ensures the tree is consistent with disk before the plugin starts receiving data.
|
||||
|
||||
### 3. Broadcast Corrections to Already-Connected Plugins
|
||||
|
||||
If multiple plugins are connected, the corrections are pushed to the message queue, which broadcasts to all connected WebSocket clients. This ensures all plugins get the updated state.
|
||||
|
||||
## Edge Cases
|
||||
|
||||
- **Large projects**: Full re-snapshot could add 100-500ms latency on connect for very large projects. This is acceptable for one-shot mode (happens once) and live mode (happens on reconnect). Can be optimized later with targeted path validation if needed.
|
||||
- **Concurrent access**: The tree lock is held during validation. Other WebSocket handlers will block briefly. This matches the existing pattern in `ChangeProcessor::apply_patches`.
|
||||
- **No changes**: If the tree is already fresh (common case), `compute_patch_set` returns an empty patch and no work is done. The overhead is one `snapshot_from_vfs` call (directory walk + file stats).
|
||||
- **VFS consistency**: After validation, VFS events for the re-snapshotted paths will still fire (watches persist). The `compute_and_apply_changes` handler will process them as no-ops since the tree already matches.
|
||||
|
||||
## What This Fixes
|
||||
|
||||
After this change, "no changes appearing" on plugin reconnect should not happen regardless of whether VFS events were missed. The tree is always refreshed from disk on connect, matching the behavior of a fresh `rojo serve` startup.
|
||||
|
||||
## What This Does NOT Fix
|
||||
|
||||
The content "swap" issue (both directions firing during per-item selection) is a separate bug that needs further investigation. The stale tree fix ensures that even if a swap occurs, the developer can immediately reconnect and see the correct diff to recover.
|
||||
|
|
@ -13,12 +13,12 @@ local function getLogLevel()
|
|||
end
|
||||
|
||||
local function addTags(tag, message)
|
||||
return tag .. message:gsub("\n", "\n" .. tag)
|
||||
return tag .. message
|
||||
end
|
||||
|
||||
local TRACE_TAG = (" "):rep(15) .. "[Rojo-Trace] "
|
||||
local INFO_TAG = (" "):rep(15) .. "[Rojo-Info] "
|
||||
local DEBUG_TAG = (" "):rep(15) .. "[Rojo-Debug] "
|
||||
local TRACE_TAG = "[Rojo-Trace] "
|
||||
local INFO_TAG = "[Rojo-Info] "
|
||||
local DEBUG_TAG = "[Rojo-Debug] "
|
||||
local WARN_TAG = "[Rojo-Warn] "
|
||||
|
||||
local Log = {}
|
||||
|
|
|
|||
|
|
@ -197,9 +197,8 @@ function ApiContext:write(patch)
|
|||
addedCount += 1
|
||||
end
|
||||
|
||||
-- Only log summary at debug level to reduce verbosity
|
||||
if #patch.removed > 0 or addedCount > 0 or #patch.updated > 0 then
|
||||
Log.debug(
|
||||
Log.info(
|
||||
"Sending to server: {} removals, {} additions, {} updates",
|
||||
#patch.removed,
|
||||
addedCount,
|
||||
|
|
@ -207,15 +206,14 @@ function ApiContext:write(patch)
|
|||
)
|
||||
end
|
||||
|
||||
-- Log individual operations at trace level (very detailed)
|
||||
for _, removed in ipairs(patch.removed) do
|
||||
Log.trace("[Syncback] Remove ID: {}", tostring(removed))
|
||||
Log.info("[Syncback] Remove ID: {}", tostring(removed))
|
||||
end
|
||||
|
||||
for _, addedInstance in pairs(patch.added) do
|
||||
local instanceName = addedInstance.name or addedInstance.Name or "unknown"
|
||||
local instanceClass = addedInstance.className or addedInstance.ClassName or "unknown"
|
||||
Log.trace("[Syncback] Add {} ({})", instanceName, instanceClass)
|
||||
Log.info("[Syncback] Add {} ({})", instanceName, instanceClass)
|
||||
end
|
||||
|
||||
for _, update in ipairs(patch.updated) do
|
||||
|
|
@ -225,7 +223,7 @@ function ApiContext:write(patch)
|
|||
propCount += 1
|
||||
end
|
||||
end
|
||||
Log.trace("[Syncback] Update ID {} ({} properties)", update.id, propCount)
|
||||
Log.info("[Syncback] Update ID {} ({} properties)", update.id, propCount)
|
||||
end
|
||||
|
||||
local updated = {}
|
||||
|
|
|
|||
|
|
@ -21,6 +21,7 @@ local Log = require(Packages.Log)
|
|||
local RbxDom = require(Packages.RbxDom)
|
||||
|
||||
local encodeProperty = require(script.Parent.encodeProperty)
|
||||
local UNENCODABLE_DATA_TYPES = require(script.Parent.propertyFilter)
|
||||
|
||||
-- Script class names that need Source property
|
||||
local SCRIPT_CLASS_NAMES = {
|
||||
|
|
@ -234,6 +235,10 @@ encodeInstance = function(instance, parentId, _skipPathCheck)
|
|||
-- Get the full PropertyDescriptor for encoding
|
||||
local descriptor = RbxDom.findCanonicalPropertyDescriptor(instance.ClassName, propertyName)
|
||||
if descriptor then
|
||||
if UNENCODABLE_DATA_TYPES[descriptor.dataType] then
|
||||
continue
|
||||
end
|
||||
|
||||
local encodeSuccess, encodeResult = encodeProperty(instance, propertyName, descriptor)
|
||||
if encodeSuccess and encodeResult ~= nil then
|
||||
properties[propertyName] = encodeResult
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ local Log = require(Packages.Log)
|
|||
local RbxDom = require(Packages.RbxDom)
|
||||
|
||||
local encodeProperty = require(script.Parent.encodeProperty)
|
||||
local UNENCODABLE_DATA_TYPES = require(script.Parent.propertyFilter)
|
||||
|
||||
return function(instance, instanceId, properties)
|
||||
local update = {
|
||||
|
|
@ -21,6 +22,10 @@ return function(instance, instanceId, properties)
|
|||
continue
|
||||
end
|
||||
|
||||
if UNENCODABLE_DATA_TYPES[descriptor.dataType] then
|
||||
continue
|
||||
end
|
||||
|
||||
local encodeSuccess, encodeResult = encodeProperty(instance, propertyName, descriptor)
|
||||
|
||||
if not encodeSuccess then
|
||||
|
|
|
|||
|
|
@ -11,7 +11,13 @@ return function(instance, propertyName, propertyDescriptor)
|
|||
end
|
||||
|
||||
local dataType = propertyDescriptor.dataType
|
||||
local encodeSuccess, encodeResult = RbxDom.EncodedValue.encode(readResult, dataType)
|
||||
local pcallOk, encodeSuccess, encodeResult = pcall(RbxDom.EncodedValue.encode, readResult, dataType)
|
||||
|
||||
if not pcallOk then
|
||||
-- Encoder threw an error (e.g., Ref, Region3, SharedString)
|
||||
Log.warn("Could not sync back property {:?}.{}: {}", instance, propertyName, encodeSuccess)
|
||||
return false, nil
|
||||
end
|
||||
|
||||
if not encodeSuccess then
|
||||
Log.warn("Could not sync back property {:?}.{}: {}", instance, propertyName, encodeResult)
|
||||
|
|
|
|||
|
|
@ -107,12 +107,11 @@ function ChangeBatcher:__flush()
|
|||
return nil
|
||||
end
|
||||
|
||||
-- Log summary at debug level
|
||||
local addedCount = 0
|
||||
for _ in pairs(patch.added) do
|
||||
addedCount += 1
|
||||
end
|
||||
Log.debug("Two-way sync: {} updates, {} additions, {} removals", #patch.updated, addedCount, #patch.removed)
|
||||
Log.info("Two-way sync: {} updates, {} additions, {} removals", #patch.updated, addedCount, #patch.removed)
|
||||
|
||||
return patch
|
||||
end
|
||||
|
|
|
|||
16
plugin/src/ChangeBatcher/propertyFilter.lua
Normal file
16
plugin/src/ChangeBatcher/propertyFilter.lua
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
--[[
|
||||
Property type blacklist for encoding, matching the server-side
|
||||
filter_properties_for_meta behavior in src/web/api.rs.
|
||||
|
||||
Types listed here are skipped during property encoding because they
|
||||
cannot be serialized to JSON or are meaningless for syncback.
|
||||
]]
|
||||
|
||||
-- Property data types that cannot be encoded standalone.
|
||||
-- Matches the Variant::Ref / Variant::UniqueId filter in api.rs.
|
||||
local UNENCODABLE_DATA_TYPES = {
|
||||
Ref = true, -- Instance references; need the InstanceMap, not encodable standalone
|
||||
UniqueId = true, -- Internal Roblox identifiers; not meaningful for syncback
|
||||
}
|
||||
|
||||
return UNENCODABLE_DATA_TYPES
|
||||
|
|
@ -27,13 +27,12 @@ local function applyPatch(instanceMap, patch)
|
|||
-- to ensure that referents can be mapped to instances correctly.
|
||||
local deferredRefs = {}
|
||||
|
||||
-- Log summary of patch being applied (debug level for summary)
|
||||
local addCount = 0
|
||||
for _ in pairs(patch.added) do
|
||||
addCount += 1
|
||||
end
|
||||
if #patch.removed > 0 or addCount > 0 or #patch.updated > 0 then
|
||||
Log.debug(
|
||||
Log.info(
|
||||
"Applying patch to Studio: {} removals, {} additions, {} updates",
|
||||
#patch.removed,
|
||||
addCount,
|
||||
|
|
@ -124,7 +123,6 @@ local function applyPatch(instanceMap, patch)
|
|||
continue
|
||||
end
|
||||
|
||||
-- Log the update (debug level for property details to reduce verbosity)
|
||||
local instancePath = instance:GetFullName()
|
||||
local propList = {}
|
||||
if update.changedName then
|
||||
|
|
@ -138,7 +136,7 @@ local function applyPatch(instanceMap, patch)
|
|||
table.insert(propList, propName)
|
||||
end
|
||||
end
|
||||
Log.debug("[Studio] Update: {} - {}", instancePath, table.concat(propList, ", "))
|
||||
Log.info("[Studio] Update: {} - {}", instancePath, table.concat(propList, ", "))
|
||||
|
||||
-- Pause updates on this instance to avoid picking up our changes when
|
||||
-- two-way sync is enabled.
|
||||
|
|
|
|||
|
|
@ -219,7 +219,7 @@ function ServeSession:__onWebSocketMessage(messagesPacket)
|
|||
return
|
||||
end
|
||||
|
||||
Log.debug("Received {} messages from Rojo server", #messagesPacket.messages)
|
||||
Log.info("Received {} messages from Rojo server", #messagesPacket.messages)
|
||||
|
||||
-- Combine all messages into a single patch
|
||||
local combinedPatch = PatchSet.newEmpty()
|
||||
|
|
@ -227,6 +227,17 @@ function ServeSession:__onWebSocketMessage(messagesPacket)
|
|||
PatchSet.assign(combinedPatch, message)
|
||||
end
|
||||
|
||||
local addedCount = 0
|
||||
for _ in pairs(combinedPatch.added) do
|
||||
addedCount += 1
|
||||
end
|
||||
Log.info(
|
||||
"Combined patch: {} removals, {} additions, {} updates",
|
||||
#combinedPatch.removed,
|
||||
addedCount,
|
||||
#combinedPatch.updated
|
||||
)
|
||||
|
||||
-- If we're already waiting for confirmation, merge into the patch being confirmed
|
||||
-- Changes flow into the dialogue even in one-shot mode so user can see them
|
||||
if self.__confirmingPatch ~= nil then
|
||||
|
|
@ -580,7 +591,7 @@ function ServeSession:__applyPatch(patch)
|
|||
end
|
||||
|
||||
if not PatchSet.isEmpty(unappliedPatch) then
|
||||
Log.debug(
|
||||
Log.info(
|
||||
"Could not apply all changes requested by the Rojo server:\n{}",
|
||||
PatchSet.humanSummary(self.__instanceMap, unappliedPatch)
|
||||
)
|
||||
|
|
@ -696,7 +707,7 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo)
|
|||
ignoreCount += 1
|
||||
end
|
||||
end
|
||||
Log.debug("User selections: {} push, {} pull, {} ignored", pushCount, pullCount, ignoreCount)
|
||||
Log.info("User selections: {} push, {} pull, {} ignored", pushCount, pullCount, ignoreCount)
|
||||
|
||||
-- Build partial patches based on selections
|
||||
local pushPatch = PatchSet.newEmpty() -- Items to apply to Studio
|
||||
|
|
@ -710,7 +721,7 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo)
|
|||
|
||||
if selection == "push" then
|
||||
-- Apply Rojo changes to Studio
|
||||
Log.trace("[Push] Update: {}", instancePath)
|
||||
Log.info("[Push] Update: {}", instancePath)
|
||||
table.insert(pushPatch.updated, change)
|
||||
elseif selection == "pull" and self.__twoWaySync then
|
||||
-- Send Studio state back to Rojo
|
||||
|
|
@ -723,7 +734,7 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo)
|
|||
continue
|
||||
end
|
||||
end
|
||||
Log.trace("[Pull] Update: {}", instancePath)
|
||||
Log.info("[Pull] Update: {}", instancePath)
|
||||
local update = encodePatchUpdate(instance, change.id, propertiesToSync)
|
||||
if update then
|
||||
table.insert(pullPatch.updated, update)
|
||||
|
|
@ -754,7 +765,7 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo)
|
|||
|
||||
if selection == "push" then
|
||||
-- Apply Rojo removal to Studio (delete the instance)
|
||||
Log.trace("[Push] Delete: {}", instancePath)
|
||||
Log.info("[Push] Delete: {}", instancePath)
|
||||
table.insert(pushPatch.removed, idOrInstance)
|
||||
elseif selection == "pull" and self.__twoWaySync then
|
||||
-- Syncback: Create file in Rojo from Studio instance
|
||||
|
|
@ -789,7 +800,7 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo)
|
|||
|
||||
if selection == "push" then
|
||||
-- Apply Rojo addition to Studio
|
||||
Log.trace("[Push] Add: {}.{}", parentPath, instanceName)
|
||||
Log.info("[Push] Add: {}.{}", parentPath, instanceName)
|
||||
pushPatch.added[id] = change
|
||||
elseif selection == "pull" and self.__twoWaySync then
|
||||
-- Don't add in Studio, remove from Rojo
|
||||
|
|
@ -810,7 +821,7 @@ function ServeSession:__confirmAndApplyInitialPatch(catchUpPatch, serverInfo)
|
|||
for _ in pairs(pushPatch.added) do
|
||||
addCount += 1
|
||||
end
|
||||
Log.debug(
|
||||
Log.info(
|
||||
"Applying to Studio: {} additions, {} removals, {} updates",
|
||||
addCount,
|
||||
#pushPatch.removed,
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ use crossbeam_channel::{select, Receiver, RecvError, Sender};
|
|||
use jod_thread::JoinHandle;
|
||||
use memofs::{IoResultExt, Vfs, VfsEvent};
|
||||
use rbx_dom_weak::types::{Ref, Variant};
|
||||
use std::fmt;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::{Duration, Instant};
|
||||
use std::{
|
||||
|
|
@ -18,6 +19,27 @@ use crate::{
|
|||
syncback::{deduplicate_name, name_needs_slugify, slugify_name, strip_script_suffix},
|
||||
};
|
||||
|
||||
/// Wrapper that displays a path relative to a project root directory.
|
||||
struct RelPath<'a> {
|
||||
path: &'a Path,
|
||||
root: &'a Path,
|
||||
}
|
||||
|
||||
impl fmt::Display for RelPath<'_> {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
self.path
|
||||
.strip_prefix(self.root)
|
||||
.unwrap_or(self.path)
|
||||
.display()
|
||||
.fmt(f)
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns a display wrapper that shows `path` relative to `root`.
|
||||
fn rel_path<'a>(path: &'a Path, root: &'a Path) -> RelPath<'a> {
|
||||
RelPath { path, root }
|
||||
}
|
||||
|
||||
/// Processes file change events, updates the DOM, and sends those updates
|
||||
/// through a channel for other stuff to consume.
|
||||
///
|
||||
|
|
@ -54,6 +76,7 @@ impl ChangeProcessor {
|
|||
message_queue: Arc<MessageQueue<AppliedPatchSet>>,
|
||||
tree_mutation_receiver: Receiver<PatchSet>,
|
||||
suppressed_paths: Arc<Mutex<std::collections::HashMap<PathBuf, (usize, usize)>>>,
|
||||
project_root: PathBuf,
|
||||
) -> Self {
|
||||
let (shutdown_sender, shutdown_receiver) = crossbeam_channel::bounded(1);
|
||||
let vfs_receiver = vfs.event_receiver();
|
||||
|
|
@ -63,6 +86,7 @@ impl ChangeProcessor {
|
|||
message_queue,
|
||||
pending_recovery: Mutex::new(Vec::new()),
|
||||
suppressed_paths,
|
||||
project_root,
|
||||
};
|
||||
|
||||
let job_thread = jod_thread::Builder::new()
|
||||
|
|
@ -132,9 +156,17 @@ struct JobThreadContext {
|
|||
/// Paths recently written by the API's syncback. Events for these paths
|
||||
/// are suppressed to avoid redundant re-snapshots. Values are `(remove_count, create_write_count)`.
|
||||
suppressed_paths: Arc<Mutex<std::collections::HashMap<PathBuf, (usize, usize)>>>,
|
||||
|
||||
/// Root directory of the project, used to display relative paths in logs.
|
||||
project_root: PathBuf,
|
||||
}
|
||||
|
||||
impl JobThreadContext {
|
||||
/// Returns a display wrapper that shows `path` relative to the project root.
|
||||
fn display_path<'a>(&'a self, path: &'a Path) -> RelPath<'a> {
|
||||
rel_path(path, &self.project_root)
|
||||
}
|
||||
|
||||
/// Find the init file inside a directory-format script folder.
|
||||
/// Returns the path to the first `init.*.luau` or `init.*.lua` found.
|
||||
fn find_init_file(dir: &Path) -> Option<PathBuf> {
|
||||
|
|
@ -241,7 +273,7 @@ impl JobThreadContext {
|
|||
self.unsuppress_path(meta_path);
|
||||
log::error!(
|
||||
"Failed to upsert name in meta file {}: {}",
|
||||
meta_path.display(),
|
||||
self.display_path(meta_path),
|
||||
err
|
||||
);
|
||||
}
|
||||
|
|
@ -255,7 +287,7 @@ impl JobThreadContext {
|
|||
self.unsuppress_path(model_path);
|
||||
log::error!(
|
||||
"Failed to upsert name in model file {}: {}",
|
||||
model_path.display(),
|
||||
self.display_path(model_path),
|
||||
err
|
||||
);
|
||||
}
|
||||
|
|
@ -283,7 +315,7 @@ impl JobThreadContext {
|
|||
self.unsuppress_path(model_path);
|
||||
log::error!(
|
||||
"Failed to remove name from model file {}: {}",
|
||||
model_path.display(),
|
||||
self.display_path(model_path),
|
||||
err
|
||||
);
|
||||
}
|
||||
|
|
@ -314,7 +346,7 @@ impl JobThreadContext {
|
|||
self.unsuppress_path(meta_path);
|
||||
log::error!(
|
||||
"Failed to remove name from meta file {}: {}",
|
||||
meta_path.display(),
|
||||
self.display_path(meta_path),
|
||||
err
|
||||
);
|
||||
}
|
||||
|
|
@ -342,7 +374,7 @@ impl JobThreadContext {
|
|||
|
||||
log::info!(
|
||||
"apply_patches: path {} affects IDs {:?}",
|
||||
current_path.display(),
|
||||
self.display_path(current_path),
|
||||
ids
|
||||
);
|
||||
|
||||
|
|
@ -352,7 +384,7 @@ impl JobThreadContext {
|
|||
|
||||
log::info!(
|
||||
"apply_patches: no IDs at {}, trying parent...",
|
||||
current_path.display()
|
||||
self.display_path(current_path)
|
||||
);
|
||||
match current_path.parent() {
|
||||
Some(parent) => current_path = parent,
|
||||
|
|
@ -363,12 +395,12 @@ impl JobThreadContext {
|
|||
if affected_ids.is_empty() {
|
||||
log::info!(
|
||||
"apply_patches: no affected instances found for path {}",
|
||||
path.display()
|
||||
self.display_path(&path)
|
||||
);
|
||||
}
|
||||
|
||||
for id in affected_ids {
|
||||
if let Some(result) = compute_and_apply_changes(&mut tree, &self.vfs, id) {
|
||||
if let Some(result) = compute_and_apply_changes(&mut tree, &self.vfs, id, &self.project_root) {
|
||||
// If an instance was removed, schedule a recovery check
|
||||
// in case the path is recreated momentarily.
|
||||
if let Some(removed_path) = result.removed_path {
|
||||
|
|
@ -390,9 +422,9 @@ impl JobThreadContext {
|
|||
// This is intentionally verbose — it is critical for debugging
|
||||
// file watcher desync issues (e.g., rapid delete+recreate).
|
||||
match &event {
|
||||
VfsEvent::Create(path) => log::info!("VFS event: CREATE {}", path.display()),
|
||||
VfsEvent::Write(path) => log::info!("VFS event: WRITE {}", path.display()),
|
||||
VfsEvent::Remove(path) => log::info!("VFS event: REMOVE {}", path.display()),
|
||||
VfsEvent::Create(path) => log::info!("VFS event: CREATE {}", self.display_path(path)),
|
||||
VfsEvent::Write(path) => log::info!("VFS event: WRITE {}", self.display_path(path)),
|
||||
VfsEvent::Remove(path) => log::info!("VFS event: REMOVE {}", self.display_path(path)),
|
||||
_ => log::info!("VFS event: OTHER {:?}", event),
|
||||
}
|
||||
|
||||
|
|
@ -445,7 +477,7 @@ impl JobThreadContext {
|
|||
.expect("Error applying VFS change");
|
||||
log::info!(
|
||||
"VFS event SUPPRESSED (API syncback echo): {}",
|
||||
path.display()
|
||||
self.display_path(path)
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
|
@ -466,8 +498,8 @@ impl JobThreadContext {
|
|||
Ok(canonical_path) => {
|
||||
log::info!(
|
||||
"VFS: canonicalize OK for {} -> {}",
|
||||
path.display(),
|
||||
canonical_path.display()
|
||||
self.display_path(&path),
|
||||
self.display_path(&canonical_path)
|
||||
);
|
||||
self.apply_patches(canonical_path)
|
||||
}
|
||||
|
|
@ -509,7 +541,7 @@ impl JobThreadContext {
|
|||
log::info!(
|
||||
"VFS: phantom Create/Write for non-existent {} — \
|
||||
consumed pending suppression (likely stale rename event)",
|
||||
path.display()
|
||||
self.display_path(&path)
|
||||
);
|
||||
Vec::new()
|
||||
} else {
|
||||
|
|
@ -523,8 +555,8 @@ impl JobThreadContext {
|
|||
log::info!(
|
||||
"VFS: Create/Write for vanished file {} — \
|
||||
resolved via parent to {}",
|
||||
path.display(),
|
||||
resolved.display()
|
||||
self.display_path(&path),
|
||||
self.display_path(&resolved)
|
||||
);
|
||||
self.apply_patches(resolved)
|
||||
}
|
||||
|
|
@ -532,7 +564,7 @@ impl JobThreadContext {
|
|||
log::info!(
|
||||
"VFS: Skipping Create/Write for {} — \
|
||||
parent no longer exists: {}",
|
||||
path.display(),
|
||||
self.display_path(&path),
|
||||
err
|
||||
);
|
||||
Vec::new()
|
||||
|
|
@ -552,13 +584,13 @@ impl JobThreadContext {
|
|||
match self.vfs.canonicalize(parent) {
|
||||
Ok(parent_normalized) => {
|
||||
let resolved = parent_normalized.join(file_name);
|
||||
log::info!("VFS: Remove resolved to {}", resolved.display());
|
||||
log::info!("VFS: Remove resolved to {}", self.display_path(&resolved));
|
||||
self.apply_patches(resolved)
|
||||
}
|
||||
Err(err) => {
|
||||
log::info!(
|
||||
"VFS: Skipping remove event for {} — parent no longer exists: {}",
|
||||
path.display(),
|
||||
self.display_path(&path),
|
||||
err
|
||||
);
|
||||
Vec::new()
|
||||
|
|
@ -620,7 +652,7 @@ impl JobThreadContext {
|
|||
if std::fs::metadata(&path).is_ok() {
|
||||
log::info!(
|
||||
"VFS recovery: path {} was removed but has reappeared on disk. Re-snapshotting.",
|
||||
path.display()
|
||||
self.display_path(&path)
|
||||
);
|
||||
let patches = self.apply_patches(path);
|
||||
if !patches.is_empty() {
|
||||
|
|
@ -638,7 +670,7 @@ impl JobThreadContext {
|
|||
} else {
|
||||
log::info!(
|
||||
"VFS recovery: path {} confirmed removed from disk.",
|
||||
path.display()
|
||||
self.display_path(&path)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -675,7 +707,7 @@ impl JobThreadContext {
|
|||
log::info!(
|
||||
"Two-way sync: Removing instance {:?} from tree (path: {})",
|
||||
id,
|
||||
path.display()
|
||||
self.display_path(path)
|
||||
);
|
||||
}
|
||||
InstigatingSource::ProjectNode { .. } => {
|
||||
|
|
@ -753,8 +785,8 @@ impl JobThreadContext {
|
|||
if new_dir_path != dir_path {
|
||||
log::info!(
|
||||
"Two-way sync: Renaming directory {} -> {}",
|
||||
dir_path.display(),
|
||||
new_dir_path.display()
|
||||
self.display_path(dir_path),
|
||||
self.display_path(&new_dir_path)
|
||||
);
|
||||
self.suppress_path_any(dir_path);
|
||||
self.suppress_path(&new_dir_path);
|
||||
|
|
@ -917,8 +949,8 @@ impl JobThreadContext {
|
|||
if new_path != *path {
|
||||
log::info!(
|
||||
"Two-way sync: Renaming {} -> {}",
|
||||
path.display(),
|
||||
new_path.display()
|
||||
self.display_path(path),
|
||||
self.display_path(&new_path)
|
||||
);
|
||||
self.suppress_path_any(path);
|
||||
self.suppress_path(&new_path);
|
||||
|
|
@ -1094,8 +1126,8 @@ impl JobThreadContext {
|
|||
renaming {} -> {}",
|
||||
old_class,
|
||||
new_class,
|
||||
actual_file.display(),
|
||||
new_path.display()
|
||||
self.display_path(&actual_file),
|
||||
self.display_path(&new_path)
|
||||
);
|
||||
self.suppress_path_any(&actual_file);
|
||||
self.suppress_path(&new_path);
|
||||
|
|
@ -1118,7 +1150,7 @@ impl JobThreadContext {
|
|||
log::warn!(
|
||||
"Cannot change ClassName for directory {} \
|
||||
— no init file found inside",
|
||||
path.display()
|
||||
self.display_path(path)
|
||||
);
|
||||
}
|
||||
} else if old_is_script != new_is_script {
|
||||
|
|
@ -1177,7 +1209,7 @@ impl JobThreadContext {
|
|||
if let Some(Variant::String(value)) = changed_value {
|
||||
log::info!(
|
||||
"Two-way sync: Writing Source to {}",
|
||||
write_path.display()
|
||||
self.display_path(write_path)
|
||||
);
|
||||
self.suppress_path(write_path);
|
||||
if let Err(err) = fs::write(write_path, value) {
|
||||
|
|
@ -1242,7 +1274,14 @@ struct ComputeResult {
|
|||
removed_path: Option<PathBuf>,
|
||||
}
|
||||
|
||||
fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option<ComputeResult> {
|
||||
fn compute_and_apply_changes(
|
||||
tree: &mut RojoTree,
|
||||
vfs: &Vfs,
|
||||
id: Ref,
|
||||
project_root: &Path,
|
||||
) -> Option<ComputeResult> {
|
||||
// Use rel_path(p, project_root) inline for log display.
|
||||
|
||||
let metadata = tree
|
||||
.get_metadata(id)
|
||||
.expect("metadata missing for instance present in tree");
|
||||
|
|
@ -1281,12 +1320,12 @@ fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option<
|
|||
|
||||
log::info!(
|
||||
"compute_and_apply_changes: checking path {} for instance {:?}{}",
|
||||
path.display(),
|
||||
rel_path(path, project_root),
|
||||
id,
|
||||
if is_init_file {
|
||||
format!(
|
||||
" (init file, will snapshot parent dir {})",
|
||||
snapshot_path.display()
|
||||
rel_path(snapshot_path, project_root)
|
||||
)
|
||||
} else {
|
||||
String::new()
|
||||
|
|
@ -1300,7 +1339,7 @@ fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option<
|
|||
// that path and use it as the source for our patch.
|
||||
log::info!(
|
||||
"compute_and_apply_changes: path EXISTS via VFS, re-snapshotting {}",
|
||||
snapshot_path.display()
|
||||
rel_path(snapshot_path, project_root)
|
||||
);
|
||||
|
||||
let snapshot = match snapshot_from_vfs(&metadata.context, vfs, snapshot_path) {
|
||||
|
|
@ -1329,7 +1368,7 @@ fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option<
|
|||
"compute_and_apply_changes: VFS says path removed but REAL \
|
||||
filesystem confirms it EXISTS: {}. Re-snapshotting instead \
|
||||
of removing.",
|
||||
path.display()
|
||||
rel_path(path, project_root)
|
||||
);
|
||||
|
||||
let snapshot =
|
||||
|
|
@ -1338,7 +1377,7 @@ fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option<
|
|||
Err(err) => {
|
||||
log::error!(
|
||||
"Recovery snapshot error for {}: {:?}",
|
||||
snapshot_path.display(),
|
||||
rel_path(snapshot_path, project_root),
|
||||
err
|
||||
);
|
||||
return None;
|
||||
|
|
@ -1359,8 +1398,8 @@ fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option<
|
|||
log::info!(
|
||||
"compute_and_apply_changes: init file {} deleted but parent \
|
||||
directory {} still exists. Re-snapshotting directory.",
|
||||
path.display(),
|
||||
snapshot_path.display()
|
||||
rel_path(path, project_root),
|
||||
rel_path(snapshot_path, project_root)
|
||||
);
|
||||
|
||||
let snapshot =
|
||||
|
|
@ -1369,7 +1408,7 @@ fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option<
|
|||
Err(err) => {
|
||||
log::error!(
|
||||
"Directory re-snapshot error for {}: {:?}",
|
||||
snapshot_path.display(),
|
||||
rel_path(snapshot_path, project_root),
|
||||
err
|
||||
);
|
||||
return None;
|
||||
|
|
@ -1390,7 +1429,7 @@ fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option<
|
|||
"compute_and_apply_changes: path NOT FOUND on disk, removing \
|
||||
instance {:?} for {}. Scheduling recovery check.",
|
||||
id,
|
||||
path.display()
|
||||
rel_path(path, project_root)
|
||||
);
|
||||
|
||||
let removed_path = path.to_path_buf();
|
||||
|
|
@ -1407,7 +1446,7 @@ fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option<
|
|||
Err(err) => {
|
||||
log::error!(
|
||||
"Error processing filesystem change for {}: {:?}",
|
||||
path.display(),
|
||||
rel_path(path, project_root),
|
||||
err
|
||||
);
|
||||
None
|
||||
|
|
@ -1427,7 +1466,7 @@ fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option<
|
|||
log::info!(
|
||||
"compute_and_apply_changes: re-snapshotting project node '{}' at {}",
|
||||
name,
|
||||
path.display()
|
||||
rel_path(path, project_root)
|
||||
);
|
||||
|
||||
let snapshot_result = snapshot_project_node(
|
||||
|
|
|
|||
|
|
@ -23,6 +23,9 @@ use crate::{
|
|||
snapshot_middleware::snapshot_from_vfs,
|
||||
};
|
||||
|
||||
/// Set to `true` to validate on plugin connect (useful for testing, do not enable on production).
|
||||
const VALIDATE_TREE_ON_CONNECT: bool = false;
|
||||
|
||||
/// Contains all of the state for a Rojo serve session. A serve session is used
|
||||
/// when we need to build a Rojo tree and possibly rebuild it when input files
|
||||
/// change.
|
||||
|
|
@ -142,6 +145,7 @@ impl ServeSession {
|
|||
Arc::clone(&message_queue),
|
||||
tree_mutation_receiver,
|
||||
Arc::clone(&suppressed_paths),
|
||||
root_project.folder_location().to_path_buf(),
|
||||
);
|
||||
|
||||
Ok(Self {
|
||||
|
|
@ -259,6 +263,67 @@ impl ServeSession {
|
|||
.map(|rules| rules.ignore_hidden_services())
|
||||
.unwrap_or(true)
|
||||
}
|
||||
|
||||
/// Re-snapshots the project tree from the real filesystem and patches
|
||||
/// the in-memory tree to correct any drift caused by missed VFS watcher
|
||||
/// events. Called on plugin connect to guarantee the tree is fresh.
|
||||
///
|
||||
/// Skipped on freshly-started sessions (< 5 s) where the tree is
|
||||
/// guaranteed correct and the VFS watcher should handle all changes.
|
||||
///
|
||||
/// Controlled by [`VALIDATE_TREE_ON_CONNECT`]; set that constant to
|
||||
/// `false` to disable this entirely during testing.
|
||||
pub fn validate_tree(&self) -> Vec<AppliedPatchSet> {
|
||||
if !VALIDATE_TREE_ON_CONNECT {
|
||||
log::debug!("Tree validation skipped (VALIDATE_TREE_ON_CONNECT = false)");
|
||||
return Vec::new();
|
||||
}
|
||||
|
||||
// On a freshly-started session the tree was just built from the
|
||||
// filesystem and cannot be stale. Skip validation to avoid racing
|
||||
// with the VFS watcher on early file changes.
|
||||
const MIN_SESSION_AGE: std::time::Duration = std::time::Duration::from_secs(5);
|
||||
if self.start_time.elapsed() < MIN_SESSION_AGE {
|
||||
log::debug!(
|
||||
"Tree validation skipped (session age {:.1?} < {:.1?})",
|
||||
self.start_time.elapsed(),
|
||||
MIN_SESSION_AGE
|
||||
);
|
||||
return Vec::new();
|
||||
}
|
||||
|
||||
let start = Instant::now();
|
||||
let start_path = self.root_project.folder_location();
|
||||
let instance_context = InstanceContext::new();
|
||||
|
||||
let snapshot = match snapshot_from_vfs(&instance_context, &self.vfs, start_path) {
|
||||
Ok(s) => s,
|
||||
Err(e) => {
|
||||
log::error!("Tree validation snapshot error: {:?}", e);
|
||||
return Vec::new();
|
||||
}
|
||||
};
|
||||
|
||||
let mut tree = self.tree.lock().unwrap();
|
||||
let root_id = tree.get_root_id();
|
||||
let patch_set = compute_patch_set(snapshot, &tree, root_id);
|
||||
|
||||
if patch_set.removed_instances.is_empty()
|
||||
&& patch_set.added_instances.is_empty()
|
||||
&& patch_set.updated_instances.is_empty()
|
||||
{
|
||||
log::info!(
|
||||
"Tree validation complete (no corrections needed) in {:.1?}",
|
||||
start.elapsed()
|
||||
);
|
||||
return Vec::new();
|
||||
}
|
||||
|
||||
log::info!("Tree validation found stale state, applying corrections");
|
||||
let applied = apply_patch_set(&mut tree, patch_set);
|
||||
log::info!("Tree validation complete in {:.1?}", start.elapsed());
|
||||
vec![applied]
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Error)]
|
||||
|
|
|
|||
|
|
@ -3063,6 +3063,14 @@ async fn handle_websocket_subscription(
|
|||
session_id
|
||||
);
|
||||
|
||||
// Validate the in-memory tree against the real filesystem on every
|
||||
// plugin connect. This corrects any drift caused by missed VFS watcher
|
||||
// events so the plugin always sees the true state.
|
||||
let corrections = serve_session.validate_tree();
|
||||
if !corrections.is_empty() {
|
||||
message_queue.push_messages(&corrections);
|
||||
}
|
||||
|
||||
// Now continuously listen for new messages using select to handle both incoming messages
|
||||
// and WebSocket control messages concurrently
|
||||
let mut cursor = input_cursor;
|
||||
|
|
|
|||
|
|
@ -1 +1 @@
|
|||
Subproject commit a05977d6b8107c1ddc5dca7d4d26819ddd6582ba
|
||||
Subproject commit f18e6d4c523994e0eda7bf18f4947a724717e59d
|
||||
Loading…
Add table
Add a link
Reference in a new issue