mirror of
https://github.com/rojo-rbx/rojo.git
synced 2026-10-23 05:51:39 +00:00
Add server-side and test-side port retry mechanisms to handle race conditions
- Introduced a new plan to address port-in-use race conditions in both server and test environments. - Implemented a retry mechanism with exponential backoff for binding to known ports in `LiveServer::start()`. - Added a `respawn_with_new_port()` method in `TestServeSession` to handle transient port conflicts during tests. - Modified `wait_to_come_online()` to retry with a new port if the server process exits due to `AddrInUse`. - Updated `fresh_rebuild_read()` to include port retry logic, enhancing robustness against port conflicts. These changes aim to improve the reliability of server and test interactions by mitigating race conditions related to port usage.
This commit is contained in:
parent
bf56dc4733
commit
d9fa62c88e
6 changed files with 544 additions and 89 deletions
104
.cursor/plans/fix_test_port_toctou_86726cb4.plan.md
Normal file
104
.cursor/plans/fix_test_port_toctou_86726cb4.plan.md
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
---
|
||||
name: Fix test port TOCTOU
|
||||
overview: Two-layer fix for port-in-use race conditions. (1) Server-side -- retry bind with backoff when a known port is busy. (2) Test-side -- retry with a new random port when the serve process dies from AddrInUse.
|
||||
todos:
|
||||
- id: server-bind-retry
|
||||
content: Add bind retry with backoff in `LiveServer::start()` for known ports
|
||||
status: completed
|
||||
- id: respawn-method
|
||||
content: Add `respawn_with_new_port()` method to `TestServeSession`
|
||||
status: completed
|
||||
- id: wait-retry
|
||||
content: Modify `wait_to_come_online()` to detect AddrInUse and retry via respawn
|
||||
status: completed
|
||||
- id: fresh-rebuild-retry
|
||||
content: Add port-retry logic to `fresh_rebuild_read()`
|
||||
status: completed
|
||||
isProject: false
|
||||
---
|
||||
|
||||
# Fix Port-in-Use Race Conditions
|
||||
|
||||
## Problem
|
||||
|
||||
Two failure modes:
|
||||
|
||||
1. **Tests (random ports):** `get_port_number()` in `[tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs)` binds port 0, reads the assigned port, drops the listener, then passes the port to the spawned serve process. Another test can grab the same port in that TOCTOU window.
|
||||
2. **Server (known ports):** `TcpListener::bind(address).await.unwrap()` in `[src/web/mod.rs:80](src/web/mod.rs)` panics immediately if the port is in use, with no retry. A previous server instance shutting down or a transient conflict gives no grace period.
|
||||
|
||||
## Layer 1: Server-side bind retry (`src/web/mod.rs`)
|
||||
|
||||
Replace the `.unwrap()` on `TcpListener::bind()` at line 80 with a retry loop:
|
||||
|
||||
- On `AddrInUse`: retry up to 5 times with exponential backoff (200ms, 400ms, 800ms, 1600ms, 3200ms -- ~6s total)
|
||||
- On any other bind error, or after retries exhausted: return `anyhow::Error` (propagates up cleanly instead of panicking)
|
||||
|
||||
```rust
|
||||
let listener = {
|
||||
const MAX_BIND_RETRIES: u32 = 5;
|
||||
const BASE_BACKOFF_MS: u64 = 200;
|
||||
let mut last_err = None;
|
||||
for attempt in 0..MAX_BIND_RETRIES {
|
||||
match TcpListener::bind(address).await {
|
||||
Ok(listener) => { last_err = None; break; }
|
||||
Err(err) if err.kind() == std::io::ErrorKind::AddrInUse => {
|
||||
let delay = BASE_BACKOFF_MS * 2u64.pow(attempt);
|
||||
log::warn!(
|
||||
"Port {} in use, retrying in {}ms (attempt {}/{})",
|
||||
address.port(), delay, attempt + 1, MAX_BIND_RETRIES
|
||||
);
|
||||
tokio::time::sleep(Duration::from_millis(delay)).await;
|
||||
last_err = Some(err);
|
||||
}
|
||||
Err(err) => { last_err = Some(err); break; }
|
||||
}
|
||||
}
|
||||
// unwrap or propagate
|
||||
};
|
||||
```
|
||||
|
||||
This helps both tests (brief TOCTOU window) and production (previous process releasing a port).
|
||||
|
||||
## Layer 2: Test-side retry with new random port (`tests/rojo_test/serve_util.rs`)
|
||||
|
||||
If the serve process still dies after the server-side retries (e.g., the port is held long-term by another test), the test infrastructure retries with a completely new port.
|
||||
|
||||
### 2a. Add `respawn_with_new_port()` to `TestServeSession`
|
||||
|
||||
Kills the current process, picks a new ephemeral port, and respawns:
|
||||
|
||||
```rust
|
||||
fn respawn_with_new_port(&mut self) {
|
||||
let port = get_port_number();
|
||||
let port_string = port.to_string();
|
||||
let working_dir = get_working_dir_path();
|
||||
|
||||
let rojo_process = Command::new(ROJO_PATH)
|
||||
.args(["serve", self.project_path.to_str().unwrap(), "--port", &port_string])
|
||||
.current_dir(working_dir)
|
||||
.stderr(Stdio::piped())
|
||||
.spawn()
|
||||
.expect("Couldn't start Rojo");
|
||||
|
||||
self.rojo_process = KillOnDrop(rojo_process);
|
||||
self.port = port;
|
||||
}
|
||||
```
|
||||
|
||||
### 2b. Modify `wait_to_come_online()`
|
||||
|
||||
In the `Ok(Some(status))` branch (line ~266-276), detect `AddrInUse` in stderr and retry:
|
||||
|
||||
- Add a `port_retries` counter (max 3 attempts)
|
||||
- On `AddrInUse`: call `self.respawn_with_new_port()`, reset the backoff counter, continue
|
||||
- On other errors: panic as before
|
||||
|
||||
### 2c. Apply same pattern to `fresh_rebuild_read()`
|
||||
|
||||
`fresh_rebuild_read()` (line ~929) has its own inline wait loop. Wrap the spawn + wait in a retry loop that detects `AddrInUse` and retries with a new port.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `[src/web/mod.rs](src/web/mod.rs)` -- bind retry with backoff (Layer 1)
|
||||
- `[tests/rojo_test/serve_util.rs](tests/rojo_test/serve_util.rs)` -- respawn with new port (Layer 2)
|
||||
|
||||
|
|
@ -7,6 +7,7 @@ local Roact = require(Packages.Roact)
|
|||
local Timer = require(Plugin.Timer)
|
||||
local Assets = require(Plugin.Assets)
|
||||
local Theme = require(Plugin.App.Theme)
|
||||
local trueEquals = require(Plugin.Reconciler.trueEquals)
|
||||
|
||||
local ScrollingFrame = require(Plugin.App.Components.ScrollingFrame)
|
||||
local DisplayValue = require(Plugin.App.Components.PatchVisualizer.DisplayValue)
|
||||
|
|
@ -35,8 +36,7 @@ function Dictionary:calculateDiff()
|
|||
key = key,
|
||||
patchType = "Remove",
|
||||
})
|
||||
elseif newValue ~= oldValue then
|
||||
-- Note: should this do some sort of deep comparison for various types?
|
||||
elseif not trueEquals(oldValue, newValue) then
|
||||
table.insert(diff, {
|
||||
key = key,
|
||||
patchType = "Edit",
|
||||
|
|
|
|||
|
|
@ -157,8 +157,11 @@ local function trueEquals(a: any, b: any): boolean
|
|||
end
|
||||
return true
|
||||
end
|
||||
if t == "Font" then
|
||||
return a.Family == b.Family and a.Weight == b.Weight and a.Style == b.Style
|
||||
end
|
||||
|
||||
return false
|
||||
return a == b
|
||||
end
|
||||
|
||||
return trueEquals
|
||||
|
|
|
|||
|
|
@ -593,6 +593,265 @@ return function()
|
|||
end)
|
||||
end)
|
||||
|
||||
describe("Font", function()
|
||||
it("identical fonts", function()
|
||||
local a =
|
||||
Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal)
|
||||
local b =
|
||||
Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal)
|
||||
expect(trueEquals(a, b)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("different family", function()
|
||||
local a =
|
||||
Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal)
|
||||
local b =
|
||||
Font.new("rbxasset://fonts/families/RobotoMono.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal)
|
||||
expect(trueEquals(a, b)).to.equal(false)
|
||||
end)
|
||||
|
||||
it("different weight", function()
|
||||
local a =
|
||||
Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal)
|
||||
local b =
|
||||
Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Bold, Enum.FontStyle.Normal)
|
||||
expect(trueEquals(a, b)).to.equal(false)
|
||||
end)
|
||||
|
||||
it("different style", function()
|
||||
local a =
|
||||
Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal)
|
||||
local b =
|
||||
Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Italic)
|
||||
expect(trueEquals(a, b)).to.equal(false)
|
||||
end)
|
||||
|
||||
it("same reference via rawequal", function()
|
||||
local a = Font.new("rbxasset://fonts/families/SourceSansPro.json")
|
||||
expect(trueEquals(a, a)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("semibold vs regular", function()
|
||||
local a = Font.new(
|
||||
"rbxasset://fonts/families/SourceSansPro.json",
|
||||
Enum.FontWeight.SemiBold,
|
||||
Enum.FontStyle.Normal
|
||||
)
|
||||
local b =
|
||||
Font.new("rbxasset://fonts/families/SourceSansPro.json", Enum.FontWeight.Regular, Enum.FontStyle.Normal)
|
||||
expect(trueEquals(a, b)).to.equal(false)
|
||||
end)
|
||||
end)
|
||||
|
||||
describe("BrickColor", function()
|
||||
it("same BrickColor", function()
|
||||
expect(trueEquals(BrickColor.new("Bright red"), BrickColor.new("Bright red"))).to.equal(true)
|
||||
end)
|
||||
|
||||
it("different BrickColor", function()
|
||||
expect(trueEquals(BrickColor.new("Bright red"), BrickColor.new("Bright blue"))).to.equal(false)
|
||||
end)
|
||||
end)
|
||||
|
||||
describe("fallback == equality", function()
|
||||
it("true vs true", function()
|
||||
expect(trueEquals(true, true)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("false vs false", function()
|
||||
expect(trueEquals(false, false)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("true vs false", function()
|
||||
expect(trueEquals(true, false)).to.equal(false)
|
||||
end)
|
||||
|
||||
it("false vs true", function()
|
||||
expect(trueEquals(false, true)).to.equal(false)
|
||||
end)
|
||||
|
||||
it("same string content", function()
|
||||
expect(trueEquals("hello", "hello")).to.equal(true)
|
||||
end)
|
||||
|
||||
it("different string content", function()
|
||||
expect(trueEquals("hello", "world")).to.equal(false)
|
||||
end)
|
||||
|
||||
it("empty strings", function()
|
||||
expect(trueEquals("", "")).to.equal(true)
|
||||
end)
|
||||
|
||||
it("same Axes", function()
|
||||
expect(trueEquals(Axes.new(Enum.Axis.X, Enum.Axis.Y), Axes.new(Enum.Axis.X, Enum.Axis.Y))).to.equal(true)
|
||||
end)
|
||||
|
||||
it("different Axes", function()
|
||||
expect(trueEquals(Axes.new(Enum.Axis.X), Axes.new(Enum.Axis.Y))).to.equal(false)
|
||||
end)
|
||||
|
||||
it("same Faces", function()
|
||||
expect(trueEquals(Faces.new(Enum.NormalId.Top), Faces.new(Enum.NormalId.Top))).to.equal(true)
|
||||
end)
|
||||
|
||||
it("different Faces", function()
|
||||
expect(trueEquals(Faces.new(Enum.NormalId.Top), Faces.new(Enum.NormalId.Bottom))).to.equal(false)
|
||||
end)
|
||||
|
||||
it("same Region3int16", function()
|
||||
local a = Region3int16.new(Vector3int16.new(0, 0, 0), Vector3int16.new(10, 10, 10))
|
||||
local b = Region3int16.new(Vector3int16.new(0, 0, 0), Vector3int16.new(10, 10, 10))
|
||||
expect(trueEquals(a, b)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("different Region3int16", function()
|
||||
local a = Region3int16.new(Vector3int16.new(0, 0, 0), Vector3int16.new(10, 10, 10))
|
||||
local b = Region3int16.new(Vector3int16.new(0, 0, 0), Vector3int16.new(20, 20, 20))
|
||||
expect(trueEquals(a, b)).to.equal(false)
|
||||
end)
|
||||
|
||||
it("same Vector3int16", function()
|
||||
expect(trueEquals(Vector3int16.new(1, 2, 3), Vector3int16.new(1, 2, 3))).to.equal(true)
|
||||
end)
|
||||
|
||||
it("different Vector3int16", function()
|
||||
expect(trueEquals(Vector3int16.new(1, 2, 3), Vector3int16.new(4, 5, 6))).to.equal(false)
|
||||
end)
|
||||
|
||||
it("same Vector2int16", function()
|
||||
expect(trueEquals(Vector2int16.new(1, 2), Vector2int16.new(1, 2))).to.equal(true)
|
||||
end)
|
||||
|
||||
it("different Vector2int16", function()
|
||||
expect(trueEquals(Vector2int16.new(1, 2), Vector2int16.new(3, 4))).to.equal(false)
|
||||
end)
|
||||
|
||||
it("same PhysicalProperties", function()
|
||||
local a = PhysicalProperties.new(1, 0.5, 0.3)
|
||||
local b = PhysicalProperties.new(1, 0.5, 0.3)
|
||||
expect(trueEquals(a, b)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("different PhysicalProperties", function()
|
||||
local a = PhysicalProperties.new(1, 0.5, 0.3)
|
||||
local b = PhysicalProperties.new(2, 0.5, 0.3)
|
||||
expect(trueEquals(a, b)).to.equal(false)
|
||||
end)
|
||||
|
||||
it("different Instances are not equal", function()
|
||||
local a = Instance.new("Folder")
|
||||
local b = Instance.new("Folder")
|
||||
expect(trueEquals(a, b)).to.equal(false)
|
||||
a:Destroy()
|
||||
b:Destroy()
|
||||
end)
|
||||
|
||||
it("same EnumItem via fallback", function()
|
||||
expect(trueEquals(Enum.SortOrder.LayoutOrder, Enum.SortOrder.LayoutOrder)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("different EnumItems of same enum", function()
|
||||
expect(trueEquals(Enum.SortOrder.LayoutOrder, Enum.SortOrder.Name)).to.equal(false)
|
||||
end)
|
||||
end)
|
||||
|
||||
describe("attribute dictionaries", function()
|
||||
it("identical Color3 attributes", function()
|
||||
local a = { BackgroundColor = Color3.fromRGB(32, 34, 39), TextSize = 14 }
|
||||
local b = { BackgroundColor = Color3.fromRGB(32, 34, 39), TextSize = 14 }
|
||||
expect(trueEquals(a, b)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("Color3 cleanup_f32 precision", function()
|
||||
local a = Color3.fromRGB(32, 34, 39)
|
||||
local b = Color3.new(0.125490, 0.133333, 0.152941)
|
||||
expect(trueEquals(a, b)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("dict with Color3 cleanup_f32 precision", function()
|
||||
local a = { BackgroundColor = Color3.fromRGB(32, 34, 39) }
|
||||
local b = { BackgroundColor = Color3.new(0.125490, 0.133333, 0.152941) }
|
||||
expect(trueEquals(a, b)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("dict with Font values", function()
|
||||
local font = Font.new(
|
||||
"rbxasset://fonts/families/SourceSansPro.json",
|
||||
Enum.FontWeight.SemiBold,
|
||||
Enum.FontStyle.Normal
|
||||
)
|
||||
local a = { LabelFont = font }
|
||||
local b = {
|
||||
LabelFont = Font.new(
|
||||
"rbxasset://fonts/families/SourceSansPro.json",
|
||||
Enum.FontWeight.SemiBold,
|
||||
Enum.FontStyle.Normal
|
||||
),
|
||||
}
|
||||
expect(trueEquals(a, b)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("dict with mixed types matching", function()
|
||||
local a = {
|
||||
BackgroundColor = Color3.fromRGB(128, 128, 128),
|
||||
Transparency = 0.25,
|
||||
Size = 14,
|
||||
Label = "Hello",
|
||||
LabelFont = Font.new(
|
||||
"rbxasset://fonts/families/SourceSansPro.json",
|
||||
Enum.FontWeight.Regular,
|
||||
Enum.FontStyle.Normal
|
||||
),
|
||||
Corner = UDim.new(0, 8),
|
||||
}
|
||||
local b = {
|
||||
BackgroundColor = Color3.fromRGB(128, 128, 128),
|
||||
Transparency = 0.25,
|
||||
Size = 14,
|
||||
Label = "Hello",
|
||||
LabelFont = Font.new(
|
||||
"rbxasset://fonts/families/SourceSansPro.json",
|
||||
Enum.FontWeight.Regular,
|
||||
Enum.FontStyle.Normal
|
||||
),
|
||||
Corner = UDim.new(0, 8),
|
||||
}
|
||||
expect(trueEquals(a, b)).to.equal(true)
|
||||
end)
|
||||
|
||||
it("dict with one Font mismatch fails whole table", function()
|
||||
local a = {
|
||||
BackgroundColor = Color3.fromRGB(128, 128, 128),
|
||||
LabelFont = Font.new(
|
||||
"rbxasset://fonts/families/SourceSansPro.json",
|
||||
Enum.FontWeight.Regular,
|
||||
Enum.FontStyle.Normal
|
||||
),
|
||||
}
|
||||
local b = {
|
||||
BackgroundColor = Color3.fromRGB(128, 128, 128),
|
||||
LabelFont = Font.new(
|
||||
"rbxasset://fonts/families/SourceSansPro.json",
|
||||
Enum.FontWeight.Bold,
|
||||
Enum.FontStyle.Normal
|
||||
),
|
||||
}
|
||||
expect(trueEquals(a, b)).to.equal(false)
|
||||
end)
|
||||
|
||||
it("extra key in new table", function()
|
||||
local a = { BackgroundColor = Color3.new(1, 0, 0) }
|
||||
local b = { BackgroundColor = Color3.new(1, 0, 0), NewAttr = "hello" }
|
||||
expect(trueEquals(a, b)).to.equal(false)
|
||||
end)
|
||||
|
||||
it("missing key in new table", function()
|
||||
local a = { BackgroundColor = Color3.new(1, 0, 0), OldAttr = "hello" }
|
||||
local b = { BackgroundColor = Color3.new(1, 0, 0) }
|
||||
expect(trueEquals(a, b)).to.equal(false)
|
||||
end)
|
||||
end)
|
||||
|
||||
describe("type mismatches", function()
|
||||
it("string vs number", function()
|
||||
expect(trueEquals("hello", 42)).to.equal(false)
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ mod util;
|
|||
use std::convert::Infallible;
|
||||
use std::net::SocketAddr;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::Duration;
|
||||
|
||||
use hyper::body::Incoming;
|
||||
use hyper::server::conn::http1;
|
||||
|
|
@ -77,7 +78,37 @@ impl LiveServer {
|
|||
|
||||
let rt = Runtime::new().unwrap();
|
||||
let exit_reason = rt.block_on(async move {
|
||||
let listener = TcpListener::bind(address).await.unwrap();
|
||||
let listener = {
|
||||
const MAX_BIND_ATTEMPTS: u32 = 5;
|
||||
const BASE_BACKOFF_MS: u64 = 200;
|
||||
let mut attempts = 0u32;
|
||||
loop {
|
||||
attempts += 1;
|
||||
match TcpListener::bind(address).await {
|
||||
Ok(listener) => break listener,
|
||||
Err(err)
|
||||
if err.kind() == std::io::ErrorKind::AddrInUse
|
||||
&& attempts < MAX_BIND_ATTEMPTS =>
|
||||
{
|
||||
let delay = BASE_BACKOFF_MS * 2u64.pow(attempts - 1);
|
||||
log::warn!(
|
||||
"Port {} in use, retrying in {}ms (attempt {}/{})",
|
||||
address.port(),
|
||||
delay,
|
||||
attempts,
|
||||
MAX_BIND_ATTEMPTS
|
||||
);
|
||||
tokio::time::sleep(Duration::from_millis(delay)).await;
|
||||
}
|
||||
Err(err) => {
|
||||
panic!(
|
||||
"Failed to bind to {}: {} (after {} attempts)",
|
||||
address, err, attempts
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
loop {
|
||||
tokio::select! {
|
||||
|
|
|
|||
|
|
@ -256,47 +256,85 @@ impl TestServeSession {
|
|||
self.port
|
||||
}
|
||||
|
||||
/// Waits for the `rojo serve` server to come online with expontential
|
||||
/// backoff.
|
||||
fn respawn_with_new_port(&mut self) {
|
||||
let port = get_port_number();
|
||||
let port_string = port.to_string();
|
||||
let working_dir = get_working_dir_path();
|
||||
|
||||
let rojo_process = Command::new(ROJO_PATH)
|
||||
.args([
|
||||
"serve",
|
||||
self.project_path.to_str().unwrap(),
|
||||
"--port",
|
||||
port_string.as_str(),
|
||||
])
|
||||
.current_dir(working_dir)
|
||||
.stderr(Stdio::piped())
|
||||
.spawn()
|
||||
.expect("Couldn't start Rojo");
|
||||
|
||||
self.rojo_process = KillOnDrop(rojo_process);
|
||||
self.port = port;
|
||||
}
|
||||
|
||||
/// Waits for the `rojo serve` server to come online with exponential
|
||||
/// backoff. If the process dies with `AddrInUse`, respawns with a new
|
||||
/// random port (up to `MAX_PORT_RETRIES` times).
|
||||
pub fn wait_to_come_online(&mut self) -> ServerInfoResponse {
|
||||
const BASE_DURATION_MS: f32 = 30.0;
|
||||
const EXP_BACKOFF_FACTOR: f32 = 1.3;
|
||||
const MAX_TRIES: u32 = 5;
|
||||
const MAX_PORT_RETRIES: u32 = 3;
|
||||
|
||||
for i in 1..=MAX_TRIES {
|
||||
match self.rojo_process.0.try_wait() {
|
||||
Ok(Some(status)) => {
|
||||
let mut stderr_output = String::new();
|
||||
if let Some(mut stderr) = self.rojo_process.0.stderr.take() {
|
||||
let _ = stderr.read_to_string(&mut stderr_output);
|
||||
let mut port_retries = 0u32;
|
||||
|
||||
'retry_port: loop {
|
||||
for i in 1..=MAX_TRIES {
|
||||
match self.rojo_process.0.try_wait() {
|
||||
Ok(Some(status)) => {
|
||||
let mut stderr_output = String::new();
|
||||
if let Some(mut stderr) = self.rojo_process.0.stderr.take() {
|
||||
let _ = stderr.read_to_string(&mut stderr_output);
|
||||
}
|
||||
if stderr_output.contains("AddrInUse") && port_retries < MAX_PORT_RETRIES {
|
||||
port_retries += 1;
|
||||
log::warn!(
|
||||
"Port {} in use, respawning with new port (attempt {}/{})",
|
||||
self.port,
|
||||
port_retries,
|
||||
MAX_PORT_RETRIES
|
||||
);
|
||||
self.respawn_with_new_port();
|
||||
continue 'retry_port;
|
||||
}
|
||||
panic!(
|
||||
"Rojo process exited with status {}\nstderr:\n{}",
|
||||
status, stderr_output
|
||||
);
|
||||
}
|
||||
panic!(
|
||||
"Rojo process exited with status {}\nstderr:\n{}",
|
||||
status, stderr_output
|
||||
);
|
||||
Ok(None) => { /* The process is still running, as expected */ }
|
||||
Err(err) => panic!("Failed to wait on Rojo process: {}", err),
|
||||
}
|
||||
Ok(None) => { /* The process is still running, as expected */ }
|
||||
Err(err) => panic!("Failed to wait on Rojo process: {}", err),
|
||||
|
||||
let info = match self.get_api_rojo() {
|
||||
Ok(info) => info,
|
||||
Err(err) => {
|
||||
let retry_time_ms = BASE_DURATION_MS * (i as f32).powf(EXP_BACKOFF_FACTOR);
|
||||
let retry_time = Duration::from_millis(retry_time_ms as u64);
|
||||
|
||||
log::info!("Server error, retrying in {:?}: {}", retry_time, err);
|
||||
thread::sleep(retry_time);
|
||||
continue;
|
||||
}
|
||||
};
|
||||
|
||||
log::info!("Got session info: {:?}", info);
|
||||
|
||||
return info;
|
||||
}
|
||||
|
||||
let info = match self.get_api_rojo() {
|
||||
Ok(info) => info,
|
||||
Err(err) => {
|
||||
let retry_time_ms = BASE_DURATION_MS * (i as f32).powf(EXP_BACKOFF_FACTOR);
|
||||
let retry_time = Duration::from_millis(retry_time_ms as u64);
|
||||
|
||||
log::info!("Server error, retrying in {:?}: {}", retry_time, err);
|
||||
thread::sleep(retry_time);
|
||||
continue;
|
||||
}
|
||||
};
|
||||
|
||||
log::info!("Got session info: {:?}", info);
|
||||
|
||||
return info;
|
||||
panic!("Rojo server did not respond after {} tries.", MAX_TRIES);
|
||||
}
|
||||
|
||||
panic!("Rojo server did not respond after {} tries.", MAX_TRIES);
|
||||
}
|
||||
|
||||
pub fn get_api_rojo(&self) -> Result<ServerInfoResponse, reqwest::Error> {
|
||||
|
|
@ -938,71 +976,91 @@ pub fn fresh_rebuild_read(project_path: &Path) -> NormalizedInstance {
|
|||
);
|
||||
};
|
||||
|
||||
let port = get_port_number();
|
||||
let port_string = port.to_string();
|
||||
let working_dir = get_working_dir_path();
|
||||
|
||||
let process = Command::new(ROJO_PATH)
|
||||
.args([
|
||||
"serve",
|
||||
project_file.to_str().unwrap(),
|
||||
"--port",
|
||||
port_string.as_str(),
|
||||
])
|
||||
.current_dir(working_dir)
|
||||
.stderr(Stdio::piped())
|
||||
.spawn()
|
||||
.expect("Couldn't start fresh Rojo for round-trip check");
|
||||
|
||||
let mut kill_guard = KillOnDrop(process);
|
||||
|
||||
// Wait for fresh server to come online with same backoff
|
||||
const BASE_DURATION_MS: f32 = 30.0;
|
||||
const EXP_BACKOFF_FACTOR: f32 = 1.3;
|
||||
const MAX_TRIES: u32 = 5;
|
||||
const MAX_PORT_RETRIES: u32 = 3;
|
||||
|
||||
let mut info = None;
|
||||
for i in 1..=MAX_TRIES {
|
||||
match kill_guard.0.try_wait() {
|
||||
Ok(Some(status)) => {
|
||||
let mut stderr_output = String::new();
|
||||
if let Some(mut stderr) = kill_guard.0.stderr.take() {
|
||||
let _ = stderr.read_to_string(&mut stderr_output);
|
||||
let working_dir = get_working_dir_path();
|
||||
let mut port_retries = 0u32;
|
||||
|
||||
loop {
|
||||
let port = get_port_number();
|
||||
let port_string = port.to_string();
|
||||
|
||||
let process = Command::new(ROJO_PATH)
|
||||
.args([
|
||||
"serve",
|
||||
project_file.to_str().unwrap(),
|
||||
"--port",
|
||||
port_string.as_str(),
|
||||
])
|
||||
.current_dir(&working_dir)
|
||||
.stderr(Stdio::piped())
|
||||
.spawn()
|
||||
.expect("Couldn't start fresh Rojo for round-trip check");
|
||||
|
||||
let mut kill_guard = KillOnDrop(process);
|
||||
let mut info = None;
|
||||
let mut needs_port_retry = false;
|
||||
|
||||
for i in 1..=MAX_TRIES {
|
||||
match kill_guard.0.try_wait() {
|
||||
Ok(Some(status)) => {
|
||||
let mut stderr_output = String::new();
|
||||
if let Some(mut stderr) = kill_guard.0.stderr.take() {
|
||||
let _ = stderr.read_to_string(&mut stderr_output);
|
||||
}
|
||||
if stderr_output.contains("AddrInUse") && port_retries < MAX_PORT_RETRIES {
|
||||
port_retries += 1;
|
||||
log::warn!(
|
||||
"Port {} in use, retrying with new port (attempt {}/{})",
|
||||
port,
|
||||
port_retries,
|
||||
MAX_PORT_RETRIES
|
||||
);
|
||||
needs_port_retry = true;
|
||||
break;
|
||||
}
|
||||
panic!(
|
||||
"Fresh Rojo process exited with status {}\nstderr:\n{}",
|
||||
status, stderr_output
|
||||
);
|
||||
}
|
||||
panic!(
|
||||
"Fresh Rojo process exited with status {}\nstderr:\n{}",
|
||||
status, stderr_output
|
||||
);
|
||||
Ok(None) => {}
|
||||
Err(err) => panic!("Failed to wait on fresh Rojo process: {}", err),
|
||||
}
|
||||
Ok(None) => {}
|
||||
Err(err) => panic!("Failed to wait on fresh Rojo process: {}", err),
|
||||
}
|
||||
|
||||
let url = format!("http://localhost:{}/api/rojo", port);
|
||||
if let Ok(resp) = reqwest::blocking::get(&url) {
|
||||
if let Ok(body) = resp.bytes() {
|
||||
if let Ok(server_info) = deserialize_msgpack::<ServerInfoResponse>(&body) {
|
||||
info = Some(server_info);
|
||||
break;
|
||||
let url = format!("http://localhost:{}/api/rojo", port);
|
||||
if let Ok(resp) = reqwest::blocking::get(&url) {
|
||||
if let Ok(body) = resp.bytes() {
|
||||
if let Ok(server_info) = deserialize_msgpack::<ServerInfoResponse>(&body) {
|
||||
info = Some(server_info);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
let retry_time_ms = BASE_DURATION_MS * (i as f32).powf(EXP_BACKOFF_FACTOR);
|
||||
thread::sleep(Duration::from_millis(retry_time_ms as u64));
|
||||
}
|
||||
let retry_time_ms = BASE_DURATION_MS * (i as f32).powf(EXP_BACKOFF_FACTOR);
|
||||
thread::sleep(Duration::from_millis(retry_time_ms as u64));
|
||||
|
||||
if needs_port_retry {
|
||||
continue;
|
||||
}
|
||||
|
||||
let info = info.expect("Fresh Rojo server did not respond");
|
||||
let root_id = info.root_instance_id;
|
||||
|
||||
let url = format!("http://localhost:{}/api/read/{}", port, root_id);
|
||||
let body = reqwest::blocking::get(&url)
|
||||
.expect("Failed to read from fresh server")
|
||||
.bytes()
|
||||
.expect("Failed to get bytes from fresh server");
|
||||
let read: ReadResponse =
|
||||
deserialize_msgpack(&body).expect("Fresh server returned malformed response");
|
||||
|
||||
return normalize_read_response(&read, root_id);
|
||||
}
|
||||
|
||||
let info = info.expect("Fresh Rojo server did not respond");
|
||||
let root_id = info.root_instance_id;
|
||||
|
||||
let url = format!("http://localhost:{}/api/read/{}", port, root_id);
|
||||
let body = reqwest::blocking::get(&url)
|
||||
.expect("Failed to read from fresh server")
|
||||
.bytes()
|
||||
.expect("Failed to get bytes from fresh server");
|
||||
let read: ReadResponse =
|
||||
deserialize_msgpack(&body).expect("Fresh server returned malformed response");
|
||||
|
||||
normalize_read_response(&read, root_id)
|
||||
}
|
||||
|
||||
/// Assert that the live session tree matches a fresh rebuild from the filesystem.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue