mirror of
https://github.com/rojo-rbx/rojo.git
synced 2026-10-23 05:51:39 +00:00
Implement auto-download feature for syncback command
- Enhanced the `syncback` command to automatically download Roblox place files from the asset delivery API, allowing for zero-argument usage. - Updated `Cargo.toml` to include compression features for `reqwest` and moved `tempfile` to dependencies. - Introduced a new `download` argument in `SyncbackCommand` to specify place IDs for downloading. - Implemented logic to resolve input file paths, prioritizing downloads when necessary. - Added sourcemap generation after successful syncback and refreshed the git index if in a repository. - Updated tests to validate the new download functionality and sourcemap creation.
This commit is contained in:
Joe
parent
8fe908740f
commit
2b2ada219e
32 changed files with 1180 additions and 125 deletions
276
.cursor/plans/syncback_auto-downloader_0508b4bb.plan.md
Normal file
276
.cursor/plans/syncback_auto-downloader_0508b4bb.plan.md
Normal file
|
|
@ -0,0 +1,276 @@
|
|||
---
|
||||
name: Syncback Auto-Downloader
|
||||
overview: Add an auto-download feature to the syncback command that fetches the .rbxl file from Roblox's asset delivery API, with sensible defaults to make `rojo syncback` work with zero arguments.
|
||||
todos:
|
||||
- id: update-cargo
|
||||
content: Add gzip/brotli/deflate features to reqwest, move tempfile to dependencies
|
||||
status: completed
|
||||
- id: download-function
|
||||
content: Implement download_place() function with no-cache headers and cookie auth
|
||||
status: completed
|
||||
- id: update-args
|
||||
content: "Update SyncbackCommand args: --download, new defaults, --interactive flip"
|
||||
status: completed
|
||||
- id: input-resolution
|
||||
content: Implement input file resolution logic (download vs existing file)
|
||||
status: completed
|
||||
- id: sourcemap-gen
|
||||
content: Add automatic sourcemap.json generation after syncback
|
||||
status: completed
|
||||
- id: git-refresh
|
||||
content: Add git update-index --refresh for git repos
|
||||
status: completed
|
||||
- id: testing
|
||||
content: Test the complete flow with various argument combinations
|
||||
status: completed
|
||||
isProject: false
|
||||
---
|
||||
|
||||
# Syncback Auto-Downloader Feature
|
||||
|
||||
## Overview
|
||||
|
||||
Enhance the `rojo syncback` command to automatically download place files from Roblox's asset delivery API, generate sourcemaps after completion, and refresh the git index. The goal is to make `rojo syncback` work with zero arguments.
|
||||
|
||||
## Key Files to Modify
|
||||
|
||||
- `[src/cli/syncback.rs](src/cli/syncback.rs)` - Main syncback command implementation
|
||||
- `[Cargo.toml](Cargo.toml)` - Add compression features to reqwest, move tempfile to dependencies
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### 1. Update Dependencies in `Cargo.toml`
|
||||
|
||||
```toml
|
||||
# Add compression features to reqwest
|
||||
reqwest = { version = "0.11.27", default-features = false, features = [
|
||||
"blocking",
|
||||
"json",
|
||||
"rustls-tls",
|
||||
"gzip",
|
||||
"brotli",
|
||||
"deflate",
|
||||
] }
|
||||
|
||||
# Move tempfile from dev-dependencies to dependencies
|
||||
tempfile = "3.24.0"
|
||||
```
|
||||
|
||||
### 2. Add Download Arguments to `SyncbackCommand`
|
||||
|
||||
In `src/cli/syncback.rs`, modify the struct:
|
||||
|
||||
```rust
|
||||
pub struct SyncbackCommand {
|
||||
/// Path to the project to sync back to.
|
||||
#[clap(default_value = "default.project.json5")]
|
||||
pub project: PathBuf,
|
||||
|
||||
/// Path to the Roblox file to pull Instances from.
|
||||
#[clap(long, short, default_value = "Project.rbxl")]
|
||||
pub input: PathBuf,
|
||||
|
||||
/// Download the place file from Roblox. If no place ID is provided,
|
||||
/// uses the first ID from servePlaceIds in the project file.
|
||||
#[clap(long, short = 'd')]
|
||||
pub download: Option<Option<u64>>,
|
||||
|
||||
/// Interactive mode - prompts before writing. Disabled by default.
|
||||
#[clap(long, short = 'i')]
|
||||
pub interactive: bool,
|
||||
|
||||
// ... rest of existing fields
|
||||
}
|
||||
```
|
||||
|
||||
Note: Flip `--non-interactive` to `--interactive` (new default is non-interactive).
|
||||
|
||||
### 3. Download Function Implementation
|
||||
|
||||
Create the download logic with these characteristics:
|
||||
|
||||
- **URL**: `https://assetdelivery.roblox.com/v1/asset/?id={PLACEID}`
|
||||
- **Auth**: `ROBLOSECURITY` cookie from environment variable
|
||||
- **No-cache headers**:
|
||||
- `Cache-Control: no-cache, no-store, must-revalidate`
|
||||
- `Pragma: no-cache`
|
||||
- `Expires: 0`
|
||||
- **Compression**: Request `Accept-Encoding: gzip, deflate, br` (reqwest handles decompression)
|
||||
- **Temp file**: Use `tempfile::NamedTempFile` with `.rbxl` extension, keep it alive during processing
|
||||
|
||||
```rust
|
||||
fn download_place(place_id: u64) -> anyhow::Result<tempfile::NamedTempFile> {
|
||||
let cookie = std::env::var("ROBLOSECURITY")
|
||||
.context("ROBLOSECURITY environment variable not set")?;
|
||||
|
||||
let url = format!("https://assetdelivery.roblox.com/v1/asset/?id={}", place_id);
|
||||
|
||||
let client = reqwest::blocking::Client::builder()
|
||||
.gzip(true)
|
||||
.brotli(true)
|
||||
.deflate(true)
|
||||
.no_proxy() // Avoid proxy caching
|
||||
.build()?;
|
||||
|
||||
let response = client
|
||||
.get(&url)
|
||||
.header(COOKIE, format!(".ROBLOSECURITY={}", cookie))
|
||||
.header(CACHE_CONTROL, "no-cache, no-store, must-revalidate")
|
||||
.header(PRAGMA, "no-cache")
|
||||
.header("Expires", "0")
|
||||
.header(USER_AGENT, "Rojo")
|
||||
.send()?;
|
||||
|
||||
// Create temp file with .rbxl extension
|
||||
let mut temp_file = tempfile::Builder::new()
|
||||
.suffix(".rbxl")
|
||||
.tempfile()?;
|
||||
|
||||
// Stream response to file
|
||||
let bytes = response.bytes()?;
|
||||
std::io::copy(&mut bytes.as_ref(), &mut temp_file)?;
|
||||
|
||||
Ok(temp_file)
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Resolve Place ID Logic
|
||||
|
||||
When `--download` is provided without a value:
|
||||
|
||||
1. Load the project file first
|
||||
2. Extract `serve_place_ids` from the project
|
||||
3. Take the first place ID from the set (convert HashSet to Vec, sort for determinism, take first)
|
||||
4. Error if `servePlaceIds` is empty or not set
|
||||
|
||||
### 5. Input File Resolution Logic
|
||||
|
||||
In the `run()` method:
|
||||
|
||||
```rust
|
||||
// Determine the actual input path
|
||||
let input_path = if let Some(download_opt) = &self.download {
|
||||
// --download was specified
|
||||
if !self.input_exists() || download_opt.is_some() {
|
||||
// Download needed: either file doesn't exist or explicit place ID
|
||||
let place_id = match download_opt {
|
||||
Some(id) => *id,
|
||||
None => {
|
||||
// Load project to get servePlaceIds
|
||||
let project = Project::load_fuzzy(&path_old)?;
|
||||
project.serve_place_ids
|
||||
.as_ref()
|
||||
.and_then(|ids| ids.iter().min().copied())
|
||||
.context("No servePlaceIds in project and no place ID provided")?
|
||||
}
|
||||
};
|
||||
|
||||
eprintln!("Downloading place {}...", place_id);
|
||||
let temp_file = download_place(place_id)?;
|
||||
// Keep temp_file alive during processing
|
||||
DownloadedInput::Temp(temp_file)
|
||||
} else {
|
||||
DownloadedInput::Existing(resolve_path(&self.input).into_owned())
|
||||
}
|
||||
} else if self.input.exists() {
|
||||
DownloadedInput::Existing(resolve_path(&self.input).into_owned())
|
||||
} else {
|
||||
anyhow::bail!("Input file {} not found. Use --download to fetch from Roblox.", self.input.display())
|
||||
};
|
||||
```
|
||||
|
||||
### 6. Sourcemap Generation After Syncback
|
||||
|
||||
After successful syncback, generate `sourcemap.json` in the project root:
|
||||
|
||||
```rust
|
||||
// After snapshot.write_to_vfs()
|
||||
let sourcemap_path = session_old.root_project().folder_location().join("sourcemap.json");
|
||||
write_sourcemap(&session_old, Some(&sourcemap_path), filter_nothing, false)?;
|
||||
eprintln!("Generated sourcemap at {}", sourcemap_path.display());
|
||||
```
|
||||
|
||||
This requires extracting or sharing the `write_sourcemap` function from `sourcemap.rs` (likely needs to be made `pub(crate)`).
|
||||
|
||||
### 7. Git Index Refresh
|
||||
|
||||
After syncback completes, check if in a git repo and refresh the index:
|
||||
|
||||
```rust
|
||||
fn refresh_git_index(project_dir: &Path) {
|
||||
// Check if .git exists in project dir or any parent
|
||||
let mut check_dir = Some(project_dir);
|
||||
let mut is_git_repo = false;
|
||||
while let Some(dir) = check_dir {
|
||||
if dir.join(".git").exists() {
|
||||
is_git_repo = true;
|
||||
break;
|
||||
}
|
||||
check_dir = dir.parent();
|
||||
}
|
||||
|
||||
if is_git_repo {
|
||||
let result = std::process::Command::new("git")
|
||||
.arg("update-index")
|
||||
.arg("--refresh")
|
||||
.current_dir(project_dir)
|
||||
.stdout(std::process::Stdio::null())
|
||||
.stderr(std::process::Stdio::null())
|
||||
.status();
|
||||
|
||||
if let Err(e) = result {
|
||||
log::warn!("Failed to refresh git index: {}", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 8. Summary of Default Changes
|
||||
|
||||
|
||||
| Argument | Old Default | New Default |
|
||||
| ------------------- | ------------------ | ----------------------------------------------------------------- |
|
||||
| `project` | `""` (current dir) | `default.project.json5` |
|
||||
| `--input` | Required | `Project.rbxl` |
|
||||
| `--non-interactive` | `false` | Removed, replaced with `--interactive` (default: non-interactive) |
|
||||
|
||||
|
||||
## Flow Diagram
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start[rojo syncback] --> CheckDownload{--download flag?}
|
||||
CheckDownload -->|No| CheckInput{Input file exists?}
|
||||
CheckDownload -->|Yes| CheckPlaceId{Place ID provided?}
|
||||
|
||||
CheckPlaceId -->|Yes| Download[Download from Roblox]
|
||||
CheckPlaceId -->|No| LoadProject[Load project.json5]
|
||||
LoadProject --> GetPlaceId[Get first servePlaceIds]
|
||||
GetPlaceId --> Download
|
||||
|
||||
Download --> TempFile[Save to temp file]
|
||||
TempFile --> ProcessDom[Process DOM]
|
||||
|
||||
CheckInput -->|Yes| UseExisting[Use existing file]
|
||||
CheckInput -->|No| Error[Error: file not found]
|
||||
|
||||
UseExisting --> ProcessDom
|
||||
ProcessDom --> Syncback[Run syncback]
|
||||
Syncback --> WriteFiles[Write to filesystem]
|
||||
WriteFiles --> GenSourcemap[Generate sourcemap.json]
|
||||
GenSourcemap --> GitRefresh[git update-index --refresh]
|
||||
GitRefresh --> Cleanup[Cleanup temp file]
|
||||
Cleanup --> End[Done]
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Edge Cases
|
||||
|
||||
- If `--input` file exists and `--download` not specified: use existing file (no download)
|
||||
- If `--input` file exists but `--download <placeid>` explicitly given: download anyway (override)
|
||||
- If `--download` with no arg and no `servePlaceIds` in project: error with helpful message
|
||||
- If `ROBLOSECURITY` not in environment: error with helpful message
|
||||
- If download fails: error with status code and helpful message
|
||||
- If not a git repo: silently skip git refresh
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue