Work on instance loading a little bit

DESIGN.md

@@ -12,23 +12,32 @@ The aim is to rewrite most of Rojo to drop the notion of routes almost entirely
 * **Roblox Instance**: An actual Roblox object derived from `Instance` on the client.
 
 ## Initialization
-On server initialization, Rojo reads all files in each partition and runs each through our middleware stack (renamed from plugins). Middleware transform a `VfsItem` into  a list of `RbxInstance` objects with these properties:
+On server initialization, Rojo reads all files in each partition and runs each through our middleware stack (renamed from plugins). Middleware transform a `VfsItem` into a set of `RbxInstance` objects with these properties:
 
 * name: `String`
 * class_name: `String`
 * parent: `Option<Id>`
 * properties: `HashMap<String, RbxValue>`
-* route: `Route` (does not have to serialize)
+* file_route: `FileRoute`
 * id: `Id`
 * rbx_referent: `Option<RbxReferent>`
 
 The server should keep these book-keeping maps:
 
-* `HashMap<Id, RbxInstance`, to serve as an index
+* partitions: `HashMap<String, Partition>`, to read partitions from disk
-* `HashMap<Route, Id>`, to manage filesystem changes
+* partition_instances: `HashMap<String, Id>`, to track partition instances
-* `HashMap<Id, Route>`, to manage changes from the client
+* partition_files: `HashMap<String, VfsItem>`, to serve as an in-memory filesystem
-* `HashMap<Route, String>`, to disambiguate and debounce changes written to the filesystem
+* instances: `HashMap<Id, RbxInstance>`, to serve as an index
-* `HashMap<RbxReferent, Id>`, when loading `rbxmx` format models from disk
+* instances_by_route: `HashMap<FileRoute, Id>`, to handle filesystem changes
+
+As a future extension, the server should also track:
+
+* `referent_to_id`: `HashMap<RbxReferent, Id>`, when loading `rbxmx` format models from disk.
+
+`Partition` is defined as:
+
+* path: `PathBuf`
+* target: `Vec<String>`
 
 On initial sync, the client should receive:
 
@@ -38,14 +47,13 @@ On initial sync, the client should receive:
 ## File Changes
 Whenever a file is changed, the server should:
 
-* Convert its path to a `Route`.
+* Convert its path to a `FileRoute`.
 * Read the file into a `VfsItem` object.
-* Compare the contents of the file with the last-seen contents from the map from `Route` to `String`.
+* Compare the contents of the file with the last-seen contents from the map from `FileRoute` to `String`.
 	* If they're the same, that means this change was caused by the server itself, and should be discarded.
 	* Else, clear the value from the map.
-* Pass the `VfsItem` through the middleware chain to a tree of `RbxInstance` objects.
+* Pass the `VfsItem` through the middleware chain to create a tree of `RbxInstance` objects.
-	* TODO: How should this tree be represented? The normal `RbxInstance` case should just reference children by ID, but that's super inconvenient here.
+* Check the existing map from `FileRoute` to `Id`.
-* Check the existing map from `Route` to `Id`.
 	* If an existing instance exists with the root's `Id`, reuse it and replace the root's `Id`.
 	* Else, generate a new ID.
 * For each child instance:
@@ -69,7 +77,7 @@ Those changes should come in three flavors:
 * Deleted a known instance (need `id`)
 * Created a new instance (need `clientId`)
 
-Every sync period, which could be around 100ms, the plugin should send a request with every Roblox Instance mutation all at once.
+Every sync period, which could be around 50ms, the plugin should send a request with every Roblox Instance mutation all at once.
 
 In order to correctly serialize changes received from the client, some care needs to be taken.
 

server/src/file_route.rs

@@ -1,5 +1,7 @@
 use std::path::{Path, PathBuf};
 
+use partition::Partition;
+
 // TODO: Add lifetime, switch to using Cow<'a, str> instead of String? It's
 // possible that it would be too cumbersome!
 #[derive(Debug, Clone, PartialEq, Hash)]
@@ -34,4 +36,21 @@ impl FileRoute {
 
         result
     }
+
+    /// This function is totally wrong and should be handled by middleware, heh.
+    pub fn name(&self, partition: &Partition) -> String { // I guess??
+        if self.route.len() == 0 {
+            // This FileRoute refers to the partition itself
+
+            if partition.target.len() == 0 {
+                // We're targeting the game itself!
+                "game".to_string()
+            } else {
+                partition.target.last().unwrap().clone()
+            }
+        } else {
+            // This FileRoute refers to an item in a partition
+            self.route.last().unwrap().clone()
+        }
+    }
 }

server/src/lib.rs

@@ -13,6 +13,7 @@ extern crate tempfile;
 pub mod commands;
 pub mod file_route;
 pub mod id;
+pub mod partition;
 pub mod pathext;
 pub mod project;
 pub mod rbx_session;

server/src/partition.rs

@@ -0,0 +1,10 @@
+use std::path::PathBuf;
+
+#[derive(Debug, Clone)]
+pub struct Partition {
+    /// The path on the filesystem that this partition maps to.
+    pub path: PathBuf,
+
+    /// The route to the Roblox instance that this partition maps to.
+    pub target: Vec<String>,
+}

server/src/rbx_session.rs

@@ -3,60 +3,103 @@ use std::collections::HashMap;
 use std::io::Read;
 use std::fs::{self, File};
 
-use id::{Id};
+use id::{Id, get_id};
 use file_route::FileRoute;
+use partition::Partition;
 
 /// Represents a file or directory that has been read from the filesystem.
-// TODO: Keep track of file Path or FileRoute?
 #[derive(Debug, Clone)]
 enum FileItem {
     File {
         contents: String,
+        route: FileRoute,
     },
     Directory {
         children: HashMap<String, FileItem>,
+        route: FileRoute,
     },
 }
 
+// TODO: Switch to enum to represent more value types
+type RbxValue = String;
+
+#[derive(Debug, Clone)]
 struct RbxInstance {
+    /// Maps to the `Name` property on Instance.
+    pub name: String,
+
+    /// Maps to the `ClassName` property on Instance.
     pub class_name: String,
+
+    /// Maps to the `Parent` property on Instance.
     pub parent: Option<Id>,
-    pub properties: HashMap<String, String>,
+
+    /// Contains all other properties of an Instance.
+    pub properties: HashMap<String, RbxValue>,
 }
 
 struct RbxSession {
-    pub partition_paths: HashMap<String, PathBuf>,
+    pub partitions: HashMap<String, Partition>,
+
+    /// The RbxInstance that represents each partition.
     pub partition_instances: HashMap<String, Id>,
+
+    /// The in-memory files associated with each partition.
     pub partition_files: HashMap<String, FileItem>,
+
+    /// The store of all instances in the session.
     pub instances: HashMap<Id, RbxInstance>,
+    // pub instances_by_route: HashMap<FileRoute, Id>,
 }
 
-fn file_to_instance(file_item: &FileItem) -> RbxInstance {
+fn file_to_instance(file_item: &FileItem, partition: &Partition) -> (Id, HashMap<Id, RbxInstance>) {
     match file_item {
-        &FileItem::File { ref contents } => {
+        &FileItem::File { ref contents, ref route } => {
             let mut properties = HashMap::new();
             properties.insert("Value".to_string(), contents.clone());
 
-            RbxInstance {
+            let primary_id = get_id();
+            let mut instances = HashMap::new();
+
+            instances.insert(primary_id, RbxInstance {
+                name: route.name(partition).to_string(),
                 class_name: "StringValue".to_string(),
                 parent: None,
                 properties,
-            }
+            });
+
+            (primary_id, instances)
         },
-        &FileItem::Directory { ref children } => {
+        &FileItem::Directory { ref children, ref route } => {
-            RbxInstance {
+            let primary_id = get_id();
+            let mut instances = HashMap::new();
+
+            instances.insert(primary_id, RbxInstance {
+                name: route.name(partition).to_string(),
                 class_name: "Folder".to_string(),
                 parent: None,
                 properties: HashMap::new(),
+            });
+
+            for child_file_item in children.values() {
+                let (child_id, mut child_instances) = file_to_instance(child_file_item, partition);
+
+                child_instances.get_mut(&child_id).unwrap().parent = Some(primary_id);
+
+                for (instance_id, instance) in child_instances.drain() {
+                    instances.insert(instance_id, instance);
                 }
             }
+
+            (primary_id, instances)
+        }
     }
 }
 
 impl RbxSession {
     fn new() -> RbxSession {
         RbxSession {
-            partition_paths: HashMap::new(),
+            partitions: HashMap::new(),
             partition_instances: HashMap::new(),
             partition_files: HashMap::new(),
             instances: HashMap::new(),
@@ -64,7 +107,7 @@ impl RbxSession {
     }
 
     fn load_files(&mut self) {
-        for partition_name in self.partition_paths.keys() {
+        for partition_name in self.partitions.keys() {
             let route = FileRoute {
                 partition: partition_name.clone(),
                 route: vec![],
@@ -76,12 +119,25 @@ impl RbxSession {
         }
     }
 
-    fn read(&self, route: &FileRoute) -> Result<FileItem, ()> {
+    fn load_instances(&mut self) {
-        let partition_path = self.partition_paths.get(&route.partition)
+        for (partition_name, file_item) in &self.partition_files {
-            .ok_or(())?;
+            let partition = self.partitions.get(partition_name).unwrap();
-        let path = route.to_path_buf(partition_path);
+            let (root_id, mut instances) = file_to_instance(&file_item, partition);
 
-        println!("Read {:?}, path {}", route, path.display());
+            // there has to be an std method for this
+            // oh well
+            for (instance_id, instance) in instances.drain() {
+                self.instances.insert(instance_id, instance);
+            }
+
+            self.partition_instances.insert(partition_name.clone(), root_id);
+        }
+    }
+
+    fn read(&self, route: &FileRoute) -> Result<FileItem, ()> {
+        let partition_path = &self.partitions.get(&route.partition)
+            .ok_or(())?.path;
+        let path = route.to_path_buf(partition_path);
 
         let metadata = fs::metadata(path)
             .map_err(|_| ())?;
@@ -96,8 +152,8 @@ impl RbxSession {
     }
 
     fn read_file(&self, route: &FileRoute) -> Result<FileItem, ()> {
-        let partition_path = self.partition_paths.get(&route.partition)
+        let partition_path = &self.partitions.get(&route.partition)
-            .ok_or(())?;
+            .ok_or(())?.path;
         let path = route.to_path_buf(partition_path);
 
         let mut file = File::open(path)
@@ -110,12 +166,13 @@ impl RbxSession {
 
         Ok(FileItem::File {
             contents,
+            route: route.clone(),
         })
     }
 
     fn read_directory(&self, route: &FileRoute) -> Result<FileItem, ()> {
-        let partition_path = self.partition_paths.get(&route.partition)
+        let partition_path = &self.partitions.get(&route.partition)
-            .ok_or(())?;
+            .ok_or(())?.path;
         let path = route.to_path_buf(partition_path);
 
         let reader = fs::read_dir(path)
@@ -139,6 +196,7 @@ impl RbxSession {
 
         Ok(FileItem::Directory {
             children,
+            route: route.clone(),
         })
     }
 }
@@ -174,16 +232,22 @@ mod tests {
 
         let mut session = RbxSession::new();
 
-        session.partition_paths.insert("agh".to_string(), root_dir.path().to_path_buf());
+        let partition = Partition {
+            path: root_dir.path().to_path_buf(),
+            target: vec!["ReplicatedStorage".to_string()],
+        };
+
+        session.partitions.insert("agh".to_string(), partition);
 
         session.load_files();
 
         assert_eq!(session.partition_files.len(), 1);
 
+        {
             let folder = session.partition_files.values().nth(0).unwrap();
 
             let children = match folder {
-            &FileItem::Directory { ref children } => children,
+                &FileItem::Directory { ref children, .. } => children,
                 _ => panic!("Not a directory!"),
             };
 
@@ -191,4 +255,17 @@ mod tests {
             assert!(children.get("foo.txt").is_some());
             assert!(children.get("bar.tsv").is_some());
         }
+
+        session.load_instances();
+
+        assert_eq!(session.instances.len(), 3);
+        assert_eq!(session.partition_instances.len(), 1);
+
+        {
+            let folder_id = session.partition_instances.values().nth(0).unwrap();
+
+            let folder = session.instances.get(folder_id).unwrap();
+            assert_eq!(folder.name, "ReplicatedStorage");
+        }
+   }
 }