Differences

This shows you the differences between two versions of the page.

--- content:tech:rhizome_retriever [25/09/2012 19:53]
Luke Marshall created
+++ content:tech:rhizome_retriever [14/05/2013 23:18]
@@ Line 1: / Line 1: @@
-====== Rhizome for Citizen Journalism ======
-===== Mission(short) =====
-Create an application that allows to spread files over the mesh network by exchanging with servers that the handset can see (could be peers and web servers)
-===== Some design elements =====
-{{http://dl.dropbox.com/u/3505759/serval/images/design_elements.png}}
-A Rhizome logical file is constituted of three elements:
-  * The data file (ex: serval.apk): the actual data
-  * The meta file (.serval.apk.meta): contains <strong>local </strong>informations
-    * When the file appeared on our repository?
-    * Has the file been already read?<br />''''&gt; It's a personal file, never exchanged but **generated when a file gets into your repository** (ie, when imported or downloaded)
-  * The manifest file (.serval.apk.manifest)
-    * Contains static information on the file (author/version/creation date/file name/hash)
-    * It's a companion file ; when a file is exchanged, always comes with its manifest
-A file can be imported from the local FS : informations are asked to create the manifest, and the meta is automatically generated.
-A file can be downloaded from another repository (an HTTP server -- can be another batphone or a classic server). It comes with its manifests, and the meta is built when the file is actually downloaded. Here is the algorythm for choosing what to do with a proposed file.
-  * If the file exists and the version number in the new manifest is not greater, does nothin
-  * If the file no longer exist (because it has been deleted), do nothing except &hellip;
-  * If the version of the new manifest is greater, download the manifest
-  * Download the file
-  * Compute the checksum of the new file, and if it fits with the manifest, import the file in the repository.
-Every 15s, the app :
-  - Look for the peers
-  - Locate all the possible manifests
-  - Download interestings file
-In the background.
-There are three important locations on the SD card:
-  ***serval-rhizome**: the repository
-  ***serval-rhizome-export**: the export folder
-  ***serval-rhizome-temp**: the temporary folder
-===== Features =====
-==== Implemented ====
-  * List all the files in your local Rhizome repository
-  * Open (some)
-  * Allow to import from a file picker (asks for some info to build the associated manifest, and the meta is created automatically)
-  * Allow to delete a file on your Rhizome repository
-  * Allow to see manifest's informations
-  * Allow to mark the file as read (information stored in the meta file)
-  * Allow to export a received file "out" of the Rhizome system -- for instance, if you want to keep it from evoluting
-  * Download a manifest and a file from the cloud (will be another Rhizom or a standard web server)
-    * If the file exists and the version number in the new manifest is not greater, does nothing
-    * If the version of the new manifest is greater (or the phone does not have this manifest yet), downlaod the manifest
-    * Download the file
-    * Take the checksum of the new file, and if it fits with the manifest, import the file in the repository.
-  * Background looking for other peers -- WIP
-  * Background and periodic lookup for unknown -- or newer -- manifests in the neighbors and known hosts
-  * Lots of notifications -- on update, on error, on import, on the screen :) (toast)
-  * Auto refresh when a file comes from the network
-  * Creates the architecture at the launch time, remove the unnecessary files when closing
-  * Possibility of defining a static repo -- which will be tried each time, such as Serval update repository
-  * Maintain an exclusion list -- the file you've downloaded and deleted, so you dont want them to reappear 15 s after.
-  * Allow to reinit the exclusion list (in the menu)
-==== Not yet implemented ====
-  * Opening all files type &hellip; Looks complicated, need to trigger the MIME type from extension [[http://stackoverflow.com/questions/7377361/open-a-file-as-android-file-explorer-does|StackOverflow]]
-  * Message signature&hellip; I'm fighting with the keystores and algorithm. If you have an android example, i'll be glad to have a look.
-  * Peer lookup and actual manifest list fetching -- i need the phones
-==== It's-not-a-bug-it's-a-feature list ====
-  * The updates stop **after** the app is closed. This can be changed very easily if another behavior is preferred.
-===== Demonstration scenario =====
-**MAKE SURE YOUR SD CARD IS NOT MOUNTED ON A COMPUTER OR THE PROGRAM WILL CRASH WHILE STARTING!**
-Android doesn't manage to access to a filesystem which is already mounted somewhere else.
-  - Start the app
-  - Wait for the files to appear from the network
-  - Try to look their manifest
-  - Look at the content of one of the meta file (especially date)
-  - Delete one of the file
-  - The file does NOT come back, because you've deleted it.
-  - Reinit the exclusion list (in the menu)
-  - When its comes back, check the meta date; should have evolved to the new date
-  - Import a file (menu)
-  - Give informations to build the manifest
-  - Modify the .manifest to change the hash (into a wrong one)
-  - Check the Manifest view : you should see a warning saying the hash is wrong
-  - Increment the version number of one of the remote file and observe the update process (check in the manifest version number to ensure the update process went right.
-  - Delete a file, and increment the number in the remote manifest file ; the file should come back because the number is greater and you deleted the last version, not this one.
-  - Increment the version number of one of the remote manifest file and change the associated hash. The file wont update, because the hash is wrong.
-  - Export one the file to rhizhome-serval-export
-  - Quit the program, it will kill the update process and clear the cache directory.
-===== Interface =====
-In order to demonstrate how bad I am with the UI, I made this small screenshots. Romana, I know you'll hate me.
-{{http://dl.dropbox.com/u/3505759/serval/images/base_w_menu.png}}
-The base application wihen the menu key is pressed.
-{{http://dl.dropbox.com/u/3505759/serval/images/context_menu.png}}
-Contextual menu
-{{http://dl.dropbox.com/u/3505759/serval/images/deletion.png}}
-If I delete a file, here is the feedback (the file's no longer in the list)
-{{http://dl.dropbox.com/u/3505759/serval/images/import.png}}
-Import a file from a picker.
-{{http://dl.dropbox.com/u/3505759/serval/images/manifest.png}}
-Add some data to the manifest
-{{http://dl.dropbox.com/u/3505759/serval/images/manifest_view.png}}
-Later, you can check the manifest using the menu
-{{http://dl.dropbox.com/u/3505759/serval/images/update_from_network.png}}
-The file database.xml has been updated to the version 58 from the network (background update, no action from the user)
-===== Code source =====
-The code source is on GitHub : https://github.com/rbochet/Rhizome-Retriever
-You're more than welcome to fork / push if you find a bug you can fix in a second.
-===== Testing =====
-You can import easily the project in Eclipse, there are no external dependencies.
-\\
-The only _strange//thing for the moment is that if you want to test using the network updates, you'll have to have a place on the web where you can put manifests and files. My files are here:
-\\
-\\
-  * [[http://dl.dropbox.com/u/3505759/serval/.database.xml.manifest]] (and [[http://dl.dropbox.com/u/3505759/serval/database.xml]])
-\\
-  * [[http://dl.dropbox.com/u/3505759/serval/.disable-lagfix.truite.manifest]] (and [[http://dl.dropbox.com/u/3505759/serval/disable-lagfix.truite]])
-\\
-\\
-But there's more fun if you can change their version number, and play with the file hash to test the functions&hellip; So find a place on the web where you can put them and where you have writing permissions.
-\\
-If you want to try with other files, just grab files you have already imported and their manifests from the serval-rhizome directory.
-\\
-The path is to set up in the StuffDownloader class for testing purposes:
-{{https://github.com/rbochet/Rhizome-Retriever/blob/master/src/org/servalproject/rr/StuffDownloader.java#L189}}.
-\\
-----
-===== Final Comments =====
-I may have been unclear in this documentation. If you dont understand something, send a mail romain@servalproject NOSPAM org or to the developer list.

The Serval Project Wiki

User Tools

Site Tools

Differences

Page Tools