The Serval Project Wiki

User Tools

Site Tools

You are here: The Serval Project Wiki » content » Technology Roadmap » Rhizome
Trace:
content:tech:rhizome

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
content:tech:rhizome [06/05/2013 19:37]
Andrew Bettison content:technologies:rhizome renamed to content:tech:rhizome (Consolidate all technology descriptions under :content:tech) 		content:tech:rhizome [21/02/2016 22:52] (current)
Andrew Bettison Re-word [Technical documentation] section; remove link to GitHub (maintenance burden)
Line 1: Line 1:
 ====== Rhizome ====== ====== Rhizome ======
  
-**Rhizome** is a resilient file distribution system developed by [[:|The Serval Project]].  It forms the basis of all non-[[http://en.wikipedia.org/wiki/Connection_oriented|connection-oriented]] services provided by [[:content:servalmesh:|Serval Mesh]], such as [[:content:technologies:meshms|text messaging]], file sharing, voice mail, and automatic software upgrades. +**Rhizome** is a resilient content distribution system developed by [[:|The Serval Project]].  It forms the basis of all non-[[http://en.wikipedia.org/wiki/Connection_oriented|connection-oriented]] services provided by [[:content:servalmesh:|Serval Mesh]], such as [[meshms|text messaging]], file sharing, voice mail, and automatic software upgrades. 
  
 +==== Technical documentation====
 +
 +**[[Rhizome API]]** contains links to the authoritative [[:content:dev:techdoc|technical documentation]] of the Rhizome HTTP REST API, which resides in the source code of the [[:content:servaldna:development|Serval DNA]] software, and should be used in preference to the older documents on this wiki.
 +
 +Other relevant information:
 +
 +  * [[Security Framework]] describes the cryptographic features of Rhizome
 +  * [[:content:activity:naf4]] describes the original development of the Rhizome HTTP REST API (beware: some technical details are out of date; refer to [[Rhizome API]] for current documentation)
  
 ==== Purpose and features ==== ==== Purpose and features ====
  
   * Rhizome is a **[[http://en.wikipedia.org/wiki/Store_and_forward|store and forward]]** service for transporting units of data called **payloads** ("files") between **applications** running on nodes in a digital [[http://en.wikipedia.org/wiki/Telecommunications_network|telecommunications network]]   * Rhizome is a **[[http://en.wikipedia.org/wiki/Store_and_forward|store and forward]]** service for transporting units of data called **payloads** ("files") between **applications** running on nodes in a digital [[http://en.wikipedia.org/wiki/Telecommunications_network|telecommunications network]]
-  * Rhizome is designed for [[http://en.wikipedia.org/wiki/Wireless_mesh_network|wireless mesh networks]] such as [[:content:technologies:mdp|MDP]] in the [[:content:servalmesh:|Serval Mesh]], but can also be used within a fixed network infrastructure+  * Rhizome is designed for [[http://en.wikipedia.org/wiki/Wireless_mesh_network|wireless mesh networks]] such as [[MDP]] in the [[:content:servalmesh:|Serval Mesh]], but can also be used within a fixed network infrastructure
   * Rhizome guarantees that payloads will remain intact wherever they are transported   * Rhizome guarantees that payloads will remain intact wherever they are transported
   * Rhizome makes no guarantees about delivery time or eventual arrival   * Rhizome makes no guarantees about delivery time or eventual arrival
Line 20: Line 28:
   * Users do not use Rhizome directly, but //indirectly via [[http://en.wikipedia.org/wiki/Application_software|applications]]// which use Rhizome as their data transport   * Users do not use Rhizome directly, but //indirectly via [[http://en.wikipedia.org/wiki/Application_software|applications]]// which use Rhizome as their data transport
   * Current Rhizome applications:   * Current Rhizome applications:
-    * [[:content:technologies:meshms|MeshMS]], the simple text messaging service of the [[:content:servalmesh:|Serval Mesh]] app for Android+    * [[MeshMS]], the simple text messaging service of the [[:content:servalmesh:|Serval Mesh]] app for Android
     * [[:content:servalmaps:|Serval Maps]], an Android app produced by the [[:|Serval Project]], uses Rhizome to share geo-tagged data like photos and documents     * [[:content:servalmaps:|Serval Maps]], an Android app produced by the [[:|Serval Project]], uses Rhizome to share geo-tagged data like photos and documents
     * the [[:content:servalmesh:|Serval Mesh]] app for Android uses Rhizome to distribute software upgrades     * the [[:content:servalmesh:|Serval Mesh]] app for Android uses Rhizome to distribute software upgrades
     * the file sharing service of the [[:content:servalmesh:|Serval Mesh]] app for Android is a simple UI for sharing, browsing and saving/viewing non-MeshMS content in Rhizome     * the file sharing service of the [[:content:servalmesh:|Serval Mesh]] app for Android is a simple UI for sharing, browsing and saving/viewing non-MeshMS content in Rhizome
-    * [[:content:technologies:servalsam|Serval SAM]] is a crowd survey app for Android that allows the user to fill in an [[http://opendatakit.org/|Open Data Kit]] form and uses Rhizome to transport completed forms to a central collection server +    * [[Serval SAM]] is a crowd survey app for Android that allows the user to fill in an [[http://opendatakit.org/|Open Data Kit]] form and uses Rhizome to transport completed forms to a central collection server 
-    * [[:content:technologies:sensorlogger|Sensor Logger]], an Android app produced by the [[:|Serval Project]], uses Rhizome to send continuous accelerometer logs to the [[:content:seniordevs:servalhq|Serval HQ]] central collection server +    * [[Sensor Logger]], an Android app produced by [[:content:about]], uses Rhizome to send continuous accelerometer logs to the [[:content:seniordevs:servalhq|Serval HQ]] central collection server 
   * Future applications may include:   * Future applications may include:
     * voice mail     * voice mail
Line 41: Line 49:
 ==== Rhizome's data model ==== ==== Rhizome's data model ====
  
-Every Rhizome **payload** ("file") is accompanied by some structured meta data called a **manifest**: +Every file distributed by Rhizome ("payload") is accompanied by some structured meta-data called a **[[Rhizome manifest|manifest]]**.
-  * some meta data is supplied by the originating application to accompany the payload, but is not used by Rhizome, eg, File name, Content MIME type +
-  * some meta data is created and used by Rhizome to identify, prioritise, route and expire the payload, eg, Bundle ID, Bundle Version, Payload size, Payload hash (digest) +
-  * some meta data is supplied by the originating application and is used by Rhizome to prioritise, route and expire the payload, eg, sender and recipient [[:content:technologies:sid|Serval Identities]], [[:content:technologies:rhizome:journal|Journal head]], [[:content:technologies:rhizome:boundingcircle|Bounding circle]], [[:content:technologies:rhizome:lifetime|Lifetime]]+
  
-A Rhizome **bundle** = **manifest** + **payload** +A payload plus a [[Rhizome manifest|manifest]] together form a **[[Rhizome bundle]]**.
- +
-== Rhizome manifest == +
- +
-The Rhizome manifest is a set of key-value pairs called **fields**,  A manifest is stored and transmitted in either text or binary representation: +
- +
-  * The **text manifest** is the original implementation +
-  * The original text format used [[http://en.wikipedia.org/wiki/ASCII|US-ASCII]] character encoding and may be upgraded to [[http://en.wikipedia.org/wiki/UTF-8|Unicode UTF-8]] when needed +
-  * The **[[:content:technologies:rhizome:binarymanifest|binary manifest]]** is not yet implemented +
-  * Here are some sample (historical) implementations of the text manifest from [[:content:servalmesh:releases:version_0_90|Serval Mesh release 0.90 "Shiny"]]: +
-    * [[https://github.com/servalproject/serval-dna/blob/batphone-release-0.90/rhizome_bundle.c#L87|rhizome_bundle.c]] in the [[:content:servaldna:|Serval DNA]] source code +
-    * [[https://github.com/servalproject/batphone/blob/0.90/src/org/servalproject/rhizome/RhizomeManifest.java|RhizomeManifest.java]] from the [[:content:servalmesh:development|Batphone]] source code +
-  * An example of a text manifest:<code> +
-recipient=FFA9248AF3848D3FD3FCCCF339C3AF9D574A3D6C9BA2FB3F6C587FB25C479F64 +
-sender=49259519777EF680E20C6ED05FA4B51EB86DDDC7D175A3B15B7AF2F9E446B54E +
-service=MeshMS1 +
-date=1351559461139 +
-id=E6856342C6F2DAFE560C18FE85EC1CCDAB5457D50BFDAE7FCF271D5B9C0E4123 +
-BK=8CEA7C075CD15F1CC8AA1D0F5E263ACD4ECEE6A870A87D90BBD68A4968B8939C +
-crypt=0 +
-filesize=39 +
-filehash=254CDA75BF5E8CA6B5498FE20C33F120BA50464B7BC7B09FD6C41D10722F1B0844137E55F073FCDC7A4388EA48DD282B2C5A6D42931C84E9E076A2BFA8A21BC0 +
-version=1351559461142 +
-</code> +
-  * Every manifest contains one or more cryptographic signatures to prevent tampering +
-    * Rhizome nodes will reject manifests with no signature or a signature that does not verify +
-    * Only the original creator (author) of the manifest may possess the signing key, so is the only party able to modify the manifest (publish newer Bundle Versions with the same Bundle ID) +
- +
-== Backward and forward compatibility == +
- +
-Both Rhizome manifest representations (text and binary) are designed to allow upgrades. +
- +
-[[http://en.wikipedia.org/wiki/Forward_compatibility|Forward compatibility]] arises from an extensible manifest data format that allows unrecognised fields to be skipped: +
-  * The text manifest uses a special field delimiter (newline) that cannot appear in any field value +
-  * The [[:content:technologies:rhizome:binarymanifest|binary manifest]] stores the length of every field +
-  * Every Rhizome node uses only the manifest fields it recognises and successfully parses, and ignores any it does not (possibly logging a warning) +
-  * Every Rhizome node always stores and transmits the manifest intact, without modification, including any unrecognised fields +
- +
-[[http://en.wikipedia.org/wiki/Backward_compatible|Backward compatibility]] is done using a standard replace/deprecate/delete process: +
-  * The format and meaning of existing fields is never changed +
-  * Software upgrades may introduce a new field that can coexist with the older field(s) it supercedes +
-  * For a while, software upgrades create manifests containing the deprecated field and the newer field +
-  * Eventually a software release and all following releases cease to insert the deprecated field +
-  * This provides a time window during which the [[:content:technologies:autoupgrade|automatic field upgrade]] has a chance to upgrade all the Rhizome software in the field +
- +
-== Sender and recipient == +
- +
-  * If a bundle does not indicate the [[:content:technologies:sid|Serval Identity]] of its intended recipient, it is for general consumption (broadcast) +
-  * If a bundle does not indicate the [[:content:technologies:sid|Serval Identity]] of its sender, then it is of unknown origin (anonymous)+
  
 ==== Rhizome's data transport ==== ==== Rhizome's data transport ====
Line 128: Line 85:
  
   * Rhizome nodes may transfer bundles using **[[http://en.wikipedia.org/wiki/Network_layer|network layer]]** connections over established networks, eg, [[http://en.wikipedia.org/wiki/Transmission_Control_Protocol|TCP]] or [[http://en.wikipedia.org/wiki/User_Datagram_Protocol|UDP]] or [[http://en.wikipedia.org/wiki/Http|HTTP]] over public Internet or cellular GSM   * Rhizome nodes may transfer bundles using **[[http://en.wikipedia.org/wiki/Network_layer|network layer]]** connections over established networks, eg, [[http://en.wikipedia.org/wiki/Transmission_Control_Protocol|TCP]] or [[http://en.wikipedia.org/wiki/User_Datagram_Protocol|UDP]] or [[http://en.wikipedia.org/wiki/Http|HTTP]] over public Internet or cellular GSM
-  * Adjacent Rhizome nodes may transfer bundles over **[[http://en.wikipedia.org/wiki/Data_link_layer|data link layer]]** connections, eg, [[:content:technologies:mdp|MDP]] to [[:content:servalmesh:|Serval Mesh]] devices within Fi-Fi range+  * Adjacent Rhizome nodes may transfer bundles over **[[http://en.wikipedia.org/wiki/Data_link_layer|data link layer]]** connections, eg, [[MDP]] to [[:content:servalmesh:|Serval Mesh]] devices within Fi-Fi range
  
 ==== Neutrality ==== ==== Neutrality ====
Line 138: Line 95:
   * Rhizome does not rank (prioritise) based on the content of payloads   * Rhizome does not rank (prioritise) based on the content of payloads
  
-==== Bundle integrity ==== 
  
-  * Rhizome does not modify any manifest once it is created 
-  * Rhizome does not modify any payload 
-  * Rhizome creates a random 256-bit Elliptic Curve private key for every bundle, called the **Bundle Secret** or **BS** 
-  * The 256-bit public key derived from the Bundle Secret is called the **Bundle ID** or **BID** 
-  * Rhizome does not reveal the Bundle Secret to any other application or node 
-  * Rhizome computes the 512-bit SHA512 hash of the payload, called the **Payload Hash** or **PH** (also **File Hash** or **FH**) 
-  * At injection, Rhizome inserts the Payload Hash into the bundle's manifest 
-  * Rhizome signs every complete manifest using the Bundle Secret 
-  * Rhizome nodes will only store and transmit manifests with valid signatures 
-  * Only an application possessing the original Bundle Secret can alter the manifest or payload 
- 
-The net result of Rhizome's crytpo system is that: 
-  * any party may create a new bundle (with a new Bundle ID) 
-  * only the originating party may modify the manifest or payload of an existing bundle (keeping the same Bundle ID) 
-  * thus, once a bundle is created, all new payloads with the same Bundle ID are guaranteed to be from the same original sender 
- 
-==== Payload security ==== 
- 
-  * Rhizome can optionally **encrypt** a payload during injection with a one-way asymmetric cipher using the recipient's [[:content:technologies:sid|Serval ID]] public key 
-  * Rhizome can optionally **encrypt** a payload during injection with a two-way session cipher constructed using the sender's and recipient's [[:content:technologies:sid|Serval ID]] public keys 
-  * Rhizome will **decrypt** an encrypted payload during extraction if the node possesses the recipient's [[:content:technologies:sid|Serval ID]] secret key 
-  * The Payload Hash of an encrypted payload is the SHA512 hash of the encrypted form, not the clear text form 
-  
 ==== User intervention ==== ==== User intervention ====
  
Line 176: Line 109:
 Once a file is inserted into Rhizome, it can be notoriously difficult to eradicate.  This is analogous to a biological process of vegetative reproduction: a [[http://en.wikipedia.org/wiki/Rhizome|rhizome]] is an underground plant stem with the ability to send out shoots which develop into new plants. If a rhizome is cut into smaller pieces, each piece can grow into a new organism.  Anybody who has battled the incursion of [[http://en.wikipedia.org/wiki/Elymus_repens|couch grass]] into their garden knows how obstinate a rhizome can be. Once a file is inserted into Rhizome, it can be notoriously difficult to eradicate.  This is analogous to a biological process of vegetative reproduction: a [[http://en.wikipedia.org/wiki/Rhizome|rhizome]] is an underground plant stem with the ability to send out shoots which develop into new plants. If a rhizome is cut into smaller pieces, each piece can grow into a new organism.  Anybody who has battled the incursion of [[http://en.wikipedia.org/wiki/Elymus_repens|couch grass]] into their garden knows how obstinate a rhizome can be.
  
-Curiously (but not coincidentally) some of the principles of the [[http://en.wikipedia.org/wiki/Rhizome_(philosophy)|rhizomatic]] philosophy of thought also describe the behaviour of the Rhizome file distribution system.+==== Prototypes ==== 
 + 
 +The [[Rhizome Retriever]] was the first prototype of the manifest-payload distribution system.
  
content/tech/rhizome.1367894239.txt.gz · Last modified: 06/05/2013 19:37 by Andrew Bettison

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution 4.0 International
CC Attribution 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki