User Tools

Site Tools


content:tech:rhizome

This is an old revision of the document!


Rhizome

Rhizome is a file distribution system developed by The Serval Project that enables files to be propagated over different network topologies. Rhizome can distribute files via mesh, direct connect and client-server connections.

Purpose and features

  • Rhizome is a system for transporting units of data called payloads (“files”) between applications running on nodes in a digital telecommunications network
  • Rhizome uses store and forward to propagate payloads
  • Rhizome makes no guarantees about delivery time or eventual arrival
  • Rhizome guarantees that payloads will remain intact wherever they are transported
  • Rhizome aims to make the best effort to transport payloads to wherever they are wanted, subject to the constraints of the network (topology, bandwidth) and its nodes (storage, battery), without favouring any payloads above others of equal kind
  • Rhizome is neutral with respect to originating and receiving nodes and applications
  • Rhizome uses cryptography to guarantee the integrity of payloads by preventing any party except the originator from modifying a payload or its meta data

Rhizome's data model

  • Every Rhizome payload is accompanied by some structured meta data called a manifest:
    • some meta data is supplied by the originating application
    • some meta data is created by Rhizome to identify and prioritise the payload
  • A manifest together with its payload is called a bundle
  • A manifest may be present on many nodes at a time
  • A payload may be present on many nodes at a time
  • A manifest may persist for any length of time on any node
  • A payload may persist for any length of time on any node
  • Rhizome does not modify any manifest once it is created
  • Rhizome does not modify any payload
  • A bundle may indicate the Serval Identity of its intended recipient, otherwise it is for general consumption (broadcast)
  • A payload may indicate the Serval Identity of its sender, otherwise it is of unknown origin (anonymous)
  • Rhizome enforces a set of logical conditions on every bundle it receives from any source:
    • certain fields must always be present
    • each field must have a certain length and/or format
Rhizome manifest

The Rhizome manifest is a set of key-value pairs called “fields”.

The Rhizome manifest is designed for forward and backward compatibility between different versions of the Rhizome software:

  • Every Rhizome node uses the manifest fields it recognises and silently ignores those it does not
  • Every Rhizome node preserves all manifest fields intact when it stores the manifest and when it transmits the manifest to other nodes

Rhizome nodes

  • A Rhizome node is any network node (device) that is running the Rhizome software
  • Payloads and manifests are copied directly from node to node using network transfers
  • Payloads and manifests are stored on nodes using volatile (RAM) or non-volatile (Flash, disk) local storage
  • Each Rhizome node actively transmits its recently acquired manifests to its neighbours
  • Each Rhizome node records all received manifests in a sorted list called the rank
  • Each Rhizome node attempts to acquire the payloads of its highest ranked manifests from neighbouring nodes
  • Each Rhizome node recovers local storage space by evicting its lowest ranked payloads and manifests

Rhizome applications

  • Users do not use Rhizome directly, but indirectly via applications which use Rhizome as their data transport, eg, |MeshMS and Serval Maps
  • Applications inject data into Rhizome by supplying a payload and some meta data
  • Applications extract data from Rhizome, receiving the payload and all meta data

Neutrality

  • Rhizome interprets meta data uniformly, regardless of the originating or receiving application
  • Rhizome ranks (prioritises) payloads using uniform rules that do not depend on the originating or receiving application
  • Rhizome software does not encapsulate any application-specific logic
  • Rhizome does not record the originating or transited nodes of any payload, so nodes cannot rank (prioritise) payloads based on point of origin, only on available meta data
  • Rhizome does not rank (prioritise) based on the content of payloads

Bundle integrity

  • Rhizome creates a random 256-bit Elliptic Curve private key for every bundle, called the Bundle Secret or BS
  • The 256-bit public key derived from the Bundle Secret is called the Bundle ID or BID
  • Rhizome does not reveal the Bundle Secret to any other application or node
  • Rhizome computes the 512-bit SHA512 hash of the payload, called the Payload Hash or PH (also File Hash or FH)
  • At injection, Rhizome inserts the Payload Hash into the bundle's manifest
  • Rhizome signs every complete manifest using the Bundle Secret
  • Rhizome nodes will only store and transmit manifests with valid signatures
  • Only an application possessing the original Bundle Secret can alter the manifest or payload

The net result of Rhizome's crytpo system is that:

  • any party may create a new bundle (with a new Bundle ID)
  • only the originating party may modify the manifest or payload of an existing bundle (keeping the same Bundle ID)
  • thus, once a bundle is created, all new payloads with the same Bundle ID are guaranteed to be from the same original sender

Payload security

  • Rhizome can encrypt a payload during injection with a one-way asymmetric cipher using the recipient's Serval ID public key
  • Rhizome can encrypt a payload during injection with a two-way session cipher constructed using the sender's and recipient's Serval ID public keys
  • Rhizome will decrypt an encrypted payload during extraction if the node possesses the recipient's Serval ID secret key
  • The Payload Hash of an encrypted payload is the SHA512 hash of the encrypted form, not the clear text form

User intervention

  • Users may improve the local capacity of Rhizome by operating nodes with high storage capacity
  • Users may improve the reach of Rhizome by periodically moving a node between distant locations

Network connections

  • Rhizome nodes may communicate using network layer connections over established networks, eg, TCP or UDP or HTTP over public Internet or cellular GSM
  • Adjacent Rhizome nodes may communicate over data link layer connections, eg, MDP to Serval Mesh devices within Fi-Fi range

Architecture

Rhizome is currently implemented as part of the Serval DNA daemon.

The name "Rhizome"

Once a file is inserted into Rhizome, it can be notoriously difficult to eradicate. This suggested an analogy to a biological process of vegetative reproduction: a rhizome is an underground plant stem with the ability to send out shoots which develop into new plants. If a rhizome is cut into smaller pieces, each piece can grow into a new organism.

Curiously, but not by coincidence, the principles of the rhizomatic philosophy of thought also resemble the high-level requirements of the Rhizome file distribution system.

content/tech/rhizome.1363760514.txt.gz · Last modified: 19/03/2013 23:21 by Andrew Bettison