User Tools

Site Tools


content:tech:rhizome

This is an old revision of the document!


Rhizome

Rhizome is a resilient content distribution system developed by The Serval Project. It forms the basis of all non-connection-oriented services provided by Serval Mesh, such as text messaging, file sharing, voice mail, and automatic software upgrades.

Technical documentation

Purpose and features

  • Rhizome is a store and forward service for transporting units of data called payloads (“files”) between applications running on nodes in a digital telecommunications network
  • Rhizome is designed for wireless mesh networks such as Mesh Datagram Protocol (MDP) in the Serval Mesh, but can also be used within a fixed network infrastructure
  • Rhizome guarantees that payloads will remain intact wherever they are transported
  • Rhizome makes no guarantees about delivery time or eventual arrival
  • Rhizome makes its best effort to transport payloads to wherever they are wanted, subject to the constraints of the network (topology, bandwidth) and its nodes (storage, battery), without favouring any payloads above others of equal kind
  • Rhizome is neutral with respect to originating and receiving nodes, applications, and payload content
  • Rhizome uses cryptography to preserve the integrity and privacy of payloads
  • Rhizome protects anonymous senders by not storing or transmitting information that may allow senders to be traced, even on the originating device
  • Rhizome functions without centralised infrastructure such as well-known servers or a persistent network connection
  • Rhizome can be configured to use central servers to improve efficiency and reach

Rhizome applications

  • Users do not use Rhizome directly, but indirectly via applications which use Rhizome as their data transport
  • Current Rhizome applications:
    • MeshMS, the simple text messaging service of the Serval Mesh app for Android
    • Serval Maps, an Android app produced by the Serval Project, uses Rhizome to share geo-tagged data like photos and documents
    • the Serval Mesh app for Android uses Rhizome to distribute software upgrades
    • the file sharing service of the Serval Mesh app for Android is a simple UI for sharing, browsing and saving/viewing non-MeshMS content in Rhizome
    • Serval Sam (app for Android) is a crowd survey app for Android that allows the user to fill in an Open Data Kit form and uses Rhizome to transport completed forms to a central collection server
    • Sensor Logger, an Android app produced by The Serval Project, uses Rhizome to send continuous accelerometer logs to the Serval HQ central collection server
  • Future applications may include:
    • voice mail
    • disaster response situational awareness
    • medical or environmental sensor data collection
    • collaborative text editing and whiteboarding
    • distributed document management
    • citizen journalism
    • classroom and remote teaching tools
    • group chat
    • Wikipedia caching
  • Applications inject data into Rhizome by supplying a payload and some meta data
  • Applications extract data from Rhizome, receiving the payload and all meta data

Rhizome's data model

Every file distributed by Rhizome (“payload”) is accompanied by some structured meta-data called a manifest.

A payload plus a manifest together form a Rhizome bundle.

Rhizome's data transport

  • A Rhizome node is any network node (device) that is running the Rhizome software
  • A Rhizome node exchanges manifests and payloads directly with adjacent nodes on the network
  • Rhizome bundles are transferred between nodes automatically and autonomously, ie, without user intervention, using any available network connection
  • A manifest may be stored on many nodes at a time
  • A payload may be stored on many nodes at a time
  • A manifest may persist for any length of time on any node
  • A payload may persist for any length of time on any node
  • Each Rhizome node advertises its recently acquired bundles to its neighbours
Rhizome store
  • A Rhizome node contains a Rhizome store
  • A Rhizome store is a database containing payloads indexed by PH (Payload Hash) and manifests indexed by BID (Bundle ID)
  • A Rhizome store may reside in volatile (RAM) or non-volatile (Flash, disk) local storage
  • The original implementation of the Rhizome store used SQLite for the database
Rhizome Rank
  • Every Rhizome node maintains its own ordered list of all the manifests in its store, called the Rank
  • Every Rhizome node attempts to fetch missing payloads from neighbouring nodes, in order of rank
  • Every Rhizome node recovers local storage space by evicting its lowest ranked payloads
  • A bundle's position in the rank depends on meta data in the manifest and local node state, which may include:
    • own Serval Identities
    • Serval Identities of recently reachable nodes
    • current time
    • current or recent geographic location
Network connections

Neutrality

  • Rhizome interprets meta data uniformly, regardless of the originating or receiving application
  • Rhizome ranks (prioritises) payloads using uniform rules that do not depend on the originating or receiving application
  • Rhizome software does not encapsulate any application-specific logic
  • Rhizome does not record the originating or transited nodes of any payload, so nodes cannot rank (prioritise) payloads based on point of origin, only on available meta data
  • Rhizome does not rank (prioritise) based on the content of payloads

User intervention

  • Users may improve the local capacity of Rhizome by operating nodes with high storage capacity
  • Users may improve the reach of Rhizome by periodically moving a node between distant locations

Architecture

Rhizome is currently implemented as part of the Serval DNA daemon.

The name "Rhizome"

Once a file is inserted into Rhizome, it can be notoriously difficult to eradicate. This is analogous to a biological process of vegetative reproduction: a rhizome is an underground plant stem with the ability to send out shoots which develop into new plants. If a rhizome is cut into smaller pieces, each piece can grow into a new organism. Anybody who has battled the incursion of couch grass into their garden knows how obstinate a rhizome can be.

Prototypes

The Rhizome Retriever was the first prototype of the manifest-payload distribution system.

content/tech/rhizome.1446433694.txt.gz · Last modified: 01/11/2015 19:08 by Andrew Bettison