User Tools

Site Tools


content:tech:rhizome_journal

Rhizome journal

A journal is like a restricted kind of file that can only be appended to, like a log file, but existing content cannot be modified. This means that consumers of the file need not re-examine the entire file, only the end, to discover new content.

MeshMS was first implemented by treating the payload of a Rhizome bundle as a journal. Each new MeshMS message sent from one party to another was appended to the list of messages already sent, so the payload size increased in an unbounded manner.

From the outset, it was planned that MeshMS would take advantage of a new mechanism in Rhizome which would explicitly mark a bundle as a journal, and treat journal bundles differently:

  • new versions of the bundle can only append content, but cannot alter existing content
  • the Rhizome protocol would allow a node to fetch only the new part of a journal payload that it did not already possess, ie, the data beyond the filesize of the payload already stored, thus keeping network load constant while still allowing the journal to grow
  • the version and filesize manifest fields become one and the same
  • a new tail manifest field indicates the offset into the journal at which the payload contents begin
  • an application (like MeshMS) must explicitly advance tail when it decides that old content is no longer of interest, which causes Rhizome to discard all data prior to the tail offset from the payload
  • the file hash is computed on the data between the tail and filesize offsets

This scheme is described in more detail in this paper: Gardner-Stephen, P., Lakeman, J., Challans, R., Wallis, C., Stulman, A. and Haddad, Y., MeshMS: Ad Hoc Data Transfer within Mesh Network, Int'l J. of Communications, Network and System Sciences, vol. 5 issue 8 (August 2012).

This approach is currently only theoretical. This project seeks to answer the question of whether this approach is feasible by attempting an implementation.

Progress to date

This technology was first developed in the first half of 2013 as a student project.

content/tech/rhizome_journal.txt · Last modified: 14/05/2013 23:18 (external edit)