content:activity:openitp2

This is an old revision of the document!


OpenITP - Contractor Agreement #??-OITPDOS2013 “Grant 2”

In December 2013, The Serval Project commenced a second round of work for the OpenITP (Internet Tools Project) to release software images for the Serval Mesh Extender device and the Serval Mesh app for Android by the end of March 2014.

Section 1: Work to be Performed (Scope of Work)

  1. Release Mesh Extender software image
  2. Release Serval Mesh Android application

Attachment I

From the grant proposal:

Specifically, we [The Serval Project are requesting OpenITP's support in completing work unit 3, that together with an existing grant under consideration by the NLnet Foundation, and existing work that are have [sic] currently resourced, will allow us to complete stages 1 and 2, and thus release an easy to use and effective Mesh Extender software image, and an integrated version of the Serval Mesh Android application. This will allow the general public to form their own mesh telephony networks over interesting distances using our open-source software.

COMPLETION REPORT

Summary

The work was completed in November 2014, six months later than the contracted date of April 30, 2014. The delay was due to staff absences, more-than-expected technical hitches in the Mesh Extender firmware installation process, and demands of other tasks such as preparation of grant submissions to sustain the Project financially and completion of the following contracts:

During December 2014 and January 2015, three pairs of Mesh Extender units were tested and shipped in working condition to the “Bleeding Edge” supporters of the Speak Freely crowdfunding campaign.

Work items performed

The following work items were performed to satisfy the contract:

The following work items were performed to demonstrate fulfilment:

  • W6. Flash, test and ship three working pairs of Mesh Extenders
  • W7. Provide technical support for Mesh Extender early adopters

<BOOKMARK:W1>

W1. Build and release Serval DNA for OpenWRT

The Mesh Extender firmware is an OpenWRT image, so during February and March 2014, formal OpenWRT build and release support was added to Serval DNA:

  • The new sp-openwrt-release script was added to the serval-tools repository, to automate the process of releasing Serval packages for OpenWRT instead of developers making manual changes to the Serval OpenWRT feed repository.
  • The Serval DNA development Makefile was introduced to support an OpenWRT edit-compile-test development cycle using the local working copy of Serval DNA instead of its GitHub repository.

<BOOKMARK:W2>

W2. Improve Serval Mesh "Peer List" screen

The “Peer List” screen (activity) of the Serval Mesh app for Android was unresponsive and sometimes failed to update, which made it difficult for users to spot the presence of Mesh Extender access points.

From January to March 2014, we changed the “Peer List” activity's back-end interface with Serval DNA. Instead of synchronous (blocking) calls through the JNI interface, a new, asynchronous Java API was developed for MDP network client processes, and the connect activity was altered to send and receive Distributed Numbering Architechure (DNA) queries and replies without blocking.

We used this opportunity to improve the software architecture by adding a Java API layer to the Serval DNA component itself so that other applications can easily embed it and communicate with it, independently of the Serval Mesh application. This had the immediate advantage of bringing the new Java API code under the scope of the Serval DNA testing framework. New test cases were written to specify the semantics and contracts of the new Java API and to prevent regressions. The existing Java code in the Serval Mesh application for communicating with Serval DNA was replaced by the new Java API, which effectively increased the test coverage of the Serval Mesh app for Android.

<BOOKMARK:W3>

W3. Improve Serval Mesh "Connect" screen

The “Connect” screen (activity) of the Serval Mesh app for Android was designed for experts, and was not very usable.

In order to make it easy for users to connect to Mesh Extenders, we redesigned and simplified the “Connect” activity:

  • the “Auto Cycle” checkbox was removed, leaving only the “Enable Services” checkbox
  • the list of all Android Wi-Fi networks was replaced by a single “Wi-Fi” checkbox and a settings icon that opens the Android “Wi-Fi Settings” activity where the user can select from a list of available access points
  • the Personal Hotspot option was placed on a “Portable Wi-Fi hotspot” checkbox and a settings icon that opens the Android “Personal Wi-Fi Hotspot Settings” activity
  • the Ad Hoc Wi-Fi option was placed on an “Ad Hoc Mesh” checkbox which is initially greyed out (unavailable).

During February and March 2014, we improved the “Connect” activity:

  • commit batphone 21576d3 contains the bulk of the changes
  • commit batphone a137c12 added the “Settings” icons to the options
  • commit batphone 1464c99 relegated the HERE BE DRAGONS dialog to behind the “Ad Hoc Mesh” option
  • commit batphone 5680b64 tweaked the wording of the informative texts on the options to help users understand their options and give the UI more polish

Pressing the “Ad Hoc Mesh” label the first time initiates the process of probing for Ad Hoc capability, and pops up the HERE BE DRAGONS! warning. If the user chooses to Test, then the Serval Mesh application will attempt to use Super-User privilege to re-install the system Wi-Fi driver, which will pop up a dialog asking if the user will grant Super User privilege to Serval Mesh. By relegating these scary warnings and pop-ups behind the “Ad Hoc Mesh” button, we were able to remove them from the app's first-time start wizard, which was a significant obstacle to adoption.

The new “Connect” screen is much simpler than its predecessor, yet still requires more usability work. Users still need to know that they have to connect to mesh.servalproject.org in order to connect to a Mesh Extender. However, the design of the new “Connect” screen was directed towards eventually adding an extra row labelled “Mesh Extender”, which would be greyed out (unavailable) unless one or more Mesh Extender access points were within range. Checking the “Mesh Extender” option would automatically enable Wi-Fi and associate to the mesh.servalproject.org access point. We did not implement this because there are usability and implementation issues yet to be resolved, such as:

  • the presence of Mesh Extenders cannot be detected unless Wi-Fi is first activated (Wi-Fi Direct might help solve this problem)
  • if Wi-Fi is already activated, associating to a Mesh Extender would disconnect from any network the user had already connected to, which violates the UX principle of “no unpleasant surprises”
  • we want to garner feedback on the new “Connect” screen before proceeding to change it

<BOOKMARK:W4>

W4. Release Version 0.92 of Serval Mesh

The Serval Mesh Version 0.92 encapsulated the improvements made in W2 and W3, as well as many bug fixes, performance and stability improvements. The Release Notes list all the changes and known issues, and acknowledge the funding support of OpenITP.

The 0.92 release process lasted eight months from March to October 2014:

  • on March 1, the first release candidate 0.92-RC1 was created
  • on March 24, the second release candidate 0.92-RC2 was created
  • on March 24, a message was sent to the Serval Project Developers list encouraging Beta testers
  • on July 28, the third release candidate 0.92-RC3 was created and announced
  • on August 15, the fourth release candidate 0.92-RC4 was created and announced
  • on October 1, the fifth release candidate 0.92-RC5 was created and announced
  • on October 27, the final release 0.92 was created and announced

<BOOKMARK:W5>

W5. Mesh Extender firmware

From July to November 2014, scripts and documentation for installing the Mesh Extender firmware on an TP-LINK MR2030 from scratch were developed and committed to the new Mesh Extender Builder GitHub repository. This work was complicated by several technical issues:

  • The develop-test cycle was significantly slowed by the need to re-flash and reboot a Mesh Extender on every single change.
  • The Sandisk USB flash pen drives that we had purchased to use in the Mesh Extender were defective: after some number of disconnects without a proper sync and unmount (ie, on every Mesh Extender hard reset), they would permanently revert to read-only mode. We contacted Sandisk, who confirmed they were aware of the issue but did not intend to correct it.
  • The OpenWRT image produced by W1 turned out to be slightly too big to fit into the MR2030 flash memory, so the serval-dna package had to be excluded from the core image and built as a module instead.
  • The original RFD-900 radio module serial flash loader utility was a Python script, which slowed down the MR3020 boot process and required the OpenWRT image to include the Python interpreter, so we developed new serial flash loader in C which reduced the boot time and allowed us to shrink the OpenWRT image by removing Python.
  • The RFD-900 radio module could boot into different serial settings, making detection of the module's firmware version slow and unreliable, so we decided to always upload the radio firmware on every boot.
  • The Auto Upgrade feature could not be tested while any other Serval Mesh nodes were active within Wi-Fi range, because upgrade images stored on other devices would interfere with the test.

<BOOKMARK:W6>

W6. Test and ship Mesh Extenders

The Speak Freely crowdfunding campaign, which ran from July to September 2013, raised 13,465 USD for Mesh Extender development, which included donations from four “True Believer” supporters who are now credited in the Serval Mesh app, and from three “Bleeding Edge” supporters, to each of whom we have now shipped a pair of working Mesh Extender MR3020 prototype units and, as a bonus for late delivery, a pair of Huawei IDEOS X1 u8180 phones with Serval Mesh Version 0.92 pre-installed.

Preparation of these Mesh Extender prototypes was complicated by technical difficulties such as defective RFD-900 radio modules, serial cable solder joint failures caused by repeated extraction and insertion into the 3D-printed case, and defective Sandisk USB flash drive units. The shipments were finally sent in January 2015.

<BOOKMARK:W7>

W7. Mesh Extender technical support

content/activity/openitp2.1420664725.txt.gz · Last modified: 07/01/2015 13:05 by Andrew Bettison