| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
content:activity:openitp2 [07/01/2015 07:41]
Andrew Bettison Write [W1. Build Serval DNA for OpenSRT] section |
content:activity:openitp2 [18/01/2015 22:06] (current)
Andrew Bettison link to Paul's blog post on shipping mesh extenders |
| * [[naf6]] | * [[naf6]] |
| |
| During December 2014 and January 2015, three pairs of Mesh Extender units were tested and shipped in working condition to the "Bleeding Edge" supporters of the [[https://www.indiegogo.com/projects/speak-freely|Speak Freely crowdfunding campaign]]. | During December 2014 and January 2015, four pairs of tested and functional Mesh Extender units were prepared to ship to the "Bleeding Edge" supporters of the [[https://www.indiegogo.com/projects/speak-freely|Speak Freely crowdfunding campaign]]. |
| |
| ==== Work items performed ==== | ==== Work items performed ==== |
| |
| Several work items were performed to satisfy the contract: | The following work items were performed to satisfy the contract: |
| |
| * **[[#W1]]**. Add support for [[:content:meshextender:building_on_openwrt]] | * **[[#W1]]**. [[:content:meshextender:building_on_openwrt]] and [[https://github.com/servalproject/openwrt-packages|formal OpenWRT versioning and release procedures]] |
| * **[[#W2]]**. Establish [[https://github.com/servalproject/openwrt-packages|formal OpenWRT versioning and release procedures]] | * **[[#W2]]**. Improve the "Peer List" screen in the [[:content:servalmesh:|Serval Mesh app for Android]] |
| * **[[#W3]]**. Improve the "Connect" screen in the [[:content:servalmesh:|Serval Mesh app for Android]] | * **[[#W3]]**. Improve the "Connect" screen in the [[:content:servalmesh:|Serval Mesh app for Android]] |
| * **[[#W4]]**. Release [[:content:servalmesh:releases:version_0_92]] | * **[[#W4]]**. Release [[:content:servalmesh:releases:version_0_92]] |
| * **[[#W5]]**. Build and install firmware for the [[:content:meshextender:]] [[:content:meshextender:prototyping_on_mr3020]] | * **[[#W5]]**. Build and install firmware for the [[:content:meshextender:]] [[:content:meshextender:prototyping_on_mr3020]] |
| |
| The following work items were performed to demonstrate fulfilment: | The following supplemental work items demonstrate fulfilment: |
| |
| * **[[#W6]]**. Flash, test and ship three working pairs of Mesh Extenders | * **[[#W6]]**. [[http://servalpaul.blogspot.com.au/2015/01/bleeding-edge-mesh-extender-kits.html|Ship four working pairs of Mesh Extenders]] |
| * **[[#W7]]**. Provide technical support for Mesh Extender early adopters | * **[[#W7]]**. Support our Mesh Extender early adopters |
| |
| <BOOKMARK:W1>== W1. Build Serval DNA for OpenWRT == | <BOOKMARK:W1>=== W1. Build and release Serval DNA for OpenWRT === |
| |
| The Mesh Extender firmware is an [[http://www.openwrt.org|OpenWRT]] image, so during February and March 2014, formal OpenWRT build and release support was added to [[:content:servaldna:]]: | The Mesh Extender firmware is an [[http://www.openwrt.org|OpenWRT]] image, so during February and March 2014, formal OpenWRT build and release support was added to [[:content:servaldna:]]: |
| |
| * An official [[https://github.com/servalproject/openwrt-packages|Serval OpenWRT feed repository]] was created and its branching policy established and documented, with explicit support for building a Mesh Extender firmware compatible with any official release of [[:content:servalmesh:|Serval Mesh app for Android]]. | * An official [[https://github.com/servalproject/openwrt-packages|Serval OpenWRT feed repository]] was created and its branching policy established and documented, with explicit support for building Mesh Extender firmware compatible with any [[:content:servalmesh:releases:|official release]] of [[:content:servalmesh:|Serval Mesh app for Android]]. |
| |
| * The new [[https://github.com/servalproject/serval-tools/blob/master/doc/sp-openwrt-release.md|sp-openwrt-release]] script was added to the [[https://github.com/servalproject/serval-tools|serval-tools]] repository, to automate the process of releasing Serval packages for OpenWRT instead of developers making manual changes to the Serval OpenWRT feed repository. | * The new [[https://github.com/servalproject/serval-tools/blob/master/doc/sp-openwrt-release.md|sp-openwrt-release]] script was added to the [[https://github.com/servalproject/serval-tools|serval-tools]] repository, to automate the process of releasing Serval packages for OpenWRT instead of developers making manual changes to the Serval OpenWRT feed repository. |
| * The [[https://github.com/servalproject/serval-dna/blob/development/openwrt/packages/serval-dna/Makefile|Serval DNA development Makefile]] was introduced to support an OpenWRT edit-compile-test development cycle using the local working copy of Serval DNA instead of its GitHub repository. | * The [[https://github.com/servalproject/serval-dna/blob/development/openwrt/packages/serval-dna/Makefile|Serval DNA development Makefile]] was introduced to support an OpenWRT edit-compile-test development cycle using the local working copy of Serval DNA instead of its GitHub repository. |
| |
| | * OpenWRT build and release steps were added to the [[:content:servalmesh:release:|Serval Mesh release procedure]], to ensure that the OpenWRT platform is not neglected. |
| |
| <BOOKMARK:W2>== W2. OpenWRT release procedures == | <BOOKMARK:W2>=== W2. Improve Serval Mesh "Peer List" screen === |
| |
| <BOOKMARK:W3>== W3. Improve Serval Mesh "Connect" screen == | The "Peer List" screen (activity) of the [[:content:servalmesh:|Serval Mesh app for Android]] was unresponsive and sometimes failed to update, which made it difficult for users to spot the presence of Mesh Extender access points. |
| |
| <BOOKMARK:W4>== W4. Release Version 0.92 of Serval Mesh == | From January to March 2014, we changed the "Peer List" activity's back-end interface with [[:content:servaldna:]]. Instead of synchronous (blocking) calls through the JNI interface, a new, asynchronous Java API was developed for MDP network client processes, and the connect activity was altered to send and receive [[:content:tech:DNA]] queries and replies without blocking. |
| |
| <BOOKMARK:W5>== W5. Mesh Extender firmware == | We used this opportunity to improve the software architecture by adding a Java API layer to the [[:content:servaldna:]] component itself so that other applications can easily embed it and communicate with it, independently of the [[:content:servalmesh:]] application. This had the immediate advantage of bringing the new Java API code under the scope of the [[:content:servaldna:]] testing framework. New test cases were written to specify the semantics and contracts of the new Java API and to prevent regressions. The existing Java code in the [[:content:servalmesh:]] application for communicating with Serval DNA was replaced by the new Java API, which effectively increased the test coverage of the [[:content:servalmesh:|Serval Mesh app for Android]]. |
| | |
| | * the commits [[https://github.com/servalproject/batphone/commit/ff2303ce593cf74d05ae4e2c521c5b8eda8012cc|batphone ff2303c]], [[https://github.com/servalproject/batphone/commit/26673efa9b71739001ae0d4d0594aa29ced8a404|batphone 26673ef]], [[https://github.com/servalproject/serval-dna/commit/99fb8b6108021842ddaa82d95e17fe1f2f1347b8|serval-dna 99fb8b6]], [[https://github.com/servalproject/serval-dna/commit/46ef40cf237c5941847c857a7748779756ba28e5|serval-dna 46ef40c]] and [[https://github.com/servalproject/serval-dna/commit/63db7b4513c2fb049fb1a87eeb24aed7fe5d72e1|serval-dna 63db7b4]] moved the existing Java API from the [[https://github.com/servalproject/batphone|Batphone repository]] to the [[https://github.com/servalproject/serval-dna|Serval DNA]] repository |
| | * commit [[https://github.com/servalproject/serval-dna/commit/20494a90d5cd42bfb2f8418cbb605fd1e6fda390|serval-dna 20494a9]] created the new Java API for MDP clients |
| | * commits [[https://github.com/servalproject/batphone/commit/0e21c9971571918c057e5d71512c8ada295e06b0|batphone 0e21c99]] and [[https://github.com/servalproject/batphone/commit/6e71aeaaa132747d484da9d745ed0fe21c7b26a2|batphone 6e71aea]] upgraded the "Peer List" activity and all other places peer details are displayed to use asynchronous MDP instead of synchronous JNI |
| | |
| | <BOOKMARK:W3>=== W3. Improve Serval Mesh "Connect" screen === |
| | |
| | The "Connect" screen (activity) of the [[:content:servalmesh:|Serval Mesh app for Android]] was designed for experts, and was not very usable. |
| | |
| | In order to make it easy for users to connect to Mesh Extenders, we redesigned and simplified the "Connect" activity: |
| | * the "Auto Cycle" checkbox was removed, leaving only the "Enable Services" checkbox |
| | * the list of all Android Wi-Fi networks was replaced by a single "Wi-Fi" checkbox and a settings icon that opens the Android "Wi-Fi Settings" activity where the user can select from a list of available access points |
| | * the Personal Hotspot option was placed on a "Portable Wi-Fi hotspot" checkbox and a settings icon that opens the Android "Personal Wi-Fi Hotspot Settings" activity |
| | * the Ad Hoc Wi-Fi option was placed on an "Ad Hoc Mesh" checkbox which is initially greyed out (unavailable). |
| | |
| | During February and March 2014, we improved the "Connect" activity: |
| | * commit [[https://github.com/servalproject/batphone/commit/21576d33435c63c9345dc11ccea031d599160979|batphone 21576d3]] contains the bulk of the changes |
| | * commit [[https://github.com/servalproject/batphone/commit/a137c120082033f07478652a12879e8d3fb49e4a|batphone a137c12]] added the "Settings" icons to the options |
| | * commit [[https://github.com/servalproject/batphone/commit/1464c99592e38dd9e797b4f760ad7eb7dcaa5425|batphone 1464c99]] relegated the HERE BE DRAGONS dialog to behind the "Ad Hoc Mesh" option |
| | * commit [[https://github.com/servalproject/batphone/commit/5680b649e82990bbe10cf95b60fea4f1598a811c|batphone 5680b64]] tweaked the wording of the informative texts on the options to help users understand their options and give the UI more polish |
| | |
| | Pressing the "Ad Hoc Mesh" label the first time initiates the process of probing for Ad Hoc capability, and pops up the HERE BE DRAGONS! warning. If the user chooses to Test, then the Serval Mesh application will attempt to use Super-User privilege to re-install the system Wi-Fi driver, which will pop up a dialog asking if the user will grant Super User privilege to Serval Mesh. By relegating these scary warnings and pop-ups behind the "Ad Hoc Mesh" button, we were able to remove them from the app's first-time start wizard, which was a significant obstacle to adoption. |
| | |
| | The new "Connect" screen is much simpler than its predecessor, yet still requires more usability work. Users still need to know that they have to connect to //mesh.servalproject.org// in order to connect to a Mesh Extender. However, the design of the new "Connect" screen was directed towards eventually adding an extra row labelled "Mesh Extender", which would be greyed out (unavailable) unless one or more Mesh Extender access points were within range. Checking the "Mesh Extender" option would automatically enable Wi-Fi and associate to the //mesh.servalproject.org// access point. We did not implement this because there are usability and implementation issues yet to be resolved, such as: |
| | * the presence of Mesh Extenders cannot be detected unless Wi-Fi is first activated (Wi-Fi Direct might help solve this problem) |
| | * if Wi-Fi is already activated, associating to a Mesh Extender would disconnect from any network the user had already connected to, which violates the UX principle of "no unpleasant surprises" |
| | * we want to garner feedback on the new "Connect" screen before proceeding to change it |
| | |
| | <BOOKMARK:W4>=== W4. Release Version 0.92 of Serval Mesh === |
| | |
| | The [[:content:servalmesh:releases:version_0_92]] encapsulated the improvements made in **[[#W2]]** and **[[#W3]]**, as well as many bug fixes, performance and stability improvements. The [[https://github.com/servalproject/batphone/blob/0.92/CURRENT-RELEASE.md|Release Notes]] list all the changes and known issues, and acknowledge the funding support of OpenITP. |
| | |
| | The 0.92 release process lasted eight months from March to October 2014: |
| | * on March 1, the first release candidate [[https://github.com/servalproject/batphone/tree/0.92-RC1|0.92-RC1]] was created |
| | * on March 24, the second release candidate [[https://github.com/servalproject/batphone/tree/0.92-RC2|0.92-RC2]] was created |
| | * on March 24, a [[https://groups.google.com/d/msg/serval-project-developers/grV0INjiQHE/K50jqTQBfZgJ|message]] was sent to the Serval Project Developers list encouraging Beta testers |
| | * on July 28, the third release candidate [[https://github.com/servalproject/batphone/tree/0.92-RC3|0.92-RC3]] was created and [[https://groups.google.com/d/msg/serval-project-developers/2XW_rwbPR_M/gqMc-K8AFFsJ|announced]] |
| | * on August 15, the fourth release candidate [[https://github.com/servalproject/batphone/tree/0.92-RC4|0.92-RC4]] was created and [[https://groups.google.com/d/msg/serval-project-developers/lq-5COzLfB8/tghco97qRucJ|announced]] |
| | * on October 1, the fifth release candidate [[https://github.com/servalproject/batphone/tree/0.92-RC5|0.92-RC5]] was created and [[https://groups.google.com/d/msg/serval-project-developers/g2CxodAmw18/8w47H2imPSUJ|announced]] |
| | * on October 27, the final release [[https://github.com/servalproject/batphone/tree/0.92|0.92]] was created and [[https://groups.google.com/d/msg/serval-project-developers/8gwm_53ouyM/WxT_uY7o1K8J|announced]] |
| | |
| | <BOOKMARK:W5>=== W5. Mesh Extender firmware === |
| |
| From July to November 2014, scripts and documentation for installing the Mesh Extender firmware on an TP-LINK MR2030 from scratch were developed and committed to the new [[https://github.com/servalproject/mesh-extender-builder|Mesh Extender Builder]] GitHub repository. This work was complicated by several technical issues: | From July to November 2014, scripts and documentation for installing the Mesh Extender firmware on an TP-LINK MR2030 from scratch were developed and committed to the new [[https://github.com/servalproject/mesh-extender-builder|Mesh Extender Builder]] GitHub repository. This work was complicated by several technical issues: |
| * The [[:content:tech:Auto Upgrade]] feature could not be tested while any other Serval Mesh nodes were active within Wi-Fi range, because upgrade images stored on other devices would interfere with the test. | * The [[:content:tech:Auto Upgrade]] feature could not be tested while any other Serval Mesh nodes were active within Wi-Fi range, because upgrade images stored on other devices would interfere with the test. |
| |
| <BOOKMARK:W6>== W6. Test and ship Mesh Extenders == | The repository's [[https://github.com/servalproject/mesh-extender-builder/blob/master/README.md|README.md]] contains instructions for creating a USB flash drive containing the Mesh Extender software image: |
| | |
| | * **STEP 1.** On a Linux workstation, the user runs the [[https://github.com/servalproject/mesh-extender-builder/blob/master/make_image|make_image]] script, which: |
| | * downloads the OpenWRT 12.09 ("Attitude Adjustment") image builder tarball for the MR3020 target, |
| | * uses the OpenWRT image builder to compile a working, generic Mesh Extender firmware image from source, |
| | * inserts the [[https://github.com/servalproject/mesh-extender-builder/tree/master/image-files|image files]] into the generic firmware image, which includes all the configuration and settings needed to boot a Mesh Extender, plus the radio module serial flash loader executable binary and radio firmware blob. |
| | |
| | The generic OpenWRT image builder does not build [[:content:servaldna:]]. Instead, the pre-built **servald** executable binary is contained in the repository. The [[https://github.com/servalproject/mesh-extender-builder/blob/master/make_image|make_image]] script could be improved to compile [[:content:servaldna:]] from source during STEP 1, but limited time and budget did not permit further refinement. |
| | |
| | * **STEP 2.** The user runs the [[https://github.com/servalproject/mesh-extender-builder/blob/master/gather-image-files|gather-image-files]] script, which: |
| | * downloads the latest [[:content:servalmesh:]] release APK from the Serval Project server, |
| | * downloads the source code for all Serval software components from GitHub and rolls them into tarballs, which will be copied to the Mesh Extender to provide off-line access to all technical documentation, offer transparent evidence of software validity, and to comply with GPL licensing terms |
| | * collects some OpenWRT packages that were built as modules, and not included in the main firmware image |
| | * assembles a staged directory that will be copied verbatim to the USB flash drive |
| | * assembles an upgrade file, **serval.up**, that will be copied verbatime to the USB flash drive |
| | |
| | * **STEP 3.** The user connects the target Mesh Extender unit to the workstation by Ethernet cable and flashes the new firmware image to it using one of two options: |
| | - If the unit is //virgin// (factory settings; ie, never had the Mesh Extender firmware installed before) then the user runs the [[https://github.com/servalproject/mesh-extender-builder/blob/master/flash-virgin-mr3020|flash-virgin-mr3020]] script, which uploads the firmware image via HTTP to the IP address 192.168.0.254 (the MR3020 default factory IP address), then issues another HTTP command to initiate the flash upgrade. |
| | - If the unit has already been flashed at least once with the Mesh Extender firmware, then the user runs the [[https://github.com/servalproject/mesh-extender-builder/blob/master/reflash-mesh-extender|reflash-mesh-extender]] script with the IP address of the unit as its first argument, which uploads the firmware image via SSH to the given IP address, then executes the //sysupgrade// command on the unit using SSH. |
| | |
| | After STEP 3 the Mesh Extender unit will reboot, and will be running the new firmware kernel. The following steps install all the staged files to the Mesh Extender. |
| | |
| | * **STEP 4.** The user inserts a USB flash drive into the workstation and runs the [[https://github.com/servalproject/mesh-extender-builder/blob/master/partition-memory-stick.sh|partition-memory-stick.sh]] script, which partitions the USB flash drive (erasing existing partitions). The user must then remove and re-insert the USB flash drive to ensure that the workstation detects the new partitions. If re-using a USB flash drive from a previous run, then this step can be omitted, since it will already be partitioned correctly. |
| | |
| | * **STEP 5.** The user runs the [[https://github.com/servalproject/mesh-extender-builder/blob/master/populate-memory-stick|populate-memory-stick]] script, which formats the new partitions of the USB flash drive and copies the staged directory and Serval DNA configuration file to their respective partitions. It also creates the NOROOT file in the root of one of the partitions. |
| | |
| | The user may then remove the USB flash drive from the workstation, insert it in the Mesh Extender unit, and power it up. The Mesh Extender will automatically upgrade itself from the contents of the USB flash drive then reboot. The Mesh Extender unit is now fully functional. |
| | |
| | The presence of the NOROOT file in its corresponding USB flash partition prevents the Mesh Extender from accepting logins via SSH, for security reasons. Anybody with physical access to the unit can re-enable SSH logins by extracting the USB flash drive, inserting it in a workstation, removing the NOROOT file, re-inserting it in the Mesh Extender unit, and powering up the unit. The unit will re-create the NOROOT file and accept SSH logins until its next reboot, so enabling SSH logins by this method is not permanent. |
| | |
| | The user may repeat STEP 3, STEP 4 and STEP 5 as often as desired to flash new Mesh Extender units from the same firmware image and create new USB flash drives for them. |
| | |
| | <BOOKMARK:W6>=== W6. Test and ship Mesh Extenders === |
| | |
| | The [[https://www.indiegogo.com/projects/speak-freely|Speak Freely crowdfunding campaign]], which ran from July to September 2013, raised 13,465 USD for Mesh Extender development, which included donations from four "True Believer" supporters who are now credited in the [[:content:servalmesh:]] app, and from four "Bleeding Edge" supporters, to each of whom we will give a pair of working Mesh Extender MR3020 prototype units. As a bonus for late delivery, we will also give them each a pair of [[:content:devices:huawei_ideos_x1_u8180:]] phones with [[:content:servalmesh:releases:version_0_92]] pre-installed and tested in conjunction with the pair of Mesh Extender units. |
| | |
| | During December 2014 and January 2015, we assembled and tested four pairs of Mesh Extender prototype units. This was complicated by technical difficulties such as defective RFD-900 radio modules, serial cable solder joint failures caused by repeated extraction and insertion into the 3D-printed case, and defective Sandisk USB flash drive units. Once these issues were overcome, the units performed correctly. The units and phones were [[http://servalpaul.blogspot.com.au/2015/01/bleeding-edge-mesh-extender-kits.html|shipped in January 2015]]. |
| | |
| | <BOOKMARK:W7>=== W7. Support Mesh Extender early adopters === |
| | |
| | The Mesh Extender is still very much in prototype stage, and we are now focussed on getting as many as possible into the hands of early adopters. The [[https://groups.google.com/forum/#!forum/serval-project-developers|Serval Project Developers]] group is our primary means for receiving feedback and providing technical support. |
| |
| <BOOKMARK:W7>== W7. Mesh Extenders technical support == | The following message threads reveal the presence of a few Mesh Extender early adopters: |
| |
| | * November 5, 2014 -- [[https://groups.google.com/d/msg/serval-project-developers/8I8MoOtkrn4/8cA6GXIKb0UJ|mesh extender firmware]] |
| | * October 31, 2014 -- [[https://groups.google.com/d/msg/serval-project-developers/8CR4IVphxvM/R-UfMbr3JvAJ|mesh extender firmware update]] |
| | * October 24, 2014 -- [[https://groups.google.com/d/msg/serval-project-developers/AWVOy7fuhwo/TCCvuLAkauMJ|mesh extender firmware]] |
| | * October 2, 2014 -- [[https://groups.google.com/d/msg/serval-project-developers/GpAm14UlB1I/dbCP6AaNQSsJ|mesh-extender-builder]] |
| | * June 26, 2014 -- [[https://groups.google.com/d/msg/serval-project-developers/xouZ2LqJ-S8/ys6HThvFpxIJ|clever/foolish mesh extender ideas]] |
| | * June 4, 2014 -- [[https://groups.google.com/d/msg/serval-project-developers/zKfzisNKH34/Tx_FX1RPJ9gJ|Mesh Extender 400 MHz]] |
| | * October 30, 2013 -- [[https://groups.google.com/d/msg/serval-project-developers/5_oFPWfKmqk/DeYpOLkadZMJ|alternative mesh extender config]] |
| |
