Embedded/Distributed Architecture

Presently the components of OpenVisualizer operate within a single process. However, in a realistic sensor networking implementation, we do not wish to deploy a PC as an LBR!  This page presents the first steps toward segmenting the components of OpenVisualizer across processes and machines, using an architecture as shown in the diagram below.

As a first step toward this goal, we plan to separate the GUI to run on a PC, and an LBR to run on an inexpensive, low-power, embedded Linux device.

Messaging

Presently we implement EventBus messaging with PyDispatch, which provides communication between threads in a single process. However, we must split the GUI from the LBR when the LBR is headless, like an embedded Linux device. In addition to host-to-host communication, we also may wish to segment the components in the LBR into separate processes, for example to reimplement serial communication in C.

Remote GUI host and LBR

We wish to minimize the processing required by the LBR due to its limited resources, particularly for a sub-Linux implementation. Therefore, the LBR pushes information to the Remote GUI. However, the GUI also may signal to the LBR, for example to toggle DAGroot status.

Although the LBR is the messaging client, the GUI must make the initial contact to the LBR to establish a tunnel for communication. It simply is not feasible to expect a PC user to open a port for access, even ports 80/443. However, we plan to use a reverse SSH tunnel, as shown in the diagram above. The reverse nature of the tunnel allows the LBR to access the server port on the GUI host.

We implement messaging with the bi-directional WebSockets (RFC 6455) protocol. As a standard, it supports OpenWSN's implementation approach. The LBR includes a UI Proxy component that pushes mote status reports and event counts, and receives notification of UI events. We encapsulate this messaging in JSON, as described in detail later on this page.

Implementation Tools

LBR

RoleToolNotes
WebSocket clientwebsocket-clientv0.11; Installed with pip.

GUI Host

RoleToolNotes
WebSocket servergevent-websocketv0.3.6; Includes dependencies below
server dependencygeventv0.13.8; Python network library; statically linked with libevent
gevent dependencygreenletv0.4.1; Python concurrency library
Tunnel to LBRPuTTY

Command line:
plink -ssh -N -T -R 8000:localhost:8000 -pw <xxx> user@lbrAddr

For production use, replace password with public key authentication.

For reference, installed the GUI host on Windows 7 Home Professional.

Booting WebSocket Connection

The GUI host must signal to the LBR when it wishes to connect. This signal must trigger the LBR to connect to the GUI hosts's WebSocket server. We leverage the POSIX signal facility to accomplish this goal.

Remote GUILBR
 On startup, create a signal handler on SIGUSR1

At some later time...
On startup:

  1. start WebSocket server on port 8000
  2. run plink to forward port 8000 to LBR
  3. run 2nd plink to execute a remote command on LBR, which sends the signal
    pkill -SIGUSR1 -f "python openVisualizerCli"
 
 Signal received, and handler connects to WebSocket server on forwarded port
ws://localhost:8000/lbr

The signal above assumes the LBR is in the form of the openVisualizerCli module, which probably is the simplest solution.

JSON Messages

JSON provides a convenient and widely accepted mechanism for implementation of the structured messages we must pass between LBR and GUI host.

Messages include:

MessageDescription
mote-inventoryAn array of the connected motes. Sent just after establishing the connection to the GUI host.
mote-statusAn object with the elements of a Mote Status report. Like OpenVisualizer, these reports are generated at some defined rate.
event-countAn object with the current count of each event.

Messaging Between Processes

As future work for the embedded LBR, we plan to use the native POSIX message queue for inter-process messaging, due to its built-in availability. The posix_ipc module for Python provides access to this functionality.

Alternative Messaging Approaches

We researched the projects below, but they are not our first choice. We may investigate further at a later date.

Nanomsg and ZeroMQ provides a common messaging API for a variety of endpoints – threads, processes, and machines. They are open source, but not standards. However, it would be efficient to use a single tool for messaging across all domains.

Nanomsg is developed by Martin Sústrik, the original author of ZeroMQ. Sústrik left ZeroMQ due to a dispute over trademarks, as described in a Linux Weekly News article.

Celery provides a distributed message queue, but is more heavy-duty and requires a message broker.