During development of the simulation cross-compile framework, we saw a significant number of desynchronization errors when running the simulation. We also saw process memory usage increase. This page provides test results to quantify these issues and decide whether to issue a pull request for the cross-compilation work.
Based on these results, I plan to proceed with the pull request. However, I recommend the amd64-windows firmware be labeled experimental and that we solicit user feedback. I also plan to file a bug for the memory usage.
Test Runs
A simulation test run used a three-mote simulation, with two nodes one hop away from the RPL root. The link quality between each node to the root was set to a 1.0 PDR. Each test run lasted approximately six hours. We used the OpenVisualizer develop_SW-112 branch to run the simulation.
Guide to table columns:
- Arch-OS, CPU Firmware architecture, and the Intel CPU architecture on which the test was run. Windows tests were run on Windows 7.
- FW Branch Most tests run on the FW-177i branch of the kb2ma repository, but a couple of runs used the current openwsn-fw develop branch.
- Elapsed ASN Difference between ASN at end vs. start of run, on RPL root.
- Memory Consumed Process memory use as reported by Windows Task Manager or Linux pidstat. Top value is normalized on elapsed ASN.
- Desync Errors Count of desynchronization errors reported in OpenVisualizer log. Top value is normalized on elapsed ASN.
| Arch-OS, CPU | FW Branch | Elapsed ASN | Memory Consumed | Desync Errors |
|---|---|---|---|---|
| x86-windows, i3 | FW-177 | 14.9M slots | 132K slots/MiB 113 MiB | 877K slots/error 17 errors |
| x86-windows, i3 | develop | 16.0M slots | 133K slots/MiB | 593K slots/error 27 errors |
| x86-windows, i7 | FW-177 | 26.3M slots | 199K slots/MiB | 560K slots/error 47 errors |
| amd64-windows, i7 | FW-177 | 27.7M slots | 73.8K slots/MiB 375 MiB | 577K slots/error 48 errors |
| amd64-windows, i7 | FW-177 | 27.2M slots | 73.9K slots/MiB 369 MiB | 567K slots/error 48 errors |
| amd64-windows, i3 | FW-177 | 15.7M slots | 73.7K slots/MiB 214 MiB | 563K slots/error 28 errors |
| amd64-linux, i5 | FW-177 | 12.6M slots | 68.6K slots/MiB 184 MiB | 843K slots/error |
| amd64-linux, i5 | FW-177 | 10.5M slots | 68.9K slots/MiB 152MiB | 955K slots/error 11 errors |
| amd64-linux, i5 | develop | 10.5M slots | 68.7K slots/MiB | 501K slots/error 21 errors |
The first run of amd64-windows on the i3 CPU failed after around four hours. When we observed the run after six hours, ASN was not increasing, but process memory was fluctuating within a few KB, and CPU usage was 0. There was no output on the terminal to indicate an error occurred.
Discussion
The x86-windows FW-177 firmware performed no worse than the develop firmware. The error rate on the amd64-windows was consistently at 500K - 600K slots/error, but some of the x86-windows and amd64-linux runs showed this same error rate.
There is clearly a memory leak in the OpenVisualizer process, regardless of the firmware used, architecture, or operating system.
It is surprising that the i5-based Linux simulations ran significantly slower than the i3-based Windows simulations.