Prev-Talk | Next-Talk | All-Talks | Talks-Sorted

Talk-01: Advanced MOOS-IvP Autonomy through Embedded Computer Vision with Merlin

Anthony Spears, Prioria Robotics Inc.

This presentation discusses development of the Merlin embedded navigation and computer vision system and its power to provide advanced capabilities to MOOS-IvP autonomy. Complex algorithmic tasks such as SLAM, computer vision, sonar processing, and advanced autonomy can provide powerful capabilities to autonomous systems. However, such capabilities are often computationally prohibitive and therefore limited on standard embedded computing systems (e.g. Gumstix). When designed into autonomous platforms, these standard COTS embedded computing solutions also require integration effort for control (e.g. actuators, thrusters) and navigation (e.g. GPS , INS) hardware and software.

An embedded navigation, vision and autonomy computing solution, Merlin, has been developed to specifically target the limitations of such autonomous platform applications. An overview of this system and an example target-track and follow MOOS-IvP application will be presented here. This solution tightly couples the capabilities of a Pixhawk PX4 autopilot with an NVIDIA Tegra X2 GPU/CPU embedded processor to facilitate development of advanced autonomy applications in these systems. This tightly-coupled Merlin solution provides the ability to integrate computationally-intensive (e.g. vision- based) applications into the autonomy GNC loop in real time. The use of open source and COTS building blocks in this design allows for rapid development, flexibility, and a wide support base.

The autopilot side of the Merlin system provides RC control, navigational sensors (e.g. inertial, GPS) and UARTs, CAN, I2C, servo PWM, etc. for external actuators and sensors as well as firmware to support GNC including navigational sensor fusion. The embedded computing side of Merlin provides a Tegra X2 with a 256-core GPU, quad-core ARM CPU, as well as USB3 and Ethernet interfaces. A target-track and follow MOOS-IvP mission was developed in which the computer vision application is run in real-time on the Tegra X2 processor (at up to 30-60 fps) while publishing the heading of the target to the MOOSDB. The MOOS-IvP autonomy then makes GNC decisions based on the relative heading between ownship and the target. Other example applications of interest will also be presented.


  • Computer Vision
  • ASVs
  • Cross Domain UxVs
  • Command and Control, Mission Planning