Prev-Talk | Next-Talk | All-Talks | Talks-Sorted | MOOS-DAWG'19

Talk-13: Virtual Reality Integration with MOOS-IvP for Manned-Unmanned Teaming

Michael Novitzky, MIT

We present our latest integration of a 3D environment and virtual reality headset for manned-unmanned teaming leveraging MOOS-IvP.† This work is motivated by the desire for an immersive manned-unmanned teaming experience when access to field experiments is untenable due to resource or weather constraints.† Our manned-unmanned teaming testbed called Project Aquaticus is powered by MOOS-IvP.† We have created new applications for human-robot interaction so that a human can communicate with their autonomous robot teammates.† Certainly, pMarineViewer - the default MOOS-IvP simulation visualization and mission command and control application, is a fantastic tool.† However, its top-down view does not simulate the embedded experience the human participant has during on-water human-robot interaction while in their own vehicle. This fully observable representation of the world is well suited for managing operations from the shore and teaching new players game mechanics and strategies; however, it does not encapsulate the limited and almost chaotic view of the world a participant experiences while in their motorized kayak on the water and therefore the results would not transfer between simulation and on-water experiments. In this work, we present our latest iteration of creating a 3D visual simulator along with integration with a virtual reality headset with different yet widely available tools in order to keep costs down, have access to supported hardware, and have a system that is modifiable and accessible so that we can import our 3D models and adapt the code as we see fit. We have previously presented our capability to replay logged experiments from a participantís view using MOOS-IvP logged data (.alog files) using both the 3D environment at a screen and the VR headset using Google Glass. We present our ongoing integration of this VR setup with real-time simulation within MOOS-IvP so that a participant can have a manned-unmanned teaming experience in a virtual world that mimics their real-world experience.


  • Human-Robot Teaming
  • Simulation
  • Aquaticus