Prev-Talk | Next-Talk | All-Talks | Talks-Sorted | MOOS-DAWG'22

Talk-19: MassMIND: A semantically segmented labeled dataset of long wave infrared images in marine environment

Speakers: Shailesh Nirgudkar (UMass Lowell), Michael DeFilippo (MIT), Michael Sacarny (MIT), Michael Benjamin (MIT) and Paul Robinette (UMass Lowell)

Advances in deep learning technology have accelerated the autonomy in ground vehicles. Surface vehicles used for monitoring, surveillance can benefit from such autonomy. Historically radar has been used to sense the surrounding of the vehicle and avoid obstacles. But deep learning when applied to computer vision/perception has produced excellent outcome resulting in surge of optical sensor usage on all types of vehicles. These sensors have small form factors and great resolution and the results from them are easily interpretable. However, optical sensors do not work well in extreme light conditions. Marine environment is replete with bright light, reflections, glitter in the water during the day and very low light during nighttime. Long wave infrared sensors depend only on thermal emission of the objects and do not depend on the visible spectrum and thus give great results in such extreme light conditions. So, they can complement optical sensors quite effectively. In this presentation, we describe the first semantically segmented labeled dataset of long wave infrared images captured in busy surrounding of Boston harbor. The dataset known as MassMIND (Massachusetts Marine Infrared Dataset) contains 2900 labeled images. The raw images were shortlisted from the recordings done on Philos R/V during 2019-2021. Each image is segmented into 7 classes - sky, water, background, obstacle, living obstacle, bridge and self. These classes capture the most meaningful features of marine environment. The dataset is open for researchers to develop the state-of-the-art domain specific deep learning models which can be used in real time for obstacle avoidance.


  • USVs,
  • Sensing/Perception