Factors like geographical location, water condition, and ocean depth can present significant challenges in diver-operated systems for fish ecosystem analysis. To overcome these hurdles, remote operated vehicles (ROVs) have presented themselves as valuable tools for collecting fish data in the lesser-documented areas of the ocean. Previous E4E systems, like FishSense Lite, target the monitoring of fisheries through citizen science. With the introduction of Scout, our horizons are broadened beyond waters safe for human deployment. Aside from aiming to increase accessibility to those under-documented waters, Scout also hopes to reduce the need for trained experts in the process and allow for longer deployments in broader areas, working towards the goal of more streamlined data collection.

Scout explores the deployment of the existing fish detection, length estimation, and identification pipeline on embedded-edge computing devices paired with underwater camera technology for real-time computation of fish length and species. With a new underwater camera loaned to us from Deep Water Exploration that is specifically designed to correct for underwater aberrations, our first objective for Scout has been to collect and analyze data from this camera. We began by reviving an old watertight FishSense enclosure from a previous FishSense Lite product, along with the Jetson TX2 for the collection of camera calibration data.

DWE’s StellarHD Aquagon, specifically designed for correcting for underwater optics, with a laser mount attached

To calibrate and test the camera’s performance underwater, we have performed weekly pool tests for data collection. Students Theo Darci-Maher and Nykolas Rekasius took the camera along with the old FishSense enclosure into a pool and collected data for camera calibration. Students Annabelle Chen and Sijan Shrestha worked on the pool deck with their PhD mentor, Chris Crutchfield, to provide support and debug with the camera and the TX2.

Theo handling the enclosure, pointing the camera towards the calibration board

Example footage captured from the test, featuring Nyk holding the calibration board

After this initial pool test, we analysed the collected camera footage and determined the need for a better calibration target. Hence, during our second round of testing, we used a segmented calibration board as opposed to a checkered one, making sure to position it in each corner so that we could later quantify the amount of distortion occurring at the edges of the frame. As a fair amount of color distortion was visible in the previous test (see above), we also incorporated calibration of a color card. Since motion blur was a possible concern in the results of the first test, we weighed the enclosure down and moved the calibration boards in front of it rather than moving the enclosure itself.

The segmented calibration board, positioned to capture distortion in the lower left corner of the frame

The color collection card positioned at the center of the screen

After these first few pool tests, the TX2 stopped booting due to a flash error. In response, we swapped the TX2 out for Qualcomm’s Rubik Pi 3 to begin our work towards on-device computation. Because of the substantial power draw and, hence, generated heat from the board, our next pool tests focused on how to best manage the heat of the Pi within the underwater enclosure. The existing enclosure was adapted by Theo to accommodate for the size and structure of the new board, and its accompanying battery pack. To consider the maximum amount of heat that could be generated, we have run CPU stress tests on the board in different physical environments. During these tests, we monitored the onboard temperature and analyzed that data to consider the most effective and reasonably feasible option for heat transfer given the existing enclosure.

The Rubik Pi 3 and its power bank attached to the enclosure's endcap

The Rubik Pi enclosure setup, along with camera

Plotted data from a 30 minute long stress tests

Future work with Scout includes further stress testing of the Rubik Pi’s remaining processing units (GPU, NPU), analysis of our current camera data, testing in more realistic conditions, and potentially exploring other commercially available computer-operated cameras. With these changes leading to future deployments and collaborations, we will expand FishSense’s ability to improve accessibility of data collection in underdocumented ocean ecosystems!

A (very official) post-pool testing debriefing session