September 8, 2020 — The following was released by NOAA Fisheries:
The Northeast Fisheries Science Center’s annual sea scallop research survey uses a towed sampling device called the HabCam. It collects approximately 5 million images of the ocean bottom off the Northeast United States. Scientists and volunteers then manually examine an astonishing 100,000 of these images, roughly 2 percent of the number gathered. They focus on identifying just four targets: sea scallops, fish, crabs, and whelks.
So, a wealth of data is going uncollected owing to the sheer volume available and just how labor-intensive pulling it out of images can be.
Researchers have turned toward finding ways for machines to help identify sea life in these images, faster and more efficiently than humans can. This would improve population data for sea scallops. By more thoroughly examining each image, all kinds of information about other sea life and their habitats can also be captured.
Enter Dvora Hart, an operations research analyst at the Northeast Fisheries Science Center. Hart has been at the forefront of estimating sea scallop populations numbers from images taken by devices like the HabCam. The upcoming 2020 sea scallop assessment will once again use improved population data collected from images.
“HabCam gives us photos of animals in their natural environment without disturbing them; however, much of the information in the images is not collected because human annotators can only mark a small percentage of the available images,” said Hart. “Automated annotators can mark all the images and, given proper training, can identify a multitude of different targets—not just sea scallops, fish, crabs and whelks.”
Hart is part of an interdisciplinary team that developed the world’s first advanced automated image analysis software for the marine environment. The Video and Image Analytics for the Marine Environment, VIAME for short, uses convolutional neural networks—a recent advance in artificial intelligence. These networks teach computers to recognize species and features of their habitats in the images taken by the HabCam.
The work is so significant that Hart and her team won a 2019 Department of Commerce gold medal for their work. This recognition is the highest honor award offered to department employees.