Meet Stanford students using drones and artificial intelligence to track sharks: Here's how it works

PALO ALTO, Calif. (KGO) -- If you're lucky enough to score a diving trip to Costa Rica, who wouldn't snap a few pictures or take a few videos. But a team of student researchers from Stanford's Robotics believe their images offer a powerful technique for expanding shark conservation.

The expedition to Santa Elena Bay was funded by a grant from the Stanford Woods Institute for the environment.

The team used drones to study Pacific Nurse Sharks and their habitat. They also tracked their field work during an expedition and assembled it into a video, hoping to document the power of drone photography coupled with artificial intelligence.

We assembled the shark chasers back at Stanford to take us behind the scenes.

Field team member Dakota Riemersma says the computer science is as important as the drone hardware.

"I totally agree that one of the strengths of the model and incorporating machine learning and A.I. is that it is generalizable, and that it's also kind of infinitely specializable," says Riemersma.

The targets varied from the habitats around Santa Elena Bay to the marine life like Nurse Sharks that inhabit it.

To test their drone-powered platform, the team first hit the water with diving gear to survey the area close up. Then they compared that data to images they collected with the drone footage in a sort of 3D viewing model.

Team member Mark Leone says their advancing-recognition technology could give the aerial technique even more capabilities.

"We were developing sort of a framework where we can use object detection models, and now we're using something even more advanced called some foundation models that we combine together, to actually automatically detect and track the sharks in the imagery," he says.

A sort of face-to-fin recognition, including size, motion, and potentially behavior.

Colleague Chinmay Lalgudi says the platform could work with multiple drone models or other types of photo-survey technologies. With the potential to fast-track field research and make it easier to work in remote areas.

"Now you don't need all this human expertise and effort and time it takes to annotate this data instead. You could directly apply our pipeline and a future version of our pipeline to then automatically detect marine organisms from sharks to whales to et cetera," says Lalgudi.

But the team believes it all holds the promise of unlocking secrets, by being in the right place, at the right time, with airborne algorithms that can recognize what' they're seeing. And team member Jaden Clark believes those abilities will only become more powerful as the system improves.

"And then it goes out, finds a very rare animal, kind of observes this behavior and then goes back and read it itself. And then continues the survey later. I think that's really a pie in the sky idea that a lot of ecologists have that are using drones. But I think we're on the way to getting there," says Clark.

Dreaming big. With a project that could someday help accelerate our understanding of the marine environment around us.