AI Uses Reinforcement Learning to Navigate Oceans

Engineers at Caltech, ETH Zurich, and Harvard are working on an artificial intelligence (AI) that can enable autonomous drones to use ocean currents to aid their navigation. With this approach, the drones don’t have to fight through the currents.

The research was published in Nature Communications on December 8.

John O. Dabiri is the Centennial Professor of Aeronautics and Mechanical Engineering and one of the authors of the research. 

“When we want robots to explore the deep ocean, especially in swarms, it's almost impossible to control them with a joystick from 20,000 feet away at the surface. We also can't feed them data about the local ocean currents they need to navigate because we can't detect them from the surface. Instead, at a certain point we need ocean-borne drones to be able to make decisions about how to move for themselves,” says Dabiri.

Testing the AI

The engineers tested the AI’s accuracy with computer simulations, and the team developed a small robot that runs the algorithm on a computer chip, which could power seaborne drones on Earth as well as other planets. Eventually, they could develop an autonomous system that monitors the condition of the planet’s oceans, and it would do this by combining it with prosthetics previously developed to help jellyfish swim on command. 

For this approach to work, the drones must make decisions on their own about where to go and how to get there. They will likely have to rely on the data they collect themselves, which would be in the form of information about the water currents they’re experiencing.

The researchers used reinforcement learning networks to address this, and they wrote software that can run on a small microcontroller. 

The team was able to use a computer simulation to teach the AI to navigate. The simulated swimmer only had access to information about the water currents at its immediate location, but it was able to quickly learn how to exploit vortices in the water to coast toward a target. 

This type of naivation is common among eagles and hawks, which ride thermals in the air while extracting energy from air currents to maneuver. This allows them to move towards a target while saving energy. 

Effective Navigation Strategies

According to the team, their reinforcement learning algorithm could also learn navigation strategies that are more effective than those used by fish in the ocean.

“We were initially just hoping the AI could compete with navigation strategies already found in real swimming animals, so we were surprised to see it learn even more effective methods by exploiting repeated trials on the computer,” says Dabiri.

The researchers will now look to test the AI on each different type of flow disturbance it would encounter in the ocean. They will achieve this by combining their knowledge of ocean-flow physics with the reinforcement learning strategy.

Peter Gunnarson is a graduate student at Caltech and lead author of the paper.

“Not only will the robot be learning, but we'll be learning about ocean currents and how to navigate through them,” says Gunnarson.