Human Brain’s Light Processing Ability Could Lead to Better Robotic Sensing

The human brain often serves as inspiration for artificial intelligence (AI), and that is the case once again as a team of Army researchers has managed to improve robotic sensing by looking at how the human brain processes bright and contrasting light. The new development can help lead to the collaboration between autonomous agents and humans.

According to the researchers, it is important for machine sensing to be effective across changing environments, which leads to developments in autonomy.

The research was published in the Journal of Vision. 

100,000-to-1 Display Capability

Andre Harrison is a researcher at the U.S. Army Combat Capabilities Development Command’s Army Research Laboratory. 

“When we develop machine learning algorithms, real-world images are usually compressed to a narrower range, as a cellphone camera does, in a process called tone mapping,” Harrison said. “This can contribute to the brittleness of machine vision algorithms because they are based on artificial images that don’t quite match the patterns we see in the real world.” 

The team of researchers developed a system with 100,000-to-1 display capability, which enabled them to gain insight into the brain’s computing process in the real-world. According to Harrison, this allowed the team to implement biological resilience into sensors.

The current vision algorithms still have a long way to go before becoming ideal. This has to do with the limited range in luminance, at around 100-to-1 ratio, due to the algorithms being based on human and animal studies with computer monitors. The 100-to-1 ratio is less-than-ideal in the real world, where the variation can go all the way up to 100,000-to-1. This high ratio is termed high dynamic range, or HDR.

Dr. Chou Po Hung is an Army researcher. 

“Changes and significant variations in light can challenge Army systems — drones flying under a forest canopy could be confused by reflectance changes when wind blows through the leaves, or autonomous vehicles driving on rough terrain might not recognize potholes nor other obstacles because the lighting conditions are slightly different from those of which their vision algorithms were trained,” Hung said. 

The Human Brain’s Compressing Capability

The human brain is capable of automatically compressing the 100,000-to-1 input into a narrower range, and this is what allows humans to interpret shape. The team of researchers set out to understand this process by studying early visual processing under HDR. The team looked toward simple features such as HDR luminance. 

“The brain has more than 30 visual areas, and we still have only a rudimentary understanding of how these areas process the eye’s image into an understanding of 3D shape,” Hung continued. “Our results with HDR luminance studies, based on human behavior and scalp recordings, show just how little we truly know about how to bridge the gap from laboratory to real-world environments. But, these findings break us out of that box, showing that our previous assumptions from standard computer monitors have limited ability to generalize to the real world, and they reveal principles that can guide our modeling toward the correct mechanisms.” 

By discovering how light and contrast edges interact in the brain’s visual representation, algorithms will be more effective at reconstructing the 3D world under real-world luminance. When estimating 3D shape from 2D information, there are always ambiguities, but this new discovery allows for them to be corrected.

“Through millions of years of evolution, our brains have evolved effective shortcuts for reconstructing 3D from 2D information,” Hung said. “It’s a decades-old problem that continues to challenge machine vision scientists, even with the recent advances in AI.”

The team’s discovery is also important for the development of AI-devices like radar and remote speech understanding, which utilize wide dynamic range sensing. 

“The issue of dynamic range is not just a sensing problem,” Hung said. “It may also be a more general problem in brain computation because individual neurons have tens of thousands of inputs. How do you build algorithms and architectures that can listen to the right inputs across different contexts? We hope that, by working on this problem at a sensory level, we can confirm that we are on the right track, so that we can have the right tools when we build more complex Als.”