Brain Machine Interface Types Mental Handwriting 

Researchers from the Howard Hughes Medical Institute have successfully enabled a brain machine interface to type out the mental handwriting of users for the first time ever. The team deciphered brain activity associated with writing letters by hand to achieve the outcome.

The participant was an individual with paralysis who had sensors implanted in his brain, and the team relied on an algorithm to identify letters as he attempted to write them. The system was able to analyze this and display the text on a screen in real time. 

According to Krishna Shenoy, a Howard Hughes Medical Institute Investigator at Stanford University, this development could enable people with paralysis to rapidly type without the need of their hands. Shenoy was joined by Stanford neurosurgeon Jaimie Henderson. 

The work was published in the journal Nature on May 12.

The participant was able to type 90 characters per minute, which is more than twice the amount previously recorded with a type of brain-machine interface. 

Jose Carmena, a neural engineer at the University of California, Berkeley, says “it’s a big advancement in the field” that can help many different types of people with disabilities. Brain-computer interfaces enable thought to be converted into action. 

“This paper is a perfect example: the interface decodes the thought of writing and produces the action.”

Injuries and Neural Activity

Even though a person might suffer an injury or disease that results in them not being able to walk, grasp, or speak, the brain’s neural activity for such actions remains. Because of this, researchers can use this activity to create systems that benefit these individuals. 

Shenoy’s team has been working on decoding neural activity associated with speech for years now, and they have developed a way for participants to implant sensors and use their thoughts to move a cursor on a screen. 

However, there has been no real effort to do the same for handwriting. 

“We want to find new ways of letting people communicate faster,” says Frank Willett, a neuroscientist in the group.

BCI and Implanted Sensors

The team collaborated with a 65-year-old participant that was enrolled in a BrainGate2 clinical trial. BrainGate2 is testing BCI safety for devices that relay data directly from the brain to a computer. 

Two tiny sensors were implanted by Henderson into the part of the brain responsible for controlling the hand and arm. This enabled the individual to move a robotic arm or cursor through attempts at moving their own paralyzed arm.

The sensors picked up signals from individual neurons when the participant imagined writing, and the machine learning algorithm recognized brain patterns when producing each letter. This system allowed the man to copy sentences and answer questions almost as quickly as someone his age typing on a smartphone. 

According to Willett, the BCI operates fast due to each letter eliciting a highly distinctive activity pattern that can be easily distinguished by the algorithm.

The team will now turn its attention to a participant that cannot speak, and the researchers believe this new system can greatly benefit individuals suffering from paralysis brought on by various conditions.