AI Behind GPT-3 Could Help Detect Alzheimer’s

The artificial intelligence (AI) that powers the ChatGPT program could eventually help medical professionals detect Alzheimer’s Disease in its early stages. ChatGPT has been receiving a lot of attention for its ability to generate humanlike written responses.

The new research comes from Drexel University’s School of Biomedical Engineering, Science and Health Systems. It demonstrated that OpenAI’s GPT-3 program can identify clues from spontaneous speech that are 80% accurate in predicting the early stages of dementia.

The research was published in the journal PLOS Digital Health.

Using Language Diagnostic Programs

For many, the challenge of diagnosing Alzheimer’s Disease has been its lack of a one-size fits all test, but new research is offering therapists hope by introducing language diagnostic programs that provide an effective way to quickly screen for symptoms associated with dementia — from hesitation in speech and difficulty expressing oneself properly to forgetting words or their meanings. Such tests could make early diagnosis simpler than ever before.

Hualou Liang, PhD, is a professor in Drexel’s School of Biomedical Engineering, Science and Health Systems and a co-author of the research.

“We know from ongoing research that the cognitive effects of Alzheimer’s Disease can manifest themselves in language production,” Liang said. “The most commonly used tests for early detection of Alzheimer’s look at acoustic features, such as pausing, articulation and vocal quality, in addition to tests of condition. But we believe the improvement of natural language processing programs provide another path to support early identification of Alzheimer’s.”

OpenAI’s GPT-3

GPT-3, OpenAI's third iteration of their General Pretrained Transformer (GPT), has harnessed the power of deep learning to revolutionize language tasks. With this algorithm trained on a wide array of data from online sources that highlight how words are used and fit together, GPT-3 produces responses comparable with those created by humans -from responding to questions to creating poems or essays.

Felix Agbavor is a doctoral researcher and lead author of the paper.

“GPT3’s systemic approach to language analysis and production makes it a promising candidate for identifying the subtle speech characteristics that may predict the onset of dementia,” Agbavor said. “Training GPT-3 with a massive dataset of interviews — some of which are with Alzheimer’s patients — would provide it with the information it needs to extract speech patterns that could then be applied to identify markers in future patients.”

The researchers tested their theory by training the program with a set of transcripts that came from a portion of a dataset of speech recordings created with the support of the National Institutes of Health. These transcripts were specifically for the purpose of testing the ability of natural language processing (NLP) programs to predict dementia. The program captured certain characteristics of the word-use, sentence structure, and meaning from the text, which helped it produce an “embedding,” or a characteristic profile of Alzheimer’s speech.

Creating a Screening Machine for Alzheimer’s

The team then re-trained the program with the embedding, which turned it into a screening machine for Alzheimer’s. The program was tested by reviewing dozens of transcripts from the dataset to decide whether or not each one was from someone who was developing Alzheimer’s.

The group found that GPT-3 performed better than two other top NLP programs in terms of accurately identifying Alzheimer’s examples, identifying non-Alzheimer’s examples, and with fewer missed cases.

A second test uses the textual analysis of GPT-3 to predict the score of various patients from the dataset on a common test for predicting the severity of dementia. This common test is called the Mini-Mental State Exam (MMSE).

GPT-3’s prediction accuracy was compared to that of an analysis using just the acoustic features of the recordings, which includes pauses, voice strength and slurring, to predict the MMSE score. GPT-3 was able to ahcive about 20% more accuracy in predicting patients’ MMSE scores.

“Our results demonstrate that the text embedding, generated by GPT-3, can be reliably used to not only detect individuals with Alzheimer's Disease from healthy controls, but also infer the subject's cognitive testing score, both solely based on speech data,” the team noted. “We further show that text embedding outperforms the conventional acoustic feature-based approach and even performs competitively with fine-tuned models. These results, all together, suggest that GPT-3 based text embedding is a promising approach for AD assessment and has the potential to improve early diagnosis of dementia.”

The researchers now plan on developing a web application that can be used at home or in a doctor’s office as a pre-screening tool.

“Our proof-of-concept shows that this could be a simple, accessible and adequately sensitive tool for community-based testing,” Liang said. “This could be very useful for early screening and risk assessment before a clinical diagnosis.”