Scientists Detect Loneliness Through The Use Of AI And NLP

Researchers from the University of California San Diego School of Medicine have made use of artificial intelligence algorithms to quantify loneliness in older adults and determine how older adults might express loneliness in their speech.

Over the past twenty years or so, social scientists have described a trend of rising loneliness in the population. Studies done over the past decade in particular have documented rising loneliness rates across large swaths of society, which has impacts on depression rates, suicide rates, drug use, and general health. These problems are only exacerbated by the Covid-19 pandemic, as people are unable to safely meet up and socialize in person. Certain groups are more vulnerable to extreme loneliness, such as marginalized groups and older adults. As MedicalXpress reported, one study done by UC San Diego found that senior housing communities had loneliness rates approaching 85% when counting those who reported experiencing moderate or severe loneliness.

In order to determine solutions to this problem, social scientists need to get an accurate view of the situation, determining both the depth and breadth of the issue. Unfortunately, most methods of gathering data on loneliness are limited in notable respects. Self-reporting, for instance, can be biased towards the more extreme cases of loneliness. In addition, questions that directly ask study participants to quantify how “lonely” they feel can sometimes be inaccurate due to social stigmas surrounding loneliness.

In an effort to design a better metric for quantifying loneliness, the authors of the study turned to natural language processing and machine learning. The NLP methods used by the researchers are used alongside traditional loneliness measurement tools, and its hoped that analyzing the natural ways people use language will lead to a less biased, more honest representation of people’s loneliness.

The new study’s senior author was Ellen Lee, assistant professor of psychiatry at the School of Medicine, UC San Diego. Lee and the other researchers focused their study on 80 participants between the ages of 66 to 94. Participants in the study were encouraged by the researchers to answer questions in a way that was more natural and unstructured than most other studies. The researchers weren’t just asking questions and classifying answers. As the first author Ph.D. Varsha Badal, explained that using machine learning and NLP allowed the research team to take these long-form interview answer and find how subtle word choice and speech patterns could be indicative of loneliness when taken together:

“NLP and machine learning allow us to systematically examine long interviews from many individuals and explore how subtle speech features like emotions may indicate loneliness. Similar emotion analyses by humans would be open to bias, lack consistency, and require extensive training to standardize.”

According to the research team, individuals who were lonely had noticeable differences in the ways they responded to the questions compared to non-lonely respondents. Lonely respondents would express more sadness when asked questions regarding loneliness and had longer responses in general. Men were less likely to admit feeling lonely than women. In addition, men were more likely to use words expressing joy or fear than women were.

The researchers of the study explained that the results helped elucidate the differences between typical research metrics for loneliness and the way individuals subjectively experience and describe loneliness. The results of the study imply that loneliness could be detected through the analysis of speech patterns, and if these patterns prove to be reliable they could help diagnose and treat loneliness in older adults. The machine learning models designed by the researchers were able to predict qualitative loneliness with approximately 94% accuracy. More research will need to be conducted to see if the model is robust and if its success can be replicated. In the meantime, members of the research team are hoping to explore how NLP features might be correlated with wisdom and loneliness, which have an inverse correlation in older adults.