A group of researchers has recently published a study in Nature, detailing their attempts to use Facebook data to identify possible psychiatric illnesses. As reported by Wired, the researchers were able to construct an AI model that can successfully predict a diagnosis of a mental illness based on messages sent up to 18 before the diagnosis was made official.
In order to create the predictive model, the research team collected data from 223 volunteers. The volunteers agreed to give the researchers access to messages they had sent and images they had posted. The researchers trained a Random Forest model on features extracted from the collected messages and images. The models’ goal was to determine if a participant had a mental health diagnosis, grouping instances into mood disorder diagnoses, schizophrenia spectrum diagnoses, or no mental health diagnosis.
When the researchers analyzed the results, they found that several different features were correlated with mental health disorders. When it came to images, blue colors were associated with a diagnosis of mood disorders. High use of swear words was generally indicative of mental illness, while words like hear, feel, and see (perception words) were associated with a diagnosis of schizophrenia.
In order to determine the success of the AI model, the researchers compared false positives and false negatives. The research team reported that their success rate was between 0.65 to 0.77, with 1 being a perfect score and 0.5 being the average success of a model that randomly guesses. The more recent the messages were, the better the success of the model. However, even when the team of researchers limited themselves to messages that were dated to over a year before a diagnosis, the model still performed much better than chance.
The interesting thing about this level of accuracy is that it’s approximately equivalent to the accuracy of the PHQ-9. PHQ-9 is a diagnostic tool used to screen for depression, asking the test subject 10 questions. If an AI model trained on Facebook data can reliably perform as well as the PHQ-9, it could potentially be used as a diagnostic tool, augmenting currently existing tools used by clinicians.
The study’s lead researcher was an assistant professor at the Feinstein Institutes for Medical Research in Manhasset, New York, Michael Birnbaum. According to Wired, AI tools that use social media data have the potential to make a major difference in how psychiatric illnesses are diagnosed and treated. As Birnbaum was quoted by Wired:
“We now understand this idea that cancer has many different stages. If you catch cancer at Stage I, it’s drastically different than if you catch it once it metastasizes. In psychiatry, we have a tendency to start working with people once it’s already metastasized. But there’s the potential to catch people earlier.”
Essentially, mental illnesses can take different forms at different times and more varied sources of data can help researchers and clinicians triangulate a person’s state of mental health. The advantage of using social media data is that it serves as a continuous record of an individual’s thoughts and feelings. This data could be used to complement the long interviews that clinicians rely on to diagnose a patient.
Birnbaum expects that AI models based on social media data could assist therapists in monitoring patients over the long-term course of their treatment. Birnbaum explained that therapists typically only get a “snapshot” of a person’s life once a month or so and that the ability to use social media data lets clinicians get a more complete, representative understanding of the trends in a person’s life. Birnbaum hopes that within five to ten years the use of social media data in mental health assessment will become more commonplace.