Researchers at the University of Colorado and Duke University have developed a neural network to accurately decode images into 11 different human emotion categories. The research team at the universities included Phillip A. Kragel, Marianne C. Reddan, Kevin S. LaBar, and Tor D. Wagner.
Phillip Kragel explains neural networks as computer models that are able to map input signals to an output of interest by learning a series of filters. Whenever a network is trained to detect a certain image or thing, it learns the different features that are unique to it like shape, color, and size.
The new convolutional neural network has been named EmoNet, and it was trained on visual images. The research team used a database that had 2,185 videos and included 27 different emotion categories. From the collection of videos, they extracted 137,482 frames that were divided into training and testing samples. They were not just basic emotions, but they included many complex ones as well. The different emotion categories included anxiety, awe, boredom, confusion, craving, disgust, empathetic pain, entrancement, excitement, fear, horror, interest, joy, romance, sadness, sexual desire, and surprise.
The model was able to detect some emotions like craving and sexual desire at a high confidence interval, but it had trouble with other emotions such as confusion and surprise. To categorize the different images and emotions, the neural network used things such as color, spatial power spectra, and the presence of objects and faces in the images.
In order to build on the research and the neural network, the team studied 18 different people and their brain activity after showing them 112 different images. After showing the real humans the images, the researchers showed the same ones to the EmoNet network to compare the results between the two.
We already use certain apps and programs every day that read our faces and expressions for things like facial recognition, photo manipulation through AI, and to unlock our smartphones. This new development takes that a lot further with the possibility of not only reading a face’s physical features, but now reading a person’s emotions and feelings through their faces. It is an exciting but also concerning development as privacy concerns will surely arise. We already worry about facial recognition and what can happen with that data.
Aside from the dangerous potential regarding privacy concerns, this new technological development can help in many areas. For one, many researchers often rely on participants reporting on their own emotions. Now, researchers can use the image of that participant’s face to learn their emotions. This will reduce the errors in the research and data.
“When it comes to measuring emotions, we’re typically still limited only to asking people how they feel,” said Tor Wagner, one of the researchers on the team. “Our work can help move us towards direct measures of emotion – related brain processes.”
This new research can also help transition mental health labels like “anxiety” to brain processes.
“Moving away from subjective labels such as ‘anxiety’ and ‘depression’ towards brain processes could lead to new targets for therapeutics, treatments, and interventions.” said Phillip Kragel, another one of the researchers.
This new neural network is just one of the new and exciting developments in artificial intelligence. Researchers are constantly pushing this technology further, and it will make an impact in every area of our lives. The new developments in AI are taking it deeper into the different areas of human behavior and emotion. While we mostly know of AI dealing in the physical realm including muscles, arms, and other parts of the body, we are now going into the human psyche with the technology.
Artificial Intelligence Algorithm Used to Predict Agriculture Yield
It is predicted that the precision agriculture market will reach $12.9 billion by 2027. With this increase, there is a need for sophisticated data-analysis solutions that are capable of guiding management decisions in real-time. A new methodology has been developed by an interdisciplinary group at the University of Illinois, and it aims to efficiently and accurately process precision agricultural data.
Nicolas Martin is an assistant professor in the Department of Crop Sciences at Illinois and co-author of the study.
“We’re trying to change how people run agronomic research. Instead of establishing a small field plot, running statistics, and publishing the means, what we’re trying to do involves the farmer far more directly. We are running experiments with farmers’ machinery in their own fields. We can detect site-specific responses to different inputs. And we can see whether there’s a response in different parts of the field,” he says.
“We developed methodology using deep learning to generate yield predictions. It incorporates information from different topographic variables, soil electroconductivity, as well as nitrogen and seed rate treatments we applied throughout nine Midwestern corn fields.”
The team used 2017 and 2018 data from the Data Intensive Farm Management project to help develop their approach. In that project, seeds and nitrogen fertilizer were applied at different rates across 226 fields. Those fields were in different areas of the world, including the Midwest, Brazil, Argentina, and South Africa. High-resolution satellite images were provided by PlanetLab, and they were paired with on-ground measurements in order to predict yield.
The fields were digitally separated into 5-meter squares. The computer was given data on soil, elevation, nitrogen application rate, and seed rate for each square, and it then began to learn how the yield in that square is determined by the interaction of the factors.
In order to complete their analysis, the researchers relied on a convolutional neural network (CNN). A CNN is a type of machine learning or artificial intelligence. While some types of machine learning get computers to add new data into existing patterns, convolutional neural networks do not take existing patterns into account. CNN’s look at data and learn the patterns that are responsible for organizing it, and it works in a similar way to how humans organize information through neural networks within the brain. The CNN approach was able to predict yield with a high accuracy rate, and it was compared to other machine learning algorithms and traditional statistical techniques.
“We don’t really know what is causing differences in yield responses to inputs across a field. Sometimes people have an idea that a certain spot should respond really strongly to nitrogen and it doesn’t, or vice versa. The CNN can pick up on hidden patterns that may be causing a response,” Martin says. “And when we compared several methods, we found out that the CNN was working very well to explain yield variation.”
The use of artificial intelligence to analyze data from precision agriculture is a new field, but it is one that is growing. Agriculture is one of the major industries that will be drastically changed by artificial intelligence, and the use of it is continuing to increase. According to Martin, this experiment is just the start of CNN’s being used in a variety of different applications.
“Eventually, we could use it to come up with optimum recommendations for a given combination of inputs and site constraints.”
Deep Learning Used to Find Disease-Related Genes
A new study led by researchers at Linköping University demonstrates how an artificial neural network (ANN) can reveal large amounts of gene expression data, and it can lead to the discovery of groups of disease-related genes. The study was published in Nature Communications, and the scientists want the method to be applied within precision medicine and individualized treatment.
Scientists are currently developing maps of biological networks that are based on how different proteins or genes interact with each other. The new study involves the use of artificial intelligence (AI) in order to find out if biological networks can be discovered through the use of deep learning. Artificial neural networks, which are trained by experimental data in the process of deep learning, are able to find patterns within massive amounts of complex data. Because of this, they are often used in applications such as image recognition. Even with its seemingly enormous potential, the use of this machine learning method has been limited within biological research.
Sanjiv Dwivedi is a postdoc in the Department of Physics, Chemistry and Biology (IFM) at Linköping University.
“We have for the first time used deep learning to find disease-related genes. This is a very powerful method in the analysis of huge amounts of biological information, or ‘big data’,” says Dwivedi.
The scientists relied on a large database with information regarding the expression patterns of 20,000 genes in a large number of people. The artificial neural network was not told which gene expression patterns were from people with diseases, or which ones were from healthy individuals. The AI model was then trained to find patterns of gene expression.
One of the mysteries surrounding machine learning is that it is currently impossible to see how an artificial neural network gets to its final result. It is only possible to see the information that goes in and the information that is produced, but everything that happens in-between consists of several layers of mathematically processed information. These inner workings of an artificial neural network are not yet able to be deciphered. The scientists wanted to know if there were any similarities between the designs of the neural network and the familiar biological networks.
Mike Gustafsson is a senior lecturer at IFM and leads the study.
“When we analysed our neural network, it turned out that the first hidden layer represented to a large extent interactions between various proteins. Deeper in the model, in contrast, on the third level, we found groups of different cell types. It’s extremely interesting that this type of biologically relevant grouping is automatically produced, given that our network has started from unclassified gene expression data,” says Gustafsson.
The scientists then wanted to know if their model of gene expression was capable of being used to determine which gene expression patterns are associated with disease and which are normal. They were able to confirm that the model can discover relative patterns that agree with biological mechanisms in the body. Another discovery was that the artificial neural network could possibly discover brand new patterns since it was trained with unclassified data. The researchers will now investigate previously unknown patterns and whether they are relevant within biology.
“We believe that the key to progress in the field is to understand the neural network. This can teach us new things about biological contexts, such as diseases in which many factors interact. And we believe that our method gives models that are easier to generalise and that can be used for many different types of biological information,” says Gustafsson.
Through collaborations with medical researchers, Gustafsson hopes to apply the method in precision medicine. This could help determine which specific types of medicine patients should receive.
The study was financially supported by the Swedish Foundation for Strategic Research (SSF) and the Swedish Research Council.
Deep Learning System Can Accurately Predict Extreme Weather
Engineers at Rice University have developed a deep learning system that is capable of accurately predicting extreme weather events up to five days in advance. The system, which taught itself, only requires minimal information about current weather conditions in order to make the predictions.
Part of the system’s training involves examining hundreds of pairs of maps, and each map indicates surface temperatures and air pressures at five-kilometers height. Those conditions are shown several days apart. The training also presents scenarios that produced extreme weather, such as hot and cold spells that can cause heat waves and winter storms. Upon completing the training, the deep learning system was able to make five-day forecasts of extreme weather based on maps it had not previously seen, with an accuracy rate of 85%.
According to Pedram Hassanzadeh, co-author of the study which was published online in the American Geophysical Union’s Journal of Advances in Modeling Earth Systems, the system could be used as a tool and act as an early warning for weather forecasters. It will be especially useful for learning more about certain atmospheric conditions that cause extreme weather scenarios.
Because of the invention of computer-based numerical weather prediction (NWP) in the 1950s, day-to-day weather forecasts have continued to improve. However, NWP is not able to make reliable predictions about extreme weather events, such as heat waves.
“It may be that we need faster supercomputers to solve the governing equations of the numerical weather prediction models at higher resolutions,” said Hassanzadeh, an assistant professor of mechanical engineering and of Earth, environmental and planetary sciences at Rice University. “But because we don’t fully understand the physics and precursor conditions of extreme-causing weather patterns, it’s also possible that the equations aren’t fully accurate, and they won’t produce better forecasts, no matter how much computing power we put in.”
In 2017, Hassanzadeh was joined by study co-authors and graduate students Ashesh Chattopadhyay and Ebrahim Nabizadeh. Together, they set out on a different path.
“When you get these heat waves or cold spells, if you look at the weather map, you are often going to see some weird behavior in the jet stream, abnormal things like large waves or a big high-pressure system that is not moving at all,” Hassanzadeh said. “It seemed like this was a pattern recognition problem. So we decided to try to reformulate extreme weather forecasting as a pattern-recognition problem rather than a numerical problem.”
“We decided to train our model by showing it a lot of pressure patterns in the five kilometers above the Earth, and telling it, for each one, ‘This one didn’t cause extreme weather. This one caused a heat wave in California. This one didn’t cause anything. This one caused a cold spell in the Northeast,'” Hassanzadeh continued. “Not anything specific like Houston versus Dallas, but more of a sense of the regional area.”
Prior to computers, analog forecasting was used for weather prediction. It was done in a very similar way to the new system, but it was humans instead of computers.
“One way prediction was done before computers is they would look at the pressure system pattern today, and then go to a catalog of previous patterns and compare and try to find an analog, a closely similar pattern,” Hassanzadeh said. “If that one led to rain over France after three days, the forecast would be for rain in France.”
Now, neural networks can learn on their own and do not necessarily need to rely on humans to find connections.
“It didn’t matter that we don’t fully understand the precursors because the neural network learned to find those connections itself,” Hassanzadeh said. “It learned which patterns were critical for extreme weather, and it used those to find the best analog.”
To test their concept, the team relied on data taken from realistic computer simulations. They originally reported early results with a convolutional neural network, but the team then shifted towards capsule neural networks. Convolutional neural networks are not able to recognize relative spatial relationships, but capsule neural networks can. These relative spatial relationships are important when it comes to the evolution of weather patterns.
“The relative positions of pressure patterns, the highs and lows you see on weather maps, are the key factor in determining how weather evolves,” Hassanzadeh said.
Capsule neural networks also require less training data than convolutional neural networks.
The team will continue to work on the system in order for it to be capable of being used in operational forecasting, but Hassanzadeh hopes that it eventually will lead to more accurate forecasts for extreme weather.
“We are not suggesting that at the end of the day this is going to replace NWP,” he said. “But this might be a useful guide for NWP. Computationally, this could be a super cheap way to provide some guidance, an early warning, that allows you to focus NWP resources specifically where extreme weather is likely.”
“We want to leverage ideas from explainable AI (artificial intelligence) to interpret what the neural network is doing,” he said. “This might help us identify the precursors to extreme-causing weather patterns and improve our understanding of their physics.”