Volvo Designs Driving Simulator Using Video Game Engine To Train Autonomous Vehicles

More autonomous vehicle researchers and engineers are turning to video games to enhance the performance and reliability of autonomous vehicles. According to TNW, Volvo has begun to utilize video games to assist in the development of autonomous vehicles and the AI algorithms that will drive them.

Video games, artificial intelligence, and cars have an interesting history together. AI algorithms have been used to automate the behavior of vehicles in car racing games, and video games have been used to train autonomous vehicle systems in the past.

In order to safely operate in the real world, an autonomous vehicle must be trained to safely navigate a wide breadth of events and scenarios. The vehicles can be trained on either real-world or virtual data. Many companies, like Waymo, Tesla, and Nuro train their vehicles in real-world testing environments, exposing their AI models to a number of scenarios that occur in the real world. The assumption is that as AI experiences more and more scenarios, it will become better at handling unexpected events when operating in the real world. Research into the efficacy of using synthetic training data like video games shows that fine-tuning synthetic data can yield training results that are at least as good as results derived using only real-world data.

Obviously, there are a number of safety concerns that must be addressed when training and testing an autonomous vehicle in the real-world. Training autonomous driving agents in a virtual environment is clearly safer than training them in a real-world environment. The drawback to using virtual training environments is that they aren’t as accurate. In order to deal with the accuracy problems associated with using a virtual training environment, Volvo has employed the latest gaming technology to design a new training simulator for its autonomous vehicles.

Volvo has constructed a device intended to be the “ultimate driving simulator”. The simulator is equipped with a VR headset, a moving driving seat, and a steering wheel that provides haptic feedback. The virtual environment that the hardware interfaces are based on the Unity gaming engine. The driving simulator was created in order to help Volvo engineers simulate a variety of different road scenarios with a high degree of accuracy without endangering anyone should a failure occur.

The new, virtual training environment is highly lifelike, able to simulate a variety of different weather and lighting conditions. The system can also test how a vehicle might respond to sudden hazards like the sudden appearance of a moose crossing the road. The virtual environments are also capable of mimicking subtle human actions and behaviors. The goal is to have a highly realistic environment where scenarios that are too dangerous or too rare to test in real life can be reliably tested.

Beyond the capability to test autonomous vehicle components, the system can be used to test how humans react to certain scenarios through the use of the haptic feedback and VR system, enabling a better understanding of human-computer interactions with the vehicle.

Volvo’s new driving simulator is just the latest project in a long line of projects to use video games to train AI driving models. A variety of academic researchers have used games like Grand Theft Auto 5 to train autonomous driving and computer vision algorithm. However, these AI models existed in a strange legal space, due to using previously existing software to train on. Many AI research projects that made use of GTA5 were later sent cease-and-desist letters by Take-Two, the parent company of Rockstar Games, developer of GTA5. Volvo’s new driving simulator is likely proprietary to avoid such similar legal problems.