As reported by Nature, a new AI competition will be occurring soon, the MineRL competition, which will encourage AI engineers and coders to create programs capable of learning through observation and example. The test case for these AI systems will be the highly popular crafting and survival video game Minecraft.
Artificial intelligence systems are have seen some recent impressive accomplishments when it comes to video games. Just recently an AI beat out the best human players in the world at the strategy game StarCraft II. However, StarCraft II has definable goals that are easier to break down into coherent steps that an AI can use to train. A much more difficult task is for an AI to learn how to navigate a large, open-world sandbox game like Minecraft. Researchers are aiming to help AI programs learn through observation and example, and if they are successful they could substantially reduce the amount of processing power needed to train an artificial intelligence program.
The participants in the competition will have four days to create an AI that will be tested with Minecraft, taking up to eight million steps to train their AI. The goal of the AI is to find a diamond within the game by digging. Eight million steps of training is a much shorter time span than the amount of time needed to train powerful AI models these days, so the participants in the competition need to engineer methods that drastically improve over current training methods.
The approaches being used by the participants are based on a type of learning called imitation learning. Imitation learning stands in contrast with reinforcement learning, which is a popular method for training sophisticated systems like robotic arms in factories or the AIs capable of beating human players at StarCraft II. The primary drawback to reinforcement learning algorithms is the fact that they require immense computer processing power to train, relying on hundreds or even thousands of computers linked together to learn. By contrast, imitation learning is a much more efficient and less computationally expensive method of training. Imitation learning algorithms endeavor to mimic how humans learn by observation.
William Guss, a PhD candidate in deep-learning theory at Carnegie Mellon University explained to Nature that getting an AI to explore and learn patterns in an environment is a tremendously difficult task, but imitation learning provides the AI with a baseline of knowledge, or good prior assumptions, about the environment. This can make training an AI much quicker in comparison to reinforcement learning.
Minecraft serves as a particularly useful training environment for multiple reasons. One reason is that Minecraft allows players to use simple building blocks to create complex structures and items, and the many steps needed to create these structures serve as tangible markers of progress that researchers can use as metrics. Minecraft is also extremely popular, and because of this, it is comparatively easy to gather training data. The organizers of the MineRL competition recruited many Minecraft players to demonstrate a variety of tasks like creating tools and braking apart blocks. By crowdsourcing the generation of data, researchers were able to capture 60 million examples of actions that could be taken in the game. The researchers gave approximately 1000 hours of video to the competition teams.
Use the knowledge that humans have built up, says Rohin Shah, Ph.D. candidate in computer science at the University of California, Berkeley explained to Nature that this competition is likely the first to focus on using the knowledge that humans have already generated to expedite the training of AI.
Guss and the other researchers are hopeful that the contest could have results with implications beyond Minecraft, giving rise to better imitation learning algorithms and inspiring more people to consider imitation learning as a viable form of training an AI. The research could potentially help create AIs that are better capable of interacting with people in complex, changing environments.