Google Researches Create System To Evolve New AI Algorithms From Mathematical Building Blocks

Machine learning enables a computer application to become proficient at a wide variety of tasks, but it often takes a long time to design a machine learning architecture from scratch and to then train that algorithm. As reported by ScienceAlert, researchers at Google Brain have recently experimented with new ways of creating AI programs, employing techniques based on mutations that allow AIs to “evolve” organically.

Google’s AutoML system has been automatically creating AI programs for a while now, and many of these programs achieve better performance than models designed by human engineers. However, Google researchers were able to expand upon this system, releasing research that implies the system could be utilized to “discover” new, more efficient and powerful algorithms through a process that mirrors evolution. This process is based on the mutation of mathematical functions and it could also help reduce the human bias that often finds its way into AI systems through data.

The Google research team pre-published a paper last month of arXiv, entitled “Evolving Machine Learning Algorithms From Scratch”. In it, the research team discusses their new system, dubbed AutoML-Zero. Auto-ML Zero operates by tweaking basic mathematical operations, using them as the building blocks for new, sophisticated algorithms. Systems designed with AutoML-Zero could potentially be more powerful and accurate than many other AIs, but the research team tested the process with a specific goal in mind – the fixing of human bias within generic machine learning models and datasets. The researchers describe the problem in their research paper:

“Human-designed components bias the search results in favor of human-designed algorithms, possibly reducing the innovation potential of AutoML. Innovation is also limited by having fewer options: you cannot discover what you cannot search for.”

AutoML-Zero operates with a three-step approach: setup, predict, and learn. AutoML-Zero begins by taking 100 algorithms created through the random combination of simple mathematical operations, then the algorithms are pitted against one another. Once the best performing algorithms are identified, small tweaks are made to these algorithms and then another round of trials is carried out. This process of competition and mutation mimics a “survival-of-the-fittest” selection process.

Reportedly the entire process can be carried out rather quickly, as the system is capable of processing up to 10,000 possible algorithms a second per processor. It can also carry out these trials more or less 24/7, continuing to experiment with very little input from human operators.

Many of the most impressive algorithmic systems today are just slight variations of algorithms that have a long history in computer science and AI, scaled up. According to Haran Jackson, as quoted by Newsweek, the most interesting thing about the new paper is that the system could potentially discover brand new algorithms that radically differ from those that are most widely used.

“There is a sense amongst many members of the community that the most impressive feats of artificial intelligence will only be achieved with the invention of new algorithms that are fundamentally different to those that we as a species have so far devised,” said Jackson. “This is what makes the aforementioned paper so interesting. It presents a method by which we can automatically construct and test completely novel machine learning algorithms.”

AutoML-Zero is still in the proof of concept phase and much more work will need to be done on it before its capable of producing algorithms that are as useful as those that drive today’s most cutting edge AI applications. Nonetheless, the research conducted on the system might prove useful even before AAutoML-Zero is finished, informing how other algorithms are designed by engineers.