What Is the Law of Accelerating Returns? How It Leads to AGI

In a recent interview when asked when he expects to see the advent of Artificial General Intelligence (AGI), Elon Musk replied “3 to 6 years”. Google's DeepMind CEO Demis Hassabis now believes AGI is “a few years, maybe within a decade away” as stated at the The Wall Street Journal’s Future of Everything Festival.

These numbers are considered to be optimistic compared to most AI industry pundits who believe that AGI is often a decade, if not a century away. Some of this pessimism is from fear of committing to a shorter timeline to only be eventually proven wrong. After all in 1956, at the Dartmouth Summer Research Project the term “Artificial Intelligence” was coined and started as a field, with the expectation that a machine as intelligent as a human being would exist in no more than a generation (25 years).

Others such as Geoffrey Hinton who is known as the godfather of AI have a slightly more nuanced view. “Until quite recently, I thought it was going to be like 20 to 50 years before we have general-purpose AI. And now I think it may be 20 years or less.”

The AI industry has advanced rapidly over the past few year thanks to the rapid development of deep reinforcement learning algorithms, many that power today's Large Language Models (LLMs).

Nonetheless, all of these breakthroughs have only led to narrow AI applications such as chatbots, and language translation. This is in comparison to AGI, a type of artificial intelligence that possesses the ability to understand, learn, and apply knowledge across a wide array of tasks at a level comparable to that of a human being.

The missing link to AGI for many seems unattainable, but to a few who believe in what is called “The Law of Accelerating Returns”, it is inevitable that we will eventually build an AGI.

The Law of Accelerating Returns was conceptualized by none other than Ray Kurzweil, author, inventor, and futurist. He is involved in fields such as optical character recognition (OCR), text-to-speech synthesis, speech recognition technology, and he was hired by Google after publishing his AI Book “How to Create a Mind”. This groundbreaking book illustrates how we need to understand the human brain in order to reverse engineer it to create the ultimate thinking machine. This book was so instrumental to the future of AI, that Eric Schmidt recruited Ray Kurzweil to work on AI projects after he finishing reading this seminal book. 

The most relevant Ray Kurzweil book is none other than “The Singularity is Near“, since being published in 2005, its predictions have mirrored technological growth over the past 2 decades. Most importantly Ray Kurzweil predicts that we will achieve AGI by 2029, a timeline that is in line with the recent opinion shared by Elon Musk and Demis Hassabis.

The law posits that the rate of change in a wide variety of evolutionary systems (including but not limited to the growth of technologies) tends to increase exponentially.

In the context of technological growth, the law implies that we can expect rapid technological advances in the future because the pace of technological innovation is itself accelerating. Ray Kurzweil argues that each new generation of technology builds on the previous one, increasing the potential for innovation at an exponential rate.

This law showcases how an explosive growth of accelerating technologies, which is currently led by Generative AI, will ride other waves of other converging exponential technologies such as chip manufacturing, and 3-D printing.  This convergence is the catapult for AI to become the most powerful application ever built.

In 2001, Ray Kurzweil predicted the following:

An analysis of the history of technology shows that technological change is exponential, contrary to the common-sense “intuitive linear” view. So we won’t experience 100 years of progress in the 21st century — it will be more like 20,000 years of progress (at today’s rate). The “returns,” such as chip speed and cost-effectiveness, also increase exponentially. There’s even exponential growth in the rate of exponential growth. Within a few decades, machine intelligence will surpass human intelligence, leading to The Singularity — technological change so rapid and profound it represents a rupture in the fabric of human history. The implications include the merger of biological and nonbiological intelligence, immortal software-based humans, and ultra-high levels of intelligence that expand outward in the universe at the speed of light.

This technological explosion is due to Moore's Law which predicted that the number of transistors on a given chip would double approximately every two years. This compounded with other technological breakthrough illustrates that the Law of Accelerating Returns is thriving.  These are Ray Kurzweil observations for what this will mean for the future of humanity:

• Evolution applies positive feedback in that the more capable methods resulting from one stage of evolutionary progress are used to create the next stage. As a result, the
• rate of progress of an evolutionary process increases exponentially over time. Over time, the “order” of the information embedded in the evolutionary process (i.e., the measure of how well the information fits a purpose, which in evolution is survival) increases.
• A correlate of the above observation is that the “returns” of an evolutionary process (e.g., the speed, cost-effectiveness, or overall “power” of a process) increase exponentially over time.
• In another positive feedback loop, as a particular evolutionary process (e.g., computation) becomes more effective (e.g., cost effective), greater resources are deployed toward the further progress of that process. This results in a second level of exponential growth (i.e., the rate of exponential growth itself grows exponentially).
• Biological evolution is one such evolutionary process.
• Technological evolution is another such evolutionary process. Indeed, the emergence of the first technology creating species resulted in the new evolutionary process of technology. Therefore, technological evolution is an outgrowth of–and a continuation of–biological evolution.
• A specific paradigm (a method or approach to solving a problem, e.g., shrinking transistors on an integrated circuit as an approach to making more powerful computers) provides exponential growth until the method exhausts its potential. When this happens, a paradigm shift (i.e., a fundamental change in the approach) occurs, which enables exponential growth to continue.

Readers should read Kurzweil's blog, afterwards they should reflect on the implications of this exponential growth, and how it matches and differs from what they have personally experienced since the blog was initially published.

The Law of Accelerating Returns while not as popular as Moore's Law, remains as relevant today as when it was initially published.