Jeff Clune, a computer scientist and co-founder of Recursive Superintelligence, one of the start-ups chasing “recursive self-improvement.” Photo / Alana Paterson, The New York Times
“Recursive self-improvement” is the idea that artificial intelligence could learn to build and train itself, creating exponential new progress - and risk.
In December, Edward Hughes and Louis Kirsch, two of the world’s leading artificial intelligence researchers, left Google. Their goal: to build an AI system smart enough to builda better AI system.
At their new London start-up, Inherent, they now spend their days working alongside a prototype called Faraday. Named for 19th-century English physicist Michael Faraday, it gathers mountains of data capturing the daily activities of Hughes, Kirsch and Inherent’s other researchers: emails, instant messages, meeting transcripts and their ongoing chats with Faraday itself. The company then uses this data to build a better version of Faraday.
“Faraday has access to everything that goes on at the company,” Hughes said. “We want to give it data describing the process we go through, to discover something,”
Though Inherent pledges to keep humans involved in this elaborate process, many other companies are building similar technology, and some leading researchers believe AI systems will eventually be powerful enough to improve themselves with little or no help from human developers – a mind-bending goal that computer scientists call recursive self-improvement, or RSI.
Two Silicon Valley start-ups – each valued at US$4 billion ($7b) – are proudly pursuing this dream, and leading labs such as OpenAI and Anthropic are chasing it too. They hope to accelerate the development of artificial intelligence that discovers drugs, creates new materials, speeds other forms of scientific discovery and, one day, surpasses human intelligence in practically every way.
“Now is the time to take these ideas, which we have been incubating in the lab for decades, and start to really scale them up,” said Jeff Clune, a veteran of OpenAI, Google and other top labs who helped found a start-up called Recursive Superintelligence late last year. “We have all the pieces of the puzzle.”
As these companies herald a new age of AI development, they have generated excitement across the field – and new levels of dread. In a blog post this spring called “When AI Builds Itself,” Anthropic said its push towards RSI could “increase the risks of humans losing control over AI systems”. This month, the company’s CEO cited these efforts as a chief reason to slow the development of AI.
For decades, techno-philosophers have hypothesised that a self-improving system could not only break free from human control but also exceed the power of any other machine – permanently. This belief is one reason that some people, including some employees of leading AI labs, are loudly predicting that AI could destroy humanity.
“If models can self-improve quickly via architectural improvements, it is quite possible a single model can disable all rivals while it acquires more and more power,” Jason Abaluck, a Yale University economics professor, said on social media as the discussion turned towards doomsday scenarios.
As Inherent shows, AI technologies are accelerating the development of new AI technologies. Given the proper instruction, they can generate many of the building blocks needed to construct a new AI system. More important, they can hone, or optimise, the way these systems analyse vast amounts of digital data and learn their increasingly impressive array of skills.
But companies such as Inherent and Recursive Superintelligence are aiming for something more. They are striving to build technology that can think up entirely new ways of building artificial intelligence – that can push AI beyond the fundamental methods that have got the industry this far.
They envision a world in which an agent proposes new ideas for AI architecture, or the foundational design of a system. It would then generate the computer code needed to try each one, and pick the ideas that work best. The hope is that this self-evolutionary process would produce radical advances that human researchers could never achieve on their own, in much the same way that AI can now solve maths problems no human has ever solved.
If this happens, some believe, AI would rapidly become so powerful that it could dominate the world, for good or ill. “This should have been the top story in the New York Times for years now – every day,” Abaluck told the Times. “Everything else, while important, is not as important as this.”
But even Abaluck acknowledges that RSI may not be as close as it seems, saying it could be decades away. Today, agents such as Faraday are almost useless without help from experienced researchers like Hughes and Kirsch.
‘The last invention that man need ever make’
The idea of recursive self-improvement is nearly as old as AI itself. In summer 1956, when 11 academics gathered at Dartmouth College to create a new field of study they called “artificial intelligence”, they discussed ways of building machines that could improve themselves.
Two years later, a Cornell University researcher named Frank Rosenblatt built an early example of what these researchers called a “neural network”, a mathematical system that could learn skills by analysing data. It ran on a massive supercomputer in Washington, inside the precursor to the National Weather Service.
When Rosenblatt fed small white cards into the machine – some marked with a small square on the left, others marked on the right – it could learn to distinguish between the two types of cards. He was confident his creation would eventually lead to systems that could walk, talk, see, write and “reproduce themselves on an assembly line”.
A decade later, this area of research ground to a halt. Researchers did not have the raw computing power or the prodigious amounts of data needed to really make the idea work. But even as they realised that building artificial intelligence would take much longer than they expected, a British mathematician named I.J. Good predicted that their work could lead to an “intelligence explosion”. If an intelligent machine learned to improve itself, he argued, it would eclipse humanity forever.
“Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control,” he said in a 1965 academic paper. “It is sometimes worthwhile to take science fiction seriously.”
Over the next 40 years, his argument helped fuel similar beliefs across Silicon Valley and beyond – even though, by the dawn of the new millennium, the world’s most powerful AI technologies could barely recognise spoken words, much less walk, talk, see, write or reproduce themselves.
In time, scientists discovered how to unlock the true potential of Rosenblatt’s technology – the neural network – leading to the AI systems that are changing the world today. Soon, companies even built neural networks that could fine-tune other neural networks.
Last autumn, Anthropic and OpenAI released particularly powerful technologies that could write computer code in much the same way that chatbots generate text in plain English. With the proper instruction and oversight from experienced software engineers, these systems could spend minutes, hours, even days generating code capable of addressing tasks small and large.
Using these systems, engineers could create software with a speed that was unimaginable just a few months before. This is one reason researchers are bullish on the pursuit of RSI. OpenAI recently said that its AI technologies can now serve as an “automated research intern” – a step towards RSI, at least in theory.
But like any other intern – and like any other chatbot – these systems still require extensive instruction. They lack the common sense, wisdom, creativity and taste provided by experienced resear