Researchers at Google and DeepMind have built a method called Dream-RSI that helps self-improving AI agents explore more efficiently by replaying their past search runs instead of launching costly new ones, finding better solutions across coding, math, and GPU tasks while using far less compute.<\/p>\n
Self-improving AI agents are designed to discover new algorithms, solve harder math problems, and write faster code on their own. They follow a basic loop: propose a solution, evaluate it, learn from the result, and try again. Over thousands of attempts the system refines its output.<\/p>\n
For complex problems the search space can balloon quickly. The agent has to keep deciding which approaches look promising, which to run in parallel, and which to abandon. That exploration step often decides whether the run succeeds or just burns compute chasing dead ends.<\/p>\n
Dream-RSI changes how the agent searches without touching the underlying AI model. Existing approaches handle exploration in two ways. A fixed search strategy never learns from experience, so the agent can hit the same dead ends over and over. Adapting the strategy during a search avoids that rigidity, but testing every alternative would mean re-running long, expensive computations.<\/p>\n
The Dream-RSI team proposes reusing data from a completed search to test alternative strategies inside the space the agent already explored. As the agent searches, it records each attempt and its outcome, building a replayable log of decisions.<\/p>\n
The researchers compare this to finding your way through an unfamiliar area. On the first visit you hit dead ends, double back, and struggle to find a route. Once you have a mental map, you can plan another route without visiting every spot again.<\/p>\n
Rather than testing a new strategy in a live run, the agent runs it against stored results. It can check what would have happened if it had pursued a different branch first, abandoned some earlier, or rerouted through another path. The system does not invent entirely new solutions during replay. It tests different decisions inside the recorded search tree.<\/p>\n
Because all results are already stored, the agent does not need to generate or evaluate solutions again. Thousands of alternative strategies can be tested without calling the model or evaluator. The researchers call this process “dreaming.” The agent plays through thousands of variations and picks the best one before putting it to work in a live search.<\/p>\n
The process repeats in a loop. After each search, the agent uses the recorded results to test better strategies through replay, then applies the improved strategy to its next live run. Throughout this cycle, only the search strategy changes. The model that generates the candidate solutions stays the same.<\/p>\n
The researchers tested Dream-RSI with Gemini 3.1 Pro and Gemini 3.7 Flash on eight tasks across three areas. Each comparison used a baseline with the same starting conditions but a fixed search strategy.<\/p>\n
p><\/p>\n
One task asked the system to write the fastest possible program for a statistical calculation commonly used in genomics and finance. Dream-RSI’s program ran faster than the established libraries sklearn and glmnet on all six test datasets. With Gemini 3.1 Pro, average runtime fell from 3,587 to 2,931 milliseconds, and the number of attempts dropped from 550 to 317. Dream-RSI also beat a competing system called SimpleTES, which needed 51,200 runs compared to Dream-RSI’s 317 attempts on the same task.<\/p>\n
The same pattern held on math optimization tasks and on writing efficient GPU kernels, with comparable or better results at much lower computational cost. On two GPU tasks, Dream-RSI matched performance while cutting the number of runs by a factor of up to 2.43. On two others, it delivered up to 2.09 times the performance within the same budget.<\/p>\n
The learned strategy also adjusted its effort over time. As performance improved, it initially reduced the number of attempts. When progress stalled, it increased the search effort again, and that increase coincided with further gains.<\/p>\n
In a follow-up analysis, the researchers tested another way to use search histories. Instead of replaying them to test strategies, they condensed the histories into written instructions telling the agent where to search. On one GPU task, the instruction-based version performed worse than the version that used replay instead.<\/p>\n
The researchers suggest that overly specific directions can narrow the search space too much, keeping the agent from exploring a broader range of approaches. The result points at a real tradeoff: guidance can help, but it can also box the agent in.<\/p>\n
Recursive self-improvement has drawn growing attention. Google DeepMind introduced AlphaEvolve in 2025, using the same basic principle: Gemini Flash generates code proposals, Gemini Pro analyzes them, and an evolutionary algorithm selects the best versions. Dream-RSI works one level above that process by optimizing the search strategy itself.<\/p>\n
Other systems take related approaches. AutoTTS uses a coding agent to search for algorithms in a simulated environment, and those algorithms decide when a language model should start, expand, or abandon reasoning paths. The resulting methods beat manually designed methods while using less compute.<\/p>\n
Google Research recently presented WikiSkill, which records failures and successes in a wiki and turns them into reusable instructions. Dream-RSI’s follow-up analysis suggests that explicit instructions like these can restrict exploration on open-ended search tasks. Meta goes further with Hyperagents, which lets agents rewrite the mechanism that controls how they improve.<\/p>\n
The researchers have shared code and more details on GitHub.<\/p>\n
Dream-RSI is a method from researchers at Google and DeepMind that lets self-improving AI agents test thousands of alternative search strategies by replaying past runs, instead of re-running them. The agent picks the best strategy and applies it to the next live search.<\/p>\n
On a genomics-style coding task, Dream-RSI cut average runtime from 3,587 to 2,931 milliseconds with Gemini 3.1 Pro and lowered the number of attempts from 550 to 317. It reached the same goal as a competing system called SimpleTES that needed 51,200 runs. On GPU tasks it matched performance with up to 2.43 times fewer runs, or delivered up to 2.09 times the performance within the same budget.<\/p>\n
When the team condensed past search histories into explicit instructions for the agent, the instruction-based version performed worse than the replay-based version on one GPU task. The researchers suggest that overly specific directions can narrow the search space and limit exploration on open-ended problems.<\/p>\n