Google Deepmind's Dream-RSI helps AI agents improve by “dreaming” about past attempts

Researchers at Google and Deepmind have developed a method that helps AI agents tackle difficult search tasks more efficiently. It uses past search runs to test new strategies without repeating costly computations.

Self-improving AI agents are supposed to one day discover new algorithms, solutions to math problems, or faster code on their own. They follow the same basic process of proposing a solution, evaluating the result, learning from it, and trying again. Over thousands of attempts, they gradually work toward a good result.

For complex tasks, the search space can grow enormous. The agent must constantly decide which promising approaches to pursue, which to try in parallel, and which to abandon. This process, called exploration, can determine whether the search succeeds or wastes compute chasing the wrong ideas.

A research team from Google and Deepmind has introduced "Dream-RSI" to improve those decisions. The method changes how the agent searches, not the underlying AI model.

Existing approaches generally handle exploration in two ways. A fixed search strategy can't learn from experience, so the agent may repeatedly hit the same dead ends. Adapting the strategy during a search avoids that rigidity but comes at a cost. It takes many attempts to find out whether a strategy works, and testing countless alternatives would mean repeating long, expensive runs.

Replaying past searches makes new strategies cheaper to test

The researchers propose reusing data from a completed search to test alternative strategies within the space the agent has already explored. The agent records its attempts and their results as it searches, providing the data needed to replay those decisions later.

The researchers compare this to finding your way through an unfamiliar area. On your first visit, you hit dead ends, double back, and struggle to find a route. Once you have a mental map, though, you can plan another route without visiting every spot again.

Dream-RSI applies that principle to recorded search histories. Rather than testing a new strategy in a live run, the agent runs it against stored results. This lets it check what would have happened if it had pursued other approaches first or abandoned some earlier. The system doesn't invent entirely new solutions during replay; it tests different decisions within the recorded search tree.

Because those results already exist, the agent doesn't need to generate or evaluate solutions again, avoiding the expensive computations a live run would require. That makes testing new search strategies much cheaper. The researchers call this process "dreaming." The agent plays through thousands of variations and selects the best one before putting it to work in a live search.

The process repeats in a loop. After each search, the agent uses the recorded results to test better strategies, then applies the improved version to its next live run. Throughout this cycle, only the search strategy changes; the model generating the solutions remains untouched.

Dream-RSI finds better solutions with fewer attempts

The researchers tested Dream-RSI with Gemini 3.1 Pro and Gemini 3.7 Flash on eight tasks across three areas. Each comparison used a baseline with the same starting conditions but a fixed search strategy.

One task asked the system to write the fastest possible program for a statistical calculation commonly used in genomics and finance. Dream-RSI's program ran faster than the established libraries sklearn and glmnet on all six test datasets.

With Gemini 3.1 Pro, average runtime fell from 3,587 to 2,931 milliseconds, while the number of attempts dropped from 550 to 317. Dream-RSI also outperformed a competing system called SimpleTES, which needed 51,200 runs, compared with Dream-RSI's 317 attempts.

The same pattern held for math optimization tasks and efforts to write efficient GPU kernels, with comparable or better results at much lower computational cost. On two GPU tasks, Dream-RSI matched performance while cutting the number of runs by a factor of up to 2.43. On two others, it delivered up to 2.09 times the performance within the same budget.

Explicit instructions can limit exploration

In a follow-up analysis, the researchers tested another way to use search histories. Instead of replaying them to test strategies, they condensed them into instructions telling the agent where to search.

On one GPU task, the version with these instructions performed worse than the version without them. The researchers suggest that overly specific directions can narrow the search space too much, preventing the agent from exploring a broader range of approaches.

The same analysis showed how the learned strategy adjusted its effort. As performance improved, it initially reduced the number of attempts. When progress stalled, it increased the search effort again, which coincided with further gains. The researchers have shared code and more details on GitHub.

Recursive self-improvement has drawn growing attention lately. Developments in this field are part of why Anthropic CEO Dario Amodei recently warned about the pace of AI research.

Google Deepmind introduced AlphaEvolve in 2025, using the same basic principle. Gemini Flash generates code proposals, Gemini Pro analyzes them, and an evolutionary algorithm selects the best versions. Dream-RSI works one level above that process by optimizing the search strategy itself.

AutoTTS takes a related approach, using a coding agent to search for algorithms in a simulated environment. These algorithms decide when a language model should start, expand, or abandon reasoning paths. The resulting methods beat manually designed methods while using less compute.

Google Research recently presented a different way to reuse past runs with WikiSkill. That system records failures and successes in a wiki and turns them into reusable instructions for the agent. Dream-RSI's follow-up analysis suggests that explicit instructions like these can restrict exploration on open-ended search tasks.

Meta goes further with Hyperagents, allowing agents to rewrite the mechanism that controls how they improve.