DeepMind's DiscoRL: AI learns to design its own learning algorithms

By TheSequence · Tue, 20 Oct 2026 11:03:20 +0000.

Issue #954 of The Sequence Knowledge tackles recursive self-improvement (RSI) through a concrete case: DiscoRL, DeepMind's work. The email received contains only the summary (TLDR) and the opening of the article; the full text is on the publication's website, so this summary is limited to what appears in the body.

The starting thesis is that most AI systems get better at their tasks without getting better at learning. The example the text offers is a system that becomes a better chess player without becoming better at learning chess: with more training its moves improve, but the machinery that turns experience into improvement may stay the same. Recursive self-improvement forces a look at that machinery and raises the question of what happens when the learning algorithm itself becomes something a machine can learn.

According to the email, DiscoRL works as follows: a neural network represents the update rule, and a population of agents supplies the experience used to discover rules that produce better learners. The discovered rule, called Disco57, was obtained on Atari57 and used to train new agents on ProcGen and DMLab-30 without additional meta-training. The text presents it as a concrete case of intelligence applied to the procedure that creates intelligence, and notes that the resulting algorithm performs robustly across several reinforcement learning benchmarks, including environments outside its discovery set.

The full article, according to its summary, traces the lineage of ideas from MAML, learned optimizers and Learned Policy Gradient. It then unpacks the two-level credit assignment problem within the DiscoRL loop and analyzes what zero-shot transfer of a learning rule really tests.

It also addresses the economics of the approach, which boil down to amortizing the cost of discovery across many training runs, and the importance of interpretability before anyone adopts a learned rule.

Finally, the text sets out what a truly recursive step would have to demonstrate: that the discovered improvements keep producing measurable gains once compute and evaluation effort are accounted for. The email ends before developing any of these points, so no further details are available in the body.