The anatomy of a good training environment: when verifiability isn't enough

By TheSequence · Opinion #892.

This opinion piece from the TheSequence newsletter (Opinion #892 edition) opens with a personal note from the author, who recalls that he has been publishing the newsletter for more than two years without sponsors, despite constantly receiving proposals from sponsors. He presents it as a personal project he has maintained for more than five years, with the aim of preserving its character of technical depth, objectivity and original content on AI, and he invites readers to support it through a free or paid subscription.

The main body of the text stems from an idea the author says he took from a recent episode of Dwarkesh Patel's podcast with Grant Sanderson, in which the latter argues that 'grindability' (something like a domain's capacity to be 'worked' or exploited incrementally by a training system) is as important as verifiability when it comes to determining whether a domain is conducive to AI progress. That idea, according to the author, helped him crystallize a reflection he had been developing for months.

The question the article poses is, in his own words, deceptively simple: what makes a domain 'good' for AI? Not in the sense of being commercially interesting, but in the sense that, if a modern training pipeline is aimed at that domain, the model's capability actually accumulates and improves over time.

The standard answer to that question is usually verifiability (that is, the possibility of automatically checking whether a solution is correct), and the author does not dismiss it, but he maintains that it is just a single axis within a higher-dimensional space of properties. According to his argument, the domains in which AI has positively surprised —mathematics, programming, board games— are precisely those that score high simultaneously on all of those axes. By contrast, the domains where progress has been slow and disappointing —agents' use of computers, robotics, and open or poorly structured knowledge work— tend to be strong on one or two of those axes but fail silently on the rest.

The author promises to go through those axes one by one in the rest of the article, providing in each case an example of a domain that possesses the property in question and another that notoriously lacks it, arguing that it is precisely in that contrast where the useful intuition lies. He suggests that understanding this framework of properties helps explain several current phenomena in the sector that are otherwise confusing: why reasoning models became competent at mathematics before everyday tasks such as managing email (using the inbox as an example), why an industry of startups dedicated to building reinforcement 'environments' (RL environments) that handle budgets of billions of dollars has suddenly emerged, and why the author anticipates that some of those environment companies are going to disappoint their buyers/investors.

The email, as received, cuts off just as the author begins to develop the first axis of his framework —verifiability itself— so the detailed content on each of the additional axes (beyond the aforementioned verifiability and 'grindability') is not available in the provided text.