Claude writes code and mathematically proves it is safe, according to Anthropic and Theorem

Anthropic has used Claude to mathematically prove that existing code is correct. According to The Decoder, the approach tackles a central problem with AI-generated software, though it has strict practical limits.

Until now, proving that a program works without errors under all circumstances has been prohibitively expensive. A team of specialists needed 20 person-years to formally verify seL4, a small operating-system kernel. As a result, the method was reserved for extreme cases such as Airbus avionics software.

Anthropic's Frontier Red Team and its partner Theorem had Claude take over the work of writing the proofs. In a few weeks, the model produced about 180 million lines of mathematical proof code. It worked from test programs in 123 open-source libraries from Google's OSS-Fuzz security project.

Claude did not write new code. Instead, it verified existing software: it turned bug tests into mathematical problems, analyzed the compiled machine code and generated extensive proofs. An independent verification program, Rocq, checks these proofs automatically, and according to Anthropic the process needs very little human review. The researchers describe it as "fully autonomous" proof generation, although people at Theorem built the tools and formal foundations.

Along the way, Claude found hundreds of bugs and was allowed to fix them. Many would have gone unnoticed in conventional testing because they only show up under very specific conditions, such as unusually large input files or a particular sequence of accesses. While trying to verify a widely used PDF reader, the team found as many security bugs in one week as are usually reported for that software in a whole year. In another application, Claude caught a logic error that had gone undiscovered for more than a decade. Anthropic notes that even when a full proof ultimately fails, the attempt still uncovers real flaws.

Anthropic acknowledges that "formally verified" does not mean "bug-free." The proofs only hold under strict conditions: a specific processor architecture, single-threaded programs and inputs under 64 MB. In practice, these libraries run on other hardware and with other configurations. Compute costs are also high. Each proof can consume hundreds of millions of tokens, and Anthropic does not disclose the total cost of the project.

A smaller test showed that the proofs can be reused and that newer models are more efficient. According to the article, Opus 5.5 solves half of the tests using roughly 40% fewer tokens than Opus 5.

The broader takeaway is that if AI can verify its own code, developers could trust AI-generated software more and spend less time reviewing it. The question would shift from "can AI code?" to "can AI prove that the code it writes is safe?" Anthropic hopes AI-driven formal verification will become as practical as standard testing. Theorem's roadmap already targets critical infrastructure such as glibc, OpenSSL and cURL.

Note: the downloaded text is the full The Decoder article, which is based on Anthropic's announcement. The figures cited are those published by that outlet.