Claude Generates 180 Million Lines of Code to Prove Software Safe
Anthropic's model verified 123 open-source libraries in weeks, finding bugs that traditional testing missed.
By Chioma Kuznetsov ·
Developers who rely on open-source software may soon have more confidence in the code they use. Anthropic announced that its Claude model has mathematically verified the correctness of existing software libraries, a task that previously required years of specialized human effort. The work identifies hidden flaws that standard testing often misses.
The project involved generating approximately 180 million lines of mathematical proof code. Claude analyzed test programs from 123 open-source libraries in Google's OSS-Fuzz security project. The process took several weeks. For comparison, verifying a small operating system kernel called seL4 required 20 person-years of specialist work. That historical cost kept formal verification limited to high-stakes fields like aviation.
Anthropic's Frontier Red Team partnered with Theorem for this effort. The team did not ask the AI to write new software. Instead, Claude converted crash tests into mathematical problems and analyzed compiled machine code. A separate program called Rocq automatically checks these proofs. The researchers described the proof generation as fully autonomous, though humans at Theorem built the underlying tools and formal foundations.
The verification process uncovered hundreds of bugs. Researchers noted that many of these errors only appear under narrow conditions, such as unusually large input files. In one specific case, the team found as many security bugs in a PDF reader in one week as are usually reported in a full year. In another application, Claude identified a logical error that had remained unnoticed for more than a decade. Anthropic stated that even when a complete proof fails, the attempt can still reveal real defects.
The company emphasizes that formal verification does not mean the software is error-free. The proofs hold only under specific conditions, including a particular processor architecture and inputs smaller than 64 megabytes. Most real-world deployments run on different hardware configurations. Cost remains a factor as well, with individual proofs consuming hundreds of millions of tokens. Anthropic has not published the total project cost.
Efficiency is improving in smaller test runs. Opus 5.5, a newer model version, solves half the proofs using about 40 percent fewer tokens than Opus 5. Anthropic argues that if AI can verify its own code, developers can spend less time on manual review. Theorem's roadmap now targets critical infrastructure libraries, including glibc, OpenSSL, and cURL.
