A study of 100,000 engineers finds AI agents generate more code, but not more finished software

By Zendoric · October 10, 2026.

The figure circulating since yesterday is striking: after adopting AI coding agents, companies produce 30% more lines of code, 20% more commits (each save of changes to the repository) and 23% more pull requests (code proposed for inclusion in the project). But the metric that really matters — how many tasks get closed — did not move in any statistically significant way. That is the central finding of a preliminary study by Fiona Chen and James Stratton, researchers at Harvard, reported by Ars Technica.

The authors used aggregated data from Jellyfish, a platform that measures engineering team activity, covering 100,000 engineers at 500 companies between 2021 and March 2026. They applied a difference-in-differences technique — comparing each company before and after adopting agents, since not all of them started at the same moment — to isolate the effect of AI from other industry trends. With that methodology they distinguished between assistants (which support a human programmer who is writing) and agents (which draft and submit code more autonomously from instructions). The results are best read as associations observed in the data, not as settled causal proof.

The figure that explains the paradox lies in the phase after the code is written. The time between submitting a pull request and having it merged into the project rose 49%. The share of requests that received change requests nearly doubled, comments per request grew 35%, and the percentage of employees devoted to reviewing code rose 14%. By March 2026, 80% of the companies in the study were already using some form of AI-assisted code review, but that same AI generated only 23.3% of review comments and took part in 10.8% of requests: review remained, for the most part, human work.

The study also looked for an employment effect by cross-referencing Jellyfish records with LinkedIn data from the same companies, and found no significant change in headcount attributable to the adoption of agents. Put another way: agents neither triggered mass layoffs in programming during this period, nor freed up enough time for headcount to shrink through efficiency. It is also worth remembering that by the time Ars Technica published its coverage, adoption already reached 95% of the companies analyzed, compared with the period measured; and that the agents themselves have received significant updates after the March data cutoff, so these numbers portray a specific phase of a technology that kept moving.

Our reading: this is not an indictment of coding agents, it is confirmation that measuring productivity by volume of code is measuring badly. The bottleneck did not disappear with AI, it simply shifted from writing to review, which still demands human judgment to tell a correct contribution from one that merely looks correct. It fits something we keep repeating in our coverage of AI and employment by sector: what resists automation is not the mechanical task — writing lines — but the judgment that validates the result. In programming, that judgment is still not delegated.

There is also a historical pattern behind this finding worth naming: the so-called productivity paradox, which economist Robert Solow summed up in the 1980s when he observed that computers were everywhere except in the productivity statistics. Major general-purpose technologies take time to translate into measurable gains because they first require redesigning processes, not just adding a tool on top of the same workflows. Most of the companies in this study appear to have inserted agents into a review process designed for a human volume of code, without redesigning that process for the volume a machine generates.

That opens a window of opportunity rather than a verdict. If review is the bottleneck, the next productivity leap will not come from agents that write faster, but from tools that reliably validate what those agents produce, and from teams that learn to decide which tasks to delegate and which not. It is exactly the kind of organizational adjustment that, by our underlying thesis, characterizes the hard stretch of every real technological transition: available capability running ahead of the practice that knows how to use it. In the medium term, the combination of more mature automated review and better judgment about task allocation should narrow that gap; meanwhile, any company boasting of AI-driven productivity gains based solely on the volume of code it generates is telling half the story.