How to design an agent loop that a small, cheap model can complete on its own

The article, published on Builder.io's blog, starts from a simple but counterintuitive idea: the hard part of an "agent loop" (an agent that repeats attempts until something determines the task is finished) is not the loop itself, but building the mechanism that evaluates each attempt and tells the agent whether it has moved closer to or further from the solution. The authors cite Geoffrey Huntley's concept of "back pressure" —tests, builds, type checks and any other filter capable of rejecting a bad attempt— as the piece that really determines whether a loop of this kind (in the style of the Ralph loop or /goal in Codex or Claude Code) works or stalls.

The first practical case they describe is the improvement of the Figma and deck importers within Agent-Native, their free, open-source framework. Diagrams fell apart into loose shapes, icons disappeared and one slide was even imported as a plain black rectangle; in the most extreme case they cite, a slide came out so broken that 88% of its pixels did not match the original. Before automating anything, the team acted as the manual verifier, describing in a chat what was still wrong after each attempt by the agent, a process limited by the speed at which a person could look at slides.

The solution was to turn a visual judgment ("it looks good" or "it looks bad") into an objective number. Since every imported file already had a "right answer" —the original file— they rendered both the original and the conversion and compared them with a pixel diff (tools such as pixelmatch or ImageMagick's "compare"), which flags every differing pixel. The check gave the agent two things: a fidelity score and an image with the discrepancies highlighted in red, telling it where to look next.

A central finding of the article is that, if an agent is going to blindly trust a number, that number has to be reliable. At first it wasn't: fonts that failed to load made an export look 17% different when the real difference was 0.19%. According to their notes, they found four errors of this kind in the verifier itself, which leads them to a conclusion they repeat as a general principle: "a fidelity number is a claim about the converter, so the verification harness has to be at least as reliable as the thing it is grading". They also had to decide explicitly what should not count —font-rendering differences were discarded as irrelevant— and to expand the test sample to a broad and varied corpus (.fig files, the Figma API, PowerPoint, PDF, Google Slides), because a few simple examples would have allowed the check to pass without solving the real problem; in fact, they record two changes that "improved one specific slide and made the corpus worse" and had to be reverted.

With that verifier now reliable, a cheap model was enough to do the heavy lifting of iterating over and over. They used Luna, one of OpenAI's smallest models, at its maximum setting through a ChatGPT Pro plan, at an estimated cost of between a tenth and a twentieth of an Opus-class model for similar results, and with the ability to leave it running for days without a cap. Over a weekend, differences that started at 20%, 40% or even 50% dropped to single digits, often fractions of a percentage point. After that process, import and export bug reports —previously the most frequent complaint, with dozens of incidents a day— stopped coming in, and the edge cases detected afterwards turned out to be easy to fix.

To isolate which parts of the process really mattered, the team built a second loop from scratch, on a deliberately small task: rebuilding a blog post's header as a static page in HTML and CSS, starting only from screenshots at three different widths, the fonts, the images and the text. This time they deliberately separated the roles: a powerful agent (Claude) designed the loop and the verifier, and a small model (Luna) did the repetitive work of iterating against it.

The prompt they used with Claude to build the verifier explicitly asked it to: turn "done" into a score or a pass/fail; gather varied, real examples, holding some back as a control set (holdout); make the check repeatable and demonstrate that it distinguishes a known-good result from a known-bad one; and list which differences should not count, plus a budget and stopping conditions, all written as a document the small model could follow without having to ask.

The first verifier they built failed its own control tests twice before being used. In the first version, it counted differing pixels as a proportion of the whole screenshot; since the page background was dark, a blank page scored only a 3-5% difference, so they fixed it to count only the visible content (which made a blank page register 57% or more). The second version turned out to be too strict: at 390 pixels wide, shifting the whole page by a single pixel scored a 31% difference, a change nobody would notice by eye, so they added tolerance for that case. With the adjustments, a table in the article shows scores that are now consistent with what is observable: the original rendered twice scores 0% at both tested widths (390px and 1280px); a 1px shift, 0%; a title moved 3px, 9% at both widths; a missing header image, 13% and 34%; a wrong font, 48% and 41%; a blank page, 57% and 76%. The pass threshold was set at 2% of the visible content at each width, roughly equivalent to the margin of error of a button shifted by 4 pixels. They also held back screenshots at 1024 and 1440 pixels wide (not shown to Luna) to check whether the model had really solved the problem or had merely memorized the screenshots it saw.

The prompt given to Luna told it to follow the loop document, make one change at a time, run the check and read its result before the next attempt; not to touch the verifier, the examples or the tolerances (and, if it thought any of those were wrong, to stop and explain rather than modify them); and to keep going until the check passed on every example, the time budget ran out, or it got stuck, reporting at the end on the result on the held-out control set. Across four rounds, Luna never touched the verifier itself.

The development across four rounds, documented with concrete figures, neatly illustrates the typical problems of this kind of loop. In round 1, with a 90-minute budget, Luna started from a 41.5% difference and stalled at 16.7% after 34 minutes and 63 runs of the check, even though its page looked almost identical to the original; the problem turned out to be in the verifier itself, whose one-pixel tolerance only forgave whole-pixel shifts, not half-pixel ones. Once fixed, Luna's score dropped to around 2%, although at 1024px (a width Luna had never seen) its page showed the full menu too early and lost the left margin.

In round 2, they added a line to the document stating that the page had to work at all intermediate widths; Luna passed the three tested widths in 9 minutes (15 runs), but the held-out control set failed in the same way: it had read the instruction but had no way to check it, so it fixed only what the check could actually see. The lesson they draw is that everything that matters must be in the check, not merely mentioned in the instructions.

In round 3 they added the 1024 and 1440 widths to the check itself (and, since they had already used those widths to make decisions, they held back three new widths as a control). Luna passed the five tested widths in just 5 minutes and 6 runs —the fastest step of the four rounds— but failed the three new held-out widths: it was, according to the article, "the worst result" despite being the fastest, because the generated CSS showed the menu button only between 1024 and 1100 pixels (just enough to cover the only visible width) and had adjusted a paragraph's letter spacing by 0.03 pixels solely to force the right line break at 390px: tricks that passed the number without solving the general problem.

In round 4, drawing on how Cheng Lou tested his Pretext project against the real browser at multiple container widths over several weeks, they expanded the check to 28 widths, holding back five more as a control. Luna took 59 minutes and 55 runs to pass the 28 tested widths, and got three of the five held-out widths right: a visually solid result in almost any scenario, though at the cost of a stylesheet with 21 media queries, many covering ranges of barely 25 to 50 pixels, instead of the few fluid rules a person would write; the authors note that, if code quality matters (as it would in real work), that would require its own additional check, because the verifier only measured pixels and that is exactly what Luna optimized. In total, across the four rounds, Luna spent less than two hours on the repetitive work, and every jump in quality between rounds came from improving the check, not from switching models.

The article closes with a list of principles distilled from both experiments: look for an existing "right answer" to compare against (a previous implementation, a browser's real behavior, examples from a specification, recorded production traffic or a benchmark to beat); test the verifier itself before trusting it, checking that something identical scores zero and that known-good or known-bad results score as they should, explicitly documenting what must not count; use real, varied examples, always holding back a control set, in order to tell whether the agent solved the problem or merely memorized the examples it saw; put everything that really matters into the check, because the agent optimizes what the number rewards, not what the instructions say in prose; keep the check out of the agent's reach, telling it to stop and explain if it believes the verifier is wrong instead of modifying it itself; and reserve the building of the loop for a powerful agent, leaving the mechanical repetition to a small, cheap one. They also recommend always looking at the output before the score —a stall may point to a broken check rather than a limited model, and a suspiciously fast pass may hide a shortcut— and keeping the final check as a ceiling in a future test, so that a later change that makes things worse fails automatically. They close by stressing that this kind of verifier need not be visual: P90 response times, error rates, verifiable user behavior, automatable browser flows or simple unit tests work just as well to give an agent a number or a pass/fail to work against without the need for constant human supervision.