3JSBench: the benchmark that reveals why AI still can't rest a lamp on a table in 3D

By Zendoric · October 9, 2026.

A small startup has published an uncomfortable exam for the most powerful language models: building a simple 3D object. General Context Labs —creator of instaplay.ai, an AI video game generation platform that it says hosts more than 40,000 games, according to the company itself— this week unveiled 3JSBench, a benchmark (a standardized test for comparing the performance of different models) that measures whether large language models (LLMs) are capable of generating coherent 3D objects written as code in Three.js, the most widely used JavaScript library for 3D graphics in the browser. The test bench evaluates 10 models across 100 tasks, with 697 automatic, deterministic checks —verified by objective rules, not by another model judging screenshots, the usual method until now—.

The result, according to the thread's own data: when an object has to rest on, hang from or lean against another —a lamp on a table, a door on its hinges—, almost half of the attempts fail. The models build the individual pieces well, but they do not assemble them: geometries that intersect each other and flickering textures appear. When the task asks for a word to be written on a sign, 44% of attempts make mistakes in the letters; the thread details that one of the models evaluated, called Astra, fails in 3 out of 30 attempts, compared with 10 out of 30 for Grok 4.7.

There is an even more uncomfortable figure for anyone who wants to rely on automatic metrics alone: General Context Labs also gathered more than 9,500 votes from more than 1,000 video game creators who compared objects generated by different models by eye. The result is that they slightly prefer the objects from a model called Opus 5.5 over those from Astra, even though Astra passes more geometric correctness checks. Passing the technical exam and pleasing the human eye are not the same thing.

It is worth qualifying the sentence with which the company opens its own thread: it contrasts models "solving Millennium Problems" with their inability to build 3D objects. That is a rhetorical license that should not be taken literally —no model has actually solved any of the six Millennium Problems that remain open, and the Clay Mathematics Institute's one-million-dollar prize for each one is still unclaimed—; the company is probably referring to advances in competition mathematics or other demanding reasoning tests. That said, the underlying contrast it points to is real, and it is exactly the one we keep repeating at Zendoric: AI capability advances along a jagged line, not a straight one. A model can excel at abstract mathematical reasoning and fail at something a child solves without thinking, such as working out that a chair needs all four legs touching the ground at the same time.

It is also a small but clean example of something we often argue: benchmarks evaluated "by eye" —with another model judging a screenshot— tend to reward what looks impressive and to inflate over time; those that verify with deterministic rules, however more tedious they are to build, are the ones that really say where the system fails. Here the company itself underlines this by contrasting its method —a "linker" that identifies each object within the generated scene and applies objective checks— with standard practice in multimodal evaluation.

The relevance does not stop at the niche of AI-generated video games. If the industry wants to use language models to produce real 3D assets —in video games, simulation, robot training, product design—, geometric reliability is what decides whether the output is usable or waste, not whether the model understands the request in general terms. And the fact that the model humans prefer is not the one that passes the most automatic tests is a warning: measuring formal correctness alone, or human preference alone, yields two different rankings. Any company building a real product on AI 3D generation will need both yardsticks, not just one. It is a piece with little echo —the original thread barely gathered 510 views and no comments on Hacker News—, but of the kind worth filing away: the next generation of AI applications that "generate worlds" will depend on solving precisely these decidedly unglamorous details of everyday geometry.