GPT-6 Astra posts near-perfect reverse-engineering scores, but a saturated benchmark tells us more about risk than rank

GPT-6 Astra reportedly scored 88% on single-attempt reverse-engineering tasks on the "S Bench" benchmark, according to the short note we received. With four attempts per task, the score rose to 99.2%. Reverse engineering means working out how compiled software works without access to its source code.

The headline numbers need context. With 99.2% at four attempts, the benchmark is nearly saturated: almost every task gets solved. A saturated test can't show how far ahead one model is, or how much harder tasks remain. We saw the same thing with Cybench, where multi-attempt scores near 100% stopped telling models apart. Only the 88% first-try figure still says much. We only have a brief note, so we don't know what the tasks look like, how hard they are, how they were scored, or how other models compare.

Our take: reverse engineering is dual-use. It helps defenders analyze malware and audit software they can't see inside. It can also help attackers find weaknesses. We see results like this as a capability and a risk to be managed, not as a ranking of offensive tools. Over time, models that can quickly understand any piece of software could make systems much safer. The question that matters now is how this model does on harder, unguided expert tasks, which today's benchmarks don't capture.