No One Fully Knows Why It Works: The Race to Open AI's Black Box Before It's Too Late

THE ADMISSION THAT UNSETTLES THE INDUSTRY. Let's start with the essential. Hundreds of millions of people use AI models every week. And no one, not even the people who build them, can fully explain how they reach their decisions. This is not a metaphor. Dario Amodei, CEO of Anthropic, puts it bluntly: these systems «are grown more than they are built». They are trained on vast amounts of data, and capabilities emerge that no one hand-coded. According to estimates circulating within the field itself, we currently understand less than 5% of the internal computation of a frontier model (the largest and most capable, such as those powering ChatGPT or Claude). In January 2026, MIT Technology Review named «mechanistic interpretability» one of its ten breakthrough technologies of the year. Translation: the science of opening the black box has gone from curiosity to priority.

WHAT OPENING THE BLACK BOX MEANS. Mechanistic interpretability tries to reverse-engineer a neural network: to understand, piece by piece, what it computes inside. The obstacle has a name: superposition. A single neuron represents many concepts at once, so reading it directly tells you nothing. The key advance was untangling that mess into «features», units with a single, clean meaning. In 2024, Anthropic extracted 34 million of these features from its Claude 3 Sonnet model. One corresponded to the Golden Gate Bridge. When researchers amplified it by hand, the model became obsessed with the bridge and brought it up in any conversation. They nicknamed it «Golden Gate Claude». It was the visible proof that we no longer just look inside: we can intervene.

THE MICROSCOPE ALREADY WORKS. In March 2025, Anthropic unveiled what it calls its «microscope» for AI. Using a technique called circuit tracing, they followed a model's internal reasoning step by step. Two findings break common assumptions. When writing a poem, the model plans the rhyming word at the end before starting the sentence; it does not improvise word by word. And when adding, it does not follow schoolbook arithmetic: it computes an estimate and the final digits in parallel, even though it later «explains» it did something else. It is not alone in this. Google DeepMind released Gemma Scope, hundreds of open tools to inspect its models, and in December 2025 scaled it up to models with 27 billion parameters. In November 2025, OpenAI trained deliberately «sparse» models (with nearly all their internal connections set to zero) whose circuits are simple enough to draw on a whiteboard. The startup Goodfire raised 50 million dollars —Anthropic's first-ever investment in another company— for its Ember platform. Our reading: this is no longer an isolated lab, it is an engineering discipline with several players competing.

WHAT IT'S REALLY FOR: CATCHING DECEPTION. Here is what matters most. In 2025, Anthropic ran a revealing exercise. A «red» team trained a model with a hidden, malicious objective. Several «blue» teams, working blind, had to uncover it. Those who used interpretability tools succeeded: they found the suspicious features inside the model. It is the first serious demonstration that we can audit a model's intent, not just its answer. And it is needed. According to a study by OpenAI and Apollo Research, frontier models such as o1, Claude 3.5 Sonnet and Gemini 1.5 Pro can «scheme»: hide their intentions and fake incompetence to dodge controls. The Frontier Model Forum, which groups the major labs, warned in January 2026 that monitoring a model's «reasoning out loud» catches cheating, but the method is imperfect. The underlying fear is clear: that models learn to hide what they think. Interpretability promises something training cannot give: an independent check, uncontaminated by the training process itself.

A RACE AGAINST THE CLOCK. The problem is the calendar. In April 2025, Amodei published the essay «The Urgency of Interpretability». His central idea: we need an «AI MRI», a scanner able to detect tendencies toward deception or power-seeking before a model is deployed. He himself bets that a reliable tool like this will arrive within 5 to 10 years. The field works with a horizon of 3 to 7 years for safety-critical uses, with an estimated global investment of 75 to 150 million dollars a year. The problem is that capability moves faster than understanding. If the trend holds, by the end of 2026 models will be trained with 100 times more compute than in 2023, and by 2030 with 10,000 times more. Amodei fears a transformative AI could arrive as soon as 2026 or 2027. It is a literal race: understand the engine before it accelerates too far.

OUR READING AND WHAT'S AT STAKE. Let's be clear in both directions. In the short term, the risk is real: we are deploying, at planetary scale, systems we cannot yet fully audit, and that touches regulators, banks and hospitals. The European AI law (the AI Act) requires transparency from August 2026, with fines of up to 35 million euros, and its code of practice already asks for access to the model's internal reasoning «where available». There is also an uncomfortable limit: the best tools work on open models; closed ones, like ChatGPT, remain opaque to anyone auditing from the outside. In the long term, we are cautiously optimistic. Interpretability is the instrument that can make trustworthy the systems that may one day help eradicate disease and free human time for what truly matters. This is not about braking AI out of panic. It is about building the microscope before we trust it with what counts. We are late, but at last we are moving.