AI that reads the invisible in a biopsy: Stanford predicts a tumor's ecosystem without raising diagnostic costs

By Stanford Medicine · July 6, 2026.

Every patient who undergoes a biopsy or tumor removal generates an H&E slide: the pink-and-purple-stained tissue that pathologists have been examining under the microscope for more than a century. It is the gold standard of diagnosis, but also a radically underused source of information: that slide conceals, without displaying it, how cancer cells, immune cells and the stromal cells surrounding the tumor interact with one another. Knowing those cellular "conversations" matters because it predicts very concrete things: how aggressive the cancer will be and whether it will respond to an immunotherapy. The problem is that truly measuring them requires a technique called CODEX —developed at Stanford itself by Garry Nolan's lab—, capable of detecting dozens of proteins and cell types with a precise spatial map, but slow and expensive, unfeasible for large-scale application in routine daily clinical practice.

Ruijiang Li's team, publishing on June 16 in the journal Cell, has trained an AI called CANVAS to bypass that bottleneck: instead of running CODEX on every patient, the AI learns to infer those same cellular neighborhoods directly from the cheap, universal H&E slide. To achieve this, they built an atlas of more than 18 million cells from 457 patients with non-small cell lung cancer, overlaying the CODEX results onto the microscope images cell by cell, and they drew on MUSK, an earlier model from the same lab trained on 50 million pathology images and more than a billion fragments of medical text. The result is 10 discrete cellular "neighborhoods," each defined not only by which cells are present and where, but by which proteins they produce and what signals they send to one another —some with an immunosuppressive profile, others associated with blood vessels or the tumor core.

The most clinically relevant finding came when validating the model in more than 5,000 patients across nine different cancer types: one of those neighborhoods, rich in neutrophils expressing metastasis-facilitating proteins, correlates with worse prognosis and poorer response to immunotherapies such as anti-PD-1. And here is the figure that turns this into something more than basic science: according to the researchers, the presence of that neighborhood predicted the response to immunotherapy more accurately than the criteria used today to decide whether a patient is a candidate for that treatment. In other words, the model not only describes biology, it points to improving a real clinical decision —who to treat and with what— using an image that already exists in the file of any oncology patient.

Our reading is that this work embodies with particular clarity the underlying thesis of long-term optimism about AI in medicine: it is not about replacing the pathologist or inventing a new data point, but about extracting the maximum value from information that is already generated routinely and that until now was read incompletely. That pattern —turning a cheap, universal data point (a microscope photo) into a proxy for an expensive, scarce one (a complete proteomic map)— is replicable in many other areas of medicine, and is probably one of the most realistic paths toward diagnostic abundance: not everyone will have access to a lab with CODEX, but almost any hospital on the planet already produces H&E slides. If tools like CANVAS prove robust in clinical trials, the ceiling of diagnostic quality that today is reached only by reference centers such as Stanford, the Broad Institute or MD Anderson could move closer to hospitals with far fewer resources.

That said, it is worth keeping our feet on the ground about timelines: the authors themselves stress that the next step is to validate CANVAS in prospective clinical trials, something that in oncology usually takes years, not months, and that requires demonstrating not just correlation but real utility by safely changing treatment decisions. Moreover, a model trained mainly on non-small cell lung cancer, even if later extended to nine tumor types, will need additional scrutiny in diverse populations and hospital settings before becoming a routine tool. The short-term honesty here is simple: this is a research advance with solid correlation data, not yet an approved clinical product that will change practice at the bedside tomorrow. But the direction —using AI to read layers of biology invisible to the human eye in data that already exist— is exactly the kind of quiet, cumulative progress that, added up over years, keeps narrowing the gap between what medicine can diagnose today and what it would need in order to treat cancer with the precision that long-term abundance promises.