A lung cancer AI system learns to say 'I don't know' before recommending a treatment

By Bioengineer.org · September 27, 2026.

A team from the University of Washington and the Fred Hutchinson Cancer Center has built a prototype clinical decision support system that, as well as predicting whether a lung cancer patient will respond to chemoimmunotherapy, tells the oncologist when that prediction is unreliable. The study, published in the journal Journal of Translational Medicine, draws on data from the PET-BRIGHT clinical trial, with 35 patients with metastatic non-small cell lung cancer — the most common and lethal type of lung cancer — treated with the standard combination of carboplatin, pemetrexed and pembrolizumab, an immune checkpoint inhibitor that 'releases the brake' on the immune system so it can attack the tumor.

The starting point is uncomfortable for current clinical practice: the biomarker that guides treatment choice today, PD-L1 expression in the tumor (measured by immunohistochemistry), reached, according to the authors themselves, an area under the ROC curve — AUROC, the standard metric that summarizes how well a test distinguishes between two groups, where 0.5 is equivalent to chance and 1.0 to perfect prediction — of only 0.58. Barely better than tossing a coin to decide whether a patient should undergo months of toxic treatment.

Against that, the researchers combined three distinct biological signals, taken before treatment began and again after three weeks (a single cycle of therapy): PET/CT imaging with a radioactive glucose tracer (FDG), which measures how much energy the tumor consumes; the diversity of the T-cell receptor (TCR) repertoire, a proxy for how broad the immune arsenal capable of recognizing the tumor is; and a panel of cytokines, the signaling proteins that reflect the patient's overall inflammatory state. With conservative cross-validation — necessary given the small sample size — and logistic regression instead of opaque models, the team trained individual and combined models for each point in treatment.

The results show something more interesting than a simply 'better' model: the hierarchy of biomarkers changes over time, as the article itself reports. Before treatment, TCR diversity was the most powerful signal (AUROC 0.80), ahead of PET (0.76) and far ahead of cytokines (0.54); combining PET and TCR raised discrimination to 0.85. After three weeks of therapy, the order is reversed: cytokines become the strongest signal (0.78) while TCR loses power (0.66), and the combination of PET and cytokines achieves the study's best overall result, 0.86. Thirteen of the twenty-two combinations of model and timepoint passed a permutation test, the statistical check that rules out the result being down to chance.

The most novel piece, however, is not the accuracy but the built-in statistical honesty. The system uses conformal prediction, a framework that, instead of forcing a binary answer, sets a target confidence level — here 80% — and, when it does not reach it for a given patient, returns a set containing both possibilities rather than a single label. The share of 'single-class' predictions — the cases in which the model does venture a unique answer — thus becomes a direct, patient-by-patient measure of how much the doctor can trust that prediction. Adding TCR to PET imaging before treatment, for example, raised those confident predictions from 78% to 89%, according to the study's data.

The authors themselves are clear about the limits: 35 patients at a single center are a proof of concept, not a change in clinical practice, and they describe it as hypothesis-generating rather than ready to decide treatments. Validation in much larger multicenter cohorts, with different tumor types and populations, would be needed before a system like this comes close to an oncology clinic.

Our reading: what matters here is not the AUROC of 0.86 — a modest, fragile figure obtained with 35 people — but the architecture of the problem this team chose to solve. In medicine, a model that is confidently wrong is more dangerous than one that acknowledges its doubts; the track record of AI applied to diagnosis is full of systems with good average accuracy that fail silently precisely on the atypical patient, the one who most needs help. That the design builds in a formal mechanism to say 'I don't know' from the outset — and not as a later add-on — is the kind of engineering that separates an eye-catching headline from a real clinical product.

This connects with something we have been pointing out at Zendoric: the road toward medicine capable of turning today's lethal diseases into chronic or eradicated ones does not run only through bigger or more accurate models, but through systems that can calibrate their own certainty and through trials that, like PET-BRIGHT, measure biology at the right moment. That the relevant predictive signal migrates from the baseline immune repertoire to circulating inflammation in just three weeks is itself a biological finding, not a mere model adjustment. The current standard biomarker, PD-L1, has been in use for years despite performance close to chance; replacing it with something better calibrated, even with data from 35 patients, is the kind of incremental step — slow, verifiable, peer-reviewed — that genuinely moves the needle in oncology, as against the noise of 'AI that cures cancer' announcements that do not survive serious scientific review.

In the short term, the message for the health sector is one of caution: neither this nor any similar system is ready to decide treatments, and overselling the results of pilot studies is a real risk to public trust in medical AI. In the long term, the underlying logic — fusing heterogeneous biomarkers, quantifying uncertainty patient by patient and using it to decide who needs closer monitoring and who can avoid months of toxic, ineffective treatment — is exactly the kind of tool that, scaled and validated in broader cohorts, brings oncology closer to more precise and less burdensome decisions for each patient.