Google rolls out an AI assistant that flunks child safety and that no one can turn off

By The College Investor · July 17, 2026.

Common Sense Media's AI Child Safety Institute has given the AI features of Google search —AI Overview and AI Mode— its lowest rating: "unacceptable risk" for children and teenagers. The evaluation, published on July 14 after seven weeks of testing (May 19 to July 1) covering more than 2,600 interactions on accounts registered as belonging to 11- and 15-year-olds with SafeSearch enabled, concludes that both features failed the five "red lines" of serious harm the institute uses as its minimum threshold. The numbers are concrete: AI Overview referred to a helpline or a medical resource in only 58% of the cases where it clearly should have (against a required minimum of 95%); AI Mode literally completed 100% of 180 school assignments deliverable as-is on school devices; and only half of the questions with false premises were rejected, with cases as flagrant as describing a nonexistent Supreme Court ruling or citing a $400 million FTC penalty that never existed. Almost a third of the more than 2,100 audited citations came from Reddit, Facebook or forums with no editorial control whatsoever.

There is a design nuance that aggravates all of the above: while the standalone chatbot Gemini can be disabled through Family Link, AI Overview appears automatically in search by default —on school Chromebooks, phones and library computers— and there is no setting to turn it off. It is, according to Common Sense Media's own 2026 census, the gateway to AI for 75% of U.S. teens and preteens, which turns a product flaw into an educational-infrastructure problem. Google has reviewed a draft of the report and says it has corrected some of the tested prompts, but the institute clarifies that it has not independently verified those changes, so it's best to treat that nuance as a promise, not a done deal.

The economic angle adds a layer that The College Investor has been documenting for two years: Google demands of itself a higher accuracy standard for "your money or your life" topics (health, financial stability, safety) and its own AI fails to meet it. Its 2024 study found 43% of financial answers to be wrong or misleading; the 2025 follow-up on the same 100 questions dropped only to 37%, with institutions invented out of thin air ("Hustle Digital Credit Union," "Sally May") and outdated information about student loans. That this margin of error persists two years later, on the same repeated question, says more about the problem's priority than any one-off headline.

Our reading: this is not a capability failure of the language models —it's a product-governance failure, and the distinction matters. As we have noted in other sensitive sectors (classrooms, child protection, oncology), the recurring brake on these technologies is usually not the technical side but the lack of regulation, of age-segmented safety data and of control mechanisms before mass deployment. Google has done exactly the opposite: it deployed a feature that cannot be disabled in the default information channel for minors, without the reliability threshold it demands of its own financial content. The result is a paradox worth naming: 43% of contradictory answers to the same repeated question not only misinforms in the moment, it erodes the confidence that a verifiable correct answer even exists.

In the long term we remain convinced that well-governed AI systems can mentor learning, detect risk signals in minors before an adult does, and democratize access to reliable information about health and money for families that today have neither a financial advisor nor a school counselor. But that future requires exactly what this report documents is missing today: real off-switches, verifiable accuracy standards and independent auditing, not promises of "we've already fixed it" without data to back them up. Common Sense Media's requests —an off option for schools and families, standardized crisis protocols, quarterly reassessment with age-based data— are exactly the kind of minimum governance that would separate the AI that helps children from the one that simply exposes them, at scale, to an answer engine that not even its own creators manage to make reliable.