AI giants accept a training opt-out: a useful gesture, but late and on an uneven timeline

By Thurrott.com · October 30, 2026.

Anthropic, Google, Microsoft and OpenAI have agreed to support a common system that lets website owners exclude their content from AI model training. According to the article, Anthropic has already implemented it, while Google, Microsoft and OpenAI will do so in 2027.

The mechanism originated in an IETF (Internet Engineering Task Force, the body that sets internet standards) working group and will be proposed as an international standard once complete. It lets a website tell all AI crawlers that it does not want to take part in training, separately from search engine indexing and archiving.

That distinction is the practical core. Until now, many publishers felt they had to choose between appearing in search results and letting their texts feed generative models. Microsoft's Krishna Madhavan puts it this way: publishers should not have to choose between being discovered on Bing and deciding how models use their work. Microsoft speaks of separate preferences for training, "grounding" (retrieving content in real time to support an answer) or both. OpenAI, for its part, describes it as a voluntary effort that allows a single preference to be communicated to several providers at once.

Journalist Paul Thurrott calls it a "largely symbolic" gesture, because it comes years after these companies trained on web content at an unprecedented scale. That is his assessment, and each company would dispute the term he uses to describe that training; what is verifiable is that the opt-out only works going forward.

Our take: Thurrott is right about the past but falls short on the future. An opt-out does not undo what has already been trained, and its voluntary nature means it depends on crawlers' good faith. Even so, a shared standard is worth more than four proprietary mechanisms: it lowers the cost for publishers, creates a benchmark against which to measure non-compliance and fits a trend, already visible in the governance of agentic standards, of handling the industry's plumbing in a neutral way.

What we don't know is what matters most: how many crawlers beyond these four companies will respect it, and whether open models and Chinese players will join. If they don't, the opt-out will offer little protection. There is also a timing mismatch: until 2027, Anthropic is the only one of the four providers that honors it, so publishers will have only partial protection for months.

In the short term, the transition for media outlets and creators remains tough: their content is already inside the models and their traffic-based business is suffering. In the long term, an AI that underpins scientific breakthroughs and abundance needs a sustainable pact with those who generate knowledge. This standard does not settle that, but it is a reasonable piece of that pact; what matters is that it be enforceable, not merely declarative.