On October 30, 2026, the four largest Western AI companies — OpenAI, Google, Microsoft and Anthropic — announced they will honor a single shared signal that lets website owners opt out of AI training. For the first time, the labs that build the world's most widely used AI models have lined up behind one opt-out mechanism, rather than each demanding that publishers learn and maintain its own separate rule.

The signal is a new robots.txt policy token, __usage_ai-training. A publisher that wants its content kept out of training data adds two lines to its robots.txt file: a user-agent line naming the token, followed by a disallow rule for the whole site. Publishers can also carve out exceptions, allowing training on some sections — a press kit, for example — while blocking the rest.

One line instead of a dozen rules

Until now, opting out meant blocking crawlers one company at a time: one token for OpenAI's GPTBot, another for Google-Extended, and separate rules for Microsoft's and Anthropic's crawlers. Each came with its own documentation and quirks, and a publisher that missed one left its content exposed to that company's training pipeline. The shared token collapses that patchwork into a single entry that every participating company has agreed to read the same way.

The mechanism comes from an Internet-Draft titled “Machine-Readable Training Preference Signal,” published on October 30 and written by Kevin Kelley, a technical lead for standards at Anthropic. It sits with the Internet Engineering Task Force's AI Preferences working group, but it is an individual submission, not a finished standard: the IETF notes it has no formal standing and is not endorsed by the body, and the draft expires on May 3, 2027 unless it is revised.

Timelines differ — and nothing happens overnight

Support will not arrive all at once. Anthropic, which authored the draft, says it has already begun implementing the token. OpenAI says it aims to honor the signal in the coming months, without committing to a date. Microsoft says it will begin honoring the preference on February 1, 2027, as its crawlers revisit pages — and it is going further than the shared token, adding a page-level training setting plus a separate control for AI grounding, the step where live web content is pulled into chat answers rather than baked into model weights.

Google says it aims to support the token in 2027. When a site uses the shared token without mentioning Google-Extended directly, Google plans to treat that as if the publisher had disallowed Google-Extended, the training control it has offered since 2023. All four companies say their existing, company-specific controls will keep working alongside the new joint signal, so publishers should keep their current rules in place until support actually lands.

What the opt-out does not do

The limits matter as much as the promise. The token governs training only: it is not a crawl rule, so opting out will not stop a crawler from visiting or caching a page, will not remove a page from search results, and will not keep it out of AI-written answers that quote live pages. And it is not retroactive — updating robots.txt does nothing about content already collected into datasets or already trained into model weights.

It is also voluntary, carrying no enforcement power beyond each company's word. Only companies that sign on will follow it, and the largest labs outside the four — including Meta and the Chinese labs — have not joined. Still, the coordination itself is the news: publishers have spent years asking for one switch instead of many, and for the first time the companies that train the biggest models have agreed to read the same one.