Cloudflare introduced a new setting called “Disallow AI Training” on September 15, 2026: website operators can refuse AI training without disappearing from Google search. The prerequisite is the new “Accountable” status, which is currently held by seven major tech companies. Newly created, ad-funded sites will automatically block AI training in the future.
New switch replaces blanket bot blocking
Until now, Cloudflare customers only had a rough choice: block or allow AI bots. The new switch has four levels – Allow, Disallow AI Training, Block on Ad Pages, and Completely Block – and can be set individually for each domain, as Cloudflare explains in its blog post. In July, the company had already announced that it would block training and agent bots by default on ad-supported pages starting September 15. Newly added, ad-funded domains will now automatically receive the “Disallow AI Training” setting as a default, while ad-free pages will continue to allow all bot types. Existing customer settings were automatically converted by Cloudflare to the new category by the deadline, ensuring that search engines like Google and Apple remained crawlable while pure training accesses were eliminated, reports Search Engine Journal. CEO Matthew Prince justifies the move by stating that Cloudflare wants to maintain the openness of search while giving rights holders real control over the use of their content.
Accountable status applies to seven major tech companies
Choosing the “Disallow AI Training” setting does not automatically result in a loss of visibility in search – provided the respective provider holds the new Accountable status. To obtain it, a company must meet four conditions according to Cloudflare: an opt-out option from AI training via robots.txt or a comparable standard, an opt-out option from AI-generated summaries, initially directly with the respective provider and from early 2027 bundled in the Cloudflare dashboard, insight at the URL level into training usage, and assurance that an opt-out from training does not worsen search ranking. Apple, Google, and Microsoft meet these criteria for their combined search-and-training crawlers like Googlebot and Applebot. Amazon, Anthropic, Meta, and OpenAI are also considered Accountable for their separately operated training crawlers. Therefore, anyone who does not block any of these seven providers can refuse AI training without disappearing from Google or Bing results – for all other unlisted mixed crawlers, however, the strictest chosen rule applies.
Numbers show cautious use of the new controls
Despite the new options, only a few website operators are currently using granular bot controls: According to Cloudflare’s own, independently unverified network data, less than one percent of all customers block search engine crawlers, while 17 percent restrict AI training. Mixed crawlers, which bundle search and training in a bot identifier, account for 36.6 percent of verified crawler traffic – exactly the group for which the new Accountable status is intended. Cloudflare justifies the effort with user behavior: More than 40 percent of people who read an AI summary abandon their search afterward, while visitors coming through AI search services convert three to five times more often than traditional search visitors. For operators of ad-funded sites, this creates a conflict of interest between visibility in new AI search paths and protecting their own content from unpaid training.
It remains to be seen whether the currently low usage rate of 17 percent will increase once more operators learn about the new fine-tuning – or whether the complexity of four bot categories and four Accountable criteria will deter many. A real litmus test will come only in early 2027 when Cloudflare plans to integrate the opt-out from AI summaries into its own dashboard: only then could the entire range of AI usage – training, search, and automatically generated responses – actually be controlled in one place.
