TechAI’s biggest names back a new way to opt out of scraping

OpenAI, Google, Microsoft, and Anthropic say they will adopt a shared standard that lets publishers set AI training preferences for multiple crawlers through robots.txt. Anthropic has already adopted the signal; Google and Microsoft plan support next year, while OpenAI says it will implement it in the coming months.

AIweb scrapingpublishersweb standards

TThe Verge★★☆☆☆2026-10-30 20:26Original