Major technology companies, including Google, OpenAI, Microsoft and Anthropic, are supporting a shared system that will give website owners greater control over whether their content is used to train artificial intelligence (AI) models.
The proposed system will use a common instruction in a website's robots.txt file. Currently, publishers often need to create separate instructions for different AI companies. The new approach aims to simplify this process by allowing participating providers to recognise a shared preference.
What Does It Mean for Website Owners?
The proposed control will allow publishers to restrict the use of selected website content for AI training while keeping their websites discoverable through traditional search engines.
However, the system is not yet fully operational. Anthropic has begun implementation, Google is targeting 2027, Microsoft plans to adopt it from February 2027, and OpenAI says implementation will follow in the coming months.
The move is being seen as an important step towards giving publishers more control over their digital content as AI technology continues to expand.
Published: Sunday, 1 November 2026 | 12:37 PM IST