Four of the biggest AI companies, Google, OpenAI, Anthropic and Microsoft, said they will support a shared signal letting websites mark that they do not want their content used to train models. Until now, a site wanting to block training had to add separate instructions for each company, for each differently named crawler. The new standard is meant to replace all of those with one instruction.
How does it work? Almost every site has a file called robots.txt that tells search engine and AI company robots which parts of the site they may crawl. The new standard adds a shared marker, __usage_ai-training, telling participating crawlers that the site owner does not want pages used to train generative AI. According to reports, Anthropic has already begun implementing it, Google plans to support it in 2027, and Microsoft will add separate controls for training and for citation in answers.
What does it not do? Three important things. It covers only training, not the use of content to answer questions in real time, known as 'grounding'; a chatbot may still read your article and summarize it for a user. It is voluntary and depends on the companies' goodwill. And it does not address content already collected and used for training, nor whether and how much publishers should be paid.
Why does it matter to Israeli sites, including news sites like ours? Because it is the first time the big companies have agreed on a common language on this, which makes it much easier for site owners to decide and apply a policy. But the decision itself is not simple: blocking training does not necessarily affect regular search results, and some sites actually prefer their content to appear and be cited in AI tools. The advice for site owners: decide on a policy, update robots.txt accordingly, and watch how each company actually implements the standard.
