As generative AI services such as OpenAI’s ChatGPT, Microsoft’s Bing Chat and Google Bard are increasingly used as substitutes for search engines, they have also encountered resistance from some individuals and companies. , these website developers do not want their website data to be used for AI model training.
Today, Google announced a new way for website developers to choose to allow its Bard and Vertex AI services to access their content, or to opt out of using these APIs to train models.
In a blog post, Google announced the launch of Google-Extended, a new control that website developers can use to manage their sites with the help of Bard and Vertex AI.
Websites can currently provide disavowal lists via robots.txt, and Google believes that all AI model providers should provide this similar method of transparency and control. However, as AI applications expand, websites will face the increasing complexity of managing different uses at scale. Google said it will share more information soon.
It is found from Google support documents that Google-Extended does not have a separate HTTP request user agent (UA) string, and the crawling is completed through the existing Google UA string; robots.txt user-agent token can be used Take control.
In addition to today’s announcement, Google said it will “explore other machine-readable ways to provide selection and control for sites.” It includes a link where sites can sign up for a mailing list where they will receive Additional updates about Google’s improved website controls.
