[ai-control] Background reading

Mark Nottingham <mnot@mnot.net> Fri, 15 March 2024 05:18 UTC

Feedback-ID: ie6694242:Fastmail
From: Mark Nottingham <mnot@mnot.net>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\))
Message-Id: <3BA0785E-EDE1-46B3-B844-AD19FEB44A6C@mnot.net>
Date: Fri, 15 Mar 2024 16:18:45 +1100
To: ai-control@ietf.org
Archived-At: <https://mailarchive.ietf.org/arch/msg/ai-control/SrGx2pnqx19BwftIpTS9J0Anoig>
Subject: [ai-control] Background reading
Precedence: list

I'm sure there's more, but here are some links that may be relevant / good to be aware of.

# General / overview

* Verge article <https://www.theverge.com/24067997/robots-txt-ai-text-file-web-crawlers-spiders>
* Privacy Journal <https://www.privacyjournal.net/block-llm-crawlers/>
* EFF article <https://www.eff.org/deeplinks/2023/12/no-robotstxt-how-ask-chatgpt-and-google-bard-not-use-your-website-training>
* AI blocking survey <https://palewi.re/docs/news-homepages/openai-gptbot-robotstxt.html>
* Another survey <https://originality.ai/ai-bot-blocking>
* And another <https://reutersinstitute.politics.ox.ac.uk/how-many-news-websites-block-ai-crawlers>

# Steps taken to block

* DeviantArt "noai" meta tags <https://www.deviantartsupport.com/en/article/how-do-i-change-the-noai-setting-on-my-deviations>
* Medium's stance <https://blog.medium.com/default-no-to-ai-training-on-your-stories-abb5b4589c8>
* Squarespace's controls <https://support.squarespace.com/hc/en-us/articles/360022347072-Excluding-your-site-from-AI-scans>
* Substack's controls <https://support.substack.com/hc/en-us/articles/20382615953556-How-can-I-block-AI-from-using-my-Substack-publication-to-train-their-models>
* Cloudflare announcement <https://blog.cloudflare.com/ai-bots/>

# AI vendor docs / efforts

* Google blog <https://blog.google/technology/ai/ai-web-publisher-controls-sign-up/>
* OpenAI robots.txt instructions <https://platform.openai.com/docs/gptbot>
* Common Crawl (used by some AI but also for other purposes) <https://commoncrawl.org/ccbot>
* Nothing from Anthropic

# Other

* ai.txt proposal <https://spawning.ai/ai-txt>
* Another ai.txt proposal <https://github.com/menro/ai.txt>
* Model collapse is another potential reason to label / control AI usage <https://www.axios.com/2023/08/28/ai-content-flood-model-collapse>
* There seems to be a hook for legal support of opt-out mechanisms in the EU's DSM Directive / AI Act <https://www.technollama.co.uk/the-eu-ai-act-and-copyright>
* Specifically, see Art IV (3) <https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32019L0790#d1e986-92-1>

--
Mark Nottingham   https://www.mnot.net/

[ai-control] Background reading Mark Nottingham