R
Cloudflare is offering to block crawlers scraping information for AI bots.
Tech giants are rewriting the rules on web scraping, blaming unnamed third parties for disregarding robots.txt, and seemingly claiming the right to reuse anything posted anywhere for AI.
Now, Cloudflare is telling customers on its CDN that it can find and block AI bots that try to get around the rules.
The upshot of this globally aggregated data is that we can immediately detect new scraping tools and their behavior without needing to manually fingerprint the bot, ensuring that customers stay protected from the newest waves of bot activity.
Most Popular
Most Popular
The video game industry is mourning the loss of Game Informer
Google pulls Gemini AI ad from Olympics after backlash
Intel is laying off over 15,000 employees and will stop ‘non-essential work’
The Pixel 8 Pro has hit an all-time low ahead of Google’s next hardware event
EcoFlow’s Power Hat is a floppy, phone-charging solar panel for your noggin