Web giant Cloudflare to block AI bots from scraping content by default

Core Viewpoint - Cloudflare will block AI crawlers from accessing content without website owners' permission or compensation by default, impacting AI developers' ability to train their models [1][3]. Group 1: Cloudflare's New Policy - Starting Tuesday, new web domains signing up to Cloudflare will be asked if they want to allow AI crawlers, giving them control over data scraping [2]. - This move builds on a tool launched in September last year that allowed publishers to block AI crawlers with a single click, now making it the default for all websites [6]. Group 2: Impact on AI Development - Approximately 16% of global internet traffic goes through Cloudflare's CDN, indicating its significant role in online content delivery [3]. - AI crawlers have been accused of depriving publishers of traffic and revenue by collecting data without directing users to original sources [5]. - If effective, this development could hinder AI chatbots' ability to harvest data for training, potentially impacting the viability of AI models in the long term [8]. Group 3: Industry Reactions - OpenAI declined to participate in Cloudflare's plan, arguing that it adds a middleman to the system [6]. - AI crawlers are viewed as invasive and have been criticized for overwhelming websites and affecting user experience [7].