Workflow
Data scraping
icon
Search documents
Web giant Cloudflare to block AI bots from scraping content by default
CNBC· 2025-07-01 10:07
Core Viewpoint - Cloudflare will block AI crawlers from accessing content without website owners' permission or compensation by default, impacting AI developers' ability to train their models [1][3]. Group 1: Cloudflare's New Policy - Starting Tuesday, new web domains signing up to Cloudflare will be asked if they want to allow AI crawlers, giving them control over data scraping [2]. - This move builds on a tool launched in September last year that allowed publishers to block AI crawlers with a single click, now making it the default for all websites [6]. Group 2: Impact on AI Development - Approximately 16% of global internet traffic goes through Cloudflare's CDN, indicating its significant role in online content delivery [3]. - AI crawlers have been accused of depriving publishers of traffic and revenue by collecting data without directing users to original sources [5]. - If effective, this development could hinder AI chatbots' ability to harvest data for training, potentially impacting the viability of AI models in the long term [8]. Group 3: Industry Reactions - OpenAI declined to participate in Cloudflare's plan, arguing that it adds a middleman to the system [6]. - AI crawlers are viewed as invasive and have been criticized for overwhelming websites and affecting user experience [7].
Reddit sues AI firm Anthropic over alleged unlawful data scraping
Proactiveinvestors NA· 2025-06-05 14:50
Company Overview - Proactive is a financial news publisher that provides fast, accessible, informative, and actionable business and finance news content to a global investment audience [2] - The company has a team of experienced and qualified news journalists who produce independent content [2] Market Focus - Proactive specializes in medium and small-cap markets while also covering blue-chip companies, commodities, and broader investment stories [3] - The content includes insights across various sectors such as biotech and pharma, mining and natural resources, battery metals, oil and gas, crypto, and emerging digital and EV technologies [3] Technology Adoption - Proactive is recognized for its forward-looking approach and enthusiastic adoption of technology to enhance workflows [4] - The company utilizes automation and software tools, including generative AI, while ensuring that all content is edited and authored by humans [5]