Cloudflare has accused AI startup Perplexity of deliberately circumventing anti-scraping protocols, including Robots.txt directives and bot detection systems, in order to access web content that site owners had explicitly restricted. In a newly published report, Cloudflare detailed how Perplexity allegedly masked its identity by manipulating user-agent strings and rotating autonomous system numbers (ASNs) to appear as legitimate browser traffic.
The activity, observed across millions of daily requests, prompted Cloudflare to de-list Perplexity’s bots and implement new countermeasures to block them. Cloudflare said it initiated the investigation following complaints from customers who reported Perplexity scraping their sites despite implementing safeguards.
Perplexity denied the allegations, but the dispute underscores growing tensions between AI firms reliant on large-scale data extraction and internet infrastructure companies seeking to enforce content access boundaries.
