There’s a new conflict on the internet between Perplexity AI and Cloudflare. Where the internet infrastructure company accusing the AI search startup of using undeclared crawlers to evade website restrictions. However, Perplexity has fired back, calling the claims technically flawed and misleading.
The whole story started when Cloudflare alleged that Perplexity is bypassing websites‘ robots.txt directives and firewall blocks by rotating IPs and disguising its crawling activity behind generic browser user agents. According to Cloudflare, this “stealth crawling” enabled Perplexity to scrape content from websites that had explicitly opted out of such access.
Cloudflare said it tested the behavior by setting up newly created domains that blocked all bot activity. Despite these protections, Perplexity was still able to provide answers based on the restricted content, leading Cloudflare to conclude that Perplexity was using hidden, undeclared crawlers to access data it wasn’t supposed to.
In response, Perplexity has strongly denied the accusations. In a detailed blog post, the company claimed that Cloudflare fundamentally misunderstood how AI assistants work, especially the difference between traditional bots that scrape data at scale and user-driven agents that fetch information in response to specific questions.
“When Perplexity fetches a webpage, it’s because a user asked for it. The content isn’t stored or used for training; it’s used immediately to answer a real person’s question,” the company said. Perplexity argued that such behaviour is no different from how search engines like Google retrieve content on behalf of users during previews.
Perplexity also disputed Cloudflare’s traffic analysis, saying the 3-6 million daily requests Cloudflare attributed to it actually came from BrowserBase, a third-party browser automation service that Perplexity uses only occasionally for specialised tasks. The company estimated that less than 45,000 of those requests were actually tied to its operations.
Cloudflare’s claims reflect either a serious technical error or a publicity stunt. If you can’t tell a helpful digital assistant from a malicious scraper, then you probably shouldn’t be deciding what counts as legitimate web traffic.
Perplexity
However, Cloudflare has removed Perplexity from its list of verified bots and updated its bot management rules to block what it describes as stealth crawling. The company says millions of sites now use its tools to restrict or block AI-driven access, and it continues to collaborate with global standards bodies to define acceptable bot behavior.
Perplexity, on the other hand, is pushing back against what it sees as gatekeeping that could hurt smaller companies and limit user access to public information. The company called for a more nuanced understanding and clear guidelines around how modern AI agents should be treated.
The dispute is unlikely to be the last, as more content owners and infrastructure providers reassess their relationships with AI services, and as the definition of a “bot” continues to evolve. So if you want to stay updated with all the developments in this space and want to get all the latest tech and AI news, then join us on WhatsApp.
