logoalt Hacker News

kentonvyesterday at 6:29 PM0 repliesview on HN

You can trivially block Cloudflare crawl via robots.txt. You don't need to buy Cloudflare's bot protection -- this is not a malicious bot.

https://x.com/CloudflareDev/status/2031745285517455615

(Disclosure: I work for Cloudflare but not on this product. I get pretty tired of the conspiracy theories TBH.)