logoalt Hacker News

yunuserturk • today at 6:53 AM • 0 replies • view on HN

Hosting providers and cloudflare already block bots using bot lists. What user-agent does their agent identify itself as? Does it follow robots.txt, or is there a separate signal? What happens when it gets caught by cf's bot protection?