logoalt Hacker News

kees99today at 9:52 AM1 replyview on HN

> Cloudflare, Google, Anubis, and all these other captcha services trying to see if you're a human.

Yep. IMO, this is so far the biggest AI-inflicted damage to the web. A bit of anecdata - wikipedia (and all other wikimedia sites) are blocking my Firefox since about a week, with a "please respect our bot policy" message. Outright block, not even a captcha.

It took me a while to figure out they don't like me disabling some SSL ciphers, so now "JA4 browser fingerprint" is not matching user-agent. Funnily enough curl (what I would imagine a bot would use) pulls exact same URLs from exact same client IP, just fine.


Replies

ImaCaketoday at 11:39 AM

Maybe the curl thing is because they are happy to let you do some light scraping. What they want to avoid is bots directly crawling the page interactively. No one seems to be blocking chatGPT when I promot it to use it's web search skill anyway.