logoalt Hacker News

data-ottawayesterday at 8:46 PM1 replyview on HN

reCaptcha is a pretty strong wall to allow only Google to index websites, especially now that you need device verification. Throw in Cloudflare too.

There’s not much room to squeeze in when your competitors hold the keys to 15 million top websites.


Replies

xmcp123yesterday at 11:22 PM

I write a lot of scrapers. Both of those are pretty trivial to bypass at scale.

show 1 reply