Because you don’t need TPUs and anyone can scrape the internet
I think that has been well proven so far
Many sites deliberately whitelist Google IP's for their search crawler and ignore crawlers who just uses the Http UserAgent of GoogleBot.
alt Hacker News
Many sites deliberately whitelist Google IP's for their search crawler and ignore crawlers who just uses the Http UserAgent of GoogleBot.