logoalt Hacker News

PunchyHamsteryesterday at 1:35 AM1 replyview on HN

Given that the scraping doesn't do any rate limiting and pisses on robots.txt, yes it is abuse


Replies

singpolyma3yesterday at 2:41 AM

Is there any evidence OpenAI has been ignoring robots.txt for scraping purposes? AFAIK the main sources of that traffic are still unknown.