I used to roll my own (respectful) scrapers. AI and vibecoders have ruined it for me. Everyone has super sensitive rate limiting and not checking now.