You need to account for the fact that a lot of scraping is delegated by the big players to smaller players who can take loss of reputation and can use questionable methods (residential IPs). Some of the scrapers from these AI companies have been written very poorly from performance perspective.

From what I understand the pressure has created even Google to be a lot more aggressive than what it was before. Not entirely sure but I believe google has two categories of scrapers, the regular one and a new one for AI

One of the defining characteristics of AI turned out to be utter and profound facelessness. The scraping, the data centers, the slop content, it's like it all comes from thin air. Nobody is talking about who is really doing all this or why, because it's always just coming from... somewhere else outside of our place.

I sometimes still wonder exactly who these scrapers are

This has zero to do with AI (it's just a convenient scapegoat that fits the narrative). Absolutely everything to do with those pushing for centralised control of the Internet.

Most of the nonsense comes through botnets/residential proxies, so it's very hard to say.

Well, some websites claimed China is behind it, which could make sense (I would not know either way). At the same time, though, I kind of doubt your carte blanche here for all those companies. Why would you think none of them are responsible for the AI slop spam?

> Our largest offenders seems to be mostly limited to South-East Asia, so probably mostly Chinese AI projects, but that's speculation

Ok. So you also don't know. Well, I don't know either, but I don't make a speculation by claiming x, y, and z companies to be exempt. In my book they are all responsible.