Which is weird, because Google is in the best position for getting training data: they have YouTube (which I guess is why Google's models are good at multimedia), as well as the Google crawler. Which effectively websites cannot opt out of if they want to be Google searchable. So Google's bot probably is the _only_ bot (scraper) that hasn't been blocked from a large portion of the internet.
I think raw base text training data may be decreasingly important vs good RL environments/gyms, and maybe less clear that Google wins by default there?
Not only that, they pioneered some of the key technologies... Who isn't still on transformer models? Also funny when you consider Meta is in a weird similar boat, introduced key things, so far behind the frontier.
Google legal refuses to allow them to do so.