Would love to find out they're overfitting for pelican drawings.
Even if not intentionally, it is probably leaking into training sets.
The estimation I did 4 months ago:
> there are approximately 200k common nouns in English, and then we square that, we get 40 billion combinations. At one second per, that's ~1200 years, but then if we parallelize it on a supercomputer that can do 100,000 per second that would only take 3 days. Given that ChatGPT was trained on all of the Internet and every book written, I'm not sure that still seems infeasible.
OpenAI claims not to: https://x.com/aidan_mclau/status/1986255202132042164