alt Hacker NewsI think the reason the pelican example is great is because it's bizarre enough that it's unlikely that to appear in the training as one unified picture.
If we picked something more common, like say, a hot dog with toppings, then the training contamination is much harder to control.
Replies
rvnx • yesterday at 8:14 PM
It's the most common SVG test, it's the equivalent of Will Smith eating spaghettis, so obviously they benchmax toward it
I think it's now part of their training though, thanks to Simon constantly testing every new model against it, and sharing his results publicly.
There's a specific term for this in education and applied linguistics: the washback effect.