"Benchmarks" for LLMs are a total hoax, since you can train them on the benchmarks themselves.
I would assume a good benchmark has hidden tests, or something randomly generated that is harder to game
alt Hacker News
I would assume a good benchmark has hidden tests, or something randomly generated that is harder to game