logoalt Hacker News

jrflotoday at 3:50 PM0 repliesview on HN

The conflict of interest is real there, but some benchmarks really ought to be closed source. Otherwise, the second your benchmark is public labs will overfit their new models on it and it will cease to be useful.