logoalt Hacker News

Corence • today at 6:23 PM • 0 replies • view on HN

Yeah, MaxxBench was shared here a while ago and showed most models think about how a grader will evaluate their work on DeepSWE in 95%+ of traces. Strong models definitely realize they're being graded and start to game their output.

https://maxxbench.unlap.ai/research/why-maxxbench/