logoalt Hacker News

seizethecheese • today at 3:49 AM • 1 reply • view on HN

Does anyone really believe the cost data for benchmarks? Pretty sure all models will behave differently at this point when they get benchmark shaped questions.

(And I’m sitting on data showing Sol 6.1 and Qwen 3.8 Max cost roughly the same per task, for real tasks.)


Replies

killingtime74 • today at 3:54 AM

I mean not believing intelligence score is reasonable but cost is just reporting what they spent?

➕ show 1 reply