alt Hacker NewsIf I read your results correct it solved 5% more problems and used up between 3-5% fewer tokens, with a faster response time?
Replies
astro1234 • today at 8:23 PM
Uncertainty from count statistics are far larger than this. These are two perfectly consistent results there’s no statsig difference
Correct. But the 5% more solves and 3-5% fewer tokens are within the noise of previous eval runs I've done on models. The faster response time is not.