alt Hacker NewsCorrect. But the 5% more solves and 3-5% fewer tokens are within the noise of previous eval runs I've done on models. The faster response time is not.
Correct. But the 5% more solves and 3-5% fewer tokens are within the noise of previous eval runs I've done on models. The faster response time is not.
Someone using an actual statistical test to quantify if there's actual measurable improvement?! An angel among men.