logoalt Hacker News

nharzirotoday at 2:58 AM0 repliesview on HN

I built a benchmark myself to track this and confirmed the same thing

https://gist.github.com/nharziro/aed0c364ce2f295a493494c6f1b...

Very similar performance to 4.6 and codex 5.3 but slow and token inefficient. Still wildly impressive. Initially I didn't believe the results because 3.6 27b couldn't complete the benchmark so this is a massive leap in capability.