logoalt Hacker News

ariwilson • today at 7:12 PM • 0 replies • view on HN

I ran it through my own coding eval today using pi/OpenRouter (11 small bug fixes from my own projects). It solved 10 of 11. Compared to Sonnet 5.5 and GPT-6.1 Sol, a median run cost me $0.52 vs $0.18 for Sonnet and $0.09 for Sol, and took ~4 minutes vs 32 seconds and ~1.5 minutes.

It thinks a ton, ~24k output tokens a run vs ~3k for Sonnet. It missed a solution by thinking its way into a 10 min timeoout.

Something to look out for - if you're using pi, it doesn't have Argon in its catalog yet and reports cost at Kimi's rates, about a third of what OpenRouter actually bills.