alt Hacker NewsWe have it slightly ahead of Fable in our multi-agent coding evaluations.
Fable's main advantage is that its average solution size is smaller. However, GPT 5.6 Sol is a substantial improvement from GPT 5.4/5.5 which would write verbose, defensive code. 31KB for GPT 5.4/5.5 down to 26KB for GPT 5.6 Sol, with better performance for Sol.
Fable scores slightly lower, but with an average solution size of 12.2 KB.
This looks like a good benchmark. Time and time again I keep giving OpenAI models the chance to win me back, but Opus (and Fable especially) just writes more elegant code and is a significantly more productive rubber duck for interactive discussions. I feel vindicated seeing your description of verbose and defensive code, and I’m a bit disappointed that 5.6 Sol’s solution is still >5x longer than the human solution and 2x as verbose as Fable’s. Do you have any insight whether any of that is comments?
I wonder why nobody has tried to optimize for actual code size or complexity metric, or at least why I haven’t seen more benchmarks that display this. GPT5.5 just keeps pushing more and more pointless indirection into every function it writes in my main project, it’s borderline negative productivity.
P.S. I’d be curious to see Cursor’s composer models in there, they seem to be among the best performing low cost models: https://artificialanalysis.ai/articles/cursor-composer-2-5-c...