logoalt Hacker News

hellisotherstoday at 7:52 PM3 repliesview on HN

I thought Sol was on par with Opus, so comparing it to Fable is apples and (very expensive) oranges?


Replies

rybosworldtoday at 8:07 PM

I've used all three extensively.

Most of the benchmarks have exceeded their usefulness. Opus 5 beats fable 5 on many of them. Anyone who has used both models will notice immediately that this doesn't translate to the real world. Opus 5 is nothing short of a regression from Opus 4.8. Fable is genuinely a great model so long as you don't trigger a guard rail and it downgrades.

Sol in my experience isn't significantly different than fable ignoring that Sol burns usage 10x faster but the end result is hard to differentiate.

GLM 5.3 is a hair behind these two.

An anecdote but not an original one from the people I talk to.

rafaelmntoday at 9:16 PM

Sol routinely catches stuff in review that fable misses for me. It's impossible to compare them meaningfully because it's a complete dice roll - but in practice using both in my projects I can get work done with both, and Opus 5 is far more tedious.

But Fable security false positives and pricing just make it not worth compared to Sol IMO.

YuechenLitoday at 8:11 PM

Yeah, it's pretty much apples to oranges, and I don't consider GPT and Claude to be interchangeable at all. From my anecdotal experience, GPTs generally codes more creatively and verbosely but Claudes tend to code more carefully and precisely, so the result is that GPTs generally finds more creative solutions to problems but also writes buggier code, which is why I converged on the setup of GPT/Codex for implementation and Claude for debugging, which feels more like a force multiplier than using each model individually.