alt Hacker NewsAll these "intelligence" benchmarks miss something extremely important when using an LLM in a code-agent harness: How it communicates with you about what it did.
Opus-5 is practically unusable (for complex tasks) in this sense - its updates are voluminous, and dense with cryptic language (there are numerous reddit threads complaining about this, so it's not just me). I often have to ask it to re-state concisely in plain terms.
For a fairly gnarly task, after fighting with with Claude-Code + Opus-5, I ported my session to Codex + GPT-5.6-sol, and it was like a breath of fresh air.
Arguably a key aspect of intelligence is concise, clear communication, and current benchmarks miss that, at least as far as I'm aware. I would think some arena-type benchmarks where humans rate responses would measure this, though I'm not sure which those are.
Replies
GPT 5.6 has similar language quirks that makes its comments nearly unusable.
I wonder if this is a side effect of MoE models — they can write excellent prose, but not simultaneously with writing code.
Damn I thought it was my extra instructions, I swear everything it writes is in some shorthand with direct references to variables that literally nobody could figure out unless you literally just wrote that code 5 minutes ago. I had it stop writing comments altogether cause it was always four lines of complete and utter nonsense, and it doesn't even obey that rule half the time. Despite doing an extensive back and forth to make a complete plan, 5 seconds into the implementation it changes its mind and makes another assumption, adding some extra thing that tends to break the entire approach and needs follow-ups to repair or cleanup. Instruction following is basically non-existent compared to Fable, it just does whatever the fuck it wants.
Is that what it feels like when the models get smarter than us?