alt Hacker NewsI had a moderately complex review in a large C/C++ codebase that Codex/GPT-5.6-sol already cleaned up so I threw it at Opus 5. 4 errors found. That seemed odd, so I handed it back to GPT. All were false. Opus doesn't seem to look at the wider context and understand which functions were called in certain contexts. I gave GPT's analysis back to Opus and it admitted its mistake. Maybe it's good for writing code, but as far as analysis it seems like it needs some work.
Replies
That has always been the major strength of GPT, that's the model you use for checking. It often nearly isn't as good for creation though.
A bit worrying that at no point in here did you say you investigated the errors. LLM 1 is disagreeing with LLM 2. Shouldn’t you be the tie breaker?
Did you ask ChatGPT the same question you asked Claude from a fresh context?
> I gave GPT's analysis back to Opus and it admitted its mistake
How do you know if it was not mistakenly admitting its mistake?