alt Hacker NewsSometimes that'll turn up real bugs, sometimes just overengineered designs, premature-optimization, and 1-in-a-million possibility "bugs".
And sometimes it's not about the model, it's just about refining the search space. E.g. I've had Opus write tests and GPT 5.5 write the implementation passing all the tests. Then ask about that specific implementation and find some real corner cases. Add those to tests, etc.
But the other fun trick that's been working better and better on the GPT-5.6 series is that even the lower-end models can find the things they didn't think of first when inspecting the already-written output.
I think there's still a bit of hard-to-quantify "creativity" to the bigger models - especially when trying to untangle (a) is this edge case that the model built a complicated way to avoid real/worth worrying about and also (b) even if it is real, is there not a better way to mitigate it? But it might be confirmation bias, in a way that definitely didn't use to be true about GPT-5.3 for planning and Composer 2 for implementation, say.
Replies
They do tend to overengineer. The other day 5.6 Sol generated a while loop around a uuid4 call to make sure the generated ids were unique...
I wonder if that's also how I'd behave if I had a reinforcement learning harness around me that dived hard on and punished me for every small mistake.
> even the lower-end models can find the things they didn't think of first when inspecting the already-written output
I use Deepseek V4 Pro for my hobby project (an OpenCode client). The economics will obviously be different at work.
It’s worse than the GPTs, but indeed, if you focus it on reviewing its own code (essentially spending more reasoning and changing perspective), it’s also quite capable at improving its own approaches.
The GPTs are better at general architecting, but I think a lot of the performance gains also came from my more careful prompting (“okay, this is a hard problem, let’s think this through …”) to make using the more expensive model worth it. I use them and they are worth it on subsidized rates, but not at API prices. Since my Codex sub ran out I sometimes miss the models, but it really hasn’t devastated me.