alt Hacker NewsIs it only possible to have success with paid versions of these LLMs?
Google's "Ask AI" and ChatGPT's free models seem to be consistently bad to the point where I've mostly stopped using them.
I've lost track of how many times it was like "yes, you're right, I've looked at the code you've linked and I see it is using a newer version than what I had access to. I've thoroughly scanned it and here's the final solution that works".
And then the solution fails because it references a flag or option that doesn't even exist. Not even in the old or new version, a complete hallucination.
It also seems like the more context it has, the worse it becomes and it starts blending in previous solutions that you explained didn't work already that are organized slightly different in the code but does the wrong thing.
This happens to me almost every time I use it. I couldn't imagine paying for these results, it would be a huge waste of money and time.
Replies
I personally didn't get good results until I got the $100/mo claude plan (and still often hit $180/mo from spending extra credits)
It's not that the model is better than the cheaper plans, but experimenting with and revising prompts takes dozens of iterations for me, and I'm often multiple dollars in when I realize I need to restart with a better plan.
It also takes time and experimentation to get a good feel for context management, which costs money.
Yes, unfortunately the free version of Claude, Gemini or ChatGPT coding models can't compare with the paid ones, and are just not that useful. But, there are alternatives like GLM and Grok that can be quite useful, depending on the task.
At least from what I’ve seen, yes you do have to pay for anything useful. But just the cheaper plans seem worth the price.
It depends.
Google's AI that gloms on to search is not particularly good for programming. I don't use any OpenAI stuff but talking to those that do, their models are not good for programming compared to equivalent ones from Anthropic or google.
I have good success with free gemini used either via the web UI or with aider. That can handle some simple software dev. The new qwen3.5 is pretty good considering its size, though multi-$k of local GPU is not exactly "free".
But, this also all depends on the experience level of the developer. If you are gonna vibe code, you'll likely need to use a paid model to achieve results even close to what an experienced developer can achieve with lesser models (or their own brain).