alt Hacker NewsSo you're not wrong in most of these conclusions, however this is very surface level use case for these tools when it comes to scientific research.
> I would never trust an AI to make a scientific conclusion, but they're very good at some things
> In this case, usually it will screw up and overfit, but it's a good smoke test.
> it's being used for tedious things that provide some value
I see a lot of comments like this about AI use in general, and I suspect there’s a kind of learned helplessness combined with anchoring to earlier experiences. Someone tries a model, encounters a few failures, and forms a fairly permanent view of what these systems can do.
My concern is that people sometimes turn an expectation of failure into a permanent ceiling on how much they’ll even experiment with these tools. Their limited use then keeps confirming their limited expectations.
EDIT: This came off as criticizing the user, but this is entirely to validate my own thesis that for quite a long time nobody has been able to crack the usability challenge when it comes to non-coding knowledge work for these harnesses. It's fascinating because it seems like it's such an easy problem to solve, but in reality it's a complex product problem.
Replies
Experimenting with these tools, checking their work and spending that time is not free of cost.
> this is very surface level use case for these tools
Correct. As of Q4 2026, in my experience they are only trustworthy for surface level use cases in the domain of computational chemistry / in-silico high throughout screening, unless you are working on a highly constrained, immediately verifiable and well-studied problem like protein docking or structure-property relations.
> anchoring to earlier experiences.
I have observed the models improving over time. I am basing my post on things at the level of Opus 5.5 / GPT-6.1 Sol / Fable 5.1
> Their limited use then keeps confirming their limited expectations.
Every time a new model comes out or a harness I use gets a new feature, I do try to push it until it breaks. A swarm of GPT Lunas at max context is highly effective at finding information or condensing highly disparate data sources; that wasn't even a thing a year ago in most harnesses.
But even the smartest models like Fable will get sidetracked, or make bad decisions because they do not have enough context of the problem domain. As of Q4 2026.
I'd put their autonomous intelligence in this particular domain at the level of a well-read first year grad student. Phenomenal at knowing things, but unable to question their own assumptions or invalidate their own work unless you hold their hand in the form of a prompt, skill, or direct instruction.