> this is very surface level use case for these tools

Correct. As of Q4 2026, in my experience they are only trustworthy for surface level use cases in the domain of computational chemistry / in-silico high throughout screening, unless you are working on a highly constrained, immediately verifiable and well-studied problem like protein docking or structure-property relations.

> anchoring to earlier experiences.

I have observed the models improving over time. I am basing my post on things at the level of Opus 5.5 / GPT-6.1 Sol / Fable 5.1

> Their limited use then keeps confirming their limited expectations.

Every time a new model comes out or a harness I use gets a new feature, I do try to push it until it breaks. A swarm of GPT Lunas at max context is highly effective at finding information or condensing highly disparate data sources; that wasn't even a thing a year ago in most harnesses.

But even the smartest models like Fable will get sidetracked, or make bad decisions because they do not have enough context of the problem domain. As of Q4 2026.

I'd put their autonomous intelligence in this particular domain at the level of a well-read first year grad student. Phenomenal at knowing things, but unable to question their own assumptions or invalidate their own work unless you hold their hand in the form of a prompt, skill, or direct instruction.