> "one-shotting" is fun for a low (human) effort toy, but it's a mistake to get overly hung up on LLMs getting everything right on the first try, since a lot of that is the harness and the internal prompting that makes it burn extra tokens to test everything to get it in that one "shot" (ui turn).

I'd be more specific and state that these models are now way far from being LLMs and instead they are full blown agent orchestration systems capable of running multi-layer ReAct loops and tool-calling, and with your whole session history as context. It doesn't make any sense to refer to them as glorified chatbots as the LLM part is nowadays far from being the component that delivers the highest value.