>You literally ask the agent to do something and it does it

It's not that simple, you have to remember at the end of the day these things are just doing next token prediction. If you don't give it the proper tokens to attend to, then your outputs won't be satisfactory.

You can get stellar outputs from LLMs, but it really is a function of how well you manage your input tokens.