Making systems fully deterministic ignores the entire purpose of having agents involved.

IMHO the best of both worlds option is agents working with deterministic CLIs. Where the agent does the reasoning (and text generation) but uses CLIs to carry out all of the actions (issuing refunds, unblocking accounts, or whatever).

It's possible to get very reliable and consistent work out of agents when they're using well written prompts with well designed CLIs.