I think it's the loss of control.

Even if you can achieve awesome things with LLMs you give up the control over tiny details, it's just faster to generate and regenerate until it fits the spec.

But you never quite know how long it takes or how much you have to shave that square peg.