Yep the whole thing is retarded. You cannot trust that a non-deterministic program (i.e. an LLM) will ever do what you actually tell it to do. Letting those things loose on the command line is incredibly stupid, but people out there don't care because they think "it's the future!".