It appears they don't need to reason or be intelligent to be able to produce working solutions for code. But sure let wild and unmonitored? I wrangle my LLMs like the code monkeys they are. They help materialize code and then you need to sculpt it (and test harness of varying sorts)

It really can be useful. It's very different from old world programming.