In our case we're completely allowed to use AI, but if you do, you carry the responsibility for it's output (not in a legal sense obviously). So if your LLM says "Sure, go ahead and run that code" and it deletes the production database, that's your fault, just as much as if you reviewed it manually and said the same thing.
> you carry the responsibility for it's output (not in a legal sense obviously)
why not in a legal sense? If someone asks me what cleaning supplies are safe to mix, and i just ask some chatbot, don't vet the output, copy the response, and they end up poisoning themselves, am I not responsible?
If I'm a lawyer, and pass unvetted AI legal advice to my client, and they go to jail, should i keep my license?