alt Hacker News"There's insurance for that, another domain that's a harness around a model"
Yes, but the PE's at engineering firms have the ability to say no and demand redesigns since it's their name on it, unlike programmers.
I frequently see firms design way beyond the bare minimum so that there's no question and it's an easy sign off rather than spending months on a back and forth.
Head architects at software firms had the ability to say no to generated code and demand redesigns. They were pushed out for being "backwards thinking".
As to your second point, that just means that the training corpus is already designed with a healthy margin of error.