I get your point, but I’d say they do set goals, they’re just do bad at achieving them that it’s hard to tell.

Hopefully AI would help us better achieve our goals, but they still need to be our goals. I’m just not sure what that means. I don’t think anybody does.

That’s a major problem here, if we can’t reliably articulate our goals in unambiguous terms, how in earth can we expect AI to help us achieve them? The chances that whatever they end up achieving will match what we will actually like after the fact seems near zero.