"human experts work with AI to decide what to prove"
Why can't ai do this too? Is this the fundamental limit of ai?
Because "what to prove" is fundamentally bound in a value judgment, not merely the mechanistic application of reasoning.
Because that’s a design question. The AI has to know what it’s actually supposed to be building.
we are still involving humans in the loop, it helps them acquiesce
Because humans are the ones who care? At some point, you have to make the wish, or the genie is not going to do anything.