There is a popular movie starring Tom Cruise that explores the consequences of this question.
In Minority Report (spoilers), the precrime department is shut down because of a single error proving the system is fallible, thus making the plot preposterous in the age of AI.
In Minority Report (spoilers), the precrime department is shut down because of a single error proving the system is fallible, thus making the plot preposterous in the age of AI.