Agree that chess is not the ideal example to use for the AI domain.. but used for building a quick mental image.. reason being that in chess, AI does have an objective truth built into its training.. and that is to checkmate.

The real world open ended research doesn't always have an evaluation function. A computer can calculate 10 million options to solve a problem, but it cannot tell you if you're solving the right problem from the human context. That's why centaur setups fail in closed games, but remain essential everywhere else.

Another loose example just to build a mental model .. AI driven cars do a great job in most driving situations but often fail to park in a good parking spot for most people! Good parking spot is partly human initiated and subjective.