Therefore Einstein couldn't have had the idea of relativity because it wasn't in his training data.

Joking aside, the reality is that a pre-Einstein in 1850 couldn't have had the idea of relativity because Maxwell wasn't in his training data.

And a pre-Maxwell couldn't have had the idea of electromagnetism in 1820 because Faraday wasn't in his training data.

What we call "new ideas" are actually just deduced from the available training data, which includes all the previous ideas and all experimental results. AI can do the exact same thing (I assert). Perhaps not today's AI, but some AI, probably not too far in the future.