I'd have thought the same but this article from yesterday blew my mind https://www.amazon.science/blog/why-dont-machine-learning-re...

As it essential implies that these models compress knowledge well in a way that what remains is what's generalizeable more so than remembering every specific detail... Anyway more understanding necessary but thought provoking