From what I understand, and I might be oversimplifying here; many of the differences between Indo-European languages today are a result of the languages that were there before. So the reason Proto-German and Latin were different is in part because of the pre-IE languages in each of those regions. But we don’t have a ton of information on those languages, so scholars are not sure.

Of course there were other reasons languages changed, and the inheritance is only one factor.

But in some sense there are many lost languages hiding inside contemporary ones.