alt Hacker News> I imagine lack of training data would be a problem
I imagine that's an understatement. Using only the written works available from before Einstein (let alone Newton or Plato), you'd probably need a whole new technology that could train itself without any training data other than observation of the world.
arguably it is feasible in principle to filter out that knowledge from later writings, but you can always say it leaked in a subtle way - "this chonky cat is bending space around it" (this is exaggerated, can be much more subtle and hard to notice)