I haven't read this paper in details, but I have read papers about the platonic hypothesis in the past. Usually one of their weakness is about the design an experiment proving that "neural networks converges towards the same latent space when scaling". Usually authors are effectively proving a subset of the hypothesis, that can be explained by the size of the model and the transfer learning part, leaving not definite answer.

Here it seems that their definition is more robust (the representation is defined up to a rotation), but I am wondering how much more re-training they need to perform.