Ya, super interesting research area the authors explored of basically trying to answer the question: "Is there a canonical/intrinsic way that concepts/representations/information are 'stored' in the universe/reality?".

They tested that by performing "spectral analysis of over 1100 models - including 500 Mistral-7B LoRAs, 500 Vision Transformers, and 50 LLaMA-8B models ... by applying spectral decomposition techniques to the weight matrices of various architectures", and concluding that "deep neural networks trained across diverse tasks exhibit remarkably similar low-dimensional parametric subspaces", showing that "neural networks systematically converge to shared spectral subspaces regardless of initialization, task, or domain".

Not just philosophically interesting but also has practical implications for being smarter about how to reuse models, model merging, developing more sustainable training and inference algos, etc.

Paper source: https://arxiv.org/abs/2512.05117