can they be considered to have compressed the entirety of their training data into their weights?
If you consider them lossy compression, then yes.
The goal would be to find the minimum model that, with a fixed seed, would exactly reproduce your text.
alt Hacker News
If you consider them lossy compression, then yes.
The goal would be to find the minimum model that, with a fixed seed, would exactly reproduce your text.