alt Hacker NewsI'm more interested in the converse question: how well does an LLM perform as a compressor, compared to gzip (ignoring its insanely lower speed)?
Replies
anax32 • today at 8:44 AM
hallucinations are lossy compression artefacts
Ohentis • today at 8:29 PM
Much better
alienbaby • today at 6:06 PM
can they be considered to have compressed the entirety of their training data into their weights?
➕ show 1 reply
Top contestant in the Hutter Prize uses a neural network for compression. So fair to say, LLMs would perform pretty well compared to gzip.