Compression is prediction

i do agree with that point of view. I often referred to models as 'modern mp3s' storing a lossfull but lookalike version of information in order to counter that 'AI is totally new and not violating copyright by storing information in a magic fashion' argument.

Author's bio:

> Annie Sexton is a Developer Educator at ngrok with a passion for nerd-sniping developers.

So a company with an wildly superior compression algorithm stumbling into a society-breaking AI isn't so far-fetched. Jesus...Silicon Valley really was ahead of its time.

I mean, save for the part where the founders recognised the threat it posed to society and acted responsibly rather than unleashing it on the public and sucking down billions in VC money.

did the SSL cert expire? i'm getting a big scary warning about this blog

Reminds me of this...

In Silicon Valley, Richard Hendricks creates a revolutionary lossless data compression algorithm for his startup, Pied Piper.

This is simply wrong.

Compression requires prediction.

The better the prediction, the better the compression, whether you are measuring fidelity or result size.

This doesn't mean that compression is prediction.

I was rushing to post this and then found out somebody had already

There

Perhaps a similar observation; https://news.ycombinator.com/item?id=48703636 :

> Compression, Predictive modeling, or Complexity?

Perhaps a bad example: https://news.ycombinator.com/item?id=38400380 :

> "78% MNIST accuracy using GZIP in under 10 lines of code" (2023) https://news.ycombinator.com/item?id=37583593

Not really, compression doesn't require a world model, it's mathematically pure. Any AGI system must periodically reset it's prediction since the world is inherently stochastic. When we look at prediction markets they only seem to work in the long term because human language is abstracted away from the real world, again it's mathematically pure. That's why we get bugs in code and disputes with prediction outcomes.

A better title, you can improve your compression if you make an accurate prediction. Much like how a branch predictor can make a CPU do the same work in less time. Or when your symbols are true uncompressable rules of reality (which is probably meaningless both semantically and physically again due to inherent randomness)

The main difference between minimalist and maximalists are how much that set of uncompressable rules gives you. I suspect the search space is too large. What we see in practice is that lossy rules let you cover more ground but eventually you hit a wall and have to move to a lower level of abstraction to make progress. There are 10^360 paths in a go tree, but something like 10^300,000 for molecular chemistry and that's not even all the way up (or down, say 10^3000 for the standard model of physics that's 10^900,000,000 if you want to do chemistry without chemistry abstractions.). Just semiconductor fab is 10^(10^11) so 10^(10^16) with molecular chemistry (think finding an implementation for some sort of desired self assembly outcome). AI can be way way way smarter than humans and there just not be enough energy in the universe to find these needles. So we definitely need abstractions, but those are at odds with predictions and the choice of symbols often introduces abstractions that the designer didn't consider.

[flagged]

[flagged]

[dead]

[dead]

[dead]

I'm surprised no one has mentioned the recent 3 Blue 1 Brown video on this topic: https://www.youtube.com/watch?v=l6DKRf-fAAM