alt Hacker Newstheir MLA architecture cuts KV cache by ~5-13x vs standard attention. that's why inference is actually cheaper to run, not just a price war to gain market share.
Replies
vitorsr • today at 7:51 PM
Yes. The discount was most likely a "post-market trial" of how efficient the caching works for the new generation models.
➕ show 1 reply
hmaddipatla • today at 6:30 PM
[dead]
That's also a game changer for local inference. It unlocks long contexts, batched inference and storing the KV cache to disk on ordinary consumer platforms.