alt Hacker News> The kernel work helped reduce the end-to-end cost of serving the model by 20%, while its experiments increased token-generation efficiency by more than 15%.
If the cost of serving GPT-5.6 just dropped by 20%, does that add up to literally billions of dollars in savings per month?
We know Anthropic spend $1.25 billion renting inference capacity from SpaceX (in two Colossus datacenters) from the SpaceX IPO, but we don't know how much of Anthropic's inference capacity that is (presumably a small fraction, since they were operating on top of AWS and other providers before the SpaceX deal.)
I've not seen any numbers that hint at OpenAI's per-month inference bill, but surely that has to be in the multiple billions of dollars as well.
So 20% is a really, really big deal.
Replies
~2 years ago gemini2.5 helped write better kernes for itself and (only) reached 1% efficiency gains. Today we're at 20%.
imagine writing that on your resume
> reduced inference cost by 20 percent saving company x billion dollars per month
Some numbers: https://www.wheresyoured.at/exclusive-openai-financials/
If those numbers are accurate, I don't think 20% is a really, really big deal. It's like saying "we're digging our grave 20% slower." Ok, but they're still digging!
Or, different analogy, if I'm going broke because I lost my job due to executive AI psychosis, cancelling my netflix subscription doesn't really change the math of not being able to afford rent. It doesn't even really slow it. The amount of money that OpenAI is spending is so absurd that a minor cost saving is like, uh, some progress, but they'd need to do it a lot more to move the needle