logoalt Hacker News

NitpickLawyertoday at 4:10 PM1 replyview on HN

Sure, but that's for "personal" serving. I meant for 3rd party providers. Usually we get a good indication on what it costs to host this, as the prices settle on open router. That's why I said it's tougher to serve than kimi k3 on launch. As a provider you'd do fp8 if the model creator didn't do QAT on q4, or until someone does a good calibrated nvfp4. And that's usually nvda :)


Replies

MrDrMcCoytoday at 4:19 PM

That makes sense, but your specific phrasing precluded the possibility of non-QAT quantization.

show 1 reply