logoalt Hacker News

penagwin • today at 2:38 PM • 0 replies • view on HN

In my own experience the vast majority of llm consumers would prefer to pay by token, especially for large (300B+) as either their usage is below the point a dedicated cluster is viable or it’s just a lot more hands on.

With Kimi k3 specifically you need something like 1.6TB of memory which... Well you can’t just use 8x H100’s, and per token pricing becomes extremely attractive at this size for most users.