logoalt Hacker News

4by4by4 • today at 1:23 PM • 0 replies • view on HN

Does anyone have rough numbers on how this compares to vLLM in terms of price performance? Maybe with something like Dynamo for disaggregated serving and KV-aware routing.

I'm curious if it's a modest increase in performance or a significant one.