alt Hacker News>much faster.
This really depends on your infrastructure. In this case it all depends on your hardware, and there is no secret sauce, you can buy the same GPUs or better than the big players are using. Self-hostef GLM in our company is faster than any commercial model. Of course the hardware was not cheap, but completely manageable for a medium or large company.
Replies
antiloper • today at 2:46 PM
> you can buy the same GPUs or better than the big players are using
You can not buy an H100.
Speed can mean two things: rate of token generation or time to finish a task. What maters it the later and a model that takes a faster path to the result and uses fewer tokens because of that comes out ahead in practice.