logoalt Hacker News

speedping • yesterday at 6:29 PM • 1 reply • view on HN

Even if you can’t run this at home, this is still a step toward commoditization & diversification of inference and breaking up of concentrated power. Being able to serve a frontier-class model with 1M$ of capital in GPUs instead of multi-year billion dollar investment in training & research is a big difference


Replies

throw938990 • yesterday at 6:36 PM

You can run it on high end laptop at like 0.1 token per second.

But I think smaller size models like qwen 3.8 are way more important.