logoalt Hacker News

yieldcrv • yesterday at 11:09 PM • 0 replies • view on HN

other improvements in the inference engine and harness net 1.5x+ improvements in tokens/sec, so if you're doing that on your own hardware you don't really need less tokens run, you just need any output done quickly