logoalt Hacker News

pamcake • yesterday at 11:39 PM • 1 reply • view on HN

> All of those where proven harmless for LLM inference.

Could you share link(s) to those proof(s)?


Replies

fsbonetto • today at 1:42 AM

Yup, the tests already compare to a CPU hugginsface implementation, but I'm working on making running and verifying the results of those tests easier and more available. Also, working on fixing those bugs and make the code more reliable now that the underling hardware has been saturated (memory bound)