logoalt Hacker News

antirezlast Wednesday at 9:59 PM1 replyview on HN

I wish I had a Google Keyboard that could easily run on Whisper Medium. This is already great. But unfortunately would be too much inference cost, incredibly slow. The problem with Whisper is not the inference quality: medium and large are incredible. Is that the base model is not enough, and the only one with fast inference in mobile devices.


Replies

hephaes7usyesterday at 2:58 AM

FUTO keyboard is trying to do this. I think they have some kind of distillation of Whisper running on-device.

show 1 reply