logoalt Hacker News

Shershah1024 • today at 1:14 PM • 6 replies • view on HN

Hi all. I am Shahir. This was my experiment. First of all - Thanks to whoever posted this on HN. It came as a surprise.

Just to clarify - This started as just an experiment to see if we can use the fact that text and audio share a vector space in Embedding Gemma 2 to convert audio to text. Had to hack around a few things, but, it worked. Speed was just a side benefit. Was not aiming for it.

The accuracy is good, but not in the range of cloud models, but decent enough for everyday use, and this is just a couple of days work. The WER can improve a lot with more training on varied data. Will try that next.

I will opensource the whole thing soon. Might need a few days though. Also, releasing the TTS version for this soon.

Happy to answer any questions you have.

Replies

rahimnathwani • today at 4:24 PM

It was your post yesterday that made me aware of granite-speech-5.0-470m-turboctc (which I guess you used as a comparator because it's already very fast).

Thank you!

  uv tool install mlx-audio
  mlx_audio.stt.generate \
  --model ibm-granite/granite-speech-5.0-470m-turboctc \
  --audio.wav \
  --output-path transcript
(Until now I've been using mlx-community/parakeet-tdt-0.6b-v3)
➕ show 1 reply
xnx • today at 2:03 PM

Cool experiment. It will be amazing if this can match the WER of the best open weight models at better speed.

People aren't paying enough attention to how great EmbeddingGemma 2 and Gemma 4 are.

➕ show 1 reply
usern20260720 • today at 2:36 PM

can this tool transcribe latin language texts? McWhisper allegedly does it but it outputs gibberish

cabarltos • today at 1:57 PM

Thanks for putting this out there!

e12e • today at 3:44 PM

[dead]

prodigycorp • today at 1:50 PM

can you turn this into a decision model

➕ show 2 replies