alt Hacker NewsTranscribe.cpp
Comments
Congrats on shipping this. I love handy on my Mac, my phone for STT in situations where it’s not possible/poor performance of the native Model for STT(e.g apple’s thing is not upto scruff, like mistranslating words corresponding to a domain).
Noob question: How do you think about funding from a foundation(i have no clue if you need it or not, I do hope you have a way to get paid one way or another because handy is amazing) for maintenance of this? if you did or were going to get paid by asking for maintaining such a project what might be the kind of organizations you would look for to get supported and how would you do it?
For anyone looking to build on top of this. I have tried a few different STT systems, and they accurately capture what I am saying. Unfortunately, they don't support the reasonable workflow
I want to open an office document, for example, and start talking. And I want the software to continuously type what I am saying at the cursor with minimal latency. The continuous part is crucial. Many software will paste whatever I said after I have stopped recording, but that is not useful.
> Maintainer supported bindings in 4 Languages
Nice. Here's the Python one: https://github.com/handy-computer/transcribe.cpp/tree/main/b... - looks like it's not yet available as a binary wheel on PyPI with the dependency included (the library on PyPI right now uses ctypes to call a separately installed library) but that's planned for a future release.
What good timing to spot this. I've been reading more and more people talk about bringing TTS into their prompting toolkit and wanted to give that a try. The idea of rambling brain dump into a doc -> edit pass -> send to the robot loop sounds appealing.
This is an incredible contribution to the community and it's just... one guy? I kept reading expecting a Series A funding announcement at the bottom.
It's a nice reminder: You can use AI to slop cannon at maximum speed, or you can use it to scale your ambitions and build something more rigorous and lasting than ever before.
I'd build Transcribe.cpp into the apps I maintain, but I feel like this functionality should (generally) be integrated into the OS or "everywhere" via an app like Handy.
The numerical validation and WER testing are what stand out to me here. A lot of local ASR projects claim broad model support, but it is often difficult to know whether the converted models still match their reference implementations. Having one embeddable engine across Vulkan, Metal and CUDA, along with maintained language bindings, addresses a real distribution problem. How stable do you expect the C API and model format to be after v0.1? In particular, could an application eventually switch between different model families without needing model-specific preprocessing code?
Spot on:
> I think as we look forward to the future, more inference will start happening locally for one reason or the other. This brings the distribution story front and center. In order to have more applications running inference locally, we need to make running inference easier.
This makes these projects so much more trustworthy and easier to approach:
> Were any of the words here written using AI? Nope. They came from my mouth or my fingers.
Thank you! I found this to work much better then the old transcribe-rs lib. I updated my Offline Voice Input App to also use the new library and it's much faster now: https://github.com/notune/android_transcribe_app
What's the easiest way to add speaker separation to this?
Oh, I like this! I’ve been looking into locally hosting a transcription API server and came away feeling pretty close to the problem statement. The things most frequently lacking were streaming support (which I’m so glad this has!) and the support for special words to boost during recognition (which I guess there’s some hope they might add???).
Amazing, i've been looking for something like this and ended up doing transcription + diarization on a local server for now. Are you looking for contributions? Have you tried this one for diarization - https://huggingface.co/pyannote/speaker-diarization-communit... - it performed much better than Sortformer for me.
The post makes it seem like ONNX is CPU only. I've used ONNX runtime to run models on Nvidia GPUs. The runtime can even dispatch to TensorRT. I'm not sure what the performance is on Apple hardware so maybe that was the motivation for moving away from ONNX.
I'm using this in one of my side projects, Emyn ( https://github.com/terhechte/Emyn ) a macOS virtual camera app for composing camera video, app windows, backgrounds, effects, notes, and captions into a polished live presentation feed.
It works very well, the integration is much easier than before, users have model choice. So happy that this exists!
Just wanted to say I started using Handy last week and I love it. It might single handedly cure my RSI. Well, hopefully double handedly.
Nice. I did transcriptions on a casual project before that went through something like this. Transcribing videos or audio files with Whisper? Very common. But having to swap it out with Qwen3 or a different family of ASR models? Oops, not as straightforward. For Qwen for example you gotta deal with the forced aligner or it won't be good as subtitles, and then gotta deal with some requirements and considerations if you want to make use of MLX on a Mac or something.
Will definitely check this out since it sounds like it eases through the pain of dealing with these.
Well this almost seems to be to good to be true. :)
I assume this is going to make maintaining SubtitleEdit a lot easier from now on, too: https://github.com/SubtitleEdit/subtitleedit/.
Anyone know a good Windows app that's just a window that transcribes - and translates - whatever goes through your output device, and not the microphone like most apps do?
Congrats! I just tried Handy again which now uses transcribe.cpp and it works brilliantly. Love the streaming output from Parakeet Unified EN 0.6B. I remember using Handy about a year ago and it's amazing to see the improvements!
Why not include transcribe-cli in the release archives to make it easier to use for people that can't compile it themselves? I downloaded the Cuda version but it's only the dll files, I don't really want to have to deal with Cuda SDK, I doubt most people want to.
I saw that metal is almost x10 faster than vulkan? Why so much gap?
Any plans of including the VAD step into this? When whisper.cpp added Silero it really smoothed out the UX.
I'm using Handy on macOS and love it. Unfortunately, hotkeys still doesn't seem to work on Wayland, which make it unusable.
Really cool that every model is actually tested for accuracy instead of just claiming it works, I think alot of 'we support everything' tools skip that step. How are you checking accuracy for models that don't have an obvious "official" version to compare against?
Does the whisper model allow for entering context(which improves the accuracy greatly) as it does on Whisper.cpp?
So it's mostly intended to be a better replacement for whisper? Mostly? With better support for more models and maybe acceleration backends?
Excellent work, paired with the 500kb TTS model headlining today I can see the full stack coming together.
Is there something but for transcribing what you watch like videos and not your microphone? Samsung has this in my phone and it's useful for language learning. (Thought is not that accurate)
What's the best local TTS model right now? I'm running parakeet on a mac which transcribes all my uh's and aahs. I'm running whisper on linux/cuda and I by far prefer that one over parakeet.
Thanks CJ, you've put some pretty cool things out there!
handy has been invaluable in my workflow, and having a fast, local, c++-based transcription library with first-party ts bindings is incredibleee
tysm for shipping this, keep up the great work OP
One thing I find that's missing a lot or at least I haven't come across other than commercial offerings like AquaVoice is a decent injected technical vocabulary so that the initial transcript requires minimum cleanup afterwards. Because I mostly use these tools to essentially ramble at the command line with coding agents. So there's a lot of technical terms that don't translate well. Like OpenBao comes out as open bowel sometimes,lol. That necessitates significant cleanup prompt or background text available to the cleanup llm, usually in the form of screenshot or something that gets converted to text but that in turn requires good hw for speed to be almost imperceptible. For example m5 max turns cleanup into a noticeable delay while 5090 is decent.
Only way I have found that's relatively easy to inject technical vocab is to use whisper, but limited, I think to about 220 or so tokens. Whisper has sort of like a priming prompt where one can put in a bunch of technical words and it will try to recognize those. But again, that's limited to small number tokens. And that limits one use a relatively slow, by today's standards, whisper.cpp.
I benchmarked it across a bunch of different hardware that I have available, and Whisper gives decent performance as far as speed goes only on a pretty top-end GPU, such as a 5090 or 4070, like for example on Strix Halo, it's still relatively slow for longer transcriptions because I prefer just a stream of consciousness ramblings for minutes and then that being transcribed and cleaned up versus short sentences. So in that scenario something like 5090 really is good because the cleanup prompt runs fast using usually Qwen 3.6 MOE model. Whisper on 4070 itself is about 0.7 seconds for two or three minute transcription. So the total wait time for a three-minute transcription is roughly a second, or a little bit more than a second, so totally acceptable. But it does take decent hardware, and it grows to be double that on if running totally local. Well, in my case, it's all local, but it's my own hardware all over the place, but truly running on laptop, it's much faster using Parakeet, but then the cleanup is the bottleneck.
Anyway, it's just my experience messing around with this for the last year. I did start using AquaVoice, but their speed was exceptional, and tech vocab was exceptional, but they would have some annoying delays occasionally, and I didn't like paying the money and sending sensitive topics and screenshots into the cloud, and I had hardware, so my local solution is basically almost as good as commercial one. But I think they train their own model. So what I'm doing is I collect all the samples of my transcriptions, and I am slowly building my own data set that hopefully at some point when I get energy I will find some way to fine tune something.
I've been using this one for a week for local transcription, working pretty well so far
Does this support filtering of “umm”,”err”, “ugh”, or that is nit yet possible with open source models?
thanks man! Been searching for solution like yours :)
Is transcription a form of _inference_ though? I mean I see the word being thrown around and I understand what it means (or at least I think I do) in context of LLMs doing the thing that they do -- intelligently predict the next token, but do speech-to-text models do that?
three separate people in this thread independently remembered Dragon NaturallySpeaking and I think that is the funniest possible review of the state of speech recognition in 2026
Hey, thanks for this.
Excellent work
Another happy user of Handy here!
After seeing so many *subscription based* transcription apps all wrapping *open source models*, finding Handy was a real delight and I'm happy to see the author keep on building!
Has anybody experience with using this with strong dialects, like e.g. bavarian-family (German) based ones? Or other languages one too, as I'd figure basic behavior and approaches to improve detection of such is often similar in principle for dialect style variants of a language.
I mean, I naturally should try myself, and plan to do so, but slightly lower on my free time priority list and I figured someone else might have explored this already.
Nice - I'm definitely going to take a look at this. I've built my own cross-platform (Mac/Win/Linux) live captioning app on top of Nemotron, and it works well but dealing with ONNX is kind of annoying. With this having Rust support (I built it on Rust/Tauri) it should be a pretty solid candidate; I'll have to see if I can find a Silero VAD implementation that doesn't depend on ONNX, or maybe I'll see if the clankers can migrate it for me.
Congrats on delivering good value to the people. I have used transcribe.cpp a few weeks ago to do near realtime offline stt on a 10 year old phone, writing simple adhoc app for my use case, it's crazy what is happening right now.
I use Handy everyday. It’s a great project. Thank you for making offline ASR work great on modern machine.
Looks interesting, I'll give it a try. Though I'm really happy with faster-whisper on a GPU.
would love to see a demo handy is fantastic although its still behind the frontier models
Yet another happy user of Handy - one of the best applications out there, kudos! <3
Handy is an amazing cross-platform app for dictation from the author. There are other awesome open-source dictation tools as well like native macOS ones. You do not need SaaS subscription in this day and age for transcription.
I maintain this list of all the best open-source ones in this awesome-style GitHub repo. People looking for open-source dictation tools, hope you find something that works for you here:
Looks very cool. One thing I have been looking for, which this doesn't seem to cover (at least I didn't see any mention of IPA in the model documentation), is a way to transcribe unknown languages phonetically, using the International Phonetic Alphabet to spell them (sound-based spelling rather than meaning-based spelling). I know several linguists doing research on minority languages (fewer than 10,000 speakers in some cases), which are small enough that they will never have enough effort made towards training language-specific models in that language.
Are there models I'm not aware of that are trained for this task? Taking audio in an unknown language, and rather than identifying the language, just transcribing the sounds to IPA? That would not be useful to most people, but it would be a Godsend to many, many linguists working with minority languages around the world.