Agree about the latency requirement.
There's https://kyutai.org/stt, which is very low latency. But it seems not as hackable.