I applaud you on the experiment, this is a great showcase in how the latent space is shared between modalities and a great performance achieved.

You stated WER of 5% - against which benchmark?

Whistle [1] for example is a 16MB STT model but WER varies between 4-26% depending on the benchmark.

[1] https://cactuscompute.com/blog/whistle