alt Hacker NewsInteresting... is it overkill to use a multimodal model like this on just audio? Could the model size be decreased dramatically, or does the multimodality help the accuracy and speed in some way?
Replies
sd9 • today at 12:45 PM
Multimodality seems to just help with everything
The benefit is that you can take input in one "mode" and extract output in another; that is, the input is audio mode and the output is text mode. The intermediate representation is the "embedding".