It's very likely some 'good nuff' models will make it to silicon from some applications soon.
Something that could benefit from throughput/Wh but wouldn't benefit from flexibility and being cheaper than using bespoke chips feels like such a narrow set of uses though
Taalas is working on exactly that scenario.
It still won't make normal GPUs irrelevant like they did with crypto, there will always be demand for training and SOTA models while there is no demand for GPU mining