> there will always be demand for training and SOTA model
Some, but how much? Once you have a mass produced ASIC with a model that's "good enough" much (but obviously not all) of the demand for the cutting edge could evaporate quite rapidly I think.