Google rolled out TPUs in 2015. AWS released Inferentia and Trainium chips in 2020.

If companies working on ML-specific chips was evidence that large transformer models have fully saturated their potential, the field would have been done circa GPT-2.