Inference, yes, see https://taalas.com/ and https://chatjimmy.ai/ As expected, efficiency boost is extreme. But you've got to pick the model you want to enshrine in silicon months or maybe years before you actually get that silicon (and maybe there are additional difficulties if the model is large. Maybe in a few years everybody will just want to use the same four or five models, with no tectonic shifts on the horizon. Then it'll be a no-brainer to make ASICs for those models.
Training? I'm not so sure. You need something flexible so that it can be trained. Burning the weights into the board is the opposite of that.
As much as I love the idea of Taalas, their chatjimmy demo is a hard-wired (and probably very quantized) 8b LLM producing garbage extremely fast (17k tokens/s). Extrapolating from the picture of their PCIe board in their demo, I would expect a frontier model to require an IC not smaller than my desk.
At these dimensions, you are better off producing general-purpose inference ICs that can run any model, instead of baking in a single fixed LLM.
So, Taalas (now AMD) will probably target small models with these (maybe something like qwen3.8-27b, which seems too large already), if at all.
I read into Taalas when they first were putting out info and they also are wiring up massive CPU cache to the thing. It's not just the burned in models.