This talk serves as sort on update on where they are with it: https://youtu.be/r305-aQTaU0?si=j3IqG9BAT0uumPeA

At a high level: It doesn't benefit from batching to reduce memory bandwidth in the same way that a "standard" autoregressive model does, which ends up meaning that it's great at reducing latency for individual/low-concurrency users and delivers better outcomes on certain tasks but much more expensive per token at scale.