Today the closed-source giants maintain a substantial cost-per-token advantage due to proprietary inference tricks like speculative decoding kernels, ultra-optimized FlashAttention derivatives, multi-GPU pipeline orchestration and dynamic prefix-caching algorithms.

I'm not sure whether the DeepSeek's open-sourced inference engine does all of these, but it appears that plenty of inference tricks have just been revealed for the open source community to learn from and start improving upon.

What moat?

When the model is fungible and the switching cost is near zero (business logic lies in the harness), then there really isn't a moat for models.

In theory, this can allow a small US lab to compete with O/A/G and take marketshare from them with small open models deployed centrally or (and this is think is the real moat) on-prem for enterprises.