Deep seek have released their inference engine.

https://github.com/deepseek-ai/DeepInfer