alt Hacker NewsStupid question, but does this AI-generated repo actually _do_ anything?
Comparing it on Nemotron, it runs just about the same as plain old llama.cpp on the cpu (except prefill seems much faster). I don't see an option to set up DeepSeek in "modchip setup", so I haven't tested that one yet.
Edit: this is on a Ryzen 7945hx 16-core Zen 4 + 96 gb DDR5.
Yeah, I was wondering the same thing. I thought about trying to run DeepSeek v4 on my old-but-still-pretty-fast Intel Xeon server with 24 cores and 96GB of DDR4. The "check CPU readiness" script called it "Tier A", so I was optimistic. The default `modchip` app just installs Nemotron and says nothing about DeepSeek. Following the instructions to install their quant of DeepSeek, running it with `modchip start` fails with:
> llama_model_load: error loading model: check_tensor_dims: tensor 'blk.0.ffn_gate_tid2eid.weight' has wrong shape; expected 4, 129280, got 6, 129280, 1, 1
And running the bundled llama-cpp instance from the download, as shown in the readme, fails with "Illegal instruction (core dumped)". So I have no idea how to run this.