logoalt Hacker News

iJohnDoe • today at 5:40 PM • 1 reply • view on HN

Tried running the models with the latest LM Studio, llama.cpp, and Ollama. All failed.

https://huggingface.co/prism-ml/Bonsai-8B-gguf

tensor 'token_embd.weight' has invalid ggml type 41. should be in [0, 41) loader knows tensor types 0..40, but the model contains type 41

Replies

m0do1 • today at 7:54 PM

prismML provides a llama.cpp fork which is compatible with the 1 bit models:

https://github.com/PrismML-Eng/llama.cpp

After fails with Ollama and main llama.cpp the fork worked on my M5 MBA.

Edit: Typos