I wish they would release the quantized versions in a safetensor format. Many frameworks can't load PTE and GGUF.
Try quantizing yourself, just a suggestion.
https://github.com/pytorch/executorch/tree/main/examples/mod...
alt Hacker News
Try quantizing yourself, just a suggestion.
https://github.com/pytorch/executorch/tree/main/examples/mod...