logoalt Hacker News

smcleodyesterday at 8:27 PM0 repliesview on HN

Unsloth use a property dataset they don't release, however you can indeed create quantisation locally on your machine and it's pretty easy, llama.cpp comes with everything you need.