Huawei releases MACaQ, its internal quantizer for local inference

People are reproducing it and creating GGUFs https://www.reddit.com/r/LocalLLaMA/s/36atJ2nNjh

The top 3 repos sorted by stars on Huawei's github page are all for quantization. I am not an LLM scientist so I can't tell what the difference is between them, or why they need three different pipelines for model quantization.

https://huawei-csl.com/blog/macaq/

Why does some HN users seem to prefer GitHub page whenever available at expense of official site? I mean, Huawei is still sanctioned, anything they host third party outside China can disappear at any hour.

Did they release any model? Seems good for Strata

What a project name.

[flagged]

[dead]

[dead]