logoalt Hacker News

gcryesterday at 5:43 PM1 replyview on HN

how could running the qwen GGUF phone home? that would require cooperation with the inference backend (llama-cpp), or some kind of model exploit. It’d be far easier to pay the agent harness devs or supply-chain some plugin or something, that space is the Wild West anyways

I've certainly used these models without wifi without any differences.


Replies

HDBaseTyesterday at 10:55 PM

You've used Qwen with model quantization, locally without internet connection.

A lot of people are purchasing access via Alibaba Cloud directly, or indirectly by companies which host the model.