alt Hacker NewsAlso no political censorship, and it doesn’t report you to the police when it suspects you did something illegal. That really happened, by the way.
Now I want a local LLM server :( $25k is just so much money.
Replies
If you already have a machine, throwing an r9700 in it is one of the cheapest ways to get qwen3.8 27B Q6 with 100k+ context working.
Sadly, the r9700 has gone up in price, but it's still the cheapest route. I use It on an old x99 Linux desktop and it achieves 50 tps decode and 1000+ tps prefill.
I'm building a local LLM server myself. For about 8k€ you can have 170tps, ~6000 prefill.
A lot more with Haiku equivalents (Qwen3.8 27B) or gemini. It's not frontier, but not that far away.
renting GPU off cloud is also an option
>and it doesn’t report you to the police when it suspects you did something illegal
This is an alignment issue. OpenClaw with a local LLM with an email you setup for it could contact the police.
MacBooks with large RAM and optimized for local models should be Apple's medium term goal.
No political censorhip? Try asking Chinese models about Taiwan.
Some censorhip and bias towards certain ideology is baked into those models.
While it is not a problem for programming tasks, I would never use it for certain other tasks.