> My opinion on local LLMs [...] :

>You avoid the uncanny valley where the model appears to be intelligent before doing something stupid; the stupidity is self-evident and this helps calibrate your interactions.

This seems to be capturing a feeling I am finding messing with local LLMs: popular software around this seems to be slop coded by no talent hacks. It's a common experience to see a python process pegged at 100% CPU with a huge amount of memory allocated, seemingly doing something that could be a simple shell script. That is before stuff reaches the LLM.

If that runs on somebody else's machine, you don't notice. When you see it in top it's frustrating.