alt Hacker NewsWhat is the use case you see for non-technical users self-hosting? I think it’s important that tools remain available but I don’t expect it to be adopted by “average consumers.”
I’m interested in self-hosting for privacy and control. I already owned the hardware I’m testing with, so my spend is limited to time and electricity.
The “LLM pods” you describe will be loaded with spyware and adware (see: Smart TVs), and average consumers won’t max their compute around the clock so naturally data centers are able to make more efficient use of hardware by maximizing utilization.
Replies
And on top of that, I'm sure the "LLM pod" will still be sold on a subscription model so you get model updates etc.
But I wish we could actually have nice things. I imagine there's a niche for a middle ground: a privacy-preserving device that uses local-only models and doesn't spy on the user, and sells for a one-time payment with no subscription. It'll be expensive, though, likely more expensive than using a cloud-hosted model.
Agree with your point on them being loaded up with spyware etc because that's just how it is now I suppose.
In terms of maximising compute I kind of agree but also kinda not - people's laptops and phones aren't burning at 100% 24/7 either. Sure AI requires so much more compute...but not _that_ much more, especially as technology marches on.
For the general use case; I could be wrong but I'd see it sort of like a GPU/NAS/etc. "Pay once" rather than a subscription (to a service offered by a datacenter).
But tbf, the way things are now _is_ all subscription models and consumers just kinda let it happen. I would love to be able to pay a one-off fee for lightroom...but I can't because they want a subscription to "pay for all the updating we're doing". They barely update shit.