alt Hacker News>For starters, the average person isn't going to use a locally hosted model, ever.
Let's talk about that in 2-3 years. There is a very clear incentive to have models running locally, from data privacy to cost effectiveness. The software will pop up to easily power this, and if it is cheaper than non-open frontier models it will be used. This can be done today with a bit of hassle, and will only get easier. The consumer hardware vendors are clearly heading this way.
Replies
wooger • today at 12:20 PM
You're forgetting every iPhone and Android user for a start!
I'm heavily biased here. We are building a ChatGPT/Cowork like solution for data privacy concious enterprise customers.
This is already starting today, not in 2-3 years. Currently customers are using our solution with cloud-hosted LLMs, and for the meantime costs are acceptable, but we are also building capabilities to do organizational rollouts of local LLMs, so that the majority of tasks can be offloaded to the local machines of the employees (which are most of the time underutilized anyways). For a lot of typical office worker tasks, this works great. This also has the nice side-effect of allowing C-level to use AI locally for highly sensitive documents, without a risk of leakage.
I would expect that the cost-concious customers will push the broader market of harness providers to allow for solutions like this going forward. I am however not sure how much this will take hold on the consumer market.