alt Hacker NewsHere's some things you can do right now with local models on a consumer device:
- text-to-speech - speech-to-text - dictionary - encyclopedia - help troubleshooting errors - generate common recipes and nutritional facts - proofread emails, blog posts - search a large trove of documents, find information, summarize it (RAG) - manipulate your terminal/browser/etc - analyze a picture or video - generate a picture or video - generate PDFs, documents, etc (code exec) - simple programming - financial analysis/planning - math and science analysis - find simple first aid/medical information - "rubber ducking" but the duck talks back
A quarter of those don't need more than a gig of RAM, the rest benefit from more RAM. Technically you don't even need a GPU, it just makes it faster. I do half that stuff on my laptop with local models every day.
That said, it really doesn't need to be local. I like the idea that I can do all that stuff offline if I'm traveling, but I usually have cell service, and the total tokens is pretty cheap (like $2/month for all my non-coding AI use).
Replies
Please add double new lines as your formatting for the bullet point list makes it all one paragraph.
Would you share your experience of the used models? I have quite some experience with the larger models but the smaller ones tend to loop around or just fail on their tasks...
RAG on every machine, and the means for corporations / shadowy powers to query it.
>it really doesn't need to be local
I urge you to reconsider this attitude. If AI has a tenth the significance people claim, you're signing away your life; your ideas, your privacy, your very sovereignty of mind, all under someone else's control and revocable at any moment. Don't move your brain to the cloud.
Imagine an alternate timeline where we never had personal general purpose computers, only dumb terminals to access corporate servers on subscription. Don't vote for that world with your wallet, today.
Don't be a cloudhead!