logoalt Hacker News

alerighi • today at 11:41 AM • 2 replies • view on HN

Because maybe you don't want to send your private data to a company that could use it for whatever it wants?


Replies

barrkel • today at 1:13 PM

Can you explain what data is going where?

➕ show 1 reply
anonym29 • today at 1:46 PM

GP isn't proposing using closed proprietary models over an API, they're proposing having local hardware serving open weight models at your house/apartment being remotely accessible through an overlay network like Tailscale. The idea being that local inference is very hungry for power and thermal budget, so you run the inference somewhere that power and thermal headroom is abundant (your house/apartment, on a tower, or dedicated unified memory boxes like DGX Spark, Strix Halo, Mac Studio, etc) and then remotely utilize that inference from a regular laptop. You still get all of the benefits of local, private AI inference, without having to expend a laptop's limited power and thermal budget on doing the inference on the laptop itself.