alt Hacker NewsIf 256GB of RAM enables them to run on-device AI models that (for reasons) are a key feature differentiator?
Personally, I think there's no way memory heavy inference moves on-device (vs cloud) due to the economics, but it's not impossible technology + platforms go that way for currently unforeseeable reasons.
Replies
Right. I’m not arguing that Apple wouldn’t offer a 256GB model if they could make money doing it; I’m puzzled as to why they wouldn’t offer several lower-spec models as the entry-level into and then progressive upgrades within that line, since only some people need that 256GB feature differentiator of running frontier-level models on their MacBook Pro.
Think past on-device inference... imagine what on-device training could do. And that would need a lot of RAM.
I think there’s a realistic chance consumer inference moves on-device. I think it really depends on marketing.
My non-tech friends and family would probably be served perfectly fine by local models today, if they had a working web search tool. Their queries are often “soft” and don’t have an exact answer. My mom and aunt used it to pick a hairstyle, my mom used it to get an image of what a room would look like with particular drapes in it, etc. Stuff I think mid-sized local models like Gemma or smaller Qwens could do without issue. They just don’t have a device that will run them.
Businesses won’t move. They need a huge context so they can stuff a bunch of Confluence pages in it and 300 tools and it needs to read an entire codebase and yada yada. The hardware depreciation and electricity will probably make it a net zero or even cost more than paying for API access.