Local AI already is really useful. This has been true since Qwen Flash Next and Qwen 3.8 27B rolled out. The struggle with exactly 64GB of unified RAM is that you end up having to stream the engram table from SSD because you will need almost all of your 64GB to run QFN. A 27B dense model still runs a little slow, even on an M1 Max Mac. However if you have even an older gaming GPU with 12-16GB of RAM and enough system RAM you can run this model (QFN) at a very respectable token rate and intelligence level. It is a Sonnet 5 class model you can run at home. There are new quantization techniques that make these models very good at agentic tasks.

And now if you stretch you can run DeepSeek Flash 4.1.

And if you were willing to roll up your sleeves you could build your own AI lab at home and run this model and models like GLM Flash 5.3 and DeepSeek Flash V4.1 at lower quants. The budget GPUs (like V100) are kind of not budget now, but still <5K to acquire 128GB of fast VRAM that can compete with a 5090 and actually has a lot more interesting options available IMO.

That's only part of the issue - prefill is horrible prior to M5 class chips.

So it depends on what you're doing. Using it for a fun oracle without feeding it too much context per turn? Sure, an older Max chip is great.

As a coding agent doing meaningful work on a sizable project? Compared to a cheap as chips cloud option, it will take an order of magnitude more time to complete.

Puh i don't know, there might be plenty of new/better chips coming out for AI next year or in 2 years.

Isn't it also rumored that the M7 is reworked for AI?

And Xiaomi announced their AI Cube with 1.2t of bandwidth. Just because Nvidia is protecting its business by giving consumers either compute or bandwidth and Apple being something in between, doesn't mean we can't get both a lot better balanced.

It seems so wasteful to spend premium on the hardware for local ai to be shoved into macbook form though. You'd probably save a lot building a compute node you shove under your desk that you connect to with some more comfortable but cheap thin client, maybe a refurbished low spec macbook pro (inb4 I say air or neo and people say they cannot work on inferior screens).

Then again the people working on stuff involving this much local model compute demands probably make enough where they don't care about the couple grand they'd save building their own node. I guess apple will get the bag from being the nicest prebuilt out there in this case.

Running Gemma 4 26B A4B and Qwen 3.8 27B on a M4 Pro with 24 GB - Qwen 3.8 runs fine but is slow and taxes the machine a bit, Gemma 4 on the other hand, is a better experience.

Srouji is killing it on whatever he has on mind or is asked to do. Everyone is riling up on GPU and Cloud AI costs. 99.99% of what people use AI for today already processes fine on an M1 from 5 years ago.

Apple killed M6 Pro, M6 Ultra and M6 Max to focus on AI chip, so yeah that'd be a more sensible option.

>The generation doesn't mater as much to me as the memory bandwidth

I don't know man?

The generation kind of does matter. I'm not gonna lie, matrix multiplication units speed things up considerably. That hardware confers a material change in capability.

Maybe from M5 up, it doesn't matter? But you're definitely going to notice the difference between M1 and M5 for instance.

I'm running qwen 3.8 flash next with strata (q3 quant) on laptop with 5090 and 128G RAM and OMP harness. At 90t/s speed it is pretty much good enough already.

Will local AI eventually pull the proverbial rug out from under the big providers and the huge sums invested in building out data centers?

Still on my M1 Max 64GB from release date. Love it. I don't do local AI but this laptop runs so cool and fast I have no reason to upgrade. They really knocked it out of the park on the design.

It feels just as fast as day 1 and I've never had any speed issues. Say what you want about Apple and their pricing and ecosystem but I've been using their hardware for over 20 years now and it's always been top notch.

If you buy their support/AppleCare, your support and warranty is excellent too. They don't skimp. I've seen them go so far past the extra mile it's unreal.

> I'd rather have a M1 Max 64GB over a M5 Pro 64GB.

I am one of those: bought the M1 Max with 64gb thinking in running VMs, afterward bought a lighter Air with less memory and ended up using the M1 Max as a kind of luxury for LLMs and multicore computing.

Considering the insane memory price surge it's going to take years (or an eternity) before most people will consider upspecing their machines to run local over paying $20 per month to have the honor to loan a slice of Altman's hardware.

The only hope we have is the AI bubble bursting, but that will undoubtedly come with its own set of macroeconomic surprises.

Man you Apple Fans really live under a rock.

Mini PC with Occulink, EGPU dock, 750w power supply and a refurbished 3090 is 2500. It can run Gemma4:31b at 1000+ tok/sec prefill, ~40 tok/sec generation (which is slow, but a fully connected model is way better for all tasks)

if you want more specialized partial activation models like Qwen, you can get 100+ tok/sec generation.

This beats anything that Apple has by a long shot. You don't need to run large models for usefulness. Large models can get things write the first time, but thats their only advantage. Running small models that are good enough like gemma4:31b fast allows you to do faster iterations.

And if you really want, you can also do multi GPU setups if you want slightly slower model loading with usb4/thunderbolt ports.

TLDR; local ai has already been useful for quite some time if you arent brainwashed into believing that Macs are the best for running local inference.

..really useful on a Mac. FTFY