alt Hacker NewsTo put this on perspective, consider that with 128GB of unified RAM it can run even fast: https://x.com/antirez/status/2119392658661749175 Like, coding-agent acceptably fast. See later in the reply for the prefill at ~300 t/s but can go even faster for larger prefills.
Replies
This seems great. I have 128GB of unified RAM and a AMD Ryzen AI Max+ 395 that I can use but I feel I'm still not using it as well as I could...
The main/best setup I have running so far is qwen flash next with halogen-server.
But if anyone knows if and how one can run even more impressive stuff, please let me know!
I have a 128GB M5 and while I adore the fact that I can run this as well as it does, I would not consider it fast.
The happy spot for these is for long running closed loop tasks where I’m stepping away for the evening. It doesn’t matter as much if it finishes in 1 hour or 10 hours as long as it’s done by morning.
For tasks where I’m sitting at my desk, it’s not yet where I’d consider it comfortable to use.
I do think this groundwork is going to be awesome come M7, M8 and so on.
I've recently set up Ollama on my homelab server, which is a Topton N17 micro-ATX motherboard with a Ryzen 7 7840HS and 64 GiB of DDR5 RAM. It's no AMD Strix Halo, but it's a laptop CPU with a fairly beefy GPU and a decent amount of memory bandwidth.
I've had decent results with gemma4:26b (although it's more like batch processing when used as a coding agent) and my experience so far is globally in line with the first two sections of the ARCHITECTURE.md: fast with small contexts but performance degrades fast as its size increases. If the rest of the contents of that document are true, I'll need to retune my setup from scratch.
Also https://github.com/drawthingsai/local-code-community pretty stable around 30 tok/s during decode on 128GiB RAM of M5 (with ~56GiB RAM usage), prefill is around 400 tok/s with 4k chunk.
That video seems to show it takes ~4 seconds to process 43 input tokens. Considering the amount of tokens coding-agents chew through, even when just exploring and not even implementing anything, feels far away from "acceptably fast". Say one file is 500 tokens (small file), it'd take ~a minute just to process it?
You mention "See later in the reply for the prefill at ~300 t/s" but I didn't see that in the video nor in any of the Twitter replies, could you link where that is supposed to be happening? Is that warm context perchance?
It takes 116 seconds to process 43 input tokens and produce an answer to a very simple request (translation). That's pretty unusable, to be honest. Is overthinking a trait of DeepSeek, or it's a low quantization issue?
For comparison, the whole thing takes 1.8 sec on my RTX 6000 rig with GLM5.3-Flash, and 12.5 seconds on my RTX 5090 rig with Qwen 3.6 27b (it overthinks too, unlike GLM).
I can't imagine waiting 2 minutes for each small turn. It's like... 1 coding task per night? Sounds like, as an engineer, I can accomplish more in that time if I did everything myself manually without any AI.