> on hardware that ordinary people can afford
These days, can "ordinary people" afford 24GB of ram and half a TB of NVME ssd?
sigh
After 18y of thinkpads, this year I bouth a Lenovo yoga for... Cheap (1000€).
32G RAM, nvme 1TB, core ultra 258V.
Looking at the prices now... Wow, was I lucky.
Tried some of the 7b models locally, more than usable, around 30token/sec, not with the NPU, but using the ARC integrated GPU.
I am a noob for this, but I guess it's time to experiment more with this local setup
The very boring pair of two 16GB ddr5 6000 I had in my newegg shopping cart went from $399 to $475, so increasingly the answer will be "no".
Maybe not afford new, but they probably already had it from before the current crisis?
Ideally this engineer's approach will yield better performance on lesser equipment in the future, if they keep up the good work after they get more-capable gear to experiment with as time goes by :)
> These days, can "ordinary people" afford 24GB of ram and half a TB of NVME ssd?
You can, right now, buy a brand new Mini-PC at or above this spec for $600 at retail [1]
Of course, if you want it in a desktop format with a much faster CPU, its going to cost you more.
[1]: https://www.amazon.com/GMKtec-M6-Ultra-Upgraded-Computers/dp...