alt Hacker NewsQuirks from the article:
1) The performance numbers are simulated
2) The details say that the performance numbers require 2.7x the overall power, so around 13.5W. Power needed for all the memory is not explicit in the numbers.
3) The compiler is also simulated and it's a prototype, so it's not a drop-in replacement for anything like Nvidia ISAAC
4) 5W seems to refer to sub-4B models, so not the 100B param ones.
Replies
idiotsecant • today at 8:09 PM
I think the human brain uses something on the order of 10-20W so 13.5W seems like there's probably some room to improve on a 4b model!
➕ show 1 reply
The chip is designed to run models with 20 billion to 100 billion parameters (approximately 3 billion to 20 billion active parameters)
Doesn't seem like 100B dense models are even on the table, only MoE models. Kinda misleading.