alt Hacker NewsI had to double check those figures on Sk Hynix office web site [1], and it is not a typo or wrong capital "B".
It really is 3TB per second.
I literally paused for 5 min and thought how is this even possible.
I had to double check those figures on Sk Hynix office web site [1], and it is not a typo or wrong capital "B".
It really is 3TB per second.
I literally paused for 5 min and thought how is this even possible.
This is almost entirely dominated by the read circuitry and the data path: it’s still taking 1/6 of a second to read the whole chip, which means that the flash cells aren’t working hard at all. (And that pitting the full weights of a dense model on these chips while using anywhere near all the capacity is a nonstarter if you intent to stream the weights as you run inference.)