alt Hacker NewsMy question is: what kind of hardware do you need to run this Flash beast locally at a meaningful speed?
Replies
ekianjo • today at 8:20 AM
a beefy pc with at least 20 GPUs
My question is: what kind of hardware do you need to run this Flash beast locally at a meaningful speed?
a beefy pc with at least 20 GPUs
8x RTX PRO 6000 or 4x Spark? Or 1x M5 Ultra 512GB.
The model is theoretically FP8, but really internally its mostly FP4 already, so there won't be a cut-in-half-but-almost-just-as-good quant coming for this one.