alt Hacker NewsDon’t forget that the 8B model requires 10 of said chips to run.
And it’s a 3bit quant. So 3GB ram requirement.
If they run 8B using native 16bit quant, it will use 60 H100 sized chips.
Don’t forget that the 8B model requires 10 of said chips to run.
And it’s a 3bit quant. So 3GB ram requirement.
If they run 8B using native 16bit quant, it will use 60 H100 sized chips.
> Don’t forget that the 8B model requires 10 of said chips to run.
Are you sure about that? If true it would definitely make it look a lot less interesting.