> the dense 9B fits on a single 80GB GPU Us mere mortals cannot use this.

giancarlostoro • yesterday at 11:16 PM • 2 replies • view on HN

> the dense 9B fits on a single 80GB GPU

Us mere mortals cannot use this.

regularfry • today at 4:22 PM

Seems weird. A 9B model would normally fit unquantised on a 24GB GPU.

armarr • today at 5:05 AM

There are already quantizations available

➕ show 1 reply

alt Hacker News