They can be ran on 32GB with 8GB VRAM. I don't think these will be on 16GB for a while. (35B MoE)
I have 32GB of RAM with 16GB VRAM and I haven't had a lot of luck running larger models like this. Are you able to expand on that?
alt Hacker News
I have 32GB of RAM with 16GB VRAM and I haven't had a lot of luck running larger models like this. Are you able to expand on that?