That's an interesting size, it will fit into 2x128GB at decent (≥4bpw) quantization.
As a 14B MOE I was rather expecting it to fit into smaller environments than 2x128. Am I wrong on that?
alt Hacker News
As a 14B MOE I was rather expecting it to fit into smaller environments than 2x128. Am I wrong on that?