What do you need the extra 2 for? Tensor parallelism?
Longer context and more cache. The problem is that native format with DSpark enabled you have very little room on the VRAM.
alt Hacker News
Longer context and more cache. The problem is that native format with DSpark enabled you have very little room on the VRAM.