alt Hacker NewsI am working this out with Fable right now, for getting this running on my DGX spark homelab; it mentioned that there might be issues with the 'optimized LM-head restrictions' that unsloth NVFP4 ships with. Have you had any issues here?
Are you trying this with vLLM? Or a different engine? I am getting about 15 tok/s on my spark on my current setup using the 0.26 nvidia vLLM image and MTP.
If you are using the Unsloth nvfp4 checkpoint, you need to patch vLLM+DFlash 2 to accept the quant's FP8 `lm_head`.