logoalt Hacker News

xlayn • yesterday at 8:34 PM • 2 replies • view on HN

Snap, no gguf to download, will wait for one to appear to test, any improvement over an already good model is good news.

"18 days on 144 H200 GPUs" insane.. didn't thought so much hardware would be needed to train Qwen.


Replies

drdaeman • yesterday at 9:39 PM

Rather than waiting, ask your assistant to quantize for you. It could be not the best quantization possible but it’s relatively quick, and I had an overall positive experience.

➕ show 1 reply
cookiengineer • today at 1:10 AM

It isn't if you train the quantized model with a quantized K/V and cache. That's my approach and usually takes around a day for 100k reasoning traces (including abliteration).

Also this is on consumer hardware, meaning 96GB CPU RAM and 48GB GPU VRAM, using an optimized Vulkan concurrent shader pipeline. So costs for the machine are around 3-5k depending on RAM crisis.

[1] https://github.com/cookiengineer/qwen-reasoning-trainer