alt Hacker NewsI can't get Qwen 3.8 27B to do a simple code review on a fairly basic Python file. With thinking on it just ruminates forever and with thinking off it gives obviously bad borderline hallucinating advice.
Edit: I tried again with the 2.4T model and it still ruminates to death, but with thinking turned off, it generated genuinely useful advice.
Edit2: adding
--reasoning-budget 8000 --reasoning-budget-message "Reasoning budget exhausted; give the final answer now." --reasoning-effort low"
Replies
nialv7 • today at 9:05 PM
how are you running the 2.4t model locally if you don't mind me asking
➕ show 1 reply
One of the big learnings from 3.8 27b is adding reasoning budget really hurts the model. you need to let it spin for as many thinking tokens as it wants to to get it out. Another big takeaway is reasoning effort set to low doesn't save you tokens: low is pretty uncertain about things so it ends up thinking more (you can find some tests from folks on youtube). The final question, as always, is what quant are you running it at? KLD matters _a lot_ when it comes to its performance and it especially manifests with MTP/DFlash acceptance rate which makes those long thinking traces take a long time.