logoalt Hacker News

hellajack3dtoday at 7:37 AM1 replyview on HN

I forked llama.cpp and added some crude mechanism to keep exactly this behavior under control - essentially guiding the reasoning process by injecting text strategically at specific thresholds. This was mainly put together to rein in Qwen3.6-27B, but I'd imagine 3.8 would react similarly.

Fork can be found here - https://github.com/laurencehardman/llama-mindcontrol/tree/ma...

Of course hacks like this are not perfect and may degrade performance slightly due to injected text pushing the model slightly out-of-distribution, so the string constants need to be chosen carefully - Qwen3.5's technical whitepaper does provide some guidance in this regard. The mechanism is absolutely more of a hack than a feature, and i'd imagine will be made redundant once llama.cpp supports more appropriate reasoning controls - but for now, i've found it pretty useful.


Replies

DarmokJalad1701today at 7:45 AM

Is that similar to what ggerganov is talking about here?

https://x.com/ggerganov/status/2089214161884414147

show 1 reply