alt Hacker NewsTried yesterday on my own laptop (a UltraCore 7 255H without dedicated GPU,with 32 GB RAM), it wasn't even starting thinking, even on a small context window (65k)
Replies
mdp2021 • today at 7:58 AM
Have you tried with different amounts for the "reasoning_effort (xhigh|medium|low)" parameter?
Or the "<|think_xhigh|> | <|think_low|> | <|think_off|>" tags: apart from this template detail, it is not immediately clear if reasoning_effort is deterministic (API) or is prompt engineering.
➕ show 1 reply
petu • today at 9:18 AM
What was your prompt length? It's possible it was just processing it and it's likely not fast on your setup.
➕ show 1 reply
pdyc • today at 5:57 AM
i have same 255h and i was able to run it with low token speed 6-8tg/s with approx similar context window 60k
➕ show 1 reply
> it wasn't even starting thinking
Probably stuck in prompt processing which is compute bound especially for iGPUs.
You've mentioned 3.5 - but it's actually the same model the only differences are training and implicit MTP support (affects prompt processing - can be disabled)