alt Hacker NewsIt takes 116 seconds to process 43 input tokens and produce an answer to a very simple request (translation). That's pretty unusable, to be honest. Is overthinking a trait of DeepSeek, or it's a low quantization issue?
For comparison, the whole thing takes 1.8 sec on my RTX 6000 rig with GLM5.3-Flash, and 12.5 seconds on my RTX 5090 rig with Qwen 3.6 27b (it overthinks too, unlike GLM).
I can't imagine waiting 2 minutes for each small turn. It's like... 1 coding task per night? Sounds like, as an engineer, I can accomplish more in that time if I did everything myself manually without any AI.
Replies
That's not the speed of the inference, but the fact that the default parameters are to max thinking. You can even disable thinking for many queries, or lower it to a smaller value.
Yes, Deepseek consumes a higher number of tokens per turn that comparable models (the latest Muse for example). This is just my experience, not a scientific study.