alt Hacker NewsEven jev is not truly novel, but it's latency is, you can use a reranker and get the same things but not the same speed.
Replies
mrkn1 • yesterday at 9:55 PM
If you want something even lighter than Jev to compare against, there's also gutsy (https://github.com/kouhxp/gutsy) runs on CPU
Well, according to claude and Jevbench, Qwen 3.6 35b with ninfer on a RTX 5090@480W is like 3-5 time slower but 10%-15% better performance on the public set, I could see prefill > 15k for 700-800decode. Latency against what and which hardware? I don't really get jev...