logoalt Hacker News

guregodevo • today at 12:22 AM • 2 replies • view on HN

will definitively try it. Overall with a smarter model the improvement is in the token usage and latency. Tbh I think i could get a 10 score if i tuned the prompt and temperature a little more. The original author harness is already giving 9.6 score which is pretty good. However this harness loop can take more than 1 hour on the hardest problem at 100 token /sec. A smarter model such as 3.8-flash-next could get 10 score and resolve the hardest problem faster. That's my estimate i'll give it a try. Thx


Replies

wredcoll • today at 4:50 AM

This is somewhat tangential but speaking of ai benchmarks, I want to see one benchmarkes against passing those "filelocker" websites used for illicit downloads.

You know, the kind with 50+ ads on the page and 8 of them say download and you have to click it 5 times to get past the ads then do that on two more pages?

BrandoDev_it • today at 1:04 AM

I can't wait for qwen4 flash next or something like a 150b moe from qwen.