logoalt Hacker News

enraged_camel • yesterday at 11:59 PM • 0 replies • view on HN

The problem is that "can't tell much difference" is not going to be an objective indicator; you will be very likely to overlook the flaws of the local model and overweigh the flaws of the non-local model. The only reliable way to compare the two is to have your own set of evals based on the types of tasks you regularly perform and then run those evals on Qwen and a frontier model.