alt Hacker NewsI don't trust AA scores after the last update to the index. Specially now that they are including private benchmarks that no one can test by themselves but only trust the entity doing them.
Replies
techmunky • yesterday at 10:40 PM
i was wondering why the pareto line moved so much. i did not remember it being all openai models. good to know i dont need crazy pills yet.
andriy_koval • yesterday at 10:37 PM
we need to make semi-educated decisions to pick model somehow, and AA is the best what is available
JacobAsmuth • yesterday at 6:01 PM
You'd prefer public questions and answer sheets that anyone can train on?
I don't like it much, but I understand. It's the only way to be objective. Models are trained and benchmaxxed on any popular public benchmarks.