logoalt Hacker News

kubrickslairtoday at 2:19 AM1 replyview on HN

Many people claim that the Artificial Analysis Index is highly contaminated - I have not personally looked into it.

Though, unlike the creators of benchmarks like Terminal Bench or ARC AGI, the Artificial Analysis Index team does not seem to have deep technical or ML backgrounds. They are ex-strategy consultants, McKinsey, et. al.


Replies

tintortoday at 2:21 AM

OpenAI clearly cares about Artificial Analysis Index since they included Astra score from Artificial Analysis Index.