logoalt Hacker News

peri-cl • today at 8:37 AM • 1 reply • view on HN

Eagerly awaiting Artificial Analysis to redo their Anthropic benchmarks and kick it down 15-20 index points, in the interests of accuracy. Those guys love accuracy. They moved Heaven and Earth to kick Kimi off the #2 spot, out of their deep, abiding passion for having the most accurate benchmarks in the industry.

I have no doubt they will ensure their users get an accurate picture of Opus 5.5 within hours.


Replies

MallocVoidstar • today at 9:32 AM

If Anthropic are doing A/B testing it almost certainly only affects subscription users, not API users. Every benchmark I know of uses the API.

➕ show 2 replies