logoalt Hacker News

martianvoidtoday at 6:59 AM2 repliesview on HN

It's actually crazy to see the difference between opus 5 and the next best model on ARC AGI 3 when you actually look at the ARC AGI problems


Replies

zzleepertoday at 7:15 AM

How believable is this benchmark? EG maybe opus was training on this? (You can try to identify the IP of wherever previous ARC questions came from)

show 1 reply
Stevvotoday at 8:15 AM

Why? 30% is passing the first two problems only, which are really very simple.

show 1 reply