ARC-AGI-3 launched a few months ago which would suggest that prior models likely had no knowledge of ARC-AGI-3 or training on similar challenges.

I could be wrong, but given the large outsized jump solely in the ARC-AGI-3 score, it would suggest that the model didn't become significantly more intelligent overall, but significantly better at solving those specific problems.

This could mean one of two things (I think):

- Opus 5 was not benchmaxxed on ARC-AGI-3, but has benefited significantly from discussions about the various challenges and mechanisms deployed in ARC-AGI-3 such that it has far better heuristics to solve its challenges.

- Anthropic looking for buzz around their latest model picked a well regarded benchmark with significant room for improvement and focused some of Opus 5's training compute on ARC-AGI-3-style problems.

Or it could be some combination of both. Personally, given how much of an outlier the ARC-AGI-3 jump is I struggle to see it being the product of a significant improvement in general intelligence.