Solving an unsolved problem does not require necessitate learning, it may just require effort.

ARC is trying to test if LLMs can actually learn how to play the game.