We've had mathematical problems solved by brute force in the past.

We've then improved that through systems similar to prolog intentionally searching a tree.

Then systems added heuristics for which paths in that tree are likely to be taken.

The LLMs are just using slightly more accurate heuristics for this task.

But the real measure of understanding are tasks that are not so strictly constrained.