Yeah, I think it's not so much "math is symbolic manipulation so computers and by extension LLMs are going to be good at it". LLMs are not by construction so good at it because ML models are far more statistical approximators than formal logic systems. I think the main thing that math has going for it is sufficient formalisation through tools like Lean that it's very verifiable, and the latests hotness in improving models is RLVR which seems to really work very well when the verification is strong and relevant to the task (probably also why they're getting very good at writing code and especially finding exploits).
(I would also say that computers are not good at math in the sense that LLMs are proving to be good at it: reliable symbolic manipulation is important for correctness but it's no use for actual progress in mathematics without some very good informal thinking/creativity as well)