This is probably part of why machines are doing so well at counterexamples. They have no aesthetic commitment to the conjecture and no embarrassment about producing something ugly
This reminds me of the Go Grandmaster speaking out after losing to AlphaGo, that the model has no sense of "aesthetic play", as long as it would lead to a win within the rules.
They're trained on human data. I would expect them to emulate human biases as closely as possible.
That's not why.
It's because counterexamples are easy compared to proofs which require new mathematics.
GenAI is great at combining existing things in new ways (interpolation). It's terrible at creating new things from scratch (extrapolation).