They can be, in the same way humans can be optimizers.

In most cases, there's no explicit descent - and if any descent-like process happens at all, it's not exactly exposed or expressed as hard logic.

If you want it to happen consistently, you add scaffolding and get something like AlphaEvolve at home.