Having done all three over my career (local machines as student, University's HPC cluster as grad/postgrad and eventually cloud in the private sector), I'd 100% rent out compute. Buying a custom rig for local ML training makes zero sense in 2026. With renting you'll be able to train bigger models/datasets faster on much better hardware while paying far less in total. Especially if you're doing project-based work.
I mean, there are lots of reasons why you'd want something local, but I agree that the cost makes it unreasonable right now.
Friends of mine wanted to buy a GPU server last year for their new project. Management told them no, that there was a deal with cloud providers and that investing the budget in cloud compute was better.
The project ran out of budget roughly three months in. From what I heard, status meetings consist of everyone getting together once a month, asking "have we gotten any more budget yet? no?", and then wait until the project runs out.
Cloud providers need to make a profit too, and they've gotten very good at ensuring that their clients forget this point. The whole advantage of cloud compute is that you don't pay for the time you're not using it, but once you reach a certain point you're the one covering the overhead for everybody else.
If you use a custom disk image too, you can have it boot, apply security updates, reboot, then ssh in within minutes.
Work for however long, shut down. I do this with linode and their GPU VMs. If you shutdown nightly and on weekends, the costs drop dramatically on a monthly spend.
I only need it a few hours per day, so very cheap for custom compute.