> you can't delete your credit card information from their system

In cases like this (for me it has been Verizon and Adobe), you can usually change your credit card on file. Nowadays I use Privacy merchant-locked cards for everything online but in the cases where I used a physical credit card, I would change it to a Privacy card and then pause that card or put a monthly cap on charges (after making a reasonable effort to cancel.)