I had the same issue a few months ago on AWS. All I want is a server (that pulls a container), a database, and a load balancer. It's all going behind CloudFront so there should be no need to pay for an IPv4 address for any internal machine. Couldn't do it. Since then I saw that there was some movement on IPv6 for RDS but IIRC there was still some other blocker.