You could accidentally swap ips to one that had a port forward, some applications can ask routers to forward, etc etc. I donmt know how exactly we'd measure the various potential issues but they seem incredibly minor compared to the sheer amount of breakage created by widespread nat.
I don’t have any problems with NAT on my network.