Of course leaving the EU is hard. Membership has a significant effect on regulation and governance. The fact that something is hard also doesn’t mean you aren’t free to do it.
It being “fought” or countries being “penalised” is a matter of opinion but not one I share.