The problem is that you can't know. You can't read the minds of the people involved, so you can't know if there was pressure, coercion, or something else going on, and you can't be sure if either of them are telling the truth when they both insist it's fine and fully consensual.
So you prohibit it in all cases, punish in all cases, and then you don't have to wade into that mess.
Thats a crazy approach. By that standard no man should ever have sex with a weaker woman, because you cant rule out physical fear entirely.