Have you considered the possibility that sometimes people give their spouse permission to have an affair?

Have you also considered the fact that you can get married without taking such an oath? A wedding ceremony is not how you get married. You get married by signing a document that says you are marrying the other person who signs the document, and then filing the document with the government. Marriage is a legal arrangement that primarily covers property rights and certain legal decisions (e.g. the presumption that your spouse can make decisions on your behalf if you are incapacitated). Even in the few remaining places where cheating has any meaningful legal significance divorce lawyers often advise clients not to bother -- it is rarely worth pursuing and most people just want to finish the divorce as quickly as possible and all filing for fault will do is raise your legal fees (for little tangible benefit).