You could break the agreement and return the money. I doubt anyone could argue that a one time payment can gag you for the rest of eternity from certain speech.