I'm learning as part of my study of Chabad Chassidus now - hopefully, not too belatedly - that marriage is a precious opportunity to do the kind of daily work on our ego we can't do alone, especially in that our spouse does us the "favor" of humiliating our ego even on a daily basis. Giving up this opportunity because of all the difficulties marriage constantly poses seems akin to giving in to our ego even if this is a reaction to some egoic speech and/or action by our spouse.
But naturally, fighting against the ego of your spouse, or anyone else in this regart, with your own ego only worsens the situation. I've come to realize, again thanks to my formal study of Chabad Chassidus, that your commitment to marriage must manifest itself first and foremost as your trust in your spouse and his or her abitity to tame his or her ego and its "garments" (i.e., thought, speech and action) and your compassion for him or her when he or she is jailed in the ego-made prison and struggling to liberate him- or herself from it.
I used to think until rather recently that to give in to a "sweet whisper" of the ego and give up marriage is a kind of betrayal. But now I feel that such an action and the person who has decided to take it deserve both our compassion as it stemed from his or her ego and our trust in his or her ability to become aware of this someday.