I'm in the 'whatever works for your happiness' camp as long as all parties are on board.
I personally am not wired that way, I am a one man woman. I don't have ANY sexual attraction or thoughts about other men when I'm in a relationship. Unless you count looking at someone and knowing they are handsome. It's in the same sense I know a beautiful woman is beautiful but have no physical attraction to her or see a lovely art piece and have no need to own it, LOL
I've been in relationships where I've been cheated on, I didn't even have it in me to reciprocate. Sex is part of the bonding to me and in my heart and mind I only want the two of us in that place. It's not a religious or legislated issue.....it's how I feel and who I am just as others find and know what works for them.
Happiness....we all deserve it.