Of course partners who don't want sex have the right to object to their partners having sex elsewhere.
I think that sometimes, both men and women do withdraw sex if there are other problems within the relationship. The issue can then quite easily become a token of exchange, as in "you were 'nice' today, you 'helped' today, therefore you can have sex".
In my experience, that pretty fast degenerates to "What, because today you recognised my contribution, you're intending to reward me with sex? No thanks, I'd rather do it myself." DS and DD were separated by 2 years, and I think we had sex a couple of times on one holiday which resulted in her conception.
After that, sex became extremely rare, because it was only used as a trade item, and I didn't want to play, and in the end became functionally impotent with her.
There's an old saw along the lines of "Women need to feel loved to have sex, men need to have sex to feel loved". While it's obviously a cliché, it suggest both love and sex are pretty important.
In my case, I had an affair later, realised how unhappy I was, and after a couple of false starts and hesitation left my wife. Not for the OW, but because I was unhappy.
Fortunately for me, the OW became DW some years later.
Yes we have had issues; not once have I ever felt sex is being used as a "tip" - but yes, I recognise if I've been a work-obsessed div, I'm unlikely to find DW in a caring sharing mood. So we try and share time, hobbies and work - and the big thing is recognising we need "us time".