Betty I think the key word here is Compromise. And this can be achieved, I believe.
I am amazed at how many people think that it's absolutely fine, to withdraw all sex and intimacy in a relationship, and believe that the other Partner (who still wants intimacy), simply has to suck that up. And how dare the person who wants intimacy, actually say so. That's just them being a baby. Erm. No.
If you've previously had a good sex life, and then one person withdraws, never initiates, always says no to sex, then their partner has every right to challenge the relationship, and yes, to leave if they don't want to become celibate overnight.
I would not stay with my DH, if he suddenly withdraw all sex/intimacy with me. Because, I personally couldn't live like that.
I, for one, don't think there's an OW. If there was, he would have withdrawn from you, he would not want more sex with you, he would want less, and he would be gas lighting you. He's actually being honest with you and telling you what he's struggling with. It's not a crime to be dissatisfied with your sex life. He's not handling it well, by the sounds of it. But maybe you've had this "chat" many times, and nothing has changed? You don't say.
You don't say how often you have sex, how much you want and how much he wants? Why has your libido decreased? I am assuming it's because you are knackered, what with having a small child to contend with. That's pretty common. What can he do, to help you with that?
So, let's say :
- he'd like sex 3 times a week
- you'd like it once a month
If he takes x/y/z chores off your hands/let's you have a couple of lie in's a week, would that make you less tired/feel more rested, which could lead to you trying to be intimate say, once a week?
There are lots of Posters that would think that you shouldn't compromise at all. That you having more sex than you want, is just terrible. But this is your DH we are talking about, someone who you love and presumably still fancy, we are not talking about some random guy on the street. Marriage is full of compromises.
If you really can't face any sex at all/don't fancy him any more, then sadly it's for the best if you do split. But to me, it sounds as though you really do want to save your marriage. I read somewhere once, that the more sex you have, the more you want. Tracy Cox I think.
Good luck 