No surprises what I would say to this.
I'm trying to get into your mindset in that you've got a toddler and another on the way. The prospect of your H having an affair at this time particularly must be awful to contemplate.
But I am in no doubt from what you say that an affair is either happening now - or is about to happen. Putting your head in the sand and trying to jump through hoops isn't going to stop this. You cannot prevent infidelity in someone else - none of us can. Only your H can preserve his own fidelity and commitment to his marriage.
I don't know how much reading you've done about affairs and infidelity, but one of the truths of it is that with male infidelity, it is most often practised by the person who is over-benefitted in the relationship. The one that life must revolve around, where everything has got to be about his wants and his needs. Sound familiar?
Secondly, the birth of DCs or in fact any catalyst that forces someone to "grow up" and take responsibility at last very often sees them heading the other way into escapism and a selfish "high" like an affair.
Look at this logically. Another baby is going to reduce spontanaeity. You know better than most how unspontaneous life becomes with a baby and with a toddler in the mix too, that's only going to get worse. Grown-up parents realise this and rationalise that it doesn't last forever. So of course they plan time with their partner so that life is not all about parenthood. But nobody can be spontaneous with a baby, it's just not realistic.
You are already bargaining away his comments as being fair because this is how he feels. I don't see it that way. A man committed to his marriage would be taking joint responsibility for keeping it alive and flourishing, not pondering aloud that he wonders whether there is someone or something better than it.
Unlike you, I don't think he is being honest with you at all. I think he is telling you why he is engaged elsewhere without admitting that he is. He couldn't be telling you in clearer terms that he has started to look outside of the marriage. Just because he says he won't leave you doesn't preclude him having an affair.
I agree with the suggestion of Relate, but this only works when the clients are not with-holding secrets. It is a complete waste of time if someone is hiding an emotional or physical affair.
If you pretend that you can fix this on your own by meeting his wish-list - and you don't resolve this now then I fear you will be parenting two small children with a man who has disconnected from you and ultimately might leave you. Whereas a radical shake-up now and some insistence on honesty could prevent that happening.
Now I have to tell you that in your shoes, I would be looking for some external verification about his fidelity and that means looking for clues. Not just snooping on phones/laptops, but thinking of likely OW candidates at his workplace or the places he frequents. If he is already involved or wants to be, he will not tell you the truth.
Please don't hope for the best with this one and act passively. Please don't jump through hoops trying to be the perfect woman. Trust what he is telling you - he is not committed to his marriage or fidelity at the moment. That is not going to get better through your efforts - it has got to come from him and the changes he makes to his behaviour and attitudes.