I don't suppose the 'it's stopping you having kids' angle is going to particularly resonate with you right now. I'd guess you're using the fact that you're a childless 'career woman' as a bit of a bargaining chip over his wife who's got more on her plate, so it wouldn't be in your interests to think too much about kids and being a mother.
But it's also probably because you're childless that you believe all this rubbish he's telling you about not being able to leave his kids. You probably don't know that many couples with children who make a grand job of shared parenting, or too many single dads who've found that they actually spend more time with their kids than before.
I'd also guess you haven't got a clue how most marriages with young children are and are living in some fantasy world about his wife neglecting him in favour of the kids, so he lacked attention or somesuch rubbish.
In my experience of mates who've been in your shoes, men who are never going to leave their wives tend to weave two different stories. They either blame their wives and their marriages for their affairs or they claim that they'd have bumbled along quite happily until they met the love of their lives i.e. you. Only the seriously stupid OWs fall for the first story, so the second is the one that the most manipulative characters come out with, because it makes them sound helpless and defenceless in the face of your irresistible charms.
Of course he'll be saying that he's in it for the specal connection with you and not just the sex and will even be passing off the lack of money he's spending on you as a positive thing so that you don't feel prostituted in any way. In truth it's because he can't risk money being diverted to his affair because they'd be broke/it would be noticed - and there might even be some guilt if he did spend any.
You're saying that you don't want him to leave his wife and family, so he's got it made. You won't rock any boats, you believe all this balderdash that you are his 'one' and you're probably telling yourself that it's 'so sad that you met too late'. Hell, you probably even feel sorry for him.
If you met him some place you both go regularly, like work or at a hobby, the reason he's having an affair is because you were there and you were willing. It really is that crude and that simple. He probably quite fancied the idea of sex with someone new and ideally with someone who'd make no demands on him. He probably quite likes you as a person and he probably loves the ego boost and the attention he gets, but that's about as deep as it goes. He really would ditch you unceremoniously if his wife ever got wind of it, or if you started making any demands.
Go right ahead and test this out, if you want to find out the truth.