Similar situation op, my dh also agreed then said no after stringing the subject along for a few years. I posted on here a few months ago as the resentment had been building up and i felt so angry towards him.
My thread made me realise that i would leave over it. I made escape plans as dh is also controlling, i waver everyday but the upshot is i dont trust him or what he says, he manipulated my feelings and lied to me to control me and the situation, hes selfish and wont change, these issues all stem from the baby issue.
Then theres my biology and the fact it physically hurts sometimes when i see babies and i have two kids but the yearning is strong and im grieving for my child who will never be.
My thread woke me up a little and made me start to confront that dh is ea and controlling.
My only advice is look at your life imagine in 5 years time, with him and without him, both may have no baby in it but will you hate him, will you be resentful and more importantly will you be able to look yourself in the mirror?
Staying for your dc is all well and good but the resentment will begun to turn inwards as you become trapped in the situatiion. You will end up very bitter. This is before you consider his cheating, he has manipulated you op, he is a liar. However you are young and sound like a great mother, i think you are right to think to leave him and sooner rather than later.