Ok, I'm going to say this from my own perspective and I understand all your reasons on both sides of the argument.
But I'm only just 40 and I had a baby in January. I already have two older children, (9 and 5 when he was born) and I didn't feel as though I had 'finished' - now I do.
But it has been at some considerable cost. Not financially - that isn't a massive issue. But in terms of the children...I had an awful, awful pregnancy - my first two were pretty bad too but this was worse in a lot of ways. My body didn't seem so able to cope. I spent the first roughly 5 months in bed, taking anti sickness meds, and so tired I could barely open my eyes. And the second half in constat pain and barely able to move or walk from SPD. It was really, really bad.
I was on my own though - from about 2 months pg - which obviously makes a difference as your ds will have your DH if you are unlucky enough to be poorly again. Mine had no one, really, and got very self sufficient and I feel like I was stolen from them for 9 months. 
It really affected them.
Since having him things have improved enormously and I am fit(ter) again and coping, just about, and I don't get phased by babies so that's all good...but there has been an impact on the others.
I think it is not so much about your age but about the safety net you have in place for all of you, if it does not go so well, and as you say, how it might affect your relationship with ds.
I really hope you reach a decision you are all happy with. It is so hard xxxx