If you don't want a child you don't want a child; that is fine.
Equally, if he is broody and he really does want a child, then sadly, this could be something you and he can't get past. It happens.
I am stereotyping slightly, and I may be off the mark here, but I would wonder whether or not he knows much about the practicalities of having a baby. There is a huge difference between looking at someone else's cute baby, and actually going through the process of having one (especially if you had a hard time last time). Also (and I'm 40 and TTC), the process of conceiving at 40 can be perfectly fine, and it can also be an absolute shit-show. IME, men are often less aware than women that 'having a baby' doesn't just mean rocking up with a cute newborn - it might also mean miscarrying, or damaging your health permanently, or having to make hard decisions around termination for medical reasons, or all sorts of other really difficult things.
And even if it could be 100% guaranteed that you'd sail through conception, labour and delivery and have the most adorable baby ever, the fact that you don't want to do it is justification enough not to.
I just think what you shouldn't expect is for him to swallow down his feelings of broodiness if they are genuine. Sadly, disagreements about wanting or not wanting a baby are big; they're something relationships do not always survive.