I think 46 is just at the upper age range to start trying tbh. By the time you’ve conceived and had the baby then you’re in a situation where he’d be doing the teen taxi run while pushing seventy, especially if you want a second child.
Lots of sixty somethings are in perfectly good health, but lots aren’t, and you can’t necessarily tell in advance which will fall in each pot.
You are of course in the position you are in, and it’s unrealistic to say “dump him and immediately find a younger man who wants to have children with you” but it is a real risk. I’m assuming that he doesn’t smoke and isn’t obese, or have other health red flags.