OK so your update changed things op because we now know why he said those things. He is manipulative and wants you to feel insecure. That's why he does all that other shit too btw, like not introducing you to his family. It's BECAUSE he knows it's important to you. Think about that for a moment.
As for him being the best of a bad lot...look, a shit sandwich that is 30% shit is just as bad as one that is 50% shit. It'll still make you ill!
As for not being alone at 40...40 is probably not even half your life! Why couldn't you have a new man at 45, 60, 70 ect...? And periods single. You obviously can't have kids with this dickhead and he doesn't even want to get married. And he's obviously not going to stick around with you forever, that much is clear. So why don't you just broom him?
Instead of fearing being single, you maybe need to try it. Because if the alternative is dating shit man/men forever...that's not life! That's a jailcell! Get him gone and spend some time single, learning how to spot nasty manipulative atseholes like him so that in future you'll find yourself with GOOD men. Because they do exist. And even if they don't, never eat a shit sandwich!
Let go of this daft idea of settling down. Fovus instead on living an emotionally healthy life where you love yourself. And where any partner, is just to add joy to your life, not detract joy from it.
I mean - Do you really want to spend the next 50 years with this cunt?