OP take some time to think before you do anything rash like ending the relationship on the say-so of the numerous posters on here.
Yes, his behaviour has been shit. But you clearly had a strong enough relationship to try for a baby. We’re not talking about someone you’ve had a shit relationship with, other than your wish to get married and his wish not to, how has the relationship been?
We don’t know the nature of the argument to start saying that him driving off is a red flag. If it became particularly heated then maybe that’s the best thing for now. All couples argue and in some instances those arguments can result in one or both going off to find space.
I am definitely of the belief that one should be married before having children, but given you weren’t you were clearly happy enough in your relationship to have one, and that whole relationship presumably hasn’t hinged on you being married one day, there’s more to a relationship than that, and given you’ve been talking about it for years, presumably you already knew he was reluctant if he hadn’t proposed?
Take marriage out of the equation. What is your actual relationship like? If you never discussed marriage ever again, what would the relationship be like? That is what you need to build on.
Not giving a baby his surname is inconsequential really. And if you fail to put him on the birth certificate he will simply go to court and it will be done and he will be given parental responsibility. Actions like that are purely to prove a point and nothing more. They have no actual basis in the child’s life.
So, other than marriage, what do you want? Do you really want to leave because he won’t marry you, even though you were quite happy to have a baby with him? A baby is a far greater commitment than marriage. You can get divorced and walk away from one another, once you have a baby you will both be that baby’s parents for the rest of its life. Marriage or no marriage, that’s not going to change.