Totally agree with what the others have said. This is such a ridiculous situation that it's hard to respond to it sensibly!
Basically, you don't owe this man a relationship. He doesn't deserve to have a relationship with you. Even aside from all the weird stuff, and even if he were a lovely man and even if his mum weren't harassing you, you are still within your rights to end a relationship with him for any reason you choose.
"I don't like the underwear he chooses for himself"
"He likes a band I think are shit"
"One of his friends smells funny"
"The colour of his hair puts me off"
Now these might seem like picky reasons to end a relationship, admittedly, but it wouldn't matter. You can end it for any reason you choose. And the reasons you have ended it for are very valid, and very real. Perhaps when he has grown up a little bit he will be in a position to have a proper grown up relationship.
And maybe this will be the impetus for him to do that. But he won't do it with you. The pattern of his and your relationship is already established. And he has far too much growing up to do to be a supportive partner to you and a father.
You've given him plenty of chances. If your feelings or presence in his life were enough to 'change' him, then they would have done so by now.
Don't be fooled by "if you have this baby...". That's an attempt to appear reasonable. It's an attempt to indicate that your feelings matter in this. Oh and it's an attempt to make you believe it's about you and not the baby. They know what it looks like and, IIRC, things have been threatened..?
Seriously, do not feel guilty. If I were you, the fact that his mother turned up on my doorstep to say all this, whether he knew about it or not, would be enough to reassure me I was doing the right thing. It would not make me doubt myself.
Be strong. You won't regret it. And you'll back on this in 5, 10 year time when you are with a good and decent man and realise what a lucky escape you had.