Pots - I realise you have gone to bed now but hope you come back to this in the morning.
You have had a lot of strong advice on here, all of it valid - what are you clinging onto with this man?
He is using you completely. He is 47 - he's been like this his whole life - he is NOT going to improve. Just get shot of him now before he drags you down any further.
You have already said you don't get any money from him as it is, so you have nothing to lose on that score - and even if you get no maintenance from him for your baby, it's not going to matter. In fact, you will probably be better off because there will be no chance of you lending him any more money, will there?! (being stern because have also been in extremely stupid situation of lending money to ex because felt sorry for him, knew his sad background so made excuse after excuse for him UNTIL the point when I realised he was a selfish, deceitful, thieving CUNT and didn't need my sympathy - or my money for that matter!)
In answer to your actual title - YABU because this man is NOT a "partner" in any reasonable definition of the term. Cocklodger, parasite, even exotic pet are closer to the reality than Partner.
If the house is yours and in your name then you should have no problem getting rid of him - and if you do, call the police. If you are remotely worried about how he will react, call the police. If you need more advice on what to do before you throw him out, call Women's Aid first - they'll probably tell you to call the police if he kicks off.
The whole thing about him being good with the DC is honestly not that relevant to your godawful situation - apart from the fact that it is holding up your fantasy view of this termite and preventing you from despatching him.
As he is the father of your baby, he (if he's such a good bloke and lovely with the DC) want to see his child, and he might even deign to see the others as well.
It is Not Good for your children to see that this kind of behaviour is acceptable - and by accepting it from him you are giving them the green light to behave exactly the same way when they are teenagers. Then you'll have more than one household member using you as an open purse, and someone who doesn't need any consideration.
Your cocklodger says you are a chocolate fireguard when it comes to your children - how lucky for him! Because if you weren't, you'd have got rid of him by now as well. Tell him that you have taken this point on board and your first act as a non-chocolate fireguard is to get rid of him, as he is the biggest drain on your emotional wellbeing.
And get yourself checked for any STIs as well.