What do you get out of this relationship now?. What needs of yours are being met here by him?.
I also wonder whether your love for him is actually rooted in an unhealthy co-dependency as well.
You were never really shown a decent example of what a loving relationship is when growing up so I do not think you know what a mutually loving and respectful relationship actually is. It certainly is not this particular model of one. Your own innate desire to create a family life, the one you never really had as a child, has really cost you an awful lot.
What did you learn about relationships when growing up?. Think carefully about that and the two questions I pose to you above. From what you write your mother did not show you the best of lessons and therefore you've had no sound template or role model to work from. I think you are going to have to unlearn in time an awful lot of stuff from your past and that will need to be done through counselling. That is going to be a lot of hard work for you but I would urge you to do that.
What commitment has he shown you exactly?. Some men do find it very easy to walk away from their children; they are not the ultimate commitment by any means and nor is a mortgage. His commitment is basically to himself and to get his own needs met and at your overall expense. You've given it all to him and freely too. He's also basically threatened to get out if you do not come up with your share of the deposit towards a house. He sees his money as his and his alone; he does not want to share it with you and he will never make you his wife. Infact it would not surprise me at all that he goes onto marry someone else a lot younger also pretty quick (say within 12-18 months) after you and he part. He is not that into you.
His actions towards you are not loving ones; you have given practically all your power over to him in this relationship and these children likely have his surname as well. You in turn have no real security and no real relationship with him to speak of; the power and control here in this is really all his.
I also think you need to be apart from him now. Is this really what you want to teach your children about relationships?.