Goodness.
So you are already subsidising him, although he could and should be paying his way.
And he's one of these: he thinks I have it easy even though I have been up 4 times a night feeding since DD was born. - hmm, yes, they're always princes among men. In fact - in that vital year after you've had a child - a man coming out with stuff like this is probably the single biggest indicator of an utter, utter prick.
What does he do with his 35K? Well. It's not actually a massive salary depending on where you are in the country. But 'where you are in the country' actually translates as 'how much your rent/mortgage is'. And he may as well be living in the cheapest place in the UK and more, because he's not actually paying rent or bills. His only outgoing is more food than he'd eat as a single bloke.
So he should have a massive disposable income. He could be a complete fritterer, but hmm, if you live together and you roughly know how he spends his time then that's unlikely to be it. He's either making an utter killing saving up his own income while you shoulder the main living costs, or he gambles, or drinks more than you think, or has large debts you know nothing about.
One thing is absolutely obvious - he's rubbish with money, and believes that the world (ie you at the moment) owes him. One is heavily linked to the other.
And... he's unpleasant, unfair, lazy, a sponger, and you can't have an adult conversation with him about issues without him throwing a sulk.
Yes, you have been kidding yourself about the kind of person he is. Don't kid yourself any more. You are clearly motivated, honest, hardworking. You could do better. And by that I mean that you could have a much much happier life, and your DD too, than you will find by linking yourself to this twat.