I mean this in the nicest way, why are you accepting this? Do you consider this a partnership?
Fair enough, your children are not his, that's fine. However, you have given him a roof over his head and the MINIMUM he should do is pick up half the rent and half household running costs. His excuse is poor, if he was not on the scene yes you would still be paying but his expense would no longer exist.
If you want this to work, he either finds his own accommodation and you continue a relationship like that, or he is an equal contributor and picks up half the bills, he would be paying more if you kicked him out, remind him of that.
Overall, I rarely say this as we don't get the full insight in to relationships here but you can do better. I don't think this man will ever want to do right by you or your children, he seems to do the bear minimum for his own and that speaks volumes to his character considering his earning.
Stop letting him leech off of you, do better for you and your children. All the best to you.