You are upset because (well, because he's a game playing untrustworthy twat) you have an idea in your head. This idea is that the proper family is a woman with a man and they love each other. They parent together, happily.
The proper family is the one where the needs of the kids are seen to by a primary carer. That's you. So you know what, you have a proper family.
I know, I know. Nobody likes the truth. You're it and he gets to run away. Life is (hopefully) long though. Your kids will grow and you will be able to form meaningful relationships in the future.
Seek counselling though, if you can. Where I live, there's a place which does person centred counselling for free, this is unusual, I know. But there's a reason why you fell for this bullshit. You have 2 kids by him, I expect there's all kinds of shenanigans you've put up with.
Don't be confused. He's demonstrated his feelings towards you. He doesn't care, and the rest is up to you. All power to you, lady. X