'I just think like everyone is saying that i'm giving a lot and he isn't giving up anything.'
It's way beyond that. He is using you. He gets someone to babysit whilst he has the kids, half the bills, housework, sex. He has isolated you from your support network and his tactic is working - instead of your first instinct being, 'Hey, wait! I'm getting shafted here!' you think, 'I just need to integrate more.' NO, that's not the best solution for you. Your life will be far, far less complicated and enjoyable without this man and his baggage. Why on Earth do you think that's the best you can do and deserve? Some guy with all this baggage?
Do you realise you have NO rights just now? If you displease him, don't tow his line, he can just throw you out with no notice. Just 'This isn't working out, you'll need to leave,' and you will have to go.
He'll never marry you. If you want marriage in the future, you need to leave now.
You'll never have an equal, fulfilling and committed partnership until you learn to value yourself and your goals and what you want out of life as much as you value any man or relationship.
Get away from him and take some time out to explore your boundaries and what you want and need out of life.
This guy and his baggage are a non-starter.