I’m sorry OP, but you are not in such a bad situation and you are worrying and stoping yourself from leaving due to an hypothetical situation that may not even materialise: 50/50 residence.
Courts will focus on the children’s wellbeing, given you work from home, are the main carer of the children, and the children are so young, a 50/50 you disagree with is simply NOT going to happen. I agree he can try to take them off you out of spite and that his family could help him BUT do not underestimate your strength, you love these children and would do everything for them, Including fighting a legal battle that your good income can help you sponsor.
Also, do not underestimate his selfishness, sorry to point out the obvious but, if he is only using you for sex, he is not going to fight back much, he would probably agree with you immediately that splitting is for the best and try to get you to agree to a contact pattern that gives him the least trouble, possibly starting with alternate weekend and wednesdays as the vast majority of divorced parents.
I understand you are apprehensive of letting him take care of the kids away from you BUT, you will also need that time away to rebuild your life, and I’m not talking about getting a new partner straight away but about rebuilding the life you want for you and your children. Those alternate weekends are heaven to catch up with stuff at home, prepare for the week undisturbed and keep your sanity, it is not easy to be in charge 24/7, these “breaks” allow you to keep sane. I used those breaks to do house chores, get groceries, cook for the whole week, sometimes drive down to the coast on my own, have a coffee with a supportive friend without having to care about my child listening to the conversation or sitting in silence in the living room trying to work out who I was, what my wishes and needs were as I had spent so much time working around other people’s needs and wants I had totally forgotten what was that I wanted and needed.
I would also like to mention that divorce is a massively empowering experience. You will find your feet and with that your strength. I won’t say it will be easy (far from it) but after the initial shock and disputes, your life and the life of your children will be much much better. You really don’t want your kids growing up modelling their idea of what a relationship should be on what you and your husband have (and aiming themselves for such a relationship as that’s what they saw at home).
Be brave, if anything I can say (and I’m sure 10000000s of divorced people agree) is that it is far more difficult to take the decision to leave than dealing with the consequences of it.
Best of luck (and lots of courage)