Not making a choice, is a choice.
OP I accept that you don't want to leave your marriage. That is fair enough. So sorry that he is humiliating you by sh* on both your doorsteps.
But it is what it is.
Start investigating Radical Acceptance. You cannot expect emotional support from this person ever, and nor can your children. So sorry for her pain, poor little daughter. But lots of women grew up with emotionally unavailable fathers, make sure you are a source of warmth and love.
Therefore, focus on yourself, your family, your children and your friendship groups. Act as though you are alone, because essentially emotionally you are alone. Your friends and family NOT HIM are your sources of support, warmth, laughter and living. They NOT HIM will let you know you are a worthy and worthwhile human being.
And I would strongly advise you to start squirrelling away money/setting up your own investment platforms. Because one day he is going to fall in love with one of these women and leave you.
It is completely pointless, a waste of time and energy to be shocked, discuss him, whine about it or hope for difference. This is who he is. Perhaps you could go for therapy to help support you?