I agree, just tell him tonight - if older DCs are around ask them for a bit of space as you need to talk to dad or even ask him to come out to the garden for a chat to get some privacy. Otherwise perhaps you could 'book a time' with him tomorrow evening to say you have some things you need to talk about, but if he's the kind to sense this is coming and dodge it, I'd do it sooner rather than later and catch him off guard.
I don't know your situation so far, but if I were you (and I kind of am... H moved out yesterday!) I would present it as a fait accompli - there's no discussions to be had or ultimatums.
After yet another crappy weekend where you have felt alone, disappointed and frustrated, you have decided that this relationship doesn't make you happy and you would like him to leave. If he refuses you accept that you will have to leave, but it would be better for the DCs to remain in their home for stability, so if he cares at all about them he will do the right thing.
If he raises his voice, more fuel for the fire - he is entitled to be upset, but if he starts shouting or getting angry, is unreasonable and incapable of talking to you without raising his voice - its just another reason you are leaving him.
I would get very angry and shout at my STBXH because he didn't show any emotion and would stonewall (is that the right word?!) me, put his hands up and say "this conversation stops here" etc. As long as you are being respectful and dignified, allowing him his say, he has no real reason to shout or get angry.