Yes, don't meet him.
He doesn't get to set the timetable. Start that process now - detachment, you not being available for him to access at will.
Along with his conversation with your bro, TELLING you that he will be coming round for stuff and to talk is probably the clearest sign so far that part of what's happened here is a result of his overall, probably long term attitude -that HE is the boss, HE'S the important one - and when he liked his marriage, his family and your joint life, that was all dandy... as soon as he changed his mind, there's literally no recognition that you are in fact a separate, REAL person who, err, has an opinion of your own on it. Narcissism is bandied around a lot on this site, but that's essentially what it is - a total lack of comprehension of the humanity, the autonomy of other people.
So, you need to recognise that and snap out of the dynamic you've probably been in without even realising (remember when you said you 'assumed he'd try and get the twins a place in nursery near him'?)
What he wants is not only no longer relevant, it's something to be regarded with utter suspicion. Your default position is 'I'll think about that and let you know.' Your default reply to his baffled anger at that is 'Be quiet. You no longer have any right to ask anything of me, and I am no longer your friend.'
So - no meeting tonight - who the hell gets to tell you that you'll be having an important meeting sprung on you that very evening?! Fuck off. If you want some clothes, I'll put some in a bag for you and leave it outside. I don't want to see you. You think that's OTT/ridiculous, do you? Well tough shit. I don't like you posting on FB about your excellent life as a cheating shit, and I think that's pretty ridiculous too, but you're still doing it eh? So get used to me doing what I want to do. Hey, I guess it must be right for me - and I'd hate to lie to you, that would only hurt you more in the end - so here's my fuck off right now :)