Long backstory which isn’t particularly relevant, but to summarise, he’s not a nice person, is controlling and nasty, and has made some completely vile comments towards me over the last few weeks which mean I don’t want to be anywhere near him.
We have one DC and the majority of his contact with DC is at our house. He doesn’t have his own place. I’ve suggested multiple times that he takes DC out when looking after them, and today I told him he had to as I didn’t want him here. He point blank refused, and said if I forced him to go out of the house, he will just go home and leave DC with me. 
I don’t want to be around him and hate that he insists on intruding in my personal space. AIBU to just tell him that him spending time with DC here is no longer an option, and that he needs to make other arrangements during their time together?
Reasonable possibility he will fuck off completely from DC life if I enforce this.
If IANBU - would really appreciate if anyone can help me construct a firm but not argumentative message to tell him! TIA 