I mentioned it to him and he thinks I’m over reacting.
Brace up OP, tough love coming ...
His notions about how you ought to react to his disrespect of you & your home are immaterial.
You do not need to persuade or convince him that you are right.
You do not need his opinion, permission, or input about anything to do with you & your home.
You need to set a boundary with him that is non-negotiable & unbreakable.
Have a look at how much space you give in your OP to all the excuses why your ex "has to" have contact in your home.
They are all bullshit.
Of course he has somewhere to take his children - he's not living on the street, is he?
STOP LETTING HIM MAKE THIS YOUR PROBLEM.
Send him a text, along these lines:
"Hi Ex
I'm disappointed about the disrespect you showed in my home last week. Getting into my bed, & refusing to leave when asked, then telling me that my perfectly valid request is an "over-reaction" is unacceptable.
I am therefore giving you notice that contact will no longer happen in my home.
The DC will be ready to be collected at X o'clock on X day as usual, & you will need to organise your time with them accordingly.
Please do not contact me about anything other than making arrangements for the DC.
Regards
Looking."
Do not then communicate with him via anything other than text or email. Keep screenshots of anything he sends back, including any more bullying & boundary-testing.
Do NOT engage in any attempts to argue or negotiate.
Just keep sending back "Please do not contact me about anything other than the DC."
If he escalates, text -
"I have asked you to stop bombarding me with unnecessary communications. I only wish to communicate about the DC. If you persist in contacting me about anything that does not concern the DC, I will deem it harrassment, & be reporting it to the police."
You really, really, need to mean this OP, & follow through.
It may feel unpleasant for a few days but what's the alternative? - this twat continuing to mock & undermine you in your own home?
He is enjoying controlling you. Cut that crap right out for him.