It broke my heart that he was sat there crying that he wanted a cuddle from me but thought I’d say no
This is appalling - an animal wouldn't be treated like this. You are absolutely right consider it completely unacceptable, and it is outrageous to me that your ex would try to impose his new rule on your son in your own home - it's dreadful. And let's face it - your ex has that rule because he wants privacy with the new girlfriend. I'm sorry, but that is what it is.
So, he's sacrificing your son's security and right to be close to his parents so that he, your ex, can have sex in private with the new woman.
It is nonsense that a child that young doesn't see and feel things - absolutely bollocks. It will affect the way his emotional pathways develop - it literally will affect how his brain develops and is wired.
So, as the parent who is clearly trying v hard to do the right thing and put your son's interests first (not your adult 'right' to fxxk someone), I really think you should go to your gp, go to a solicitor, and maybe talk with a paediatric psychologist or counsellor about all of this. It doesn't seem in your son's best interests to be half the time there. Not if he is not prioritised. And your ex can F off, omg, with saying what you can and can't do with your own son - just as you are unfortunately unable to say what he should do.
I would make a big fuss. Big. And I don't like making a fuss. But this is wrong. And it WILL affect your son. (I know as split up with my ex husband when my son was 2 and thought he wasn't affected and later it all came out - he was).
You DO have a right to make sure your son is properly parented. Not allowed a cuddle in the night at 2 is pretty much tantamount to abusive, in my opinion. It's all wrong.