There can be lots of reasons why a child doesn't want to see their non-residential parent. But given what you've said about your ex bad-mouthing you in front of the children, it sounds like there is some parental alienation happening. In your situation, I would make a formal request through solicitor that your ex attends a parenting apart/shared parenting course that highlights the harms of parental alienation (it's a form of abuse). If he refuses or attends but continues his abuse of his children, then I would apply to the court for a change of residency again. Evidence shows that children fare better after separation when they have good quality relationships with both parents. Court orders will only work if there is a willingness from the other parent. Mediation is another avenue where you can explore the harm to your son by being alienated from one of his parents.
In terms of your relationship with your son - the key things are:
Don't get frustrated or angry with him. Let him know that you're sorry he doesn't want to come but you won't force him. That you love him very much.
Use other methods of communication - keep checking in on him. Even if it's just a WhatsApp message, phone, or video call. If he starts to refuse these then set up an email address for him and send messages ,photos funny thing that reminds you of him to this email. When he's back in your life, you can give him access to these and he will know you have always thought of him.
Play the long game. Be kind, respectful, warm and loving. Do not bad mouth his father in retaliation. He will remember this.
I know that in Scotland there is an organisation called Shared Parenting Scotland who have resources for people who are being alienated from a child. Might be worth a look on their website.