Courts deal with this type of scenario ALL the time. They’ve already seen and heard every manipulative trick in the book that mothers use to cut fathers out of childrens lives. The court will see that you’ve made things difficult by putting 25 miles between your son and his father, hoping he’d give up trying to visitor lose interest. They will see that you’ve agreed to further mediation in order to look good, but then pulled out of it because you had no intention of negotiating. They will see that you’ve applied to put your son into nursery on days his father is available and wanting to look after him. They will see that you’re refusing to let him take care of your son when you're at work and are registering him for nursery to spite your ex, hoping that the court won't want to disrupt his nursery placement.
With the way you are behaving, they will see a hurt, selfish and emotionally manipulative woman taking her anger out on her ex by withholding their child, and given that this is emotional abuse of not just your ex but YOUR CHILD, I wouldn't be surprised if a judge ordered 50:50 custody and for social services to watch you like a hawk.
You mentioned that your son cries when your ex brings him back home. Of course he does. He's had a wonderful time with his Daddy and doesn't want that day to end - he hardly sees him. If it they saw each other on a regular basis, your son would feel more secure coming home, knowing he would be seeing Daddy again soon.
Stop using your child as a weapon to hurt your ex.
Stop hurting your poor baby by trying to cut his daddy out of his life.
Stop keeping him away from grandparents, aunties, uncles and cousins. If you died tomorrow, wouldn't it be reassuring to know that this extended family wouldn't be brand new and scary to him? That he would at least feel comfort with some familiar faces?
If I were you, I would be shit scared of this going to court. You could end up with either a 50:50 arrangement ordered upon you, or you could (because of your manipulative behaviour) lose custody of your son altogether and get weekend visits.
All of this is avoidable if you’re brave now and agree to regular access. At the moment, you have a chance to propose how many days/nights he spends with his dad per week and you could negotiate when they are to suit you both. If you let this go to court you WILL lose control of how many days/nights and what days of the week they are. The court will decide for your son.
Wake up. Sort it out before it's too late.