OK from personal experience I have finally learnt what I should have done in the first place which is......
First of all decide yourself how often and when you want your x to see your son. Then make a schedule and mark down the days/eves you want him to have your boy on.
Then email/call/speak to him and show him your schedule and ask him if he agrees to it.
If he wants to change the days/eves then be as flexible as you can but once you've agreed on the schedule make sure you both agree it should not be changed by either party unless of an emergency.
That way you both have a say and you can both then realistically stick to the schedule, without spending money on solicitors etc.
If he doesn't stick to it then remind him politely what you agreed.
You will see over the next few months whether he is going to be comitted to seeing your son regularly. Let him be the one to show you he's not interested, and never get into a row about it. That will take all the stress off you and he'll soon realise that he's being the problem.
I assume you are getting support from him. If not, you will need to speak to him about this too.
Best of luck at this difficult time.