I'm going to be very blunt here, and spell out what other people are getting at when they say not to let your ex see your son unsupervised or at his mother's where he could just walk out with your son while he is in such an emotionally unstable state.
The awful truth is that some men decide that when a relationship is over, they will commit suicide and kill the children at the same time.
The sort of men that do this are immature and unstable, and emotionally abusive and manipulative.
Your ex seems to be all these things, and that is why people are saying that he shouldn't have any unsupervised contact with your son at the moment.
'Supervised' contact means contact at a special contact centre where he would not be allowed to leave the centre with your son.
Please go and get your son back right now. Make other arrangements for his care tomorrow or don't go into work.
Speak with your hv or ss to help you formulate a plan to keep your child safe until your ex has truly accepted the relationship is over between you and he has moved on emotionally.
You need to keep a distance from him, at least at the moment, while he sorts himself out. You can't be the one to help him do this, as you are the subject of his longing. He needs to do this himself, independantly of you, with the help of his family, friends and counsellor.