Yes. I'd contact a family solicitor ASAP. Compile any evidence of his manipulative behaviour and the fact that he said that in front of your son who fully understood is horrific. That's crossing the line. Yes he may want child contact and he's picking and choosing, that's taking a huge toll on your son subconsciously making him feel like an "option" or a tool.
He also indirectly threatened you. Please record any conversations in person now should they happen and any texts etc keep screenshots.
Sort out a childcare plan with your solicitor directly and request her to send it to court with your c100 form. This would make whatever the plan is legally binding. Eg if contact is pushed for by your ex the times etc must be stuck to. If he breaks these rules consistently you can take him back to court and make the changes you and your solicitor find appropriate in the child's best interest.
Some contact will always be pushed by family court so please provide as much of the story as well as you can to your solicitor. If your ex was abusive (he sounds like he was) emotionally please make a report to the police. With any evidence of how he was manipulative / why you left etc. Even if it was a while ago and even if nothing can be done about it. Just that you've reported it is enough it'll show up on your c100 form.
This will show the judge you made a plan and tried to encourage child contact (always good sign) and for safeguarding purposes you got in contact with a solicitor to make contact safe as possible for your child. Your ex behaviours will be easier to prove in family court depending on what the police say about any allegations you make against him.
if you haven't already fill out a CMS application. In a way it's taken from his wage directly so he can stop threatening not to pay. That's a form of post separation abuse.
I hope this helps in any way.
Also if you have reported domestic abuse you are entitled to legal aid. Free solicitor.