I agree with everyone that has said your DP's behaviour is worrying. If I had any advice now it is to trust your instincts.. If his behaviour didn't ring any bells you would have had no need to post and that niggle you had is there to protect you. Listen to that internal voice, if you think something is unfair or out of order - IT IS and don't let anyone convince you that you are a bad person for doing so and trusting your judgement. The test of your new guy is to be assertive with him with regards to him telling you what to do. That said, you need to do the same with your ex.
If he was abusive towards you, then you shouldn't ever have to spend time with him again, and you don't need to. It's good that you are on good terms with his family, but if they know he has been violent they shouldn't expect you to, either.
Your ex is behaving this way because even though you left (well done you, it's hard and you should be proud that you got out) you sound as though he still intimidates you. My ex was also like this, so I refused to see him and told his parents (through solicitors) that they needed to do the handover as I refused to take his crap anymore. 5 years on I haven't had to see my ex at all in that time, but my dd sees her dad and gp's and all is good.
You need to set boundaries for both these men. Your ex needs set visitation times, with pick ups by his family, or at a contact centre. If you haven't already, contact Women's Aid, they will help and support you with this and everything else. And with dp, set boundaries for him and dont let him manipulate you. I do think the red flags are there but if he responds to you asserting yourself with love and understanding, perhaps he is a good man after all. If he responds with anger or tries to bend you to his will, then you have your answer.
You are stronger than you think, OP, you've done the hardest bit. Now you need to heal, and figure out what happened and how to recognise the signs in men. Read the Lundy Bancroft book for all you need to know to protect yourself. Above all, start to see yours and ds's needs as THE most important and base everything around what is best for you and him.
Please do keep coming on here, there are some very kind and wise posters who will help you to trust yourself and work your way through this. Be kind to yourself and I wish you all the best.