Firstly, your position isn't less strong because it's 'only' emotional abuse, with the new laws you still have protection. What pp's have advised re documenting further abuse, from him or his family, is vital, also any evidence of alcohol abuse by him or his family.
Your aim is to build a case for supervised contact only, and not supervised by his family either, that's the best, most realistic way to keep your DC safe. You absolutely mustn't ignore your gut feeling but your best line of defence now is to gather enough evidence to support your case for him not to have the DC unsupervised.
Refuse contact for now, respond the same way to any and all requests for contact, that you feel it's in DC's best interests to have a formal arrangement in place first. Don't get into any discussions about it, just repeat that statement and record the abuse that will surely follow, it will all help your case. Is he on the DC's birth certificates and is there any chance he might try to take DD from nursery? That's the first thing you need to address with WA if so, there are things you can put in place to stop him.
Stay put at your mums for now, contact WA asap, there is a function here www.womensaid.org.uk/domestic-abuse-directory/ to search for services in your area if you struggle to get through to the national number, just scroll down and you can enter your location and contact local services direct.
Ignore suicide threats, emotional blackmail etc, you are only responsible for your own actions and the safety of your DC, what he does is on his head, no one else's. If he was a serious suicide risk he wouldn't threaten, he would just go and quietly do it, its a tool to manipulate you with, nothing more.
Stay strong OP and listen to your gut, your DC are lucky to have a mum like you 