Leave. Don't wait. I wish id left years ago. I stayed as I thought keeping family together was the priority. It doesn't get better.
Financially im better off. Emotionally at peace.
My children are happier, I am happy.
Its really hard with SEN, no support and dealing with him dipping in and out with seeing the children but I wish i had left years ago.
Edited to add: he is an alcoholic. The day i left he was looking after them in the summer holidays whilst I was at work, I came home early, he was drunk before 5pm. That was enough. Initially went no contact but eventually after months of him saying he had stopped drinking agreed to contact. Paid public activity like soft play, but I drop off and pick up. Not good for me but helped my children emotionally at the time.
But as with all alcoholics, it didn't last. We did Saturday mornings and he often cancelled due to not confirming attendance (asleep hungover) or he wasn't fit to drive to meet us. Worked well for me as this was the evidence I needed. I say now that he can take me to court (he wont). I would agree to alcohol testing as part of the court process (he would refuse) and my children remain safe with me.
Most are all talk. Stay calm. Child focused. If you haven't already keep a journal of all time offered with kids, whetger tgis went ahead or not and why. You'll forget in time but its so handy to have your own Chronology prepared in advance.
Fo us on risks to children, safety plans and not your own views.