Right Mummyfixit - how old is your DS?
I went through this with XH about 5 years ago and Mumsnet was a huge help.
He threw me out and I had to fight to get DD back - i did though eventually.
It will go against him in a huge way him not sticking to agreed times and dates.
Few pointers - put his name down at a local school, make sure you are the one who deals with dentist, doctor and hairdresser - doing all the important things - even get him a place at a play group - you will then be caring adeqautely for your son and have his future set up where you live.
Never slag off your husband to a judge or cafcass officer - if anyone asks you about his abuse tell them the truth but add that you realise that this does not stop him being a great father.
Do either of you work - if he does and you don't you need to state this very loudly - it is no point giving him custody and then the child going to a childminder or nursery when his mother is sitting at home.
If you go to court you will need some witnesses who will have to draw up statements as to why they think you should have custody - make those statements full of reasons why you are the better carer not full of hate and resentment against your X and never mention the word win - went against XH in a big way as did his drinking which my witnesses had seen.
If you have a close relative whi is a fireman/policeman get them to do a statement as it really helps if they do and when they go to give evidence they wear their uniform.
We ended up with shared residence of DD but me having custody - she goes to her father every other weekend and half of the school holidays - you need to say to the court officer and in your statement that you feel the arrangements need to be put in place now that will enable your cild to attend school when he gets to that age - you can't do the week on week off thing then so you may as well get it sorted now - IYKWIM.
I live in North Essex and if I can be of any help do let me know by CATing me. I hope that helps though - and you must stick on the side of the law no matter what your heart says - it will go against him in court what he is doing.
Good Luck XXX