What everyone who has responded seems to have missed is that this mother, says this father, wants to be seen to be a good parent, so getting a child arrangements order in place may well be effective at controlling his laxity.
He'll want to present well to the court, and once an order is in place, the mother will be able to show exactly when she has had to step in when the father fails to take up contact, or cancels (outside court agreements don't allow for that). Then it's back to court for enforcement - which is a blight on the father's perception of himself as a parent. His ego probably wouldn't stand it.
All his failures will be recorded. The mother can then challenge any CMS calculation if the father fails to take the contact that is ordered.
And whilst it is not in the court's remit to force a parent to take up contact, they can certainly punish someone for breaching an order, and that includes the non-resident parent.
But the benefit to the mother here is a) a court order defining the time he has the children, b) creating a documented list of missed visits and c) changing the child maintenance payments if he doesn't have care of the kids when ordered.
To the OP: put your application in here: https://www.gov.uk/government/publications/form-c100-application-under-the-children-act-1989-for-a-child-arrangements-prohibited-steps-specific-issue-section-8-order-or-to-vary-or-discharge
And if you have not already done so, apply for child maintenance here: www.gov.uk/child-maintenance
If your husband earns more than 156k per year (feasible if a consultant and also doing private work), then CMS only calculate up to that amount - so see a solicitor about a Schedule 1 claim instead. Your kids have additional needs, so there is potential for you to get extra this way to cater for their respite care.
If you believe you are being abused, then check Legal Aid here: www.gov.uk/check-legal-aid