I'm just going to reply to a series of things you've said, sorry if this is long
"he is potentially going to be struck off or suspended from his professional accountable body if I am open - he works with adults not children."
You're not responsible for this - if he IS struck off or suspended then that is because of HIS choices about how he behaved. Regulated professionals are regulated because the public place trust in them - if he is a nurse or a lawyer then he is seeing people at their most vulnerable. Manipulative, coercive individuals who get their way with the use or threat of violence have no place in a regulated profession.
He can make some choices of his own now: if he were to engage with a perpetrators' programme and really show insight and ability to change then he is much less likely to be struck off. You can't change his past behaviour and nor can he, and he is now the only one with control over whether the past has a knock on effect on his career for the future.
"stupid question - does it matter that he hasn’t directly hurt me since they were babies?"
It's not a stupid question and I'm sure he'll have implied or outright said that you'd be mad to suggest he's an abuser if he has stopped hitting you. It doesn't matter. He only has to get physical on rare occasions for the fear of it to control your behaviour constantly, so it still counts as coercive control. Especially when it seems he is still physically abusive but has transferred this to the children.
" I am overcome with shame and humiliation, I know I shouldn’t be but I am."
Channel Gisele Pelicot - print it out and stick it on your mirror if you need to, "it is time for shame to change sides."
"My youngest has complex needs, including LD and I expect his dad will be allowed contact with him at some point."
I would not be sure of that at all. You have a SW who thinks he is a credible threat. Even supervised contact with a child with LD might be ruled out as too risky and as not a long term and sustainable option.
"I am worried that they will think I am a bad mother for not leaving sooner. He was violent to me and then he was violent to them. This knowledge is killing me. But I worry that this demonstrates that I am a bad mother. I don’t want to hide anything but equally I don’t want to be too honest and get him in more trouble than needed. Especially when it is his word against mine - who does this actually help??"
They will think he is a bad father, for being violent to you and then to them. You are working to protect them, despite having been conditioned to obedience to your ex. That is hard to do and it is the absolute exemplar of putting their needs before your own. You seem to be giving yourself a harder time than your coercive controlling "credible threat" of an ex - you shouldn't be.
You're also not the social worker here, you are not the Guardian, you are not the judge. You don't get to decide what is the right amount of trouble for him to be in. Your sole job is to give them the facts. They can then assess the facts against his level of insight, cooperation, his background (eg was he in care / from a domestically abusive household that has normalised this behaviour), his ability and willingness to change, his attitudes towards women, etc etc etc. All of that hangs in the balance and all of that is solely up to him. He can choose cooperation and insight or he can choose abusing the social worker, and those choices may lead to different outcomes on the same facts.
As to who it helps - well. If he miraculously gains some insight, participates effectively in a perpetrators' programme, and revolutionises his attitudes to relationships then it helps him and any woman he gets involved with in future. If he doesn't, then it helps any future woman he might get involved with as she can access a Clare's Law request, it helps your children to get protection from him, it helps your children to delineate how seriously unacceptable this behaviour is in a relationship, it helps you to have the external validation that you're not mad and this really did happen, it helps the wider public who will not have to worry that their nurse (or whatever he is) might be a coercive controller, and it helps the public interest in ensuring, if only in this one instance, that violent abusive men do not get to be violent abusers with impunity.