It sounds to me as though your DH cannot stay regulated when his children are distressed.
When he becomes dysregulated, he sounds more able to get angry (and bash things) than sit in his distress (e.g. cry). That's too hard for him.
He is not able to think things through, seek help nor speak about all this when he is this dysregulated that he is hitting out at things.
When calm, is he able to speak about how hard he is finding things?
Is he able to consider the impact of his bashing on his children?
Is he only able to put his own emotional needs first? If so, is he willing to seek support? There may be a reason why he cannot (seem to) tolerate distress in others/his children, and perhaps not tolerate his own panic when he sees their distress.
When people become very dysregulated (like you describe) they will often do whatever stops them feeling so bad. At these times he is unlikely to be able to think about others' feelings. This is why he may not be able to look after the children on his own, if he tends to get angry and act in frightening ways.
However, when he is calm, he might be able to recognise the impact of his behaviour. He may be willing to seek professional help.
This is likely to be a huge step for him. Many people find it hugely shameful to need help with these kinds of feelings/behaviour. So exploring this with him will need care so that he doesn't feel shamed. That tends to make more angry feelings and shut people down to different options.
Whatever you decide based on all of this, right now, he probably is not able to look after children on his own. With professional help this could change.
This must be really tough for you. I hope the conversation goes well.