As a child, i really disliked my father. I don't know when it began, i have no memories of liking him and then not; i always did and it was common knowledge in my extended family that this was the case. There were reasons. I said similar things to my mother- leave him; but she didn't ( where would she go? How would we live was the response).
For a while i felt sorry for her, trapped, overworked (someone had to pay the bills), living with a selfish twat, having to live with the fallout of his behaviour constantly. But, still she stayed. I left home as soon as possible to get away.
I have a tenuous relationship with my father as an adult, my relationship with my mother is not as close either because i cant get over that she chose to stay with him even when children were all adults and left. She has basically tacitly approved his behaviour and still is-i cant get past that.
What i wanted to say is, listen to your children- why do they feel the way they do? What is your husband really like- through a child's eyes? Has he always been this way, have the children said this for a long, or is it recent?
It is very unusual for children to say these things and mean it, so if they mean it you need to get to the bottom of it.
this will stay with your children into adulthood- the memories of him and his behaviour in their childhood , and the memory of you and the choices you made. And it is likely to affect their adult relationship with him and with you , for better or for worse.