I'm haven't been the child in this situation.
As a mother I have prevented my ex from having access to my children. He has not fought me on it, or tried to take me to Court. He could, and would probably get some sort of access, but he has not bothered. As he is now a registered sex offender and paedophile, I'm happy with this approach and as my children get older I can explain I have done what I thought was in their best interests to keep them safe - hopefully they will understand. I can see similar parents taking a similar position where there has been violence etc...
However, I have had to deal with the aftermath of this with young children feeling abandoned by their father - they are still young so know some limited details but they will obviously have to be told more details as they get older. They are hurt and angry even now. They blame themselves - they feel that if they were better he would not have done what he did and would see them. They are angry he chose to do what he did rather than fight for them.
I cannot see them wanting any form of relationship with him - there maybe some curiosity, and they may decide they want to meet him when they are older, but I suspect not. It is such a complicated and devasting trauma for young children to be abandoned.
Taking away the good reason I feel I have, the abandonment trauma would still be there. And I suspect both children would blame both parents - however the resident parent would be the only parent figure they know and there would likely be such a strong bond there that the absent parent would get the majority of the blame for not fighting. And to be honest, I cannot see a reason why a parent would not fight tooth and nail to try and get access.
I have seen this both personally and professionally from the other side too, and know fathers that have taken mothers to court when they have been denied access for no reason at all. Contact is usually ordered in the end. Yes, its stressful, expensive and a truly awful, intrusive process - but there is nothing I wouldn't do for my children.
OP - you are getting a lot of people focusing on why your husband doesn't fight for access. You claim he has his reasons for why he doesn't, although I get the impression even you feel they are weak. Based on my experience, I'd say the children will blame your DH for not fighting for them. Because lets face it, what good reason could he possibly have that the children could understand/agree with ? It may well be that the mother is a "psycho" - but surely then it is even more important he formalises some sort of contact order and enforces it, in order to safeguard the children from her and protect his own relationship with them.