IMO, husbands (and wives, but this thread is specifically about husbands cheating) need to take responsibility for making their own marriages work, and keeping their own marriage vows and generally being decent people, and not allowing outside forces to be blamed when they deliberately choose to do otherwise.
I find it hard to believe that there are many (if any) scenarios where a woman goes out of her way to pick off a vulnerable man and rip him from his blissful family idyll. But this seems to be what the classic 'home-wrecker' stereotype looks like in may people's imaginations.
My dad had an affair once and when it came to light, when I was 15, a lot of people said really awful things about the OW, about her being a manipulative nasty bitch getting her 'claws' into my dad and not giving a shit about anyone else. Even age 15, I wondered why my dad seemed to get off so lightly while all the anger was directed at the OW (he stayed with my mum). After all, he was the one betraying his wife and family, breaking his marriage vows and telling lies in order to be with her.
Personally, I feel much more judgemental of the person in the relationship who is cheating, rather than the single person they are using to satisfy their urges. I do think there is a bit of responsibility on the OW's part not to get involved with a married man, but it should be more like 10%, with the rest directed at the husband - the person actually doing the lying and cheating.
(OP, I know you've said that in your case the anger is/was directed at your ex-husband, so this isn't directed at you - just quite shocked at some of the responses. Some women even suggesting that they felt guilty after finding out he was married, even though they didn't know* at the start. WTF??)