Yeah... I went through that, "Worthless". I'm not sure about yours, but mine could be very sweet sometimes and I'm fairly convinced could have been a nice person if he hadn't been warped by a total arse of a father. Well, sometimes I am, other times I'm just angry. In the end, though, I had to say to myself that however sorry I was for the person he might have been, I just could not live with the person he had become.
He cannot confront his issues now, far too late and he doesn't have the emotional strength; he hides behind them instead. His choice, I guess, but he does not have the right to drag five other people down with him. The thing is, those issues will be with him wherever he is and whoever he's with. He needs ongoing, heavy-duty therapy (which he will never agree to), not a wife. I couldn't make him better, and believe me, I spent well over 20 years trying. In the end I had to give up because I was slowly going mad, which is not a nice feeling.
That's the thing that shakes your sanity: cognitive dissonance. It's the difference between what you believe and the evidence before your eyes. It's a really strange feeling when you suddenly let go of the things you've been trying to make yourself accept for decades, and re-engage with reality as it is actually happening. Thus, when he says you are cold, unloving, the house is a tip, you haven't had sex with him in months etc, first you stop trying to see the state of the house with his eyes (one cardigan on the refrigerator doth not a shit-hole make), keep a diary that reminds you actually you did have sex three times last week and that you tried to put your arms around him yesterday but he shrugged you off (it feels a bit horrid, the diary, like spying, but I have a dreadful memory). Then you start to wonder whether he really is seeing things differently from you, or whether he is simply... hold onto your seat here... lying.
Finally you say it doesn't matter what he believes, basically it's shit and I won't put up with it.