In a way, the fact that he won't 'let' you talk to anyone about it is an indication of the fact that he knows what he did is unforgivable; otherwise he'd expect people to forgive him, not ostracize him.
So he's kind of validating your feelings; he's telling you that in his head, what he did is worth ostracizing him for. And you're feeling like you're ostracizing him.
Is there anything else he won't 'let' you talk about? Why do you allow him to tell you what you're 'allowed' to do? I'm wondering about what was demonstrated to you as you were growing up, about relationships? Did your parents love, respect, listen to each other? Did they love, respect, and listen to you?
Accepting being controlled by a partner as an adult often comes from not being able to validate your own feelings. You feel like your feelings are not valid ('It's just me being silly/over reacting'), so you don't act on them. Then they stuck in your head with no outlet - like you say, it's like drowning in them.
Self validation comes from being taught to recognise and respond appropriately to your feelings throughout childhood. We're not born with this.
The concept of 'changing how you feel' doesn't make sense here, even if it was possible. He lied to you. He traumatised you. You know he lied to his ex wife. He won't allow you to have the outlet you badly need. He still lies now. And you're asking why you can't seem to settle with him.
Isn't it abundantly clear?
Why do you feel you have to stay with him? You don't.