You mean, when he was repeatedly raping you? Just on what you described could mean a prison sentence.
Shouting at you and putting you in fear - someone will be along with the correct terminology, but that is a form of assault. Refusing to allow you to leave a room, that's a flag that would be of interest to the local domestic abuse team, and I think there are criminal laws against that, but don't know their names. Then there's your cards - the womensaid people or police in the local domestic violence unit could probably make a loooong list of the right words, the statues, the clauses - but the biggy is the rape.
And having sex with someone either asleep or incapacitated for some reason is legally defined as rape, even within a marriage. That has been the legal position for several years. Having sex when you are unable to consent when you have previously stated that you do not want this - unequivocal rape. Your pastor needs to think how his conscience squares with that - and it is his failings, not yours.
Sorry if this sounds harsh. You are being told over and over again things by people who you should be able to trust and who should have your interests at heart that you are in the wrong. However lots and lots of people who are not part of your husband's small circle, and the law of the land, think you are right. The part of you that your husband is trying to silence knows that you are right. But I am not trying to underplay how desperately difficult it is to get past the words from people who say that they love you and are telling you that they are acting in your best interests. And the awful feeling that the only ones close to you that you have trusted in the past and you have had faith in are not acting in your best interests but their own. And that your pain and distress are less important to them than appearances.
I suppose if you want to try and use humour, you could always try imagining your pastor in frilly pink knickers and a bra. We are all human, we all make mistakes, and some of us fail at some point. Your pastor is not infallible, and I think does not understand what is going on. Normal people do not understand how someone like your husband can behave the way he does, they cope by minimising it, trying to believe that it isn't as awful, dreadful, viciously unpleasant as it is.
I've spent a lot of time writing this, with lots of praying, and I think you are worth it and a lot more. I am sure that I have cross posted with lots of other wise words. Please take some comfort - so many of us think you have far more value than you are allowing yourself.
I am praying for you - I hope my words are helpful.
Just about to press post, saw a bit about where you go - if it gets awful, call the police, have a mobile charged and handy and on you. After his performance this morning, if you rang 101 now, asked for the domestic abuse unit and asked for your address to be flagged so that they came quickly if necessary, they would not think you were over-reacting - it is that bad! If you are married you are legally entitled to a proportion of all property/savings/pension and don't let anyone talk you out of it. And you can go to somewhere like a refuge. Hope someone comes on with better advice soon. And sorry for the looooooong post - just hope some of it helps.