OP YANBU, in my opinion. I actually don't think there is a right or a wrong here.
Your DH may not be able to help his need to cross dress, and should be free to do whatever he wants. BUT if you don't want to be a part of it, then I think he should respect that and he doesn't have any more right to make you unhappy by forcing this on you, than you have to force him to stop.
BTW- I don't think the OP forced him into suppressing this for the last 20 years. She stated the terms on which she could tolerate the secret he kept from her- he chose to stay on those terms. He could have refused. And it seems like the OP has tolerated it to some extent- going out so he could cross dress without her there, making love to him while he was wearing her underwear and so on. I don't think that's entirely "forcing him to hide this aspect of himself".
To those posters saying "well I don't have a problem with it"- that's fine, but the poster does feel uncomfortable with it and has said that she finds it a turn off. She can try and work through that, if she wants to. If she cannot overcome her feelings, then I don't think there is any shame in that.
I wouldn't be comfortable with this, and I would find it a complete turn off from a sexual point of view. I also would not like the fact that he wanted to go out dressed as a woman and be chatted up by men (even if it was only for the attention)- just as I wouldn't be happy if he was thrilled to be going out to bars and chatting up women "just to enjoy the flirting" while dressed as a man- even if he had no intention of taking it further.
OP, I think you need to have a careful think about what you can live with and what is a deal breaker for you. You need to ask your husband to do the same. Then you need to sit down and have a discussion with your husband regarding some common ground, if there is any, and where you go from here.