No, YANBU. Sexual attraction isn't a tap that we can turn on and off.
Presumably, you married your partner because you liked his personality and found him physically attractive - and presumably again, the latter attraction rested somewhat to wholly because of maleness and masculinity.
The situation has now changed - after 20 years of marriage - and you are well within your rights not to find a middle aged man, wearing Anne Summers underwear, high heels and lipstick, attractive.
(May I interject here: I had a lover for several years who occasionally wore women's underwear in a kind of amusing way and he and I found it super hot - particularly because he was a from a hypermasculine cultural background, so I guess there was a taboo element. HOWEVER, had he turned around and told me he wanted to wear women's clothes and make up all the time, etc, etc, this would be a totally different thing)
You are not being unreasonable. You are being gaslighted and emotionally manipulated and this man desperately wants your validation. This isn't a private kink, or an health issue, or a time-limited ask for support. He is asking you to redefine your relationship, put your feelings to the back, hold his hand and brave the storm by being public, and maybe later down the line spend your joint finances on surgery and hormones.
And then guilting you for not being supportive enough.
Check out the transwidow thread on this site, fix your finances, get a divorce. And do not feel shamed into having sex with him.