As ever, it comes down to whether you think men and women are defined by their sexed biology, their bodies, or whether they are defined by arbitrary, changeable, and superficial sex stereotypes.
RMW seems to think that it's the latter. RMW (when still at the part-time crossdresser stage) blogged about using the women's toilets when wearing a skirt and heels, but presumably didn't do this when wearing a man's suit.