Trans people aren't being 'caricatured as threatening' any more than men are 'caricatured as threatening' by the need for single sex spaces.
Trans people come in all varieties, just like everyone else, and some of them aren't very nice - like all the ones carrying violent signs while protesting a judgement of the highest court in the land.
My husband is lovely (and has long hair), but he doesn't insist on using the ladies.
Good men stay out so bad men stand out.