Although obviously not actively the ones in the vulnerable position, the vast majority of normal men are just as opposed as women to what the TRAs and other apologists are trying to achieve and really aren't in any way seeking to gain access women's toilets and changing rooms; and, believe it or not, most men do actually care about children's well-being and preventing them from abuse too.
So let's hear their voices then, stop standing back whilst women are abused, doxed and threatened on Social Media, in public, etc., etc. Where are their letters and challenges to politicians, councils, sports facilities, Girl Guides, et al?
I completely agree with you. Because men are not the ones in danger, they mostly don't comprehend the reality of what is happening, so many would be opposed to what the TRAs desire without necessarily fully understanding what their methods and ultimate goals are.
And whilst most would not want to go in women's toilets and changing rooms (let alone prisons), simply because they are NOT women and thus it would obviously be inappropriate, they often wouldn't have thought further than assuming it was women wanting to chat with female friends (probably about them) and re-apply makeup rather than the actual real threats of rape and other violence that women have very good reason to fear if adolescent/adult people with penises are given carte blanche to use facilities that used to be set aside for those with female bodies (and accompanied young boys).
I would love to see intense campaigns to encourage men to actively understand the dangers to women - including their mums, wives, sisters and daughters - of what we now see being worked at to become normalised and forcibly 'accepted'. I just have no idea what methods and messages would work effectively, though - and I'm angry that the current mainstream 'official' paradigm seems to be based around silencing women who are waking up rather than informing and galvanising men to stand alongside women against them - the ones with this clear agenda.