Why do you think that the majority of the TRA's focus is on trans women rather than trans men?
The TRA's aren't telling men that they need to step aside and allow trans men into their toilets and changing rooms. They aren't telling gay men that they must have sex with trans men otherwise they are transphobic bigots. They aren't telling men to STFU, move aside and give up their rights so that trans men can feel validated. They aren't encouraging trans men to redefine what it means to be a man or tell men what being a man is really about.
Of course they aren't. All of the focus is on allowing trans women to force their way into female spaces, no matter how vulnerable the women in those spaces may be and no matter how triggering/upsetting/distressing these women may find having a biological man in there with them. These women simply need to shut up and deal with it because a trans women's feelings are far more important than a woman's right to feel safe/be safe.
Sports women are being expected to move on and compete with trans women even if in some cases that is bloody dangerous for the women involved and end up getting seriously injured as a result of competing against someone with all the physical advantages of being male. But who cares about that? Just as long as the trans womens feelings are validated, that's the most important part.
IMO if these so called trans activists really give a shit about transgendered people they would be campaigning for more gender neutral/unisex facilities. They would be campaigning for trans gendered people to have their own sports categories, their own prisons and their own support services. They would be discussing what could be done if trans gendered people were unable to have their own spaces and discussing how we could decided what to do on a case by case basis rather than making a blanket policy of just letting trans women in female spaces.
There is a poster on here who works for the NHS. They have a new policy which states that trans women, or men identifying as women, must be placed on female hospital wards. This new policy states it doesn't matter how long they have been "living as a woman" (whatever that means), they must still be placed on the female wards. So Dave could decide he is going to be Davina that day and be placed with on the women's ward.
This new policy also states that if any woman objects to a trans woman, or a man identifying as a woman, being on the same ward as her (no matter how valid her reasons for objection may be) then she will be firmly told that her views are not tolerated. So STFU basically. If you're a woman who has been raped or assaulted and are finding being on the same ward as a biological male upsetting or distressing or triggering then it's tough shit basically. Either STFU or GTFO. There was no such policy change for trans men or women identifying as men to be allowed on men's wards.
There is another poster here whose workplace has turned the women's toilets into gender inclusive toilets. That may sound fine at first glance but they didn't adjust the men's toilets accordingly to match. So they now have men's toilets and gender inclusive toilets 
This is why we are pissed off. We don't hate trans people. We want them to be happy, to be accepted, allowed dignity and rights. We just don't agree with allowing them to stamp all over women's hard fought for rights.
Being concerned about women's rights and safety is not bigoted or transphobic.
Questioning why it only seems to be women who are expected to move aside and let trans women into their spaces whereas men aren't expected to do the same thing for trans men is not transphobic.