Does the prohibition on association with the wrong political parties only go one way? Is it only valid for us?
Do I have to disagree that grass is green, the earth is round and water wet because some truly evil people also think that?
Away with your bigoted nonsense (yes, bigoted means intolerant of opposing, of even just alternative views). I'm perfectly happy to accept you disagree with me and that some shockingly vile people share your views without deciding that makes you a vile person, too.
We have people from across the political spectrum arguing for and against self-declaration of legal sex and for and against the rights of women and girls to have single-sex provisions and to organise, politically and otherwise, outwith the presence of males.
Women's rights throughout history have been opposed by men from all parties, whatever side of the political divide they fell on. That's because women's rights are not party political but universal human rights. Which in a patriarchy are, of course, opposed by the male sex class. And men can have all kinds of political views while still agreeing that women should stop going on about equality. Of course, each party opposes on different grounds and different issues, but the left-wing male is no less misogynist than his right-wing counterpart. I've met plenty of them, after all.