The train of thought seemed to be if you criticise the far right it means you don't believe that far right women should have access to single-sex spaces.
That's not the "train of thought". It means that many women see this issue as a cross political, single issue that they will campaign for with people they do not otherwise agree with. Some examples of women I have amplified on this issue who I strongly disagree with on some issues: KJK, Riley Gaines, Caroline Farrow. And I support them when they are harassed and attacked because they know what a woman is.
I think a lot of left wing people are anti semitic and/or handwave it away, but I will still amplify them on this specific issue, because their other views aren't relevant to me when campaigning on this issue.