People need to stop labelling everything marginally to the right of Pol Pot 'alt-right' it has become a meaningless term. If you label everything you disagree with 'extremist' then when real extremists show up no one will pay you any mind. You will become the boy who cried wolf.*
Yes, it just reinforces the idea that things which fall under certain labels are inarguable by virtue of the fact that somehow they became associated with that faction. Given that trans ideology got as far as it has because many people have made that very assumption, that it must be progressive and good because it is associated with the left, without thinking it through, we should all be suspicious of that thinking.
And really, the idea that conservatives, or the alt-right which people seem to think are the same, are all just hoping to enforce gender stereotypes and control women's bodies just for the sake of it, is so reductive. It shows the problem with this approach it prevents real engagement and understanding of why people take particular viewpoints.