The point remains, if your entire rebuttal consists of 'your argument is invalid because you are white/a man', then you don't actually have a rebuttal.
I wouldn't go so far as to render someone's entire argument invalid on the basis of their race or sex, but [white man's] experience of the patriarchy / racism won't be the same as someone who has lived experiences of either / both. So your argument automatically carries less credibility.
I can't think of anything more cringe-worthy than trying to tell a black person that my thoughts on racism carry equal weight as theirs. But we're all different.
I agree that some people do seem to struggle with the bigger picture / landscape / strategic view. You see it on here all the time.
'If you don't like Page 3, just don't by The Sun'.
'If you don't agree with strip clubs / prostitution, don't use them'.
'I shave my legs because I want to, it has absolutely nothing to do with socialisation (or, more likely, 'what other people think').
'I'm proud to be Mrs Husband's Surname - it's my choice and that's what feminism is all about'.
'Britain's has two female (inevitably 'female'!) Prime Ministers, you have equality!'
And yes, I'm sure stripping for men is super empowering for [insert random woman] but it doesn't move women forward at all.