This isn't the problem. It might be a problem but it's dwarfed by the real problem. The real problem is people who are not marginalised (in the lingo these would be called people who have privilege) weaponising identity politics to damage those who they perceive as the enemy, who can be anyone, privileged, marginalised, whatever.
Both of these things are part of why the whole concept is problematic though. There is no awareness that human nature, people who are privileged or oppressed or somewhere in between, will use any kind of hierarchical social structure to advantage. So where we have those structures they need to be balanced in a way that protects against that.
That's not to say it is always or usually easy to achieve that, arguably it's the ongoing and never-ending project of political history. But if you actually believe at the outset that some groups of people are too pure or good or powerless and so there is no need, I can guarantee that people will take advantage of that at some point.
This kind of identity essentialism undermines the idea of universal humanity and entrenches whatever differences of identity that are named and validated as vectors for power. It's inevitable they will function that way, and also tend to encourage the creation of more categories.