No, that's missing the point.
The reason why sports is an issue, is because men have an unfair advantage. There is a reason.
As there is for those other spaces. Men are bigger and stronger than women. They constitute the predator class. They constitute the violent class.
So in situations where women are vulnerable, or they can't escape, or they're naked, that's the reason for the segregation. The same with sports. Women are vulnerable to stronger and more powerful people in terms of winning.
Where's there's a reason, we segregate. Parking the car, buying food, walking in the park, - no segregation, because there's no reason to.
Many men understand sports, because they are completely familiar with the concept of physical strength in competition. And how it highlights advantage and therefore vulnerability. They have no trouble grasping the concept of bantamweight and heavyweight. They don't have to think twice. They know a heavyweight would pulverise a bantamweight.
But they do have to think twice when it comes to the advantage men have over women, because most decent men never exploit it. Or it's so ingrained, that it never crosses their mind what it's like to be the vulnerable one in the equation.
Hence the NAMALTing when you point it out.
Which should be the same as saying of course men can be in women's boxing, because let's face it, there are hardly any of them, most men aren't like that.
The point being, that those who are, will exploit it.