We’re all on a spectrum right? Some more one way that others.
Why would all being on a sex/gender spectrum, even if we were/are (I agree gender is a spectrum), mean there is no need for categories or segregation in particular circumstances?
Age is a continuous spectrum, and our age changes, but nonetheless we have age categories, and they are arbitrary, but they are still important. There are children's hospital wards and children's sports, and age limits on buying alcohol and going to nightclubs, and driving, and having sex, and so on. Because there are some situations where categories and differences matter.
The same applies to sex. (Not gender, sex.) The sexes aren't normally segregated for everyday activities, but they are for specific situations where being female means you are vulnerable to or different from men. Like prisons, hospital wards and sports.
Anyone can do any gender stuff they like (masculine/feminine presentation, interests etc) and they do and have done for ages. That's not the same thing as sex, which like age is a physical reality.
If you think it's so regressive to have sex categories in some situations, does the same go for age categories? No kids/youth categories for sports? No separation from adults in terms of sex, driving, voting, hospital wards etc? If not why not? It's a spectrum.
This ridiculous idea that if you can argue something is a "spectrum", then anything goes, falls down at the first hurdle.