You are not wrong that sex and gender are separate things.
Sex is your biological characteristics - your chromosomes, sex organs, hormones etc. Most people have either male or female sex characteristics, with some exceptions (for example, intersex people).
Gender is about whether you identify as male or female, or neither, or somewhere in between, or in a fluid way. It is the personal sense of one’s own gender, a deeply held and innate sense. For the majority of people this corresponds with their sex. For others, it doesn’t.
Gender and sex aren’t the same thing, which is why many people support the belief that trans men are men, and trans women are women. It’s because we accept that you don’t have to have female sex characteristics to be a woman, or male sex characteristics to be a man, because while sex is biological, gender isn’t.
Segregating by sex (for things like sport and prisons) rather than gender has pros and cons. The pros are that we are familiar with it, it is largely staightforward provided you ignore the exceptions, there is a lot of historical context behind it.
The cons are that it ignores intersex people, or forces them into a binary which isn’t reflective of their experience, and it is exclusionary and often dangerous to trans people. A trans woman in a male prison will face a level of threat that isn’t acceptable. A trans person could be unfairly advantaged or disadvantaged in a sporting event. These are just two examples.
There are cons to segregating by gender too - some people fear that men will take advantage of this to invade women’s spaces. These fears aren’t unfounded, but there are usually solutions which can be found while still accepting the right of trans people to live completely as the gender they are, rather than the sex.