It's not misogynistic to define the category of woman by their biology, just as it's not misandrist to define the category of man by their biology. Women are born with ovaries/uteruses, etc, men are born with penises and testicles, etc.
In a very small number of cases, you'll get people who are born with disorders that make them intersex, but that doesn't mean that humans are not a sexually dimorphic, sexually binary species - any more than people being born with limb differences or absences chances the fact we're a bipedal species with two arms, two legs, 10 fingers, and so on.
When I think, "What makes me a woman?", I literally can't answer it without resorting to biology. It's not how I dress, my hobbies or interests, or "a feeling". They make up what I am as a person, alongside my biology.
I'm a woman because I was born female, which brings with it expectations and problems that society has enforced on women over history.