They don't usually pass; you're right. But that's the societal change that the gender-affirming movement asks for: that trans people are treated as the gender they see themselves as.
Adopting stereotypes doesn't make someone male or female, but my argument is that if a male 'passes' as female, then he is a woman; if society applies the stereotypes associated with females to you, then you are a woman. Whether you are a transgender woman or not is based on whether you are actually female or not.
You're right that we should move past these stereotypes, but they're here to stay at the moment. Throughout history and across many different cultures, various stereotypes - various roles, expectations, and norms - have been applied to men and women. Although I am a gender abolitionist - I agree with you that we all have 'individual interests, preferences and personalities' - I doubt these gendered stereotypes will just fade away.
Just to clarify, my point is not that we use stereotypes for a determination of gender, but for a determination of sex instead. What set of stereotypes we then apply to people reflects their gender.
For example, we use a woman's short height, feminine face, and dress to determine that she is female. Given we determine her to be female, we apply feminine stereotypes (stereotypes of women) to her. This means she is a woman. However, it could really be that 'society' was wrong, and that this woman is male.