Women weren't supposed to be pilots or engineers, or men primary school teachers or beauticians, because of their gender, not their physical sex.
In which case, if they had said their gender was male, presumably they would have been welcomed with open arms. But they weren't.
They still aren't. Transmen, i.e. women who identify as men, are famous for giving birth. Their gender, i.e. the tool used to oppress them, hasn't changed, any more than their sex has.
Women are oppressed on the basis of their sex, and gender is the means by which it's done.
I actually think being trans, or a gender nonconforming male draws in a certain kind of abuser, who see us either as sexually louche exotics or sort of subwomen to whom the normal rules dont apply. Much the same way some men view sex workers. So obviously there is a wide range of experiences and reasons for this, but I think my experience of male sexual violence and harassment is probably closer to someone born physically female then it is someone born physically male.
You've just said that the reason for you being a target is because you are viewed as a male, followed by 'and that's why it's more like being viewed as a female'.