One way to disentangle it - the Asimov reference is very apt - would be to think not in terms of a person's identity, but rather in terms of what kind of puberty a person has undergone. This way, it is easy to keep born men out of vulnerable spaces without wading into theological discussions about language and law.
It works like this: if trans-women are women, they are still women who have been through male puberty. As such, the rules that apply to men regarding access to vulnerable spaces should apply on grounds of anatomical power alone - not identity. Case in point: nobody objects to little boys being in women's changing rooms, DV shelters, etc. Maleness is not the issue. Muscles and man-bits are the issue. Whether or not you think you're a woman.