Years ago (15 years at least) I was doing what was then known as Biological Psychology degree (now known as Neuro Psychology) and there certainly was some research that androgen exposure in the womb does affect sexual behaviour.
So female rats exposed to androgens in the womb will display lordosis (humping) if they have a lot of testosterone and the development of their genitals can be adversely affected - it permanently masculinises the females.
Not sure this is the exact research but something similar here www.ncbi.nlm.nih.gov/pubmed/11752687
Notice that any changes in the male rats (smaller body weight) were eliminated by the time they were weaned.
This is an example of the type of work that has been going on for years around hormonal exposure in the womb and it's effect on sexuality and 'masculine' and 'feminine' behaviour.
www.ncbi.nlm.nih.gov/pmc/articles/PMC4350266/
I guess where I come from with this is that androgens do seem to make males more violent/aggressive ('masculine') and therefore that is classed as male behaviour, so lower levels of testosterone may well make a male less 'masculine' in comparison to the average female. BUT that doesn't make them female but just lower down the sliding scale of masculinity, and so it goes right back to the gender roles that are a powerful force in this.
Yes, a male may have had lower testosterone and therefore naturally is less 'masculine,' so he may feel like he does not fit into the strict societal male roles and develop gender dysphoria as a result, but he is not a female. So this is a societal and cultural issue rather than anything else - we have become more pink and blue as capitalism has marched on. In other areas of the world, e.g. Iran, it is religion setting the strict roles for men and women, and they too have high levels of trans people.
I understand TRAs say they feel like a women, but what I hear is you don't feel like a man.