If you've changed your primary and secondary sex characteristics, you have changed your actual biological sex.
Male Primary Features: Penis, scrotum, and testes.
Female Primary Features: Vagina, uterus, ovaries, and cervix.
So if you haven't changed them, you haven't changed your sex?