Age remains verifiable, regardless of whether you feel like an adult or not. As does sex, regardless of whether you feel like a man or a woman. So you base your legislation around the verifiable concepts, rather than those that are more nebulous. That's the whole issue here, isn't it?
I don't think we disagree either, I see what you're saying. If we take the starting point of, "putting aside the GRA for one second, the way certain concepts are written in law right now is mostly fine", I guess the GRA then looks like it has the power to subvert those verifiable concepts and replace them with a subjective one - i.e. legal sex change in line with one's gender.
I can see why that looks like unverifiable concepts then being enshrined in law.
There are reasons why I think that isn't a huge problem; mostly because the impact of that change was considered when introducing the GRA, and exceptions written into it where knowing someone's biological sex would be critical. So essentially you can change for legal purposes EXCEPT for sport, medical treatment, residential care etc., because in those cases it would make a difference. Seems like it's been thought about to me.
Secondly there's the fact that this is very rare, and I don't think for a second self-ID would see half the population changing legal sex. The state and national law can cope just fine with a limited number of known exceptions IMO. I'm confident a digression from the purity of "measurable concepts" on rare occasions is totally manageable, especially when doing so brings big personal benefits to a community.
That's just it really - confidence. I'm confident we're all smart enough and sensible enough to manage a little flexibility without finding ourselves completely incapable of protecting women.