The key to all this is realising that women's sport is an artificial category and what that means in terms of sport. Sport itself is pretty artificial - all the rules are essentially made up. Why are football pitches a set length? Why exactly 11 players per team? Why is one person allowed to touch the ball with hand, and only in a particular area? Sport, almost by definition, is artificial.
There are lots of artificial categories we accept easily - we have a 400m when the fastest person is clearly the 100m champion, and have different weight categories is boxing for example. The boundaries can be fluid, but it is imperative that we accept they are artificial and recognise that the question is about there to draw the (artificial) line. Comparisons to Usain Bolt having faster reactions don't help at all - because we don't have separate categories for those with faster reaction times. We are dealing with one, single line in the sand - bringing in others just muddies the waters.
If we agree that having an artificial category of "women's sport" is important (and I personally think it is) the question is then simple - who counts as a woman when we are talking about physical competition? Hearing individual stories is really difficult, but it isn't possible to please everyone. It is pretty utilitarian, but I do believe in the greatest good for the greatest number in this instance. IMO, it should be simply genetic. Women's sport is, very simply, for those who have XX chromosomes. Anyone else shouldn't be included. It is simple, clear and makes no moral judgement. But I do feel this is something that is becoming increasingly muddled. It is all part of a wider (and far more complex) question though - what is a woman?
In addition, I don't trust the world sporting bodies so "IAAF say it's okay" doesn't wash with me. I am a huge fan of sport, but the way they deal with drug cheats (Justin Gatlin for example) mean that I simply don't trust their judgement.