Well, considering the people who don't have a clear-cut biological sex.
There are degrees to which this can happen, so no clear definite '3rd' group which can be proven to be neither male or female, so where would you draw the line for that? It is rare, but it does exist. Why DO we need to declare 'boy or girl' the moment a baby is born? Why does that even need to be recoded anywhere? I'd be pretty upset if people insisted on keeping a record of my sexuality for govt files. Why do they need to know what I keep in my pants?
Also, a larger group of people would fit into 'transgendered'. Again, why do they need to be categorised? Does it matter if they have female reproductive systems but 'fit' better into a more traditional male role. Again, why do we even need to know whether they're m/f to accept them into certain roles? Why do passports need as to been assigned one or the other?
Then there's the huge number of people who are really a mix of m/f interests etc. Hormone levels during pregnancy are influential in this, as well as environmental factors. Nowadays we accept the 'tomboy' and the 'metrosexual' type of person, but that isn't true of everywhere. So if we insist in saying that we are defined to be male or female, then we are making implications about people who show signs of living outside of those definitions.
To really get beyond the idea that our bodies define our roles, we should perhaps stop trying to define our bodies in such absolute terms.
In the context of my work, we were discussing the groups that students sit in at lunch time. By having some fairly rigid patterns of behaviour, some people do feel isolated. There tends to be 2 tables of just boys, 3 tables of just girls, and one co-ed table, which is the 'indie' type people. So, a new girl joined the group. She is v feminine & pretty. She gets on really well with boys as friends. She's not really into the indie scene. Where should she sit? According to the mores of the group, she should join one of the girl tables, but she will find the conversation at the boy tables more to her style, and is likely to be welcomed most easily at the indie table.
The simple fact that we place so much emphasis on m/f differences can cause isolation & polarisation. Ideally, I would like to see us stop having definitions for what is feminine & masculine, but just to see people as people without having to give them a label according to their bodies. (Just as I would relate an anecdote about someone without feeling the need to state their race, I shouldn't feel the need to state someone's sex)