People have traits, characteristics, things that make up their personality, things that they enjoy.
Our society takes these things and more or less divides them along gender lines - "men are like this and women are like that". Both the positive and the negative.
So women get things like caring, nurturing, interested in clothes, liking daytime telly and things like bitchy, vain.
Men get things like leadership, problem-solving, liking football and cars and on the downside things like aggression, inability to express emotion.
In reality it may be the case that men fit one of these gender roles better and women the other. However most people have traits from both and more people than generally thought have a majority of traits from the opposite gender.
So if a male child is castrated and raised as a girl, but they fulfil a quite "masculine" pattern of how they are, then they will feel a "wrongness".
That's how I see it anyway.