I agree with minipie. The differences really are pretty minimal - I'll talk about the language studies FastLoris mentioned, because they're all I can offer an informed opinion on.
Women have more connections between the two hemispheres of the brain.
It's been claimed that the corpus callosum, the white matter that connects the two hemispheres, is larger in women on average after you adjust for brain size. This would be the only sexual dimorphism in the anatomy of the brain after you adjust for average size. People have concluded from this that there might be differences between men and women in how the hemispheres communicate. However, there have been a series of more recent studies, some using different measuring techniques, that find either no evidence for a sex difference or only a very weak effect. So the evidence is inconclusive at best right now.
Womens' processing of language is more evenly shared between the two hemispheres.
This was a 1995 paper that claimed it found lateralization differences (ie differences in which hemisphere is used for processing) between men and women on language processing tasks - more strongly lateralized in men. BUT - they only found a difference (and a fairly weak one at that) for one of the processing tasks they tested. In the other tasks, and in studies other people have done, no effects were found. It also wasn't clear to me that they had adequately controlled for the fact that tasks were presented visually, so you have the processing associated with reading in the mix as well; and, I think worryingly, nobody on the research team was a linguist. The lead researchers were paediatric neurologists, and they published in Nature, which is not exactly known unto linguists as a top-ranking specialist psycholinguistics journal ... Again, no conclusive evidence.
There is some interesting data about stroke victims and the like where certain parts of one hemisphere are damaged. I can't remember exactly which syndrome it was, but one of the ones where a particular aspect of language is lost but the other aspects survive. They found that men lost this capacity far more gravely than women, because the original capacity was much more focused in one hemisphere, whereas the women could just use the other hemisphere to compensate.
It's not clear that these studies were comparing like with like - there is a dispute over this result within the field because the size of the brain lesions may not have been the same in the male and female groups. And the papers also showed that women were affected more by damage to certain parts of the left hemisphere (left anterior frontal cortex) than men were. So it is not as simple as "Man brain heavily lateralized, Woman brain super duper multitasking in both hemispheres".
My guess is that if/when effects are found for language, they will be pretty minimal. And it's vair difficult to tease out the effects of socialization if you are working with older subjects.
It's important to remember when you read anything about sex differences (or, well, about anything really) that there is both a publication bias - people are less likely to publish research that demonstrates a null hypothesis - and a reporting bias - because media outlets tend to pick up only on those stories which reinforce stereotypes or which are !ZOMGWackyBrainScience stories.