Yes, it is one of the single biggest drawbacks. I remember my parents being very against any suggestion I went to the local cafe, where the cool kids mixed with the local boys school. I think there were terrified all their scraped together school fees would be wasted if I discovered boys, teenage romance, and started neglecting my studies. Or worse (in their eyes), teen pregnancy.
All it did was create a 'forbidden fruit' appeal. Lots of girls would go a bit mad and snog lots of boys at any parties we did get to go to, partly because you then wouldn't have the shame of seeing them in class the next day. My friends and I eventually turned our attentions to deeply unsuitable older men, random one night stands etc. We were all intelligent girls and not doing anything dangerous or stupid, but we were naturally curious and didn't have the opportunities to discover what boys - of our own age - we liked and didn't like.
It was pretty clear that the local boys school suffered too, and the poor boys didn't have the option of getting all dolled up aged 17 and going to a nice wine bar for some flirting, which my girlfriends and I could!
On the other hand, it did remove a lot of the peer pressure to have a boyfriend, and there wasnt' the distraction factor in class. I think I actually made more male friends than female at uni, so it evened out pretty quickly.
Personally I would love to send my kids to single sex schools up until 6th form, by which point I think mixing is as much of a life skill to develop before leaving school as driving or learning to budget. We will, however - and my DH feels the same having been in single-sex boarding for 11 years! - definitely encourage our children to have a social life, ideally with people we can get to know ourselves, so they don't run off and stick their tongues down the throat of the first willing stranger!