We have had this argument when DS was 5 months old
I went to a school in a shite area (and still live in said shite area) and true alot of my fellow pupils are now wasters however myself and my circle of friends all have good jobs and got good GCSEs and if I say so myself turned out alrght. My DP lived and went to school in a very well-to-do area and often looked down on me and my family / friends because of where we grew up. Most of his friends are unemployed, have no formal qualifications and still live at home at the age of 30 because they have no means of moving out.
I always point this fact out when he starts banging on about what a crummy area we live in and what crud schools there are around here.
Personally I believe - whats that saying I heard a while ago - a rose will bloom wherever (is it that?).
Sorry for the rant / hijack about me,me,me, I guess its been on my mind for a while