I grew up in an area of outer london which has seen a dramatic demographich change in recent years. I can't speak for anyone else, but i can list some of the reasons i decided to move away:
Lots of my old schoolfriends & many family members had already left by the time i had dc1, so no old ties.
House prices. OMG.
Pace of life.
Feeling a lack of shared history/ common ground with those around me.
Being a new parent is tough: i needed support & for whatever reason, that sense of community & cameraderie just wasn't there. It's not about race & it's not about religion, necessarily. Growing up in that area, i had sikh, hindu, jewish, christian & buddhist friends whose parents had arrived in London from all over the world. In all honesty, I never became particularly close with anyone who was muslim (not for want of trying on my part!) but that's not proof of anything. It's probably more about the fact the immigrant families i was close to were settled & long established - they weren't having to deal with the immediate aftermath of arriving in a strange country.