I'm multi-ethnic and from a multi-heritage background, and it's non-Commonwealth. I read as white, but my euroasian features mean I've been asked where I am from with the "no, really" tag since I was about 13 -- and interestingly, the people who most tended to ask were from BAME communities.
Looking back as an adult, there was evidence of 'wariness' even when I was very young. One of my primary teachers referred to me as the "olive-skinned girl" (I'm not olive skinned, btw. Just kinda yellowy.
)
Does it bother me? Nah, not really. I'll always be an outsider, and that position is both a boon and a curse. I prefer to focus on the boon aspect of it. I can be nominally read as a native across Europe, North Africa, the Middle East and Central Asia. If someone assumes that I am something I am not, it doesn't bother me because the only people who could possibly guess my ethnicity accurately would have to be specialists in a very niche area of history.
In a literary sense, my identity is very Borgesian; I'm a kind of mythic beast from a place that doesn't exist and a culture that is lost to time. 
What does bother me though is that there is no recognition that people have these complex identities that very much challenge simplistic political, social and cultural narratives about race, ethnicity, culture and belonging. I guess my background enables me to see the complexity of other people's identities in this regard, and I reckon it is that insight that allows me to understand other people's perspectives.
So when one of my white working class neighbours mentions feeling a bit bothered because they had an experience where they felt they were suddenly in another country, I get that because I've experienced that sense of dislocation my entire life. When someone from a BAME community tells me they had an experience that they think might have been influenced by racial or cultural prejudices, I get that because I've had those suspicions throughout my life as well.
But when I muse about identity and culture, sometimes I just think: "well, none of this shit gets carrots peeled, does it?"
That, I reckon, is the line. Concepts of identity and culture have to be useful, they have to deliver material benefit and value; otherwise, it's all just esoteric navel-gazing, init?
And that is the point that I think we, in the Western world, may have breached in the last decade. It's identity for identity's sake in some sections of society.