I debated on whether or not to join this conversation, as it wasn't clear whether you were looking only for people to agree with you, or to hear about their contrary lived experiences. I decided to contribute when I saw others expressing similar thoughts to the ones I had about this.
I know exactly where you're coming from on this topic @Sugarintheplum and agree with some of what @OhShutIt said.
The vast majority of my family and close friends are Black. I have one very close White friend, who I've known for over 30 years (I've talked about that friendship on BMN before), and we have a surface level relationship, which suits us both.
My main friendship circle is full of Black doctors, solicitors, bankers, head teachers and accountants. We own our homes, and are all earning very good salaries, the majority 6 figures, given our professions, and the length of time we've been working. We encourage our DC to work hard, but also to take pet in extra-curricular activities. If we were white, you'd call us middle class.
All of us are second generation, our parents came here from various parts of West Africa. Depending on where we grew up, some have Home Counties accents, some lean more toward slight South London. No-one is ridiculed for sounding white.
When not in corona times, we all take part in these so-called white pursuits, some of us ski, holiday in expensive places, go to the theatre, and National Trust properties, as well as being members of Black churches, attending Black parties and mentoring younger Black members of our communities to do well.
I don't have any problem discussing race, as a dark skinned person, the first thing anyone sees when I walk in a room is my skin colour. Secondly they see I'm a woman. These are defining characteristics of mine, and it doesn't bother me speaking about either one. Do you feel uncomfortable talking about being female (if you are) in the same way?
Due to my work (finance), I am regularly in spaces where I am the only Black person. It doesn't cause me any particular stress, though I can see people wondering what I am doing in these meetings until I start speaking, then they realise.
OP, it sounds like you would benefit from exploring why you choose to set yourself apart from Black people. Black people are all different, so I wouldn't expect everybody to be the same.
The fact is, no matter how many white friends, family or acquaintances you have, you will always be Black, and it sounds quite stressful to ignore that, or to pretend it doesn't make a difference. It reminds me of when people say they don't see colour. The casual racism which you don't feel able to challenge in your circles WILL be having an impact internally. You yourself have said being in that environment affects your mental health, which doesn't surprise me at all.