If you’re not black yourself, you don’t have any right to tell someone what’s racist or not because you’re not at the receiving end of it.
Not entirely true although your later comment was more accurate in that if you are on the receiving end of a racist tirade and you report it to the police they definitely should be handling it as racist unless it is unequivocally proved otherwise. And that is how it should be.
The problem is, as many PP have said, context can really change things. Which is how a lot of clever racists get away with saying appallingly racist things in the open and wiggle out of being found guilty of racism.
Sure, if i knew you, i would definitely refer to you by name. If i had to describe you, in a group of people where you were the only black person, i may (not always) say "sure, the black lady over there". I am, because i do try to be careful, more likely to say "the lady with the red dress/green cardigan/standing by the water cooler".
My granny might have once upon a time said "i was talking to a black lady at the bus stop" - which is unnecessary and racist. But the context of that might be that my granny came from a tiny northern village and didn't meet a black person in the flesh until the late 60s. When she learned that was offensive, she didn't say it again.
And I often get called English (tbh I am English but i take on board complaints of non-English brits and try to educate my friends/colleagues that British =/= English. That is an aside sorry) when someone has to describe me quickly. In a company of over 2,000 people I'm the only English one. It is pretty obviously a very easy to describe me.
"no, where are you really from" - yeah. That is a massive problem. If you genuinely want to know where the name is from, you could ask that (I have a non-English surname, being married to a non-English person and i changed my name on marriage). So instead of "no where are you really from" it would be better to say "interesting name, where's it from". But even then, saying that to a black person who has said they're from Yorkshire is very problematic and Othering.
And so on and so on. I don't think it is always disrespectful to refer to someone as black under certain circumstances, but if they subsequently came to me and said they didn't like that, then if that type of occasion arose again I'd find a different way to refer to them. I see POC and WOC used quite often and it seems to be a respectful way to describe people if there is a reason to use it - but again if they asked me not to, i wouldn't.
I've often been the white person in China, Korea and Japan. One colleague once called me and my colleague "the long nose" as that is how he had always called white people (it's quite common) i don't really care, but my Jewish colleague immediately asked him not to use that expression again. And so on.
the TL;DR is that yes, context is key, but racists use that as an excuse sometimes. We should all remain respectful and take it on board when someone says "don't use that word for me"