I do think the word privilege is unhelpful.
Those who have it, (eg white, male) but were disadvantaged,in other ways seem to be set on an immediate defensive when you try to discuss it.
I had greater success explaining male privilege to a chap,who was a gamer, by comparing, say his career path to,playing CoD on an easy setting and not only not realising that he was on the easy setting , but (crucially) not realising that Harder Settings even existed, let alone that many people were unwillingly playing those harder settings.
I could then make the real life comparison that no one had ever mistaken him for a secretary or receptionist instead of the manager. He hadn’t missed whole swathes of meeting by bing asked to make tea, or struggled to make contributions in technical Meetings because he was also having to take minutes. That was his ‘easy‘ setting. Yes, he had still worked hard, but there weren’t certain additional barriers for him that others did experience on a regular basis.
I’ve never tried to explain white privilege, (nor would i TBH) but I think it’s much the same. No matter how hard I have had to work, a PoC in the exact same situation would have had a whole other set of obstacles that would be utterly invisible to me.