OP, you are correct that it is not a term that is widely used in the UK. I think, perhaps, because it came about because of America's history of oppressing people who were not white. Racism was enshrined in US law to disenfranchise non white groups:
Slavery,
The Alien and Sedition Acts,
The Chinese Exclusions Act,
1942 Internment of Japanese Americans,
Jim Crow laws,
The Indian Removal Act,
Ban of interracial marriage,
Policies banning anyone other than white people and african americans becoming citizens of the US,
Operation Wetback,
The Day Law...
The list is depressingly long. These laws affected Native Indians, Japanese, Indians, Chinese, Mexicans as well as black people. So in American culture the term 'People of Color' reflects the shared struggle against the 'white oppressor'. Conveying solidarity in three words.
Bell Hooks wrote used the term in her book Killing Rage: Ending Racism
As more people of color raise our consciousness and refuse to be pitted against one another, the forces of neo-colonial white supremacist domination must work harder to divide and conquer.