If the Sri Lankan company had decided they wanted more white people to reflect the ethnic make up of their customers, and set up an event to increase white representation, then no it would not be racist for the invited speaker to talk about how it felt to be the only white person in the room.
If randomly apropos of nothing they started complaining that there weren't enough white people in their office, then yes it would be.
You must see those scenarios aren't comparable though?