I'm not sure I agree. Receiving an invite to attend a conference aimed at a particular group when you don't fulfil all the criteria is one thing, and in that case yes passing the invite on would be the right thing to do.
But being invited to be a keynote speaker is a different thing - presumably there is a reason they want you, something you do or have done that they feel will be of interest to the attendees? Unless you are being invited to talk about being from a minority ethnic group then why would the fact that you are not automatically mean you shouldn't speak? Presumably they know that you are not from a minority ethnic group? I don't think it's necessarily right to think that to be a relevant and interesting speaker for a particular group of people you have to belong to that group.
Of course it depends on what you will be speaking about and what the conference theme or subject is as well.
You could always ring the organiser and mention that you are not from a minority ethnic group and weren't sure whether they were looking for someone who in fact was, in which case you have a colleague who might be interested.