We should remember all the victims, however the holocaust was specifically designed to annihilate Jews.
And people, EDI initiatives etc too often focus on the other victims as a way of saying 'See, it wasn't antisemitism, Hitler just hated everyone who wasn't Aryan'. Which is then starting to border on holocaust minimisation....
All the groups of victims should be remembered in their own right.