I think you're missing the point. You're not alone, many people seem to be doing the same, so don't feel bad.
The joke was that nobody ever seems to think of the non-Jewish victims of the holocaust. There were many other people killed - travellers, criminals, homosexuals, the disabled. Jews of course were the main target and make up the vast majority of victims, but that's the point - the numbers were so huge, even those in "lesser" groups had large numbers of victims.
Carr was also pointing out the hypocrisy surrounding racism and how it is deemed more acceptable to be racist against some groups than against others. Travellers are more "legitimate" targets than POCs or Jews for example. As a society we can't address that issue unless we are aware of it.
People seem to forget that humour is subjective. There's no definition of what makes something funny. You might like Mrs Brown's Boys, I find it offensive (certainly more offensive than Jimmy Carr). Not all humour is a straightforward joke that is meant on a superficial and obvious level. Some humour needs an understanding of the subject in question, some humour is the presentation of the absurd as something everyday.
If you can only see humour as a straight up literal mocking of events, I feel sorry for you.
Aibu to think if it was the other way around and he said that it was a positive the Jewish people were killed nobody would dare say 'just don't watch it then if you font like it.'
He didn't say that though so it's irrelevant.
If you didn't see it, he joked that people don't talk about the positives of the Holocaust.
Thousands of people died!
"Thousands" of people?! Seriously, this just makes Jimmy Carr's point that people are ignorant of what actually happened in the holocaust. "Millions" might be a better word.