Hmmm I'm on the fence.
I don't think any joke about the Holocaust or genocide is good, no matter how "clever", it's just a no - go for me.
But, from what I'm seeing, travellers come from many different ethnic backgrounds. So if a comedian takes a pop at a travellor for their behaviour, I'm not sure I'd think that was racist, as they could be Irish, Scottish, northern India etc etc. It's not about their actual race, so I guess it would be more like a hate crime? But I'm not sure, I think being a travellor still seems fair game for a joke, if written well, if it was about behaviour.
But they are a recognised race according to law so it's a moot point I guess.