I haven't moved the goalposts. My view is based on the principle of freedom of speech. He hasn't broken the law, incited criminality, or proposed these things as policy. He made a joke. Okay, some people were highly offended and fair enough. But that's the problem with freedom of speech. People are free to say things that others find offensive. Even things they find absolutely horrendous or revolting. Unfortunately, that is the price we have to pay in return for our own freedom of speech.
To me, freedom is a more important and significant value than offence. Someone could be offended by the things you say - should they have the right to shut you down? Should you be at risk of losing your job because you make a joke about vegans or something else you personally think is funny, but others disagree? I would say no you shouldn't, and I'd defend your right to make that joke, even if I thought it was unfunny, offensive, inappropriate, rude, or ignorant.
Alternatively, do we say that no one should be allowed to say anything that another person finds offensive? Where would you draw the line? Would you apply this to kids? To under-25s? Should people be allowed to say offensive things in their own homes, but not in public places? Should penalties for offending someone stop at being sacked, or should it become a criminal offence? Is it only some people's offence that matters - should you be allowed to offend men but not women, for example? Tories but not labour voters? Should artists be allowed to make art that offends people? What about the people who are not offended? Do their opinions matter?
What I'm trying to get at is that once you start policing speech and limiting freedom of expression, you risk every kind of speech, for every person. We've already seen some of these examples in real life, e.g. with bills limiting speech within the home, or censoring dramatists and performers in Scotland.
The problem is that, imo, freedom of speech has to be absolute. Either everyone has that right, or no one does. That's why people like me defend the right to speech of people we dislike, even when the content of that speech is offensive or downright repugnant.
So no, not moving the goalposts. Just trying to explain the principle.