I think the problem with that serenity, is that it assumes that acceptance and progress is inevitable.
What we have seen historically, that there is no inevitability about it. When you describe people complaining about JC and his ilk, as wanting to "speed up" that acceptance process, what you're actually describing is the process. Without the complaints, the disgust, the discussions about it, the process wouldn't actually happen.
And what we also know, is that there is always a reaction. Every generation which fights for acceptance and liberation, gets told that it's asking for too much. When black people fought for civil rights, they were told that FGS, they weren't slaves any more, what more did they want? Then when they said they wanted their children not to be murdered at bus stops, they were told, FGS they had equal pay, what more did they want? When women fought for equal pay, they were told FGS, you've got the vote, what more d'you want? Now we're fighting the climate that enables us to be raped with impunity, we're told FGS, we've got the Equal Pay Act, what more do we want? And now disabled people have the right not to be kept hidden away in orphanages, they're being told FGS, what more do they want?
Every generation of marginalised people has a backlash in response to any battles they've won and the battle has to continue. Jimmy Carr and disgusting men like him, are part of the backlash and progressive, decent people have to pursue the battle against the forces he represents, because it's not inevitable that without engaging in the battle, it will be won by some sort of unstoppable process.