I think you are completely wrong about that.
In fact I would say that what is going on in this discussion is, to a large extent, about some people wanting to keep any discussion about things racism within a very small framework - essentially allowing only one view. The whole point is to shut down views about racism that challenge the one they support, so there is no discussion where they can both be aired.
Intellectual freedom is not separate from that. Just as we've seen with other areas of id politics, the desire to step away from the established liberal democratic understanding of free speech as fundamental to other democratic freedoms seems to be attached to the claim that anyone who has a different understanding of sex/race/sexuality/gender etc is in fact bigoted.
Not being able to write about these things in the public discourse means people don't get to think about them, to consider their merits, to ask questions. That's a bad thing in any kind of democratic state, it undermines the whole political structure. Publishing, like the media, has a role to play in this that is important and carries with it certain responsibilities.