Yeah, I think this is a problem with the way freedom to read organizations deal with questions about books.
Different libraries have different purposes and collection development plans that define how they decide what firs into that remit. That document 99% of the time is the basis for assessing challenges to books. It's not unheard of for there to be more basic challenges to the document itself, but that's not usually what happens.
Patrons have every right to challenge books that they think don't fall within that. Usually they aren't supported, but sometimes they are, and in particular they may be supported to be moved to a different part of the collection. Just as an example, within the last five years in my library there was a challenge to some books in the graphic novel collection, which resulted in that collection being divided into adult, youth, and junior sections, and there was a challenge to a Dr Seuss book that resulted in it being moved out of the children's general collection and into the HQ collection where it would still be available in the catalogue.
None of that should be seen as censorship, IMO, and even on occasion if a book is shown to be right outside the CDP, that is a solid reason to ask for removal. (I can't think of anything like that I've seen in person, in most cases books like that don't make it into the collection in the first place.)
What gets my goat is that the same librarians freaking about censorship of "queer" books seem to think it is absolutely ok to censor books they don't like. What they seem to tell themselves is that these books are "non-factual" or represent bad research.
I can only conclude that they are very poorly educated and don't understand the difference between avoiding a science book because it says that gravity is caused by gomes with magnates, and avoiding a book because it has political views they think are incorrect.