I loathe Woody Allen, I believe Dillon.
Therefore I chose not to watch his films or give my money to anything associated with him. However, he still has the right to make the film's, or in this case write the book and release it.
Free speech applies to everyone, no matter how abhorrent a human being.
As an aside, I thought the actions of the actresses in France was very powerful when they walked out of an awards ceremony when the rapist Polanski was honoured changing the narrative from his award to his crimes.