Well yes, throwing the doors to everything open to everyone is a possible solution, and I have heard it before, as I'm sure most posters on here have.
It's not that having everything mixed sex has never crossed our minds
It's just that it seems like the least good option, for many reasons.
But it would mean acceding to the disproportionate demands of a tiny number of people, not because their demands are reasonable, but because they, as you put it, stirred up a hornet's nest, with a side order of threats and violence.
For that reason alone, apart from the merits or demerits of sex-segregated spaces, I think giving so much influence over social and political issues to such a small number of people just because they are more effective hornet's-nest-stirrers is not a good option.