It's not as simple as that. It's nothing to do with the university's own actions.
The question is, should external visitors who speak to university audiences be able to request, demand or imply that the audiences be segregated? And should audiences be able to self-segregate?
I hold no brief for bearded homophobic extremists with a taste for sending money to various -istans to fund the purchase of large amounts of 7.62x39 ammunition. But I'm a free-speech advocate and think that free-association is part of that. Universities UK are pretty craven in their unwillingness to stand up to said extremists, but we should be careful what we wish for in demanding non-gendered spaces.
The objection to self-segregation is lunatic. It says that if ten people have a meeting and five men sit on one side of the room and five women on the other, the vopos university security should storm in and mix them up a bit. Seriously? So we can set that to one side: Cameron's grandstanding. If everyone's happy with the seating arrangements, it's no-one's problem but the attendees'.
The rest of it is slightly less lunatic, but still troubling. I'm quite happy for, say, a women's rape survivor group to get a room in the university or the SU and insist it be a women's space, for example, and I don't see that there's any pressing societal need to intervene to stop it (quite the contrary). There's plenty to criticise about the sexual politics of Catholicism, but the ability to hire a room for candidates for a priestly training to discuss whatever it is Catholics discuss doesn't strike me as unreasonable, and they'll by definition be men. It is also, clearly, lawful.
I have a suspicion that Cameron's either engaging in general purpose "let's make life annoying for the beardy weirdies", in which case I think he should be more upfront about what he's really trying to achieve, or hasn't thought it through. If a university lecture is segregated, that's bad, although it's not instantly obvious why universities should be held to a different standard to your local single-sex 11 to 18 school. But these sorts of events are almost exclusively room-bookings or room-hires for events happening under the aegis of campus or union societies, and I'm not sure it's as serious an issue as Cameron makes out.
There's a lot to worry about the activities of some of the extremists on campuses, and it's a matter of great concern that campus Islamic societies provide a disproportionate number of people who have or have tried to blow themselves up. But the sex segregation issue is a symptom, not a cause, and it's a purely incidental problem.