So sorry to hear about your attack, Djnoun.
But please bear in mind that what we're discussing here is not individuals (all rapes and sexual assaults are terrible, whoever the perpetrators are, that goes without saying), but public policy and safeguarding.
About 1 person a decade is killed by a cyclist on the pavement. This is a deeply terrible thing for them and their families. That doesn't mean though that because pavements are unsafe because of this we should allow motorcyclists on the pavement too.
That's the point we're trying to make with women only spaces. Yes, the occasional assault will happen, and that is terrible when it does, and the perpetrators are vile women. But that's not a reason to let men in there too, with male offending patterns, which show that we'd likely see a 50 fold increase in such incidents.
Again, I'm sorry to hear what happened to you. But we do need to talk about statistics too, as well as individual experiences. Both the individual lived experiences and the wider patterns matter.