I don't think it's a reasonable feeling to not be comfortable swimming with men in the pool
Back in the pre-DC days when I looked vaguely OK in a swimsuit I used to go to a ladies' swimming session, but it was always really busy and crowded. So one night I went when the pool was open to anyone. Big mistake. It was a changing-village set up but with separate men's and women's areas - which the men were completely ignoring. Getting changed in a locked cubicle with men nearby didn't bother me - but the fact that they ignored the signs saying it was the women's area made me wonder what other boundaries they didn't respect.
Anyway, when I actually ventured the pool it was just horrible - crowds of men in the spa pool staring at every woman who walked past - literally following us with their eyes, and I was wolf-whistled at and just generally ogled and leered at. In short I found out why the ladies night was so busy!
Now I haven't been to that pool in years - things might have changed since then - and as a middle-aged woman I am less likely to be leered at nowadays. And of course NAMALT. But as a young woman I never went back to the mixed swimming at that pool. Was it really unreasonable of me not to want to get leered at and catcalled when I already felt self-conscious in my swimsuit? And I'm not from "a conservative culture" FFS.
Until ALL men learn to behave themselves in that setting I will support the right of young women to go for a swim without being harassed.