Yes it is. It is manipulative emotional blackmail to get me to participate in a lie which is harmful to me and my interests. I'm not responsible for their feelings any more than they are responsible for mine.
No, you're not "responsible for their feelings". But you could impact them, which could potentially be very damaging! It's your choice to "participate" or not, with that taken into account.
But pointing out that it could have severe consequences for the other person when it very well could is just something you have to consider. If that makes you feel sad, well, I'm not responsible for your feelings. And it would most definitely be a lie to say it couldn't have severe results.
And, FGS @MsBeaujangles - I'm not saying women should be forced to share spaces with men if they have an intense phobia of them. I'm saying that it would be beneficial for said women to try and overcome that phobia, because it's very very difficult to go through life avoiding men completely.
It was based on the example of a woman who couldn't even hear a man's voice. If she goes out the door to collect the post she could very easily hear a man's voice as he walked down the street. It would be extremely difficult to live with. As such, counselling/therapy would help her- as there's no way that she could only exist in sex-segregated spaces. Roads, shops, workplaces etc. wouldn't be able to provide that.