Yep. Much of that is fawn response as the most socially acceptable of the risk responses.
For women confronted with a male, also in an evolutionary sense, most sensible if you cant flee. Freeze is also common. Very few women go for 'fight', for obvious reasons.
Much of this is subconscious, of course.
But in the same way I found a male driving instructor made it difficult to relax enough to learn (strange male, confined space), a lot of our responses are instinctive and unconscious. Can also be quite subtle - doesn't mean I was actively terrified of my driving instructor, just that it made the learning situation harder.
And given the point of the lesson was for me to learn, a female drivng instructor was the proportionate/legitimate thing.
Given the point of the WI is for the benefit of women, excluding men is proportionate/legitimate.
At base, the point of the WI is to help women, not men.