Only a handful of women want to work on oil rigs -> excluding them is not a problem
That is direct discrimination. Excluding them is not allowed.
Only a handful of men want to join the WI -> excluding them is not a problem
If WI was excluding all men, it would not be a problem because there is a specific exemption for associations that covers this. As they are only excluding some men, this may be indirect discrimination, although this is not guaranteed. Unlike direct discrimination, this can be justified if it is a proportionate means of achieving a legitimate aim (objective justification). In deciding how high to set the bar on whether you have objective justification, the courts look at the number of people affected. If you are excluding a lot of people, the bar for what is objectively justified is very high. If you are only excluding a few people, the bar is much lower.
That is not bonkers. It is the law as interpreted by the courts.