I just came across this post and have read it and the comments below it with interest. It never ceases to amaze me how little Cancer Research UK know about (or acknowledge?) the Race for Life's beginnings, which are where the women only principle began.
As the person who came up with the idea for the Race for Life and launched it in 1994, here is the background as explanation.
In 1993 my father was diagnosed with Prostate Cancer which, sadly, took his life. Determined to stage some kind of memorial event, I started researching what was available and where there may be any gaps in the 'market'.
I decided that a mass participation event at multiple venues was the best way to maximise fundraising and, as a runner, began looking into running events.
I discovered that at the time only 13% of runners in road running events (usually 10k or longer at the time) were female and so started asking women why? The answers I got suggested that generally women (a) disliked the 'testosterone filled' atmosphere at races and wanted something less, or non, competitive and - especially for non-runners and beginners - (b) wanted a shorter, more achievable distance.
At that time, 5k was a very rarely run distance outside of track athletics but all of the above combined with other research I undertook led me to believe that the answer was a non-competitive, women only 5k series. This serviced an unfilled gap in the 'running market' and offered an opportunity for a different, new fundraising event.
I approached what was then the Imperial Cancer Research Fund and in 1994 the first Race for Life was staged at Battersea Park.
Not the memorial event I had originally envisaged for my father but one I feel he would be proud to be the reason behind.
I hope that answers the question as to why the event started as a women only event. As I am no longer involved, I cannot answer why it has remained so while running (both for fun and racing) has changed so much in the intervening 23 years.