I have been a Guider and a Scouter, and I was involved with both organisations when the decision was made for Scouting to go mixed.
I was also a Venture Scout, rather than a Ranger Guide (neither exist now, different names and different age ranges too.
As numerous others have stated, the decision to go mixed for all ages not just over 15s was taken because numbers weren't great, especially in rural areas.
Fwiw our local units weren't happy and elected to stay all boys for as long as they could - initially it was down to individual scouting groups to make the decision, but all new groups had to be mixed.
GirlGuidingUK and WAGGGS (international association) consulted at the time and repeatedly since as to whether they should stay all girls or go mixed. And every time, the members - children and leaders - have resoundingly agreed that it must stay all female.
Like a pp I've seen girls grow down when they've been on camp for a few days. Forget the stresses of makeup and hair and looking good, and just get on with fire lighting, scrubbing blackened pans, wading through mud, getting messy, stinking yo high heaven, and yes, sitting around crafting and singing and generally being themselves without needing to look cool.
And then I've seen those same girls when we've camped on Scout sites rather than Guide ones. And nice seen them spot the boys, and suddenly showers have been taking 30 minutes not three, and they don't want to get dirty, and oh they can't forage for wood because they might get sweaty. We took the decision only to use Guiding sites after that.
We do have girls in the unit who would never be allowed to attend if boys were allowed in. That isn't because we as leaders think there is anything inherently dangerous about the boys who might join, but because the girls belong to families who will not allow their girls to mix with boys. Is it better, under those circumstances, to continue to be a place where those girls can mix with girls and women who have different views to their parents, or should we take what may well be the only activity they can do which is unconnected to religion away from them?
It does seem silly, maybe, when we are talking about very young prepubertal children. But, if you decide boys can join rainbows, do you then kick them out when they reach brownie age? Or can they stay just until Guides?
I'm sorry your son is disappointed. And I hope you can find him an activity he will enjoy. FWIW as others have said, if there really isn't space in your local scouting group, volunteering yourself is the most effective way to jump the waiting list. We're always looking for more volunteers, and it doesn't have to be during the meeting - treasurers, fundraisers, loads of admin; there's so much unseen which goes towards providing the evening meetings and other activities.
Having an all girls space was so important for me when I was going through school in a not terribly nice mixed comp. I am not a Guider any more, but I'll fight strongly for the right of GirlGuidingUK to stay a movement for Girls and Women only.