I thought, until my 20s, that I hated sport and exercise because they'd been such a joyless experience at school. Netball and hockey in the freezing cold and rain (if you think girls don't do contact sports, you've clearly never been biffed in the ankle by a hockey stick or skewered with the queen bee's talons in netball).
As an adult, turns out I adore exercise: walking, fencing, running, swimming, cycling. Just not team sports.
Both my boys are the same. They couldn't give two shits about football, hate rugby, but love climbing, cycling, walking, swimming...
I do agree that making rugby-hating boys do rugby is harsh, but this is sod all to do with gender, and solely about the crapness of school PE.