Access and money helps.
Many "mc" outdoors activities are prohibitively expensive for most people (skiing, sailing, riding).
The rich are likely to either own land or have ready access to privately owned land via friends, which opens up further activities (shooting etc) which are entrenched in wealthy life-styles.
But even learning to swim at a thd local pool is out of reach of many due to cost.
Public schools also provide early exposure to many outdoor sports and activities, which are expensive. As adults then, these children have already learned the sports so don't need to pay for lessons and coaching. Likewise, if they've not learned, they'll have been able to try a wider variety of sports so as adults they'll know what they want to spend their money on if they want to try something again when that cost can be too much for many people to even attempt (lots of equity to be bought).
State schools don't always teach children to play sports properly. All very well if the child is naturally coordinated or competitive, but for everyone else they miss out on whole activities, potentially for life, because no one had the time to teach them how to do it properly when they were young. Their formative experiences are also less likely to be enjoyable.
People without wealth have less access to open spaces, especially green spaces and especially pleasant green spaces. This further reduces their enjoyment and participation of outdoor sports. In decades gone by, they may have played more freely on the street but increased traffic, safety fears and gaming has all influenced this.
More to say but that's the gist.