I did a fair bit of skiving and getting excusing letters from my parents. But in the fourth year, we started going to a swimming pool at another school, should have been for just a half term on rotation, but I asked the games teacher (the one and only good one in the school who was our teacher in 4th year) if I could go swimming every half term and she agreed! What a result! I could just go and do my own thing, no "team" games pressure and ridicule.
That's how it should be - pupils should be free to choose what they do, not have things forced on them.
The stupid thing was that the school had a huge store room of all kinds of equipment, but it was only ever used for the school teams, i.e. archery, badminton, tennis, fencing, etc.,- no one else ever got a sniff of it. Likewise with the school gym - I think in the five years I was there, I only ever saw it set up as a gym once - the rest of the time, all the equipment was folded back to the walls and all the horses/benches etc were piled at one end - all so the gym hall itself was clear and empty for us to play volleyball which was the lazy default for the games/PE teachers.
Luckily for me, I did stage lighting, so for several weeks per year, I had a genuine/valid excuse to be excused from lessons so I could go into the hall to set up the spot lights, lighting control board, etc., and funnily enough, I always chose the games/PE lessons to be the times I "had to" go into the hall to do it! Funny that!