I'm in two minds about this tbh. I've had both a sporty go-getter, bags of confidence, good at most sports (ds, eldest, now 14) and a ND dd (8) who is prone to panic attacks - and last year had one before sports day. So I'm both sympathetic and acutely aware of the emotional impact things like this can have on a child's wellbeing.
With your youngest, op, can parents watch? Are there fun elements and team events or is it competitive every child for themselves? Ime infant school sports days usually consist of some round Robin team events, bouncy hoppers, beanbag throwing, welly wanging or whatever, followed by some 100m races.
When dd had her panic attack at the classroom door, I simply took her away from the classroom and explained to the teacher that she was present, but would likely sit it out until she was ready to participate. If that didn't happen, then she was there to cheer on her team mates, and if she did decide to participate then loads of praise for whatever limited involvement she had.
He panic attack and accompanying tummy ache subsided and she sat quietly with me watching. I made chatty conversation about it being her last sports day at infant school, so it was good she was taking part by cheering others on. Eventually, she saw a fun event she wanted to take part in (teddy on a parachute) and off she went. She returned to me again for a while, and then felt that she wanted to participate in the final 100m dash. She was proud of herself for overcoming her anxiety (providing her with a safety net and plan b was the game changer) and mentioned that as it was her final sports day at that school, she was pleased she wouldn't regret not taking part.
A lot of focus is put on building resilience, rightly or wrongly. The point though is that they do need to be supported to develop resilience where possible, with the right support in place. Because in life, we have to do things we don't want to, because opting out altogether isn't always either desirable or feasible. We begin learning this when we're very young. We don't need to throw our young children in at the deep end in order to improve their resilience (in fact doing so usually only achieves the opposite result). But i don't think it always has to be all or nothing - there's often a third way, which supports their development.
I'm already talking to dd's teachers about how we can approach sports day this year, so that dd can be supported to take part in some way.