my dd has a dairy intolerance (so not the same thing, but a similar issue wrt coming across foods she cannot have)
whenever I accept a party invite, I mention to the parent that dd has dietary issues, and offer to send a party lunch/tea with dd (if the party host prefers - they usually do, as less responsibility for them - completely understandable)
dd is 4, and knows she cannot eat everything the other children eat - she has different food at school lunch, for eg (not always, obviously, but most days she will have a different pudding, or a different type of potato, or something). she is reliable (ie will not eat anything unknown without asking an adult)
so, day of party: I pack her up a lunch (usually a sandwich, a couple of treats, a cake, some dairy free chocolate) and drop her off. I remind the host she has her lunch with her (in her bag), and bob's your uncle, tbh.
party bags usually still have loads of stuff she can't have in them (chocolate buttons, odd sweets etc) but she knows I will swap them at home for stuff from her treat drawer.
no one is bothered, or upset - it is just what we do.