I generally contact the host & say about my kid/kids and then say I'll provide them with some food anyway (even if I know the party venue can cope, it's sometimes just easier to take your own).
My kids get a take away tub to go to a party with!
I squash silicone cupcake cases in it to make little compartments for all their foods. They think it's great! (It's the only time they have it, picnics are served differently, so these are special party boxes!) it's mainly salady things (toms, Cu, olives, pepper, sugar-snap peas), fruit (grapes, orange segments), breadsticks/pretzels, mini pepperami type thing or cocktail sausages (m&s ones are now gf to!!) & a cube of "allergy cheese". Maybe a wrap with ham or jam, but they get messy, crisps (hoola hoops) as a treat are my preference!
My bag always has an innocent smoothie pouch, mini bags of haribo or mini chuppa lollies to swap out during games or add/change a party bag. I found individually wrapped marshmallow sticks in home bargains the other week, 9p each, so they are now part of the "party kit" too. I normally have mini muffins of some flavour in the freezer, so will take those out & along incase they eat cake while they're there, otherwise they get it as a pudding that evening for 'being good & behaving' at the party!! As long as it's something a bit special they don't normally get as a treat, they think that's what "party food" is :-)
Even when they're 4/5 & realise that they eat very differently at a party to pretty much every other kid, because they are used to having the 'special box' they know it's special to & just for them.
Get them used to not helping themselves from the table (start this at play/toddler groups that serve snack - you always take it to them, not them help themselves off a table), only eat what you provide, make them check with you(or other trusted adult) that they can eat what is in front of them & to politely say to someone that hands things round "sorry I have allergies, I can't have that".
It's ultimately for their own safety that they need to learn the words to say. Even if you stay (& I always have so far), you'll be catching up with friends yourself, if you've got two kids to watch, it'll be standard birthday-party chaos, you won't be able to keep eyes & concentration on your child 110% of the time, so they need to learn to be a little bit independent as soon as they can really start talking. My 3.5yr old is now having to be independent for himself, as his big bro (now in P1) is not there to say it for him! And he's figured it quite quickly & says to people he cant have wheat or 'cow milk' & always comes to me for food, never takes off a table.
It becomes part of your way of life and you get used to it, just keep smiling sweetly and never pity them, it'll get you know where. They don't know any different, they don't normally know what they are missing, they don't get why 'its a shame' all the time, so just stop saying it & stop others from saying it too.
Chin up, it gets easier!