Because I would expect sweets at a children's party, life experience has taught me that at children's parties sweets cakes and crap food are often much in evidence.Therefore I do not consider them to be innappropriate, I consider them to be something that many people offer at kids parties. I wouldn't consider a party of rice cakes, raisins and fruit innapropriate either. I would go with the flow and behave like a polite party guest because I appreciate the efforts others make to host a party, and I wouldn't dream of being rude or awkward. If I didn't want my child to have something I told them, I didn't expect other people to do my parenting for me.
My kids didn't eat sweets either, and we are vegetarians. But that doesn;t mean I expected all parties to be a meat free, sweet free zone. I simply made sure my children didn't eat anything I didn't want them to eat, either by warning the host, or by trusting my children to know our rules. At 3 they were both perfectly capable of asking 'is this meat?' if they weren't sure, and both were perfectly capable of and did put sweets away, or give them to me, so they could be rationed or disposed of.
There were certain toys I wasn't keen for my children to have either - but if we were in a toy shop, I told them 'no' and explained why, I didn't expect the shop not to stock toys that were popular with other children. And that's the difference - I don;t assume I am right, I assume I have an opinion and others may have different opinions, therefore I take control of my own beliefs and behave accordingly -I don't expect the rest of the world to accomodate me.
I think if you are going to choose a lifestyle for your children you have to talk to them about why you have chosen it and start trusting them to go along with it, and once they reach a certain age accept that they may want to choose their own way of eating and living.
I think people who go to parties looking for things to criticise are rude and ungrateful. If you don't want your kids to eat something tell the host in advance or deal with it yourself when the problem arises. Just say 'no you can't have that' to your child - that is your responsibility not another mother's.