I have done this to just one child - but I was ill, I was not prepared to deal with physical violence at a kids party instead of making sure kids have a fun time / doing the catering etc, my child was terrified of the child in question and my child was not the only who was scared. I had seen child in question 'in action' under my nose, and I knew I would not be able to calm them down effectively. At 8 it was my child's first and last class party - I had no intention of having it ruined.
I spoke to the teachers first and explained why, and they were not at all surprised (happened quite frequently apparently), but were grateful I had explained and had given my child very clear instructions that no 'bragging' about the party would be tolerated, and that invitations were to be handed out without a fuss and however the teachers told her to (this bit was not usual practice apparently). The child in question's behaviour has not improved over 4 years, and is condoned by nanny / parents. Alot of parents are constantly going into school complaining about injuries suffered by their children.
If my child had done what I have seen with my own eyes, I would not expect them to get an invite to any party, let alone the kids they are picking on, and would explain to them very clearly why. I would also try and be seen to be encouraging kind behaviour, rather than laughing it off as spirited! I am a believer in do as you would be done by. If you would be happy for your child to miss the party if they were dishing out whatever your child says is happening, then go ahead and exclude.
Another way of thinking about it is - would you invite the snide distant relative to your wedding knowing that they always kicked off and spoiled stuff, and were going to be vile to your husband / wife to be and reduce them to tears on what is meant to be one of the happiest days of their life - probably not!