DS2 has a balloon phobia, he is moderately deaf and with his hearing aids, balloons popping terrify him.
The last party he went to where there were a lot of balloons the entertainer saw his distress, and that we were about to leave and moved them all to one side for a while whilst she did her stuff. I loved her for that.
He is 5 and I would not leave him only for that reason. I can just take him out for a while or go home if it's really bad.
I always breathe a huge sigh of relief when balloons are absent at parties. It means I get to drink a cup of tea and have a chat with other parents!