His sulking and guilt-tripping and controlling behaviour are disgusting. Saying that to your child is vile... it shows you that, to him, his feelings (and him being 'right' and 'obeyed') are more important than your child's feelings.
He had every right to not want to go on the holiday, but a grown up on a healthy relationship would have fully supported you and the children going to celebrate your mum's birthday. He is angry because he hasn't got his own way (when he refused to go, he wanted noone to go, he wanted to spoil all of your fun. Now he's realised you're actually going and he hasn't won, he's angry). Angry because you've dared to 'disobey' him. He actually thinks he's superior to you and you should do as you're told.
He is showing an utter lack of respect for you, selfishness to an extreme and disregard for your children's feelings. Apart from what he said to your DD, what father would begrudge his children having a 5 day holiday with their mum and wider family?
I agree with pp - make sure you put yours and the children's passports and any other necessary paperwork somewhere safe, out of the house, or it could well 'disappear' when you're about to set off. I would also make sure he can't disappear off with the children just as you need to leave for the airport. It sounds extreme, but now he's told your DD what you're doing is illegal, it shows he's willing to do rash things to hurt you and to 'win'.
And while you're on holiday, please do some serious thinking. Do you really want to be with someone who sees you as inferior, who doesn't respect you, who sulks, and controls, and is so utterly selfish? You and your children deserve better.