How absolutely awful for you. I find if I've had even a bad row with my husband, I feel quite ill with it and upset, I can only imagine this in spades if you feel he has left you.
What this does show is that he's not a good person, to leave you alone with the children like that.
I would allow yourself a bit of a cry, perhaps get some reinforcements from family and friends by telling them, and then go about your holiday.
I wouldn't fly home as you will only feel shit there, and now you are there, the children may as well go on the beach and so on.
Dig deep, you can do this, he's an utter arse-hole and obviously thinks making your holiday as unpleasant as possible is a sensible way to proceed.
Be kind to yourself as well, you are on your own with the two children, eat nice fruit, drink bottled water, relax a bit if you can and stop contacting him.
Hope it goes ok.