How did you feel about his children before the holiday, I know you say you hadn't seen a great deal of them, but was it already on your radar that the younger one could be difficult or had things been OK up until now?
I think a holiday is not an ideal way to spend the first extended amount of time with children you don;t know very well. Some children are very stressed on holiday - unfamiliar places, unfamiliar food, overtired because the youngest generally has to try to keep up with all the older people and do lots of activities and eat later than usual etc etc. It could just be that this particular child away from his mother and with essentiall another family he doesn't know veyr well got totally ovewhelmed. Fussy eating isn;t such a big deal when they are at home because its much easier to cater for it and give them nuggets if that is all they will eat.
But it's fro you to decide - I can imagine having gone through the young child stage myself, that if a child was constantly tantrumming at meal times I wouldn't want to buy into that.
I think there are possible solutions if you feel this relationship is worth working on. You stay as a couple but you don;t have a step mother role, you don't live together and he comes over when he doesn't have his kids. Or you just have occasional days out with the kids but no extended holidays.
or You put things on hold for a couple of years until the children are a bit older and they may very well be a lot easier to be around.
But if you feel you cannot do the young children thing again in any way (and it is totally different from having grandchildren) then it is best to break up for everyone concerned. It doesn't make you horrible at all, it makes you realistic and honest. It would be far worse if you didn't enjoy being with his kids but tried to put up with it and grew resentful of them.
So maybe worth talking to him to see if you can find a solution, but be clear in your mind what you do and don't want if the relationship is to continue, and I would have a good think about his nasty texts - he has shown his true colours a bit, and that would make me very cautious.