A difficult question.
It sounds as if he will have his own accommodation, but would mainly be joining you on the journey. That does not seem unreasonable, as many 80 year olds might balk at the travelling alone. And it will be an opportunity for him to be with his family and see his DD.
Is he a total pain in the arse? We took my FIL away to France with us when his wife had just died, as we felt it might help him. He was a sexist racist bigot, but we just let that wash by us whilst making sure that the children realised it was not acceptable. He was of course a product of his upbringing and he was well into his 80s - not a lot of chance for change. We never took him again - but it felt the right and decent thing to do at that time. He might have been an old bigot but he was entitled to his grief and to have his family rally round at that point.
We never took my parents as they argued non-stop.
I am a MIL now (and widowed) and have been away with family on several occasions, sometimes with one family and others at big family get-togethers. I have tried to be unobtrusive! - and hope I have been a help, with child care and cooking. But I have been clear with them that it is not an expectation on my part, and I do not go into a decline if not asked. This year it is me asking them, as I have booked a fabulous house for all 14 of us with some money that came my way when OH died. We are all looking forward to it. I do not make sexist or racist comments!!!
I would go with the flow on this - it sounds as if it is mainly the journey, and that is hardly the most exciting part of the holiday. If he is staying separately, then that is a bonus if you do not know him well. Might be worth saying to DP that it would be good if he consulted you first on another occasion; but it does sound a bit as though he was put on the spot. It is not a bad lesson for your DC to see different generations and different viewpoints rubbing along together and being tolerant. Remember this will be you one day!! - it is worth setting a good example to your children!
With goodwill on all sides, it need not be a disaster. And you will be able to shine your halo!