Ugh. YANBU, a couple of years ago DH and I booked to take our DDs abroad. It was only the 2nd time we'd been abroad just the 4 of us, the first time was more than 10 years ago, and this trip was timed for the summer before DD1 left for uni.
We'd had a holiday with my DM about 5 years previously, and when she heard we'd booked this trip, she asked if she could join us. It was really difficult to say no, but I did. I explained that we were hoping to spend some quality time with our DDs before DD1 left for uni, and my DM seemed a bit sad, but accepted it.
Then, a month before we were due to fly, she told me she'd been looking at the hotel we were staying at and they had some really good deals. That she could book a room and just spend a bit of time with us here and there. I was really annoyed, because our DDs were 16 and 18, and had busy social lives so we didn't get to spend much time with them as it was, and we were so looking forward to getting to do that on this trip. I told her again, that no, we really wanted it to be just the 4 of us.
If she had come and we'd agreed to spend a couple of days and dinners with her, then I'd have felt constantly guilty that she was there alone, and would have ended up asking her to join us, which would have pissed off DH, and spoiled what I had been so looking forward to.
DM ended up booking a trip with a friend, so I felt less guilty about her "missing out", but I was really irritated that she pushed it, even after I'd said no the first time and explained my reasons.
@MammaBear4000 I think you need to be direct here, or you risk feeling resentful during your holiday if you do spend time with her, and guilty if you don't. Far better to deal with it now, so it doesn't spoil your trip. I hope you and DS have a brilliant time!