I have been with DBoyfriend for a few months. We have known each other 5 years. He was with his ex for 6 years, I knew her as well. Their relationship came to an end a year ago as he cheated on her with, let's call her Anne for ease!
Being friends before, I knew the reason for the split and all the gory details - one night stand with Anne after a party when ex was away on holiday, they did it twice, as soon as his ex got back from holiday he ended it, I know from being friends with them both, that their relationship had been dead for a couple of years, not an excuse but trying to paint a picture, Anne was also in a relationship at the time of the ONS, she later spilt up with her partner and now has a New relationship and lives with new partner.
Anyway that's the back ground.. The point it...
DBoyfriend told me in passing a few weeks ago that he was going away for the weekend, he mentioned it in passing a few more times and I thought nothing of it.
Now today, it came up, that he was going up their with a mutual friend of his and Annes to stay at Annes and her new boyfriends house! 
I said, it makes me feel really uncomfortable, and I wouldn't do that to him out of respect to him and his feelings but I would never try and tell him what to do or make a drama when he goes but it was very shady how he didn't mention it the first few times he said he was going away.
He said I completely understand, I thought it would be fine but as soon as the words came out of my mouth I realised how out of order and weird it was, it had been planned since before we got together and he didn't think until the time was near, then realised and got stressed at the weirdness of it and if he didn't go mutual friend couldn't go as doesn't drive. Anne lives 4+ hours drive away now. But he realises it's wrong, he doesn't want to go and isn't going. I asked how much he spoke to her and he said a occasionally as they were friends before. I don't understand why/how you could/would want to talk to someone in this situation?? His ex was heartbroken, even though their relationship was dying before it, and Anne was friends with his ex as well, as they all had to work together for 3+ months afterwards till Anne moved.
This has really shaken me to be honest. And I have no idea why. I know him so well, we spoke like reasonable adults, he completely accepted my point of view and now isn't going (his choice) I feel on edge like something bad will happen.
I've had a two very abusive relationships, I've had counsilling but could this be why? Should I be worried?