Confusing feeling

I am so much better off without my ex, my children are better off and to be honest he sees his children more now then he ever did when we were together.
It's still only early days (a few months) and I've been going really well!
He has a new partner (a few days after we separated) and he seems happy.
Today, Father's Day, I feel really upset. I don't think it's to do with him. It's to do with his family.
He took the gf and the kids to his dads place and his siblings were all there.
I just feel sad because that's what we use to do.
I miss his family. I do miss him occasionally but the old him, the person he was years ago.
I just feel like they would have been all talking about me. I know I'm probably being paranoid but it just upsets me. They were my family for so many years.
How do I get past these feelings??