I moved away from the home town 15 yrs ago
That's not a coincidence, Molly. I don't think there was much reason for you to go back, new relationship or not. In fact, I think it was the most sensible decision, for both you and your husband.
It does feel lonely and isolating, I know that. We are constantly told that blood is thicker than water and families should stick together. We are made to feel at fault in some way if we can't make our family relationships work. You only have to watch the barrage of Christmas ads on TV promoting family togetherness and happiness and love and warmth. The truth is, some families are just better kept at a distance. Just being in the same room as them is enough to send your stress levels soaring. It's sad but it's not as uncommon as you might think.
You have your DH and your children. They are your family now. Concentrate on making your home a happy home and making your own happy family memories - ones that are completely different to those of your own unhappy upbringing. It can be done.
I worry that they will think it’s for no good reason too.
Ohhh Molly, even if you explained, what are the chances they would ever see it from your point of view? Isn't that part of the problem? When have you ever had support from your family? Stop worrying about what they will think. They'll think whatever they think regardless of what you do. As long as you continue to believe that you can influence their behaviour, they will continue to upset your life. Let it go. For your own sake.