Sorry, you asked about not having contact with family. Mine was nothing to do with my DH though he supported me. I severed contact with my mother and one of my sisters after years and years of problems. I would consider both of them a bit batty to use a politically incorrect term. My mother beat us when we were young, my sister has never come to terms with this or the emotional traumas she dealt out - she used to throw away toys as a punishment for any mess (and I mean anything out of place by an inch), she had various illnesses, quite serious ones, and I have vivid memories of being told I caused her cancer and she nearly died because of me
As we all grew up, she became jealous of things we had, things we did - anything became a target for her nasty comments, I don't think she ever said anything positive to me ever. When I got married and moved a few miles away, DS1 became the battleground - my mother and sister,felt, I think, replaced in his life by my DH. They began interfering in our home, (came in from work once to find my mum digging up the plants in the garden and putting them elsewhere etc) and his life. He was at high school by then and didn't want to spend as much time with them, quite naturally, but they put a lot of pressure on him, so he began to tell them he had after school stuff to go to. They started ringing the school and trying to check up on this, then ringing his mobile - my sister would say stuff like "Well, if your nana can't come over today, she will think you hate her, so shall I tell her you hate her?"
Contact severed! It was basically enough was enough - this is the potted version, horribly long though it is. Is this any use to you at all or is your circumstances very different?