Not really answering your concerns, OP, as it doesn't sound as if it applies :) But it's a good question, so here goes!
Boundaries. Something weird about all of them - this ties up with insufficient respect, which HotDAMN rightly highlighted as necessary and lacking.
XH1, a narcissist, had an excessively (creepily!) doting mother and a dead father, whom the mother had despised. His mum really did worship him. Insufficient boundary between the pair of them, and little respect for the father.
Secretive, controlling, dishonest XH2 has a perfectly nice mother who is abnormally reserved. I'd hazard a guess she has a mild ASD - very reserved; eye contact is either avoided or intense; doesn't "do emotion" but blurts it all out under high stress; prefers dogs to people on the whole. XH's dad, a sociable extrovert, was seriously disabled for most of XH's life. That marriage must have been very strained much of the time. XH was sent to boarding school from 7 to 13 and wouldn't talk about it. Way too many boundaries in his family, and inadequate respect for emotional needs.
My family was volatile, violent and without privacy - loads of children, smallish house, both parents controlling in different ways. They were neglectful as parents but simultaneously intrusive: always poking into our affairs and criticising our thoughts. There was an absence of emotional & physical boundaries, and fear where respect should have been.
My family looks all shiny & pretty at first. People usually need to meet us three or four times before realising there's something odd about us ...!