I think that this phrase isn't very well thought out at all. To insinuate that one can only gain a family once they have kids is a huge oversight, and could also can be offensive or frustrating to those who can't or don't want kids.
- Someone's idea of a family might involve:
- Four kids; amazing!
-Their partner and 2 cats; also a fabulous idea.
- A single person, content with their immediate family; Great, go for it.
AIBU to think that people should think twice before using this phrase in everyday conversation?