Couldn't agree more with Treadsoftly.
If this is an issue now, it will be a massive issue later if & when you have children. If he's anything like my ex he will then use the DCs as an excuse - "I'll stay home with the kids, you go". Not because he's a good dad, but because it's a convenient excuse. It got to the point when I would get an invitation to something (often something significant, like a wedding or a big birthday), and I'd dread even mentioning it because I just knew I'd be disappointed by his response.
He is very rude to your family - I found I saw my family less & less when I was with ex, because he didn't want to be there, I felt awkward about going and our visits dwindled. Now I have a DP who is sociable, and we see both our families much more.
This is obviously something that matters to you, and it won't get better. He's already told you that. So, here comes that old MN sentence -
"When someone tells you who they are, listen."
He's told you he can't be bothered, he has enough friends, he doesn't really want to spend time with your significant friends and family. Listen to him, and dump his sorry arse!