You're not being antisocial, Linz2. I can empathise with how you feel about fitting in this regular socialising time, when you feel you're already struggling to keep on top of existing friendships.
I moved to a very close-knit, lots-of-meet-ups-going-on village a few years ago, which I though would be lovely, and clearly is for lots of the mums who live here. But I find it a bit suffocating TBH, and because my existing, pre-son friendships are important to me, I spend most of my "time with friends" time nurturing these - pub lunches, friends to stay (tonight), and going away to stay with old friends (tomorrow). If I then spent the mornings when DS is at school at coffee mornings, I think I'd feel maxed out with socialising, and as though I hadn't had enough time on my own, and getting jobs done.
So the downside (depending upon how you look at it) is that I don't really feel part of the village clique, although I continue to be invited to things, which is nice. But the upside is I do have the odd quiet morning to just enjoy my garden, or read a book, or get on top of paperwork. I need that time. I know other mums in the village who barely have a moment to themselves, because they whirl from one coffee morning to the next mums social engagement, and they clearly thrive on this. They also don't mind sticking the kids in front of the telly for hours while they're talking or organising a fundraiser for the school, etc. I mind doing this. I know it isn't me, and I do have to fight the odd "Is there something wrong with me?" feeling.
It's worth staying friendly and meeting up occasionally, and regularly with other mums you really click with, as this is an investment in possible future friendships for your DS too. But no use wearing yourself out socialising with people you mightn't choose as friends if it weren't for the fact that you have kids at the same nursery.