Now they're sounding a bit childish.
I would let this time go, but what about trying to arrange something yourself for you all to do nearer too where you live, in a month or so's time? Arrange it then, with a date in the diary.
That way you are making a real effort. If they say no, then you have tried, have the moral high ground, and the ball is then in their court.
Incidentally, I don't think having a 20 month old daughter should stop you having a weekend away, should you want to. Don't make her an excuse. If you can't afford it that's more understandable, but maybe you could save up a bit for it?
It's always difficult for the first person of the group when they have a baby. Different priorities come into play, and it can be hard for those who haven't experienced it to quite understand what it's like.
In general though, friendships should be able to absorb changes, and evolve with them. If they become too much like hard work then it's questionable whether they are worth it.