Personally, I have been on both sides of this and I think too much is being made of the issue.
When we got married we did not want to invite children. We made an exception only for my sister's 18 month old as was the only child of immediate family. We did ask that my nephew did not attend the ceremony. We did not want the moment we were married to be spoilt by a child shouting things out and making a noise etc. This would still have applied to a young baby too. Now I've got my own children I still think that stance is reasonable, although my sis was not happy about it AT ALL.
Since we've had a child we have been invited to the wedding of my DD's godmother which was 400 miles away. When we were on the point of booking a 2 week holiday to coincide with going I thought to ask if she was invited (only been told to save the date at that point) and was told no children. I was a bit annoyed but only because she had been encouraging me to book the holiday and had never said DD not invited.
She then said I should get my parents to come and look after her or leave her with my Mum and Dad (600 miles from wedding). We did not want to do either of these options so we decided not to go.
Well Boo Hoo - I'm sure it did not spoil her wedding in the slightest not to have me there and it did not bother us at all not to go.
In short(!) everybody should be able to invite whomever they like to their own wedding as long as they are clear from the start, but should also understand that a no children policy may mean that some people can't/won't come and should happily accept their decision.
I think neither of you should go, but don't make a huge fuss about it. I wouldn't even have tried to change their mind - it is their choice, but they should also accept your decision happily.