It's not rude at all IMO. It's entirely up to the couple how they want their day, and nobody else. If it gets harder for people to travel, or take time off work, or get a babysitter, that's the risk you take.
We did all three (sort of, though not to extremes). We only had very close children. Our venue was a little bit out of the way (though not hours away) and we got married on a Thursday.
We only had a small-ish wedding so I knew the people I was inviting were there for US and didn't mind booking a day off work or driving 40 minutes... plus they were given over a years notice.. anyone who deemed it rude / inconvenient I would have preferred them not to bother coming because that just goes to show how much they actually care about us.
Midweek weddings are a lot cheaper, also. I'd rather spend a little extra money on upgrades for my guests (a bit of money behind the bar, drinks on the tables, nicer meals, etc) than to spend a few extra thousand just to have it at the weekend. Catering for my guests who actually bothered to come was my main priority, not trying to appeal to the masses who were only willing to come for the free food.
And the kid thing..... well, I can totally understand why people don't have kids at weddings. I went to an extended family members wedding with a hundred day guests. Around a third of these were children aged around 6 months to 9 years and they totally spoiled the ceremony, photos, meal, cake cutting and first dance. It was absolute BEDLAM - reminiscent of a children's playcentre with how much they ran out shrieking and throwing things. And not one of their parents tried to keep them under control, all too busy getting pissed. Bride ended up in tears because one of the little darlings did a cartwheel in the bar and knocked hubby's pint down her dress. The whole wedding was put on standstill in the middle of the speeches because one of the kids wandered off and had to be tracked down (there was a lake onsite so obviously that was a serious worry). At least 4 kids wailed, fidgeted and yelled throughout the WHOLE ceremony - the vicar, priest, whatever he is had to stop and ask for uiet so many time the service ran over fifteen minutes. The first dance was chaos because kids kept running onto the dancefloor, followed by their drunk parents trying to round them up and was just disastrous. The song had to be restarted twice, by the time they actually got on with it, it was so awkward. I can't imagine forking out all that extra money to invite people's kids to keep them happy and then they let their kids run riot like that.
As a direct result of that wedding I only had 9 children at mine... which was small anyway to be fair, but I'd made my mind up I wasnt going to risk it. Three of those kids were my own. Four were family. Two were my hubby's best friends children. Out of the nine, three behaved pretty poorly all day (my nieces and nephew) and were often very disruptive. It's amazing how much just THREE bored, unruly kids can disrupt a wedding, so I totally get why people don't have them there. I don't take offence whatsoever when mine aren't invited to weddings. Sometimes kids can be fab at weddings and sometimes they're not. It's up to the couple whether they want to take the risk or not. Personally, I didn't. Having seen how sour it can go, I didn't fancy my chances.