In theory, its no worse than mums (or dads!) hanging out in Starbucks, or baby groups, or whatever.
There is no longer smoking.
Babies don't really know or comprehend what it is a parent is drinking.
If the parent is totally pissed, that's relevant, but that can happen at home, so its not a mutually exclusive pub going thing (sorry that makes no sense but hopefully you get me)
It might be that neglectful parents are more likely to frequent pubs; but the mums (or dads) in Tuesday morning playgroups may equally be changing nappies badly/not at all, swearing, ignoring, abusing, or not stimulating their children.
Its just the context/setting that makes it seem all above board.
Its like people who take their child to a pub have to try twice as hard just to look okay than someone who takes their child to a playgroup, by virtue of the setting.
You know your ex better than anyone though. If he's socialising with other parents and kids in a setting he is happy and comfortable with, however, YABU. Otherwise, perhaps not depending.