I actually would want to say something to the mum - "why do you keep bringing your baby to this when he hates it so much?"
Jesus Christ, Mums supporting Mums eh? I can guarantee you, if someone had said that to me when my DC were younger, they'd have been told, ever so politely, to mind their own business. Or I would have with DC2, with DC1 I'd have cried, isolated myself, with my crippling PND, and probably not gone out for a week. You see, you don't know what people are dealing with behind closed doors, and kindness is always the way forward.
So much passive aggression on this thread - give them looks, and make pointed comments. How about empathising with Mum and making a supportive comment, or asking if she'd like you to hold baby while she gets changed, or inviting Mum for a coffee and a chat after swimming.
Babies cry, if you go to a place where babies are meant to be (and no, I'm not talking about restaurants where parents can take them out, I mean baby specific classes) there will be crying. Let's not isolate Mums any more than we already do. I'm so glad people didn't stop inviting me to things just because I had a colicky, crying, baby.