No it isnt. The mans choice not to go has no impact on women, what do you care if a stranger you've never met before doesn't go to a class? It doesn't, his choice not to do doesn't impact women being helped.
Of course it does (or men, might also affect men, I have no idea of his preferences).
Because if he thinks he knows all about consent, but really doesn't, then it's anyone he sets his sights on who will be affected.
When he puts on a puppy dog face, and repeatedly mentions how much effort he put into cooking, how it's still early, have another glass of wine, come and sit next to me on the settee etc to someone he's invited round, tops up their drink a bit more, until they're just that touch to squiffy to feel they should walk home now it's 1am, they should stay over, it's fine, they'll just share a bed - the couch is really uncomfy and his housemates will be up early, and somehow, despite not really wanting to, they end up having sex.
That's happened to me with creeps, I've also been with decent blokes who have slept next to me after accepting no, and I've been with blokes where enjoyed where it went - but there's no doubt that's a dodgy consent situation - and one that bares thinking about - I wonder how much thought the consent refusers have put into that?