I don't think it's helpful to blame individual men. Yes, they benefit indirectly from the culture of women's oppression, but they are not able to do anything about it.
I also benefit hugely from the oppression of people of colour across the globe. I am white. I am not a racist, I hate racism, I call out racism when I see it. I benefit from being white in that I will not be subjected to racism or discrimination for skin colour. However, what can I realistically do to change it? And I might have other factors that mean that I don't feel privileged, such as being a woman, living in poverty, being disabled- you name it. So I may not feel particularly privileged, but I still am. And like the poster above, I just mind my own business and go about my daily life. Not much opportunity to change stuff.
This is why change needs to be at a much more fundamental, structural level. Men can't stop other men from raping, but what they can do is recognise that women are oppressed and help fight for the system to be changed by those in power. They can also start being more aware of the unequal structures and not complain that e.g. gender quotas will stop 'genuine talent' from prospering. If they organise an event, try to avoid an all-male speaking panel. Call out sexist 'banter' when you hear it, just as most normal people now would not tolerate racist banter. But at the end of the day, ordinary guys are not to blame- they just happen to benefit from it.