Well for men it seems really quite simple. Generally chests are always ok, if not they display signs to tell them:
Well I disagree with that first bit already.
Some men do walk around town on sunny days with no shirt on, and the reaction is not as severe as it would be for a woman but it's definitely disapproved of. It's maybe equivalent to a woman in a bikini top.
Actually I think that's it - a mans bare chest is pretty much always exactly as "allowed" as a woman's would be if she were wearing a bikini top.