I'm not articulating myself particularly well
Indeed, you aren't.
People are perfectly able to tell whether a white man murders another white man because he's racist against white men (not terribly likely) or because of personal or financial reasons.
Same can go for men murdering men, in general. Why would that have to be recorded as hate crime?
Men murdering women is almost always a hate crime against the woman because of her sex. Very rarely can it be plausibly said to be about anything else.
Man murders woman because she ended a relationship with him = hate crime.
Man rapes woman = hate crime.
It is very easy to tell apart from other crime because there's NO REASON to do it other than hate.
Don't see how there would be any problem with it.
(Other than men as a class not wanting it because it would expose just how much they hate women)