I have thought about this, and I think that it is a by-product of the fact that we are not only highly social, we're also very adaptable and also build culture.
People sometimes think we can avoid culture, that it separable from biology, but that's a mistake IMO (as an aside, I think this is why the idea of removing gender as we see in GC perspectives, is wrongheaded - you can't get rid of cultural reflections of material reality, including sex.)
So humans adapt to many kinds of different environments, technologies, and really sometimes it seems almost anything, and while there are continuities in our our reproductive strategies operate, it is also hugely variable. Human men were capable of being sexually aroused and attracted to a female cavewoman with crazy hair and smelly BO and hairy armpits and ticks - or in another time and place, Kim Kardashian. They can be happy to live in a one room soddy and beget six children there surrounded by their offspring, or a long house surrounded by expended family members, or on the other hand become used to total privacy and sex in a night gown to hide the evidence or arousal, or a million other things. They might like women with natural hair, or women in powdered wigs. they seem to be able to adapt to any number of normative sexual practices and positions and conventions to get them warmed up for reproduction.
All this requires a brain that is very sensitive to the surrounding cultural signals and stimuli for what is sexually attractive, and what the cultural signals are for fertile, available women and men, and even what the cultural signs are that they themselves internalize as attractive to the people they are interested in.
This seems to be particular sensitive with men, probably, as a pp said, because their reproductive strategies are a little different. Women don't actually need to be all that sexually motivated to become pregnant, but men really do.
This goes on throughout our life but is most active in youth. I think fetishes and fixations are what happens when the parts of the brain that are looking to attach sexual significance to whatever the cultural norms are around reproduction in that society, somehow get stuck on the wrong thing at a very formative period which can often be quite young - early or even pre-teen. I suspect the prevalence of certain fixations, something like rubber or vinyl outfits, is because that kind of object has some sort of similarity or enhances an otherwise more standard quality that men are sexually receptive to - maybe skin texture, or radiance, or something - it could be abstract as long as it flips the trigger that one time.
I also believe that we have really increased the incidence of this kind of targeting error doe to things like television, pornography, and advertising.