I think it is one of those beauty things that "works" psychologically because it's something that is emphasising a difference between male and female bodies.
I don't think it's really the case that the preference for these things is set in childhood or youth. In my experience if there is a widespread change, people get used to it very quickly. But it's not that common that there is a wholesale change, and just a few people doing it doesn't make much difference, it doesn't work to "reset" our sense of what looks right. And most people won't put themselves out there and change on their own so it becomes a collective action problem.
I have wondered why body hair is an area people have such a gut reaction to, because it's not the case with most other fashion and body modifications. People might think someone who doesn't do the other things isn't interested in looking good, or is a bit lazy, or whatever, but that's about it, they don't think it's gross. It seems like it elicits particularly strong feelings.
I've had similar feelings too and I can't think of anything else that compares, for me what bothers me is hairy legs with pantyhose. It's completely irrational, well beyond just thinking it looks bad, and I have really no explanation. I also had an experience once where the person I was dating did some fairly extensive manscaping, and was quite pleased thinking I'd like it, and I found it completely a turn off, actually disgusting. I had a really hard time not showing my emotion and I had to really walk delicately to try not to make him feel embarrassed/awful while not encouraging a continuation. In the end I told him the stubble iritated my skin.
I'd be very interested in any theory of why people feel so strongly about this in particular.