I don't think he's done anything wrong...it's just his opinion. I was pretty gutted when my ex went from longer, thick wavy hair and a beard to crew cut and clean shaven in one day - it felt like a different man went out to work and came home and he no longer resembled the man I fell in love with. I remember going on a romantic date with him that night and feeling really weird about it as he look SO different and I was having to get used to this "new" man...it was all a bit uncanny.
It took about a week for me to feel physically attracted to him at all to be honest and I always preferred it when his hair and stubble grew back a bit (and was sort of sad when he would mention he was off to the barbers).
I never mentioned it to him at all. He thought I preferred it, I think, as most women didn't like beards! And after the initial shock wore off, he was the same man I fell in love with of course. It was just such a big, drastic change. If my connection to him was only physical and it was a shallow relationship...who knows? I might have broke up with him. But my attraction to him was based on him as a whole person so it was just a temporary "ick".
I don't think this makes me a horrible person. Just a human.
Equally, my partner preferred me dressed down with no makeup. That's not my style. I took my physical confidence from what made me feel good, not him. That would be my advice to you. If you feel better with short hair, then it's your hair. Enjoy it. If you've a relationship that goes beyond just the physical, the short hair won't make a huge difference in the long term, even if he does prefer long hair. You're still the you he fell in love with, long hair or short hair.