H is possibly the most contentious letter in the alphabet.
Dropping Hs is seen, these days, as vulgar, used by "lower classes", probably considered by some h-users as uneducated and lazy.
There was a time where dropping H was a good thing, a positive social marker (by those who care greatly about social markers) and not that long ago. Probably, at a guess, originated from the Normans (words such as humble, hospital, hypocrisy, heritage and hotel. "An hotel" might still be used today.) I imagine this is why haitch is considered correct as "aitch". Don't know for certain.
There is no academy to rule on words or pronunciation. (No, not the dictionary, that reflects the common parlance. There's some way of recording usage and once a word gets to a certain level it can go in. Never to leave.)
When people judge object to the words other people use or the way other people say words there's is no linguistic logic. It's down to snobbery, because they dislike the social feature that word or that pronunciation belongs to.
In a nutshell "haitch", is said by a lot of people and isn't wrong.