@Justanotherusernamemydear what a load of old shit. As if 5 foot 6 is a giant. Sounds like he has a hang-up about HIS height! He may be 5 ft 10, but he probably has male relatives who are over 6 foot 2, so he is classed as the short-arse in the family.
I have known this happen in families. Several sets of male cousins (maybe 8 or 9 men in total,) and they are all 6 foot 2 to 6 foot 4, except one, who is 5 ft 10, and he is mocked for being 'short.'
That's probably what's happening here. It sounds like he is projecting his issues onto you.
I am 5 ft 5 and a half, and my DD (in her 20s,) is 5 ft 7. Never ever ever have either of us been considered tall. 
Weirdly, my SIL is 5 ft 2 and a half (so only 1 to 1.5 inches below the average height for females,) and yet she is constantly the butt of 'short jokes.' Mostly because my brother and all 3 of their sons (now adults,) are 5 ft 11 to 6 ft 2. And her sister and sister's husband are 5 ft 7 and a half, and 5 ft 10, and their 2 adult kids (one son and one daughter,) are 6 ft 1 and 5 ft 8... So they class her as a shortarse.
My nan and grandad (on my father's side) were 5 ft 1 and 5 ft 5, and all 6 of their kids were 5 ft 2 to 5 ft 7. (2 girls and 4 boys.) Not ONE of them was considered short, because it was a 'not-very-tall' family.)
Some people just need something to have a pop at. If it wasn't your height, it would be your weight, or your hair colour, or something. Ignore this dickhead.
And yeah, the average height for a woman is around 5 ft 3 to 5 ft 4, but that doesn't make 5 ft 6 'tall.' Daft as fuck.
I know many women who are 5 ft 5 to 5 ft 9. Many. (Especially women under 40, they are almost always at LEAST 5 ft 6!)
I know more women in this height range than women under 5 ft 3!