I think this is symptomatic of Harry's sense of propriety and exclusivity over William that is a repeated theme.
Harry complains that William didn't want to know him at school. William would have had his own friends and would not want a younger sibling hanging on. Perfectly normal reaction by William. He didn't stop being Harry's brother, I am sure he would have been there for Harry in a crisis, but he was no longer exclusive to Harry in the day to day.
William gets married and wants to include close friends in the wedding as well as his brother. Perfectly reasonable compromise is found that allows William's friends to do some speeches. His brother has a huge and very visible role as best man and does a speech, but he's not exclusively his brother's groomsman.
William gets married to Kate. Harry feels a disproportionate level of abandonment. He clearly still spends plenty of time with his brother, but William is no longer exclusively his.