If your brother revealed private conversations which you had an expectation of staying private, how would you feel? If your brother put you in a position where the press were asking you directly if you were a racist, how would you feel? Particularly as your brother accepts an award for fighting racism in your family and then comes out and says there was no racism. So all the time people were saying you were a racist, your brother never corrected that impression he'd given. Would that make you feel betrayed? If your brother revealed private medical treatment, or made personal comments about your looks, isn't that a betrayal?
Not being funny, but if you think all of that is done out of affection, you've got a pretty strange view of a loving sibling.
Furthermore, according to Harry, William leaks to the paper all the time. There have been reports that William feels betrayed. So if we believe Harry, they came from William.