I do think Hermione was most likely written as white. But the impression that I got was that Rowling was trying to get ahead of the predicted internet meltdown when one of the plays cast Hermione as black - her message was in support of that, more of a "I never said Hermione wasn't black", than "of course she was always black". This was just a couple of years after the online tantrum thrown because Rue from the Hunger Games (who was actually written as black) had a black actress cast in the role, after all.
That said, I do think Rowling included diversity in her own way. I don't think Dubmledore being gay was a retcon, I think he was always meant to be so, and she either didn't feel the need to make it overt on the books, or more likely couldn't do so without torpedoing the series before it began. She makes it clear that several characters are of BAME ethnicity, or non-UK origin, and never specifies the race of most other characters.
I do think white privilege plays a part in it - we tend to assume the characters are the same ethnicity of us unless specified. (Teenage me even assumed Padma and Parvati were white until the first movie came out). 
A poster on another thread said that Goblins and House elves were based on well-known pre-existing fairy tale archetypes. I can see how these could be viewed as problematic when looked through modern lenses (although the only two posters to identify themselves as Jewish on the thread both said they found the assumption that "hook-nosed, money-obsessed goblins = Jewish race" much more problematic than Rowling's characterisation of goblins itself) - but at worst I think Rowling was naive, and the writing a product of its time.
The only thing that doesn't get a "pass" is Nagini - I wasn't aware of it myself, but "sexy Asian snake/dragon lady" has been a common racist depiction since the 20s, and having the only Asian female in the series lose her humanity and become the pet of the villain is definitely problematic. I'd like to think this was thoughtless rather than intentional, but even so an apology would be the better route than trying to brazen it out.
That said, I'm willing to listen to the voices of those who are of the races/nationalities/religions concerned and feel they have been done an injustice. It's difficult at the moment to tell where the criticism is coming from - whether its people appropriating these arguments to suit their own agendas, or whether people of the races/religions concerned do consider the writing in HP to be problematic.