I'm not suggesting that the white upper middle class are usually intentionally racist- just that it's not historically a multi-cultural university, and there have been some historic racist issues which were perhaps more due to thoughtlessness than intent. Basically historically it's a homogeneous group of people who don't always think about how events such as a "tribal" themed ball could come across to others. That was in 2012 btw and some students dressed in blackface or in ways that could be seen as stereotyping other races.
So historically Exeter doesn't have a reputation as a diverse and super-welcoming place. I am sure most students there are lovely and not racist at all, but I have known people from ethnic minorities be put off applying or accepting a place because of this image. So I think this incident will just reinforce people's views of Exeter as a place that's not always welcoming if your face doesn't fit. Obviously, this causes a vicious circle.
Anyway, I'm sure it's a small minority of students who actually hold racist views, but this incidents and the other incidents over the last 5/10 years show there is perhaps a culture at Exeter where students feel comfortable to, say, wear t-shirts with an anti-Semitic slogan.