It also has to be remembered that Strictly isn't a show where you hate-vote people off: you vote for who you are invested in (normally a combination of being entertained by someone + thinking they are at risk). I liked both Ashley and Alexandra but didn't vote for them. It wasn't that I despised them, or anything (I may have voted once or twice for Alexandra - I can't quite remember). I voted like mad for Charles Venn and Mark Benton, and they were constantly in the dance-off, with me complaining that the public have no taste.
I don't think the public are actively racist when it comes to Strictly but I do think racist and sexist stereotypes can inform public expectations on a subconscious level. Whenever a youngish black female singer goes on the show the forums are full of people saying, 'Omg, she's going to be brilliant!' so anything less than brilliance leads to disappointment. Expectations for the likes of Ed Balls are so low that he just has to be not terrible for people to warm to him for 'giving it a go.' How many times per series do we hear someone on Strictly say that showing emotion or performing is harder for the men? Do they think women are born with the innate ability to feel comfortable wearing a chiffon catsuit and frotting an Italian to the strains of Dusty Springfield on live television?