I agree with those who suggested that in general, people think of violence as a deliberate act that causes physical harm to someone or something. A riot in the street is considered violent if damage is done. Otherwise it's a "demonstration" (although, to be fair, the media will sometimes still describe it as violent to make a political point . . . )
Also, I think there are folks who feel the need to minimise the impact of rape by believing that some of them "aren't as bad as others." In some cases, that's because they refuse to accept that rape is as pervasive as it is and/or feel entitled to sex and don't want what they do to be regarded as rape. In some cases though, I think it can be a sort of "self-protection" mechanism, particularly if you have been raped, but perhaps feel you'll feel better about it if you tell yourself it could have been worse (i.e. "at least I wasn't beaten up,") or conversely, I've known some folks who've experienced considerable physical damage associated with rape who feel the need to assert that theirs was a "real" rape - again, I think a sort of self-protection thing, perhaps to feel justified in their suffering or something.
I would like to see a major shift in all our understanding about the nature of harm - that there are many horrific things that happen in life that don't leave physical scars that you can see. Many people still have more understanding and sympathy when someone has a physical illness and but not a mental illness. Bullying can be devastating, even where the perpetrator never lays a finger on you or perhaps when you've never even met - like through cyberbullying.
If we could get that shift, so that more people acknowledge that harm doesn't have to result from physical damage to the person, then perhaps there wouldn't be the need to convince folks that all rapes are "violent."
All rapes are a violation, all rapes are about power and control, all rapes are wrong and the total responsibility of the rapist. The experience of the survivor will be unique to him or her, doesn't follow any set pattern and it's important to acknowledge the validity of their experience. One can be raped even where no one is brought to justice (just as you can have your car stolen or your house burnt down without anyone being convicted for the crimes.)