But, the point is that I had my phone broken by a clumsy friend. I wouldn't dream of demanding he pay. It's my phone. I shouldn't have let him borrow it.
If this person borrowed your phone and broke it - and then didn't even offer to, let alone insist on, paying for a repair/replacement, I don't think either you or he quite understand what a 'friend' actually is.
Sometimes accidents happen, things get broken. It can't be helped.
Correct, some accidents (although not the one in the OP, by the sounds of it) can't be helped and/or avoided. That's not the issue at all - it's the RESPONSE of the person who was ultimately responsible for the accident that is being discussed here.
Your stuff, your responsibility.
By logical extension, then, is it the same with your body, which is also something that you own and which is yours to look after? If he says he thought you were teasing him, having a laugh and just playing hard to get when you said "NO, STOP" - it can't have been rape, right? You should have somehow stopped it or ensured that you weren't in that place at that time. Your possession, your responsibility, isn't it? He's just the innocent victim in it all, so he couldn't possibly be blamed. 
Of course, a phone is just a phone and losing one in no way materially compares with the horror and devastation of physical assault and rape. However, this is exactly the logical conclusion of your categorical statement.