I think what I find hard is that if someone steals a wallet, the victim is believed, if the wallet is found on another person, if the victim identifies them, that is taken as strong evidence that the person stole it, and it's all pretty clear cut. It's still one person's word against the other, but it's assumed that the victim didn't want their wallet stolen, even though they may have previously given money to other people, or even to the person that stole the money this time. And that's just a wallet. Just money.
If a man rapes a woman, if there is clear evidence that sex happened, the woman is not believed that it was rape. For some reason, it's believed that she wouldn't give money away, but not believed that she wouldn't want someone to shove a penis in her.
This is where I think it's wrong. I think the burden of proof should be on the man to prove he had consent, not on the woman to prove she didn't give it. I think that women should be considered to be walking around not consenting to sex by default, just as they walk around not consenting to giving money away by default.