I think a lot of women, when growing up (especially without brothers), basically get told that men are insatiable sex beasts that will sleep with anything with ovaries if given half the chance. Even when we grow up and realise that it's not as black and white as that, there is still this underlying belief that men always want sex.
So if you are in a relationship with someone, it's easy to equate "this person finds me attractive = this person wants sex with me", so if they don't want sex, then it must be because they aren't attracted to you any more.
I think you just need to talk to your partner about it one day when you aren't feeling upset, or confused or whatever. When you are just calm. You can explain to him "I always thought this, which is why I get upset when it seems like you aren't physically interested in me, as I equate it to dis interest." and see what he has to say.
It might take a few conversations about it, but hopefully you will realise that....men are human! In ltr especially it just isn't feasible to be rutting every night of the week. Sometimes you're tired, sometimes in a bad mood, sometimes you don't feel great for some reason, or sometimes you just want to feel close in a different way. It's a horrible feeling to feel like you are obligated to have sex with your partner 4-5 times a week just to appease their own sense of self worth. It's a very selfish attitude. A week isn't a long time when you are living together.
You might want to try looking at how your partner treats you outside the bedroom. If he is loving and affectionate, then just stop harassing him. Once you relax, he will probably feel under much less pressure. Let him initiate sex a few times. It might take a while (he will probably he relieved he finally gets a break) but after a couple of weeks you should start finding a cycle that works for you both.
The only problem could be if you both have completely different sex drives, even with the pressure on him taken away. If you still want five times a week and he wants once every three months then you might have an insurmountable problem.