Just a quick thought here - and I may be totally wrong.
He is obviously aware that you are unhappy and he may feel that you will only be satisfied with a real marathon session - I don't know know what those three times were like or if you would characterise them as marathons or quickies.
But, it may well be that, for whatever reason, he is wary of initiating sex because he feels that you will be expecting a real marathon session that he doesn't feel up to, or able to, provide.
There was a recent thread here about marathon sessions:-
www.mumsnet.com/Talk/relationships/1807475-how-long-should-good-sex-take
It may well be that if you let him know that, sometimes, all you want is a quickie then he may feel less pressured to perform. Then, over time, you may be able to get him to increase his frequency as well as his length - 
of time.
.
Another thing to consider is that, although you are not having sex, he is probably still masturbating, at least weekly, while he is alone.
It can be really difficult to talk about subjects like this within a relationship and the default position is for the man to deny it. But, raising the subject in a VERY non-confrontational way can be a good opener into talking about frequency and your needs as well.
Even if it becomes as blunt as saying, look, the next time you want a wank just come and bend me over and give me a good seeing to instead.
Yes, I know that last sentence was extreme - and I wrote it to have an effect. But sometimes you do need to shock people into reevaluating things.
Another possible thing to think about is ED - erectile dysfunction. This can start to hit men in a mild form from the mid 30s onwards. It doesn't mean that he can't get it up, but that it's harder to keep it hard - if that makes sense. This can get men into a downwards spiral of worrying about their performance and being concerned if it isn't always rock hard.
Sex doesn't always have to mean PIV - or PI anything - and, if one partner is feeling pressured then saying upfront that you just want to, as my American friends would say, go to 2nd base or 3rd base then that can make things a lot more relaxed and help to remove any pressure that he might be feeling.
Don't forget, that if we don't feel massively turned on, or feel stressed about something, we can just lie back and think about the shopping list - men can't do that.
One final thing - I didn't realise how much I've written - and I may just be stating the obvious here. Alcohol and large meals don't do much to help men's libidos.