I completely identify with Biedronka, I too have pushed OH away when he has seemed "in the mood". I get really mixed feelings about it all, sod you when you want it when you don't care about me, and the only way I can control anything about our sex life is to say no, but then think if I turn this down how long will it be before another opportunity arises?
I've been thinking long and hard (pardon the pun) about our situation since reading this thread. OH does have problems with intimacy... he does not like "snogging". When, at the beginning, we were having loads of good sex I realised that nearly every time was when he came home from the pub after several pints. I did say to him that it wasn't very complimentary as I felt he needed his beer goggles on to shag me! So he said he wouldn't do it at all if he'd been drinking and that around the time the problems started. Looking back, I think I should have been thinking more along the lines of Dutch courage, rather than beer goggles...
Although he is never the type to say "woe is me, I've had all these bad times", I have pieced together over the years that he had a childhood that was verging on abusive. Punished for being bad (physical punishment without grading for severity of crime, spilled milk/broken window equally punished), and ignoring good behaviour.
His last relationship was with someone I knew, who is a total bitch to people unless she's getting something from them... she will sleep with people to get favours, and was doing this while they were together.
At times OH reminds me of a child being bullied in the playground, shouting "didn't hurt, don't care" at his attackers... hence his seemingly impassive behaviour when I admitted to my fling.
As I said upthread we did go to one session of counselling together, after a brief summary of our problems, she focused on OH. He said he felt he was being picked on, and wouldn't return. Now I can't remember much of what was said, but something has stuck with me all these years which I didn't understand at the time, but has just started to make sense to me...
The counselor turned to me and said "next week we'll focus on you, and I'll explain why you are in control of this problem".
What I now understand is this; it is my house that he lives in. I pay all the bills. I do not need him (or anyone else) to live my life.
Although I do not throw my weight around, and I'm very generous with everyone I love, I actually think that deep down he is very insecure about his position in my life. (his first two serious girlfriends just upped and left him with no explanation). However he does not talk about his feelings and I waiver between thinking he doesn't actually have any (in a what I can't see I find hard to believe sort of way) and thinking he bottles it all up. He never ever verbally starts a row, I'm the big mouth one, but I do think that I kick off in re-action to his actions/inactions.
After I first posted on here on Friday I wanted to talk to him, but he immediately got defensive, then I got angry, then we had a very ugly row that just went round and round in circles. The row simmered on all weekend and we barely spoke to each other.
Yesterday I was having a bit of a snoop on the internet history... no porn, but he's been looking at stuff about male low libido. And a relationship advice site!
And last night.... we DTD!! Woohooooooo! 
I enjoyed it... but I have no idea how he feels about it, I'm not going to spoil my post coital glow yet!