Op,
Repeat several hundred times - It's absolutely not you!
So there is some intimacy? That's something I suppose, but there should be a lot more. If you had been together 20 years, then maybe things would have slowed down, but not after 2.. 
Sex is a very important part of a relationship, and if two peoples sex drives are as badly mis-matched as yours seem to be, then maybe it's time to move on?
Have a look around the threads on here - if he is "playing away", he can easily hide it. Is he carful with his phone for instance? I hope for your sake I'm wrong... 
Personally, I wouldn't be making any further commitments until this is sorted out. Do you want to spend the next (insert appropriate figure!) years of your life with a man that wont even touch you? (I think we both know the answer).
Why won't he go to the doctors? Embarrassment? He could still get over the counter Viagra (or maybe you should?), to see if that makes a difference, although at mid 40's he really shouldn't need it.
Finally, and I'm pretty sure I'll get flamed for this, but, would you consider, er
alternative arrangements? I'm certain there are thousands of guys that would be willing.