Sorry, will shut up in a sec...
Just spotted this:
"DP's first wife had an affair and they split up so I also have to live with the guilt of doing this to him for a second time"
Think about this. Right now, with you as unfulfilled, miserable, lonely and isolated as you are. How vulnerable to an affair would you be?
Let me tell you, MASSIVELY! All it would take would be for a friendly guy to shower you with attention, tell you you were lovely, that he found you fascinating and funny and attractive and your head might very well turn.
How do you know that your P's previous wife wasn't living as you do now? Or if she didn't, perhaps because of her leaving him, he has somehow placed you on virtual lockdown to prevent you from meeting anyone and doing the same to him?
If this guy doesn't let you drastically improve your life (and I worry with the comment you made about him saying he was upset and stressed and angry when you raised the subject) then he is KEEPING YOU IN IT, for his own insecurity.
I know I could be projecting, and I hope I am wrong, but you need to see where things go next.
There is every possibility that he may say, Ooh, poor you, I didn't realise, yes of course you need to do stuff... and all will be well. If he fights your happiness, take it from me, no amount of bargaining, pleading or begging will do any good. I spent 3 years begging for a life. If I were still there, I'd still be begging and I'd still be on lifetime lockdown, in a tiny flat, for weeks and weeks at a time.
The longest period I spend in egypt without leaving the apartment we were in was 10 weeks, and I only went out then as I was haemorrhaging and needed to get to a hospital fast. Mine was an extreme situation, not everyone lives like that. But then again, not everyone lives as you do.
Do what you can. I wish you literally all the luck in the world! (((HUGS)))