Right, well in that case he's not exactly your life partner then, is he?
Christ, woman - have you READ your post???? If he is not willing to do anything to maintain the marriage, then you DO have the right to call it a day, move on, and start again.
It's NOT against the rules to admit that your marriage is over. It IS against the rules to be adulterous. It. Really. Is. That. Simple.
FWIW, I would think hard about how to exit, if you feel as you do. You will have financial support from him so that issue is resolved - whether he likes it or not.
It sounds to me as if he is mentally bullying you. Too fucking right 'it's all about you'. If he is not prepared to consider 'you' as 'you plural', then fine - it's 'you singular'. If you have done all the things that you have mentioned, then I think that you have possibly hit the buffers.
In which case, try to think about how to maintain the moral highground. I repeat myself - make NO mistake about how you will be percieved / judged (and it clearly is important to you, else you woudlnt' have bothered posting on here, non?..... ) if you have an affair. The other man in this is a non-issue. Believe me. You will only feel loved - and ultimately rejected - by him as well.
My advice is clear, I think. I think you just need to consider how and when. Sorry, but there it is. You will have support on here, make no mistake. But don't think that you will get the same support if you do have an affair beforehand. that way lies madness and lack of self-respect.
Again - I honestly wish you good luck. I think you may need it if your dh is as unwilling as it sounds, to rectify your situation. That is not fair on you or your children. Be very careful not to muddy the waters so that you lose any chance of what you can get in terms of support for them.
(nonMN style hug - sounds like you need it!)
(here's a x too!)