What you do is what makes you happy, in a couple of years your kids maybe at uni, left home have families of their own, and you will still be there at home alone with an empty nest so to speak.
Your husband has alreasy made his life, and seems quite content with it, I wonder what he gets out of this arrangement? The fear of leaving and change might be the biggest reasons for not taking the steps you feel you want to make.
You have done your duty, raised two i assume great boys, kept hearth and house together and played the dutiful wife. Now you are at a cross roads and at a certain age I assume not to far off your husbands, how much longer do you suppose you can procrastinate over your future, and even worse when it comes to his and your retirement, are you going to have anthing in common anymore, and how much is he going to get on your nerves, once he is back full time.
The thing is you dont hate or dislike him, he isnt I assume a bad man or abusive, its just that you live seperate lives, literally. The only thing thats left in common is having the same front door key.
Sit down and have an honest and frank discussion both with yourself and your dp, dont stay for the kids sake, I wouldnt factor them in to the descision to be honest, because they have their own lives now, and only require a bed and a fridge so to speak. As for the other guy, the fact you are thinking about it, shows how desperate you are to make something of your time and life right now.
I wouldnt do anything with him, and if your dp found out might damage your financial position if you did split up. if you decided to split amicably is the wa to go, throw in anger and resentment to the mix, it's a recipe for disaster.
It did occur to me as well to be honest, whether he has anyone else stashed away some where, which is the reason he is so easy going about everthing.
all the best