How old were you when you got together? How many men did you sleep with before him?
I have a theory, that people who settle down in their 20's with their first Partner, can rarely do the distance. At least that's my experience, and the experience of almost every couple that I know (all bar one couple actually, and even then the DH had an affair at one point).
I met my first H when I was 16. Got engaged at 19, and married at 20. He was my first proper boyfriend, and the only man I'd slept with. I can remember having the feelings you describe, once we were many years in and with 2 kids. However, I don't think I would have acted on it....only he was unfaithful many times, so eventually the marriage ended.
I then slept with quite a lot of men, and got it out of my system (so to speak). Then decided I wanted a long term Partner, but when I started dating I was very picky about all the physical things I wanted : things I hadn't really had the chance to pick with first H, for eg, first H was short and thin, not very manly, but I like big tall men! Anyway, I met my now DH, who is 6 ft 3 and very manly, broad shoulders, built like a rugby player etc - just my type. We are now almost 12 years in, and honestly my head would never be turned.
Long story short, I think in most cases people need to do a lot of shagging around before they settle down. If you haven't done that, I would imagine this is why you are feeling unsettled now - especially as you go through your 40's, because subconsciously you know that time is running out (kind of) for you to have loads of sexual fun.
Not sure what the answer is? I think it boils down to whether you still find your DH attractive and sexy? If so, I think that's a sign you can revive things. If not, if for eg he feels like a brother to you, then I'm not sure anything can make this work.
I will say this though, breaking up a family is heart breaking. My DD took YEARS to get over it....years and years. I had no choice really as first H was cheating so much, but boy the guilt was horrendous. And the ferrying between 2 houses was just awful. So think very carefully.
One other thing : if you do embark on any affairs, please bear in mind that men who sleep with married women do so, because the woman gives out sex with no expectation of commitment. They will promise you the earth, whilst using you for sex, and you will be one of many women he's shagging. You will not meet any decent men whilst you are still married to your DH.