PerfectPanda, there is another option here. Find someone else. Somebody who gives you the emotional connections that you need, with whom you can have an easy and honest relationship. That's the end game though. You have to put some work in to get to that...
Putting your children to one side for a minute:
First step - OM. I could (and have) understood the appeal of somebody else. Your OM is single though, has nothing to lose and still he cannot and doesn't give you what you need. He offers you nothing and you grasp that with both hands.
Would you leave your marriage for him if he offered himself to you? I think you might, like a shot - but he won't. He's not invested but very much likes the idea of your investment in him.
Second step - Husband He knows some or all of this. His feelings for you will have changed now, whether you acknowledge that or whether he does or not. They've changed. You are no longer the 'whole' person that he married. Regardless of his faults and transgressions, you won't be able to get that back so, what do you do with this information?
Come clean, make up - get to the bottom of what went wrong with you and what happened to make him hold himself back from you with regard to showing you his emotions and love for you? Possible but only if you and he would want to start anew. Do you? Putting your children aside, would you want HIM again, your husband? Would you be prepared to put in the work to find each other again and want to do so because you see a point to your marriage?
Staying together for the sake of the children is a dreadful thing to do to them. So many parents think they can do this successfully and have little regard for the resultant damage that they cause because far from doing what is best for the children, they were jointly selfish in not letting go of their marriage to become better people and fulfil their obligations (separately) to become the parents they wanted to be for their children. Stay in your marriage if you and your husband genuinely want to be in it for each other - or get out, for the sake of your children.
Assuming that Steps 1 & 2 aren't applicable then:
Step 3 - End your affair - and your marriage - and find somebody with whom you will find all that's lacking
Ending your affair is something that as pp have already said, is like getting rid of an addiction. Difficult yes but not impossible and it doesn't have to have a long lead in time to recovery. You sound a reflective person, I would do that. Invest in a good vibratory (Lelo are very good) and get ride of the OM who gives you nothing.
Ending your marriage starts with discussions with your husband and plans to separate everything that you have together - with custody and housing arrangements for your children added to the mix also. Assuming that he's as good a father as you are a mother, your children will need both of you. Get professional help to bring about a fair ending for you both.
If you're wise then, once your marriage is over and arrangements for the children have settled down, spend time reflecting on what you are as a person, who you are and what you want out of life. Then start doing those things. Do them for the wish to do them, not to find somebody else because to set about finding a partner from desperation means that you will modify your 'list' of ideals to make any partner 'fit'. Don't do that - you deserve someone for whom you are lovable and desirable and worthy of being in a loving partnership with.
We are all capable of unedifying behaviour, every one of us. This behaviour is causing as much damage to you as it is to your family and the associated guilt is an additional burden. You don't need that in your head.
Reading your later posts it sounds as if you're teetering a bit in getting rid of the OM once and for all. What's the worst that could happen? Take charge of you and your life OP and fix this. You can do it. 