sometimes good people do bad things. What separates one from the other is recognising that what you are doing is wrong and taking steps to put it right.
It is possible to fall in love with someone else but it is not possible to build that love on a backdrop of lies and deception and hurt with a successful outcome at the end.
Picture your life in two/three years time. Your husband has left you, you are seeing your children only 50% of the time. Your mutual friends have all taken your husband's side and you are now friendless, your inlaws no longer speak to you, and even your own family think that what you have done is wrong and despicable.
If you are still with the om and he has left his wife for you you will both be in the same scenario together but worse still you will be building your new relationship on mistrust and the knowledge that you are both capable of lying and deceiving the other. After all, while you were both deceiving your partners you will also have been deceiving each other, about when you were having sex with your partners, what kinds of relationships you were really both in, etc, and every time one of you gets a text, or a call, or goes out you will wonder....
And if you haven't ended up with your om and are lucky enough to be with someone else the guilt will still be there. You will look at the honest relationship you potentially have and you will look back to what you have done to others.
Is this where you want your life to be? no good can come of this. none. There is no happy ever after from an affair. Sometimes people do fall in love with other people, sometimes marriages end, and sometimes affairs happen. But if you love your husband then you need to end this affair because it is the road to destruction.
If there are issues within your marriage then you need to explore those so you don't feel compelled to go down the road of an affair again. But meanwhile you need to cut contact with the om. Change jobs if you have to. and never see or speak to him again.