Cheating cannot be condoned but I don't automatically think someone cheating is a bad person. Relationships are hard and some circumstances make cheating more forgivable than others. So without knowing your motives, no judgement from me.
As for the advice, the first thing you need to do is decide if you want to have a proper relationship with your affair partner.
If not, end it with him and end it with your husband. Probably need to do it back to back in case affair partner decides to tell husband before you get there.
If you want a relationship with the affair partner, I would have a frank discussion with him, let him know you want to leave your husband, but tell him that you need time before you can go public with your relationship. Assuming he is understanding then I would break up with husband and not tell him about the affair. It doesn't ultimately change the situation and it saves hurting him further. People will no doubt disagree because they're morally perfect and I am not, but you need to do the best for your children and yourself too.
Note that if affair partner is not understanding, reconsider if you want that future with him after all.
I'd then go through the formal process of breaking up - it's tough but must be done - and I would keep things very much on the down low with your new partner. That doesn't mean not seeing him, but it means maintaining the discretion you have had for the last nine months.
When the storm has blown over, introduce your new man as your new man.
Might people find out further down the line? Possibly. Perhaps you might even want to share the full truth with ex and / or kids further down the line. But they will all hurt less when the situation isn't so raw.
If it helps, my husband's parents broke up in childhood and it later emerged his mother was having an affair before the break up happened. MIL and affair partner are still together 30 years later and everyone got over it. His parents are actually back on pretty friendly terms. People move on.