Oh, FFS. He isnt abusive. She isnt feeling abuse.
This is a person who has found out that his spouse had an emotional affair and then left him to be with that person. Anyone who has been married and had children and then gets cheated on, lashes out and gets upset. He isnt abusive. The OP hid/lied about/ignored her sexuality, married a man, had children and then left him for someone else. He will be going through heartbreak and confusion and is losing the family he thought they built together. Life is going to look very different from here in out and that takes time to accept. He is allowed to grieve the loss of the life he thought he would have.
Women post on here all the time about their husband's affairs and they derail their reaction; shouting, begging, guit tripping. They get support and told that he deserves to hear how hurt she is etc. But a man doing it to a woman who cheated is called abusive. This sit can be disgusting sometimes.
OP, you had an emotional affair and now you've left your husband. You cant make that sound nice. You're hiding behind the romance of coming out and meeting this woman and things just clicking but it doesnt change what you did. An emotional affair, then leaving your husband for someone else. He is hurt. You're going to have to deal with that. It will pass.
You cannot carry on living with him just because his finances support you. You'll have to manage just like everyone else. You leave, you figure out your own housing problem and you come to an agreement about childcare. He is an involved and loving father and he isnt choosing to leave so please dont even think about taking the kids and only allowing limited access because you think they need you more or whatever. Even time. Do your absolute best to put their needs first and make sure they get even time with you both as much as possible.
He wants you to stay because he probably hopes for reconciliation. You know that you dont want that. Staying is cruel and will prevent him from healing and moving on. He deserves to meet someone else and be happy, just like you have.
You need to divorce. You need to leave the home and find somewhere else and sort out contact with the kids.
You've found the awakening love. That's great. But you dont get to have that and maintain your marriage and keep his financial support (child maintenance is different of course). You need to go out alone now and do the best you both can for your kids.