I left my wife and two young children for a colleague with two young children. We had an emotional affair and I told my wife about it and separated and it became physical. I was selfish, emotionally immature and then some, full of self pity because my childhood had been shite.
We moved in together straight away and the kids began staying over straight away as well. They are all mid/late twenties now. I'd say two of the four are doing OK, one not so much and my daughter is badly scarred. None of them express our affair and subsequent marriage as the cause of their trauma but I'd bet my life that it is a huge part. If we'd managed the children being with us better would things have been different for them? Maybe, maybe not. If we'd not had an affair then of course yes.
My AP was incredibly jealous from the day our new front door closed. My ex wife was called a weirdo, freak and so on (not in front of kids, even then I would not have stood for that). I was interrogated every time I had contact with her, not allowed to go into her house to collect kids. My phone was requested for checking every week. After all, she knew I had cheated and moreover had cheated herself before. Affair with a married older colleague when she was in her 20s and most of her prior relationships only ended when she'd found someone else.
We are still married 20 years on. It has been grim. In the end I confirmed her fears when very drunk as I kissed a friend. I told her straight away, I think as I was hoping it would end the toxic mess but it didn't and I was too thick/afraid/immature to just end it.
I was ashamed of the affair from the off but too spineless to end it and so doubled down to try to show the world and myself it was worth it. My wife has rewritten history to say she had told her husband the marriage was dead so her remaining married to him in a home he mainly paid for until she found someone else was fine. So I am the cheater and she never has cheated once (the previous affair she told me about cannot be mentioned. I tried once in our marriage counselling, it did not go well).
AP's family hated me and her for a long time, her ex husband was invited to celebrations in her family not me or her. We go now but it's just a thin veneer of civility over the distaste. My children still aren't invited at all. My tiny family were OK with her and her kids.
I lost most of my friends and my AP didn't really have many anyway. I never saw any of my ex wife's family again.
These days, when asked out of earshot of my wife, how we met, I tell people the unvarnished truth. Through a sordid affair. They then have the choice to work with me/befriend me, keep mw at arms length or tell me to fuck off. And at least I don't have the shame of having made some lie up to add to the shame of the affair.
I did, eventually, grow up. With a lot of hard challenging therapy and reading I learned to like myself more which makes it easier to be less selfish and more open to others' suffering. I have apologised to my children with no expectations that they forgive me. I try to be a better man. My marriage is generally grim, sexless (her choice) and dull, but unless I can leave my wife financially comfortable then I stay. She wants me to stay to pay for house, netflix etc. It's like her last marriage all over again. She watches a fuck load of romantic comedies and American hospital series and points out how unromantic I am compared to the characters. If I try to bold hands I'm pestering for sex, if I don't I'm a cold loveless man. I do the life admin, the housework and imagine myself living on my own.
I left my ex wife everything financially, house, all savings including mine, all her pensions which were far bigger than mine as I'd been a SAHD until just prior to the affair. I knew then that if I hadn't AP and I would have lost anything we had taken. She didn't have anything from her ex husband as he retained custody and their house. Hence as we started from scratch in our late 30s we don't have much now. Hence wife's reluctance for me to go.
So OP, maybe your journey will be better. Maybe not. I don't know the stats, I'm not sure anyone does, not in depth, but I'd be willing to bet my shitty little mortgaged house that most relationships that begin as affairs, whether they last or not, are overall less happy than those that begin without cheating. You may not be an awful person, not all cheaters are sociopaths, narcissists or lack any good qualities. You have done an awful thing. How you make the best of it for everyone exposed to the fallout is up to you. If I'd had a spine after I confessed the affair I'd have said to my wife that I had removed the AP from my life, given my wife the choice of what she did next without any attempt to persuade her to one course or another and abided by her decision. If I'd grown a spine a little later on I'd have never married AP, left her and focussed on being a single dad to my kids.