Yes, I think you're suffering more than he is and more than the men involved ever really do. Putting their OW 'in a box' seems very commonplace and I think that men can generally compartmentalise very well indeed. This is why when you read of a couple having a row, the woman will often be unable to sleep, playing it over and over again in her head - whilst the man, oblivious, snores away contentedly.
It's not really different for affair partners. The only difference is that women tend to fall in love, read more into feelings and behaviours - positive or negative - and possibly misinterpret, whist the male partner thinks little or nothing of anything that he's said, it's much more 'in the moment'. So when you say, 'both of us can get through this', I think you're attributing more care and concern to your OM than he actually needs. I'm sad (for you) to say that 'lip-service' really springs to mind here. He has his whole life away from you and a completely separate 'slice' of something else, with you. There's nowhere for this relationship to go and, it sounds to me as if you and he really are at the end stages of it.
Now, if you can take a leaf out of his book and draw a mental line to separate what has been and what you're going to do next, you will not drag that baggage with you. If you can see it for what it is/was, ie. an affair, a separate and distinct relationship that always had a finite lifespan, then that may help you to get over it. I'm not concerned about him because I think men process this differently. He's already over you. He may miss you, think of you sometimes but he's already done whatever little 'healing' he needed to do. You haven't got there yet by the sounds of it and I know how that feels.
You need to look at you in isolation, just you, not as he and you, a couple. What were you before you met him? Strong, independent, happy? You're going to be that way again. Your body and mind won't let you 'grieve' for him forever, that's not how we work. How about trying to cut out the 'grief' over the loss of the relationship and see it as a positive thing (if you can). This was an ADDITION to what you already had, what you already were and you have taken very positive things from it. You've learned all you needed to learn and now you're nearly ready to do something else... whatever that is.
Don't wallow if you can help it. If you must, make it time-based and reduce the time you allow yourself to do it, every day. Know that he will not be doing this or need to do it. That's just how it is. He is not suffering. He accepts that you are, because it suits his male ego to know that he matters to you (however nice a person he is), so he tell you that he is too. Don't be under that illusion though because it will hold you back. He isn't suffering - and you won't be for long either.
There's absolutely no need to give yourself a beating. That is a waste of time and energy. You've done something that you say you've benefited from in terms of confidence, (irrespective of the rights and wrongs). This was never going to be a 'forever' thing because it can't be. That's what's making you beat yourself; you feel pain and you're struggling to see where there's coming from so you are artificially manufacturing some by beating yourself. It's pointless. Stop it.
If guilt is playing a part there then face it. Acknowledge what you've done and make a decision then and there that you won't put yourself or others through this again. Then let it go. It is that easy, really, because there's nothing else to do with that guilt. It's your conscience and it's pricking you. Salve it with knowledge that you won't do this again. Protect yourself from ever doing this again.