Right, so it's becoming clear that the relationship with the OM was never built on an ordinary friendship that changed into something else. It was always built on sexual attraction so it was an affair in your heads the moment you met. I think you need to acknowledge that because I get the sense that you think the affair actually started some time later, which is why you're insistent that the DV occurred 'before anything had happened' with the OM. Something had already happened with the OM though. Just because the mutual attraction was unspoken and not acted upon, makes no difference.
What usually happens is that as soon as someone else has taken up occupation in your head, the process of withdrawal and distance from the existing relationship starts in earnest.
When the DV incident occurred, was there anything else going on in your husband's life? You say you were both drunk, so was this violence connected to drink on his part and has aggressive drunkenness been a problem in the past? In other words, was this behaviour completely atypical?
I think the only connection between this incident and your affair is that it became part of your permission-giving process to move the affair on to the next stage. That might sound very calculating, but when ostensibly 'good' people have affairs they usually need to have a rationale for doing something they instinctively know is wrong. Sometimes it's consequence focused (this doesn't have to hurt him/her and he'll never find out.) Sometimes (especially in previously good marriages) it's engineered so the disharmony and negative reactions from a partner are purely the product of the affair, but the gap is created that is necessary to act. Sometimes it's a bit of both. In marriages that have always had problems, previous incidents are resurrected and magnified and given far greater significance than they were at the time.
DV is at the extreme end of the scale and would for many, be relationship- ending in its own right. The problem for you is that the affair has impacted your judgement about it.
There are likely to be competing pulls going on here.
Had you not been having an affair and not felt some guilt about that, you might have decided this was the death knell to a marriage devoid of sexual chemistry. But your affair and guilt might have prolonged a marriage that should have ended there and then.
Then having stayed in the marriage, the DV became a permissible reason to do something that you would have done anyway i.e. progressed the affair.
I think you might need to be very honest with yourself about that. If it wasn't the DV, it would have been something else.
If you're going to end it with the OM, make it final. Any suggestions that there might be future scope if your marriages end is effectively signing those marriages' death warrants. If it's easy to avoid the OM, this really is about going cold turkey and permanently deleting any means of contact. It will require great willpower and it's likely you'll need some other prop such as your own counselling to get you through it.