The ferocity of his arguments and anger are leading you to believe he is not doing anything with this woman.
I can imagine he is acting so offended that you would dare accuse him.
Don't fall for this, he's acting.
This woman, what do you know about her, is she attractive, his type ?, is she his age, younger, newly single ? I would suggest finding out about this rival you have, your h is obviously playing KISA to her at the moment. Very hurtful and he's not going to reassure you.
Do you have grown children or a friend, could you join the gym and make them uncomfortable. He would be embarrassed infront of his own children.
Another poster suggested he may be doing this to get thrown out, well he may be at the moment, they often start bad arguments and leave under the guise of not wanting to argue. Doesn't mean they want to be gone for good, they're usually hedging their bets at this point and just need a little time to act single.
His behaviour up to now has been abusive, recognise that op, you are not unreasonable, no matter how much you scream to be heard, he's not listening and you are wasting valuble energy trying to make him see your point.
I would amass your financial information, ducks in a row and go to see a solicitor, you don't need to file yet, just get a half hour free of advice.
Tell him he's giving you no other option.
It's time he made a decision, he's not going to like it and his behaviour may well go underground but at least you will have you answer, that this man is an abuser and was never capable of reciprocal love.
At the moment, you are offering him a get out of jail free card and he is not taking you up on this, he is defiantly wanting his cake, I'm affraid it's your time to exact the consequenses now.