Op, there's clicking on gay and trans porn while on porn sites ... .. then there"s going on swinging sites (which I imagine you'd have to join up to etc. in order to browse).
To me, that's the behaviour of someone who's gone past looking at actors/amateurs passively, and moved into "seeing what/who is out there in real life" (and probably, specifically in your region).
One might say that the swinging sites are also just for fantasy.
My points would be;
A. The porn wasn't enough for fantasy?
B. When posters on here have caught their partners on swinging sites, the partners were - of course - always only looking, and it was only for fantasy. But in some cases that was later proven not to be true (sometimes there had been image and video swapping or interactive sexting or videoing, sometimes meet ups).
C. When you've signed up to a swinging site and browsed it, you're a click away from interacting with someone - whether it's messaging or exchanging pics or worse. Escalation would be easy.
I don't think the situation you unfortunately find yourself in should be dismissed or white washed by discussions of his background, his abuse, his counselling.
He has done something that many people would consider highly highly inappropriate and something that could easily lead to digital or real life cheating.
And that's just the best case scenario on the swinging site.
Personally I think he's raised this, after all these years, at least in part because (no matter what the veracity) it's a very effective tool to distract, deflect and get you to back off about him being caught on at least one swinging site.
Even if he ended up on swingings site due to abuse; he's gone from gay & trans porn ...to joining a swinging site to look at 'real life" transexuals .... what next?
Why are you supposed to put up with something (to your knowledge, which may not be complete) that pushes rather hard on the boundary of fidelity?
He's had decades to get counselling and didn't. He presumably knows he's escalated his activity (onto swinging sites) but hasn't.
But now it's your role to support?
(And that's just what is known, personally I've seen too many poster's stories on here not to worry there's more).