Hi shackled
a bit of background
My DW was married when I met her. She had been married five months, been with her (now)XH for about ten years. He was staunch RC, Knights of St Columba, the whole nine yards. They went on an engaged encounter weekend as well, although she is CoE; they got married at her mum's insistence in her CoE church and got an RC blessing the same day in his church.
None of that mattered because when we met we clicked and she moved out within a month. She had married him for the wrong reasons (it was the easy, comfortable thing to do, not the right thing to do) - her mum and dad were divorcing at about the time she got married - her dad wouldn't even stay at the reception cos her mum was there. He didn't stay even at our reception (five years later) as his XW was there, although when my SiL got married two years ago he did manage to stick around for the wedding breakfast. So she was just looking for a way out of that environment, which marriage offered. She wrote to him through a solicitor asking for divorce after 5 years as irretrievably broken down - he wrote back saying divorce now for adultery with unnamed 3rd party (ie. me) which she accepted.
I guess I'm trying to say that you're not alone in this type of situation. Get some legal advice - you may qualify for legal aid, depending on circumstances - and unless the adultery thing is a problem for you morally or on religious grounds, then who cares? Given that you have no kids or others ties to him, you should never have to see him or his family again, so - quite frankly - sod 'em. You are just as deserved of happiness as he is and he has no right to poke his nose in where it isn't wanted.
As long as you have been truthful you have nothing to be ashamed of. DW for a long time had guilty feelings - fair enough, her XH was probably hurt and confused; I would be (and I'm not proud of the fact that I was the OM; but I am proud that DW and I have made it through almost ten years and have two beautiful, healthy children to show for it) - but in time your x?h, just like my dw's, will see it is for the best and move on himself.
All the best
Saul