Oh dear. I was getting ready to tear into you for having an affair with him, but you were 18 (and he was 41!) when you got together so (and I'm sure you'll hate me saying this!) he almost took advantage of a CHILD.
You were so young. And I say that as someone his age, who moved in with a boyfriend when I was 16. No bad consequences, just looking back now, I'm old enough not to feel embarrassed or defensive at saying that 16 (and 18 as you were) just isn't that old.
He cheated on his wife with an 18 year old less than half his age. He has ALWAYS cheated. Now you've got a child, a pregnancy and a nasty bit of work for a husband.
Let me tell you this: HE DOESN'T HAVE TRUST ISSUES.
That kind of language is the beginnings of making excuses for him. Oh, poor thing, he just can't trust, he's not accusing me of cheating because he utterly disrespects me - it's because he has "issues". No, it's because he's nasty.
He's your husband. He's supposed to love you. How about you tell him, once and once only that if he has issues then he should go and see a counsellor to work those through - and in the meantime, making accusations is ABSOLUTELY unacceptable? And mean it.
And do read up on abusive behaviour. The accusations are such an obvious sign, from the outside.
You've posted here because you know this is wrong. Half his age, but more mature than he is - you can feel it's not right, and not justified.
Good luck, don't lose your life to this tosser.