because he's been disgusted by his friend who did this a lot (not friends any longer)
So, at best, he's projecting how his low moral friends acted; onto your friends and you?
He has no right to do that.
Also I wouldn't count expressed disgust/disapproval of cheating as proof of anything.
I've seen several posters on here say they found out their partner, who always expressed disapproval of cheating, had cheated on them and they couldn't get their head around it.
People can be very contrary, it could be a case of "the gentleman doth protest too much", it could he a case of making sure you never question his behaviour because he's emphasised his disapproval of behaviour like that so much.
Sometimes someone who hasn't always acted inappropriately becomes convinced parity/karma will get them and their partner will get caught up in cheating. That's the root of their paranoia.
It's not always straightforward.
For someone whose (ex) mate was a prolific cheater, he's very happy to keep going out on his own ..... But somehow your mates (presumably not ever being proven cheaters) are bad and he doesn't want you going out.