Regardless of whether he is or isn't up to anything with X, he has given you good reason not to trust him, so I'd stop viewing this as you being problematic. You're having a healthy emotional response to someone who has cheated on you and consistently lies to you. We're not meant to respond to being treated like that with 'Oh, but I trust him implicitly, regardless...', because that would make us naive idiots.
Listen to what you feel. What is it telling you? 'I don't trust him', 'I don't feel safe, emotionally, with him', 'He would rather lie than prioritise my feelings' are all likely. Once you've listened to what your feelings are saying, because whether they are right or wrong, you have to find a way to live with them. So, even if he was Mr Perfect, and you didn't feel safe with him, you still would have to leave, because it's not good for anybody to be in a situation where they don't feel safe.
This isn't about who is right or wrong. This is about you doing the right things to make sure that you feel right about your life.
Boundaries: tell the person if something they do hurts you, and leave them if they choose to keep doing it, because doing that thing is higher on their priority list than you feeling ok. That's it. You don't have to think about whether you are 'right' to feel hurt, it doesn't matter.
If you've told him it hurts you when he lies, and he carries on, then he has no respect for your feelings. Is that a sustainable relationship, for you?