I was in a similar situation last year. I knew he had a gf, but what started off as mates having a drink went too far. I think that was always his intention now, and I was so dumbstruck that he wanted me in the first place that I just went with it.
It didn't take long for me to realise that he only wanted me for sex. Even then i still went with it. Even when i wasn't allowed to be friends with him on fb, even when he'd blank me when i saw him at work and even when he more or less deny my existence.
I got sick of being invisible, but he was like a drug, and everytime we'd fall out he'd come crawling back, txting and ringing me and most times i'd give in.
Then his split with his gf, and for a brief moment i was excited, but i soon realised that it still made no difference. He didn't want people to know he would been seen dead with me, it was suddenly so obvious.
He then got a new gf, and within a few weeks of them getting together, he was back messaging me, wanting to come round.
I did the only thing i knew would get him out of my life for good. I told his new and old gf's excatly what he'd been up to.
He was angry, said that i'd always said I wouldn't tell a soul, and he thought we were friends.
It worked though, he has not contacted me since and goes out of his way to avoid me if i go to where he works.
The thing is, you have to be ready to make the change. Until you are ready to do something about it then you will keep slipping back into accepting contact from him.
Good luck