I think op I'm going to remind you this man has never once tried it on with you.
For all the things that could be friendship or flirting, the key here is he has plenty of opportunity to try something with you and never once chose to do so. He doesn't seem to be a shy insecure character from what you're portraying, so this would indicate if he wanted to make a move he would absolutely have done so. He didn't.
You're portraying him like a man who chased you, like a boyfriend, but I think that's because that's how you saw him or wished to see him.
As such, I think you've maybe misconstrued a close friendship for him wanting to be romantically and physically involved with you, but you seem unable to articulate why he never went there, and why he's happy to tell you about his relationships with other women.
If you're missing something in your life that he was fulfilling, a partner basically, then it's best to focus on that. If you cannot be friends with him if he is in a relarionship, or if this has proven to you he doesn't wish a romantic relationship with you thenyoure right to pull away. If you can accept being good mates, and him being in a relarionship with someone else, then stay friends.
I think though for your own mental health it's better to pull back.