Op, I am so sorry for you.
You fell for someone hook line and sinker.
He seems so lovely and nice and you just can't imagine he isn't.
For a moment imagine yourself 10 years down the line, when he did leave his wife, and you have kids, and then he disappears/works late once a week. Where is he? he is having an affair, just like he is doing now with you.
This relationship has no future, you know that, I think, deep down.
You need to find the strength to walk away. For your own sake.
He isn't going to leave. Every single one of us posting is saying the same thing. Why? Because this is the same story spun to thousands of women over the years, and they never do leave.
If you really and truly believe that he is different, then do this.
Leave. Cut contact.
Either he will beg you to take him back, without leaving his wife.
Or he will leave and come after you. (but he may not stay, he may swing bakc and forth between you and his wife)
Or he will let you go and look for the next woman.
But you also must ask yourself if you want to be the person to break up that marriage, and how will you look those kids in the face when they are doing their weekends with their dad, knowing that you broke up their lives?
And if you want to be long term with someone capable of this level of deceit?