I thought you lived together. I don't see that happening any time soon and that's because he doesn't want to.
He doesn't want to be a full time dad and have to deal with a baby. He's been there and done that. If he and his wife wanted more kids I'm sure they wouldn't have left an 8 year gap.
It could also be that as he doesn't live with his DD, he doesn't want to live with you and your baby, in case his DD sees this as rejection. He's feeling guilty.
You're effectively a single parent.
He's not divorced because he doesn't want to be. Maybe he's hoping for a reconciliation and is hoping that by not living with you (when he is free to do so), it shows he's not in love with you.
It seems all to easy to say his wife is behind all this, but remember this is a man who stepped out on his marriage with a colleague and got her pregnant. He's not scared of her, the current situation suits him just fine. It's easier for him to tell you his wife says xyz, when it's really him who isn't keen.
When you say easier for everyone...what evidence do you have that it's causing problems for him, his DD or his wife?
If you didn't have the baby, he wouldn't have moved out. Now he isn't mad about the situation, but he's kind of trapped...by his own foolishness.
This really proved to be a very costly extra marital shag for him, which it sounds like he will always regret.
He hadn't got to know you, to even fall for you before the pregnancy. His idea of having an affair as a means of escaping from the domestic drudgery didn't get to happen and he got into an even worse situation... only now he's financially worse off for his philandering.
I can imagine if you ended it he'd be relieved.
He'll hardly be a catch to a third woman if he tells the truth of having an affair, getting his OW pregnant and then leaving you.
Is he on your child's birth certificate?
Does he pay regular child support?
Does he enjoy time with the baby?
Do you ever see his face and demeanour when he's alone with the baby or doesn't know your watching?
I know someone in a similar position, who said the answer to the last question is why she ended her relationship. He smiled with the baby when she was around, but a few times when he didn't know she was there he looked so sad with the baby and she once heard him crying.
When she ended it, he admitted the immense guilt he felt because the baby was evidence of his infidelity and the reason his wife divorced him. She would have forgiven an affair, but not with a love child.
He just couldn't bond with her the baby), because of the history and after they split he hardly saw her...but always paid child support.